Essay Example on Big data management is the organization governance

Big data management is the organization governance and administration of large volumes of data that are both structured and unstructured The goal of big data management is to ensure a high level of data quality and ease of accessibility for big data analytics applications and business intelligence There are numerous forms of big data but the most recognizable form is the form found in Hadoop also known as Apache Hadoop Understanding that Big Data is a technology that is capable of holding massive volumes of data in inexpensive storage devices processing data via the Roman Census method and storing data in an unstructured format will assist is understanding the key management issues involved in big data management Data warehouses are central repositories used to support decision making within organizations and companies A data warehouse system is defined as a subject oriented integrated nonvolatile and time variant collection of data created for the sole purpose of aiding the decision making process by the management Khouri 2017 

The Inmon approach depicts that a normalized data model is designed first Then the dimensional data marts which contain data required for specific business processes or specific departments are created from the data warehouse Data warehouses help the higher level management to make strategic decisions by utilizing the same information used for tactical decision making at tactical levels only in greater detail Management Issues in Traditional Data Management Data Warehousing Data Warehousing and Online Analytical Processing bring multiple benefits although several limitations have been reported These benefits include but are not limited to providing fast and interactive insights ad hoc analysis multi dimensional view of data and visual exploration slice and dice drill down Traditional OLAP solutions struggle in coping with the huge increases in data volume from businesses The current business dynamics have expanded thus requiring excessive data management and additional features to foster the management level decision making process Two major management issues associated with data warehousing are identification of processes and the choice of development options Identification of processes is an issue because the development and implementation of a data warehouse is a major event and organizational mishaps are highly likely to occur

 Each of these processes has its own problems which cannot be solved by only concentrating on the data regarding each process itself Problems underlying each process that must be supported with a data warehouse need to be fixed prior to the implementation of a data warehouse Regarding choice of development there is a wide variety of software options available and choosing the most appropriate option can be tedious because data in existing operational domains of an organization may usually exist in multiple formats highlighting the need for cleansing transforming and standardizing it for access by the end users A profound determinant of effectiveness and efficiency of data warehouses is the availability of clean data which impacts the quality of management reports Management Issues in 

Big Data Apache Hadoop Apache Hadoop has proven to be one of the most effective means of data management today However this form of Big Data management still has its issues The first being that it is new to big data Since Apache Hadoop is a relatively fresh form of Big Data management a majority of managers today lack hands on expertise Due to the lack of expertise three top challenges are being faced These include inadequate skills or staffing inadequate data management infrastructure and immaturity with new data types and sources

Systems like Hadoop do not have the ability to make sure of the availability of numeric account balances and valid dates whereas relational database management systems can ensure conformity to these schematics Awadallah 2011 indicates some key management issues such as complexity in data integration lack of business sponsorship and lack of governance to be major barriers to successful management of big data today Another key management issue with regards to Big Data Management is deciding the architecture of a big data management system For instance Graham 2014 highlights Structured Query Language on Hadoop to be one of the more pressing issues for those who prefer open source Apache SQL language features throttles a manager's ability to perform self service analytics and reporting which dictates labor costs over IT for the development of reports in Java Hadoop lacks in database operators indexing strategies advanced memory management dynamic workload management and concurrency Hadoop does not meet the requirements for being a complete data warehouse and can be referred to as a technology whereas data warehouse is architecture a technology cannot replace architecture Applications Data warehouses are more dominant when it comes to applications even though the application of traditional and big data management is widespread 

The banking industry deals with large piles of data on a daily basis and utilize data warehouses to evaluate business concentration risk exposures consumer asset data spot market trends and to support management planning Government and Education industry utilize data warehouses for accounting payroll procurement HR university finance student demographics and room and course scheduling Apache Hadoop is being applied by IT related companies such as Amazon Yahoo Google IBM Quantcast Rackspace and Facebook These companies utilize Apache Hadoop for the purposes of content optimization feed processing search index machine learning spam filtering B2B data management via websites and other similar purposes Future Trends Awadallah 2011 highlights Apache Hadoop as the modern operating system which is anticipated to have a big impact on big data management However some believe that data warehousing is a complete solution for contemporary data management practices Nonetheless the Big Data revolution has triggered astounding changes with regards to collection management and analytics of data There is predicted to be a consistent place of data warehouses in data analytics in future however unless big data management through Apache Hadoop makes an effort to remove the barriers its place in future data analytics may not be assured