Big Data is the datasets that are abundant in the realms of volume, variety, velocity or variability to the extent that they cannot be processed by means of ordinary equipment and require matching tools for handling (NIST, 2016). Through these realms Big Data is differentiated from ordinary datasets such as enterprise or corporate ones. Usually, the amount of information transferred between departments inside an organization or to a different company is meant to be perceived by humans the instant they receive it. Such data includes payment orders, lists of components, tasks, employees, payrolls, and other information. Typically, organizations do not require sophisticated equipment to handle such information, as it is easily processable by office computers. Big Data, on the other hand, requires the machinery several dozen times more powerful than such that a company uses for daily information exchange and processing.
The software for Big Data is usually identified by the ability of a tool to store, access, and handle datasets within a reasonable period of time. Big Data is usually stored separated into multiple parts in different places. An adequate framework for processing Big Data is often defined by how well it performs in terms of memory management, communication and networking, integrity and reliability, security and privacy, and information representativeness (Al-Jaroodi & Mohamed, 2016, p. 428). It is paramount to note that the ability to collect and generate incomplete data is also a characteristic feature of Big Data software. All of these items define Big Data frameworks as highly sophisticated and powerful tools that outmatch any other applications by the above-mentioned parameters.
One of the practical issues that an individual or an organization may encounter is the lack of understanding of the goals or final results that they wish to achieve through the processing of Big Data. Careful planning and goal-setting is often in order to address this issue because it could save an enormous amount of resources. The establishment of Big Data management system is a cost-intensive endeavor that under inadequate guidance might inflict sizable losses. Another typical issue is matching ‘the V’s’ of Big Data and the framework that is capable of managing it (NIST, 2016, p. 4). Again, planning and calculating at least preliminary scale of operations with Big Data that are needed to achieve the desired results are essential for informed allocation of resources.
There are also ethical issues that individuals or companies might face during the management of Big Data. One of them might concern the algorithms used to process and interpret datasets about human behavior that produce inadequate or offensive results. Such pattern may be exemplified by predicting people’s choice on the basis of data correlations without the account for errors and external factors. Another problem might lie in the sphere of composing individual profiles from the datasets with different parameters. For some social groups a resulting profile might be insulting or disturbing. The usage of such results may involve generating further bias and minor social unrests. Non-recognition of the impact of each contributor, who participated in the creation of Big Data, can also be viewed as an ethical concern. With a tendency to perceive large data sets as something abstract, people often forget that some valuable parts are produced by individuals whose rights need to be managed. Finally, data collection may be violating some people’s freedoms, which puts another emphasis on the accuracy of a tool design.
References
Al-Jaroodi, J., & Mohamed, N. (2016). Characteristics and requirements of Big Data analytics applications. In 2016 IEEE 2nd international conference on collaboration and internet computing (CIC) (pp. 426–432). Pittsburgh, PA, USA: IEEE.
National Institute of Standards and Technology (NIST). (2016). NIST Big Data interoperability framework: Volume 1, definitions. Big Data Information. Web.