Adoption of Big Data Analytics: Issues and Challenges

Comentários · 2902 Visualizações

Organizations in the modern times have been flooded with vast amount of data for use in their business processes from various technological sources and that are even not the systematically evaluated one. This poses a challenge for organizational strategic knowledge-based actions. For several decades now, data mining techniques were used in this respect, but recently, a relatively newer concept ‘Big Data Analytics’ has marked the escalation of new openings in the examination and use of massive data pools. However, the appraisal of earlier literature on Big Data adoption process indicate a scarcity of literature on the subject in Indian context in comparison to the developed countries. Therefore, it is an endeavor to understand the issues and challenges that might have intervened the adoption of Big Data Analytics in Indian organizations. The investigation of extant literature confirms that organizations are not hesitant in incorporating Big Data tools and techniques into their business strategy. Moreover, the benefit of Big Data services, quality of Big data, absorptive capacity and the costs of Big Data are some of the key factors which promote adoption of Big Data Analytics in organizations. Even, the Indian organizations confirm use of the Big Data Analytics promising for their businesses.

Introduction

Today the organizations are flooded with huge volume of data for use in their business decision making processes. The unconventional cost effective technologies like expandable storage feature are enabling these organizations to yield and receive enormous data, and even creating possibility for a data binge which allows to collect data easily but not the methodically analyzed. One big challenge for the organizations is of identifying and extracting right kind of data obligatory for the management of business operations. Also, as the size of the data increases it becomes more intricate to be transformed into meaningful information in a precise time. At times when life cycle of products is continuously shortening, it becomes specifically important for organizations to be capable of converting relevant data into information, knowledge, and finally taking appropriate actions to make knowledge-based key competitive differentiation for influencing favorably the rivalry organizations. For several decades now, data mining techniques were used in this respect, however, a newer concept ‘Big Data Analytics’ has appeared to the surface for exploring data and utilizing the same in decision making activities. This technology used in capturing and analyzing pools of massive data has marked the rise of new opportunities in business industry.

The multiple data channels like internet, peer networking sites, Twitter, Facebook, peer click streams etc. have come into existence with the recent developments in information technology to explode the data and given the name ‘Big Data’. Basically, the augmented internet bandwidth, wider reach of social media, advancement of mobile devices and variety of analytical techniques have led to the phenomenon of ‘Big Data Analytics’, which is employed by organizations to examine large amount of complex data to be used for different business applications. For instance, Big Data Analytics provide insights into the market conditions and preferences of customers, which enable marketing professionals to develop a product to match the needs of an individual customer. The intent behind the Big Data Analytics is to enhance the performance of organizations in different domains like creating transparency, using automated algorithms to replace human decision making and customize offerings. These actions help organizations to gain competitive advantages.

While Big Data Analytics tools and techniques have laid a new age of business intelligence and analytics research in both developed and the developing countries, these technologies also drive demand for analytically skilled knowledge personnel. One of the study by McKinsey Global Institute in 2011 estimated a shortage in USA of 140,000-190,000 data scientists and a lack of 1.5 million managers with deep analytical skills by 2018. Similarly, a study by Gartner, Inc. in 2015 predicted that the number of connected devices will reach 20.8 billion by 2020, however, the web enabled technologies to analyze vast and continuous stream of data are still underutilized in the area of business intelligence and analytics. So many of the organizations are now focusing on Big Data and Analytics to gain exclusive understanding of data. Also, the Big Data and technology services market is growing at the rate of 27 per cent year on year. Clearly, there is a huge demand for knowledge workers who can work with data in a meaningful way and thus it has become important to understand what implications the adoption of Big Data Analytics in organizations might have on their business processes.

While scanning literature in this area, it can be seen that technology research is scarce in the context of developing economies as compared to the developed. However, developing countries have been predicted to have bigger economies than developed countries and are driving global growth. Therefore, it is vital to understand the usage and adoption of Big Data Analytics in developing economies like India. As noted by numerous researchers, digital information in India has been growing twice as fast as other worldwide rates. With more than 900 million mobile connections, India has 100 million plus active mobile data users. Recently, organizations in India have identified this opportunity and are implementing Big Data Analytics to assimilate data from consumers and devices to gain competitive advantages.

 

Big Data Analytics - a chronological and conceptual perspective

The traces of Big Data from the historical perspective start from the contribution of famous thinker F. W. Taylor’s work ‘The Principles of Scientific Management’ in the early 1990s, but the scientific management methods were inadequate to analyze huge volume of work-related data owing to the technology of the time. The advent of computers in the early 1940s and then computing technology during the 1970s placed the systems into the mainstream, facilitating organizations to collect vast amount of data, but restricted their ability to get information and actionable knowledge, on the other side, the information overload problem emerged in decision making. It was the early 1990s when MRP and ERP systems evolved which on applying the concepts of distributed computing enabled organizations to access vast data pools. From the mid-1990s, data was available on real-time basis to organizations, followed by cloud-based systems such as google drive, SkyDrive, Amazon Cloud etc. The worldwide availability and acceptance of internet and World Wide Web in the early 2000s also generated huge amounts of freely accessible data.

Over the last few years, both Practitioners and academics for the last few years are placing much emphasis on Big Data. Generally, technological advancements during recent times have made the process of developing Big Data sets from multiple sources somewhat representative, and the field of business intelligence and analytics is becoming increasingly important and challenging. In the past few years, high speed of large volume data creation has played a major role in the development of Big Data Analytics technologies like web intelligence, web analytics, and mining of unstructured data. Organizations developed analytics technologies aiming at the social media and crowd-sourcing systems to find opportunities and initiate general decision making, supply chain and logistics decision making, and business analytics.

The noteworthy fashion that triggered a substantial growth in data generations included the progress in customary transactional databases and multimedia content, and the simultaneous development of internet and social media. These trends are continuously churning out huge amount of structured and unstructured data. Businesses are facing challenges in the management and capitalization of these data to gain advantages. As a result, the demand for a new class of technologies and analytical methods has increased where Big Data Analytics have emerged as tools and techniques to manage such data.

Big Data is a vague term frequently linked with never ending accumulation of all kinds of data, most of which is unstructured and Big Data Analytics are specific tools and techniques to process vast pool of data sets that are highly complex, unstructured and organized for application in various business processes. In essence, analytics describes the application of advanced statistics to historical data, with the aim to identify behavioral patterns which eventually enable the forecasting of future behavior to some extent. Therefore, a common use of the terms Big Data and Big Data Analytics is to describe huge data sets requiring advanced and unique data storage, management, analysis, visualization technologies as well as statistical analysis. Specifically, Big Data Analytics are the application of advanced statistics to any kind of stored electronic communication, which may include but is not limited to messages, updates, and images posted to social networks, readings from sensors, and GPS signals from cell phones. The objective is to find behavioral patterns within the data, which ultimately allow forecasting future behavior to some extent.

Five V’s of Big Data -

Originally, researchers defined the three V’s of Big Data, namely, Volume, Velocity and Variety and then frequently described the Seven Pillars or Four V’s of Big Data, namely, Volume, Velocity, Variety, and Value, and now Five V’s (5V’s) which are namely, Volume, Variety, Velocity, Veracity and Value.

  1. Volume as data sets that are generated from social media, digital devices and 24-hour span transactions to obtain important knowledge.
  2. Variety as sources and types of Big Data are structured, semi-structured and unstructured data, for instance, e-mail, sensor data, graphics, audio, and video files, social media sites, click streams, and such others which have been difficult to be handled by the existing traditional analytic systems.
  3. Velocity reflects the pace at which the data are generated and analyzed from various sources like business processes, machines, networks and human interaction with things like social media sites, mobile devices, etc. to maximize the value of information and efficiency.
  4. Veracity is the degree of uncertainty about the consistency and completeness of data in order to take decisions. The lack of Veracity might lead to incorrect correlations in Big Data which further might lead to incorrect decisions for businesses.
  5. Value are the economic benefits from the available Big Data.

The techniques of Big Data Analytics to analyze structured and unstructured data in real time have been categorized in five sub-categories:

Analytics

Applications

Text Analytics

Extracting information from textual data like e-mails, online forums, blogs, social network feeds etc. to enable businesses to convert vast amount of human-generated text into meaningful abstracts by involving statistical analysis, machine learning and computational linguistics.

Audio Analytics/ Speech Analytics

Analyze and extracts information from unstructured audio data from sources such as call center, health care and such others to improve customer satisfaction in providing better experience by gaining insights into customer behavior and identifying product service issues.

Video Analytics

Monitor, analyze and extract information from video stream; however, the key challenge has been the sheer size of video data.

Social Media Analytics

Analysis of unstructured and structured data from social media channels and user generated content such as sentiments, videos, images and bookmarks to establish relationships and interactions between the network entities i.e. products, people and organizations.

Predictive Analytics

A variety of techniques to predict future outcomes by uncovering patterns and predicting relationships from current and historical data.

 

This makes Big Data Analytics as a strategic technology option for organizations to harness the data created by technologies and to use for better decision making in day-to-day operations for gaining a competitive edge.

 

Big Data Analytics – the adoption issues

The adoption of Big Data Analytics leads to cost reduction, increased revenue and better decision making in organizations whereas the risk and vulnerability are the costs to be assumed for implementation of the Big Data services. In addition, absorptive capacity implies the skills and domain knowledge for assimilating, managing and leveraging Big Data and; Big Data quality indicates the characteristics of Big Data. The earlier studies suggest that in order to adopt the Big Data Analytics, following issues must be attended to by the organizations.

  • The government and the organizations must bring diverse data sets together for managing the quality of data collection and data capture across the board;
  • Organizations are required to prefer free flow of data across the network and data sharing by various applications and systems to make the availability of the data all the times;
  • Organizations have to provide enough IT capability to maintain consistency in corporate data elements to encourage usage of available data;
  • Organizations also need to promote the experience of managing their digital data and infrastructure becomes IT capability that can be mustered for competitive advantages;
  • Strict data security practices must be adhered to and maintained in line with sensitive data by organizations; and finally the
  • Organizations should enhance the policy concerning privacy protection of users by addressing the purpose rather than prescribing the mechanism.

 

Big Data Analytics – the big challenges

The Big Data Analytics have been approved for their benefits and applications in various sectors. The development of distributed file systems, cloud computing, machine learning algorithms etc. have brought the Big Data tools and techniques into existence to handle vast amount of diverse data with multiple relationships optimally. Despite the growth in these technologies and algorithms to handle Big Data, there are few challenges, which the organizations must attend carefully. Due to the scattered nature of Big Data, it needs to be segregated and processed over different servers. But with such distributed databases there arises the complexity of privacy, fault-tolerance, security and access controls. The lack of awareness pertaining to Big Data also poses serious threats to the cyber security and is also a barrier to the socio-economic development of a country.

Issues and challenges

Data errors:

As the information technology grow, huge volume of data is generated. The introduction of cloud computing to store and retrieve data requires the use of Big Data. There are chances of errors and losses of massive datasets obtained through internet sources, and therefor are unreliable. The data sources should by comprehensively understood about using multiple datasets in order to minimize the errors. The properties and limits of the dataset should be understood before analysis to avoid or explain the bias in the interpretation of data.

Timeliness of Analysis: The value of the data drops over time. Most of the business applications such as insurance and banking, require real time or approximately real time analysis of the transactional data.

Scalability and Storage Issues: Data is increasing at much faster pace as compared to the existing processing systems and even the storage systems of the organizations are not capable enough to store these data. The processing system in organizations adopting Big Data Analytics requires to be taken care not only that serve today's needs but also future needs.

Data Analytics System: Though suitable for the structured data, the conventional relational database systems lack the scalability and expandability. The unstructured data is processed using the non-relational databases, however, their performances are problematic. It requires a mechanism be designed in a manner that the advantages of both relational and non-relational database systems be combined to ensure flexibility.

Representation of Diverse Data: The nature of the data is diverse as acquired from different sources. The conventional tools like SQL cannot be used to store and process the unstructured data such as videos and data from social media. The audio/video data is recorded at an unbelievably increasing rate by the smartphones, stressing one’s brain to operate more. But, the process of storage and processing of the audio/video data is deficient.

Privacy and Security:

The modern equipment and technologies such as cloud computing make available a path that help in accessing and storing information for further analysis. The mixing of IT equipment and technologies demonstrate bigger risk to intellectual property and security of the data. The privacy concerns will increase due to the access to personal information like buying preferences and call detail records.

Not always better data:

Many researchers have today given their attention to the social media mining. One such new prevalent source is twitter. Although users of twitter do not symbolize the population of the whole world. The researchers have to understand the how the Big Data and whole data are different. The users as well as the accounts on twitter are redundant, a single twitter account is accessed by many persons and many accounts are created by one user. Besides, users are active or passive those sign in simply to listen. Thus, accuracy of the sample size of the dataset and interpretation obtained from analysis are doubtful

Conclusion

Big Data Analytics has been conceived in different ways. There is no consensus on its definition and research has shown that it is a multi-faceted construct. Grounded on review of the extant literature, this study suggests that Big Data Analytics have crossed the chasm of mere interest and are certainly making waves with organizations. Across the world, scientific, academic, research, business, as well as, government communities are aggressively charting plans and paths for the adoption process to benefit from developments in the Big Data field. Organizations are not hesitant to incorporate Big Data tools and techniques into their business strategy. The examination of earlier studies revealed Big Data quality, benefit of Big Data services, absorptive capacity and the costs of Big Data are the biggest factor cited by organizations to adopt Big Data Analytics. As a whole, the present study infers that organizations have identified the Big Data Analytics satisfactorily favorable for their business, not costly to adopt and lastly but not the least is that a good quality of data and skill sets to handle the data helps these organizations to bring competitive advantages.

 

References

 

  • Agrawal, K.P. (2013), “The assimilation of Big Data Analytics (BDA) by Indian firms: a technology diffusion perspective”, 3rd Biennial Conference of the Indian Academy of Management (IAM), IIMA, 12-14 December.
  • Barbier, G. and Liu, H. (2011), “Data mining in social media”, in Aggarwal, C.C. (Ed.), Social Network Data Analytics, Springer, pp. 327-352.
  • Benjamins, V. R. (2014), Big data: From hype to reality? In: Proceedings of the 4th International Conference on Web Intelligence, Mining and Semantics (WIMS14). ACM, New York, NY, USA, pp. 2.1-2.2.
  • Boyd D. and Crawford K. (2012), “Critical questions for big data: provocations for a cultural, technological, and scholarly phenomenon”, Inf Commun Soc, Vol. 15 No. 5, pp. 662–679.
  • Brown, ,  Chui,  M.  and  Manyika,  J.  (2011),  “Are  you  ready  for  the  era  of  ‘big data’”, McKinsey Quarterly, Vol. 4, pp. 24-35.
  • Buhl, H.U., Roglinger, M., Moser, D.K.F. and Heidemann, J. (2013), “Big Data”, Business Information Systems Engineering, Vol. 5 No. 2, pp. 65-69.
  • Burks, A. (1989), “The First Electronic Computer: The Atanasoff Story”, The University of Michigan Press, Ann Arbor, MI.
  • Chen, C.P. and Zhang, C.Y. (2014), “Data-intensive applications, challenges, techniques and technologies: a survey on big data”, Information Sciences, Vol. 275, pp. 314-347.
  • Chen, H., Chiang, R.H. and Storey, V.C. (2012), “Business intelligence and analytics: from Big Data to big impact”, MIS Quarterly, Vol. 36 No. 4, pp. 1165-1188.
  • Chern, C.-C., Lei, W.U. and Chen, S.-Y. (2015), “A decision-tree-based classifier for credit assessment problems under a big data environment”, Proceedings of the 2015 Decision Sciences Institute Annual Meeting, Seattle, WA, November 21-24, available at: www.decisionsciences.org/Portals/16/Proceedings/AM-2015/files/p1033360.pdf, accessed on February 12, 2016).
  • Chung, W. (2014), “Bizpro: extracting and categorizing business intelligence factors from textual news articles”, International Journal of Information Management, Vol. 34 No. 2, pp. 272-284.
  • Cuzzocrea, A., Song, I.-Y. and Davis, K. C., (2011), Analytics over large-scale multidimensional data: the big data revolution! In: Proceedings of the ACM 14th International workshop on Data Warehousing and OLAP (DOLAP '11). ACM, New York, NY, USA, pp. 101-104.
  • Demirkan, H. and Delen, D. (2013), “Leveraging the capabilities of service-oriented decision support systems: putting analytics and big data in cloud”, Decision Support Systems, Vol. 55 No. 1, pp. 412-421.
  • Doan, A., Ramakrishnan, R. and Halevy, A. (2011), “Crowd-sourcing systems on the world wide web”, Communications of the ACM, Vol. 54 No. 4, pp. 86-96.
  • Dong, X.L. and Srivastava, D. (2013), “Big data integration”, IEEE 29th International Conference on Data Engineering, IEEE, Brisbane, pp. 1245-1248.
  • Dubey, R. and Gunasekaran, A. (2015), “Education and training for successful career in Big Data and business analytics”, Industrial and Commercial Training, Vol. 47 No. 4, pp. 174-181.
  • Dubey, R., Gunasekaran, A., Childe, S.J., Wamba, S.F. and Papadopoulos, T. (2015), “The impact of big data on world-class sustainable manufacturing”, The International Journal of Advance Manufacturing Technology, Vol. 84 Nos 1-4, pp. 631-645.
  • Esteves, J. and Curto, J. (2013), “A risk and benefits behavioral model to assess intentions to adopt big data”, Journal of Intelligence Studies in Business, Vol. 3 No. 1, pp. 37-46.
  • Fan, J., Han, F. and Liu, H. (2014), “Challenges of big data analysis”, National Science Review, Vol. 1 No. 2, pp. 293-314.
  • Fisher, D., DeLine, R., Czerwinski, M. and Drucker, S. (2012), Interactions with big data analytics. Interactions, Vol 19 No. 3, pp. 50-59.
  • Forrester (2012), The big deal about big data for customer engagement business: leaders must lead big data initiatives to derive value. Available at http://www.forrester.com/The +Big+Deal+About+Big+Data + For+Customer+Engagement/fulltext/-/ERES72241, accessed on January 7, 2016.

 

Comentários