Big data trends 20/21
Using data and analytics in efforts to digitise and transform business models is not a new phenomenon. Many of the most recognisable brands and companies today have relied on big data to transform and elevate their status and business model. Take Netflix, for instance. Netflix started as a DVD rental company in 1997 and, since its shift to a cloud streaming service, boasts an estimated 182.8m subscribers with a market capital of over $200bn.
While it’s been a trend for some time, it’s still on the rise. And quickly. By 2022, Gartner expects that 90% of corporate strategies will explicitly mention information as a critical enterprise asset and analytics as an essential competency. To ensure that your business is keeping up with the latest in big data, here are five trends that we expect to continue throughout the second half of 2020 and into 2021.
The increasing demand for Data Scientists and Chief Data Officers (CDOs)
The Harvard Business Review predicted in 2012 that the role of Data Scientist would be the sexiest job of the 21st Century. Has their prediction come true; is it sexy? You can decide that one, but it’s certainly lucrative.
More businesses are uncovering the value in their data and, as such, are honing a data-focused approach that reveals new solutions to old problems. The astronomical volume of data being produced – every day we generate over 2.5 quintillion bytes of data – has led to an all-time high demand for data professionals.
Clearly, the data is there. The gap, it seems, is in talent. In a global survey conducted by KPMG last year, 67% of Chief Information Officers (CIOs) reported a skills gap and subsequent struggle to recruit the right talent. Big data and analytics ranked in the top three most scarce skills.
And it’s not just data scientists. Chief Data Officers are responsible for not only leading data science, analytics, data governance, and data architecture teams, but also building and rolling out an effective organisational data strategy. As poor data quality and management can result in losses of around $15m per year, it’s important to get right. The number of CDOs has quadrupled over the last four years and, as data-led transformation will continue to be a top priority, that trajectory looks set to continue.
The increasing adoption of DataOps
As more companies become data-driven and data pipelines increase in complexity, they need greater levels of integration and governance. Businesses then look to an agile solution to increase the speed and efficiency of data management which, in turn, will more rapidly produce actionable business intelligence leading to higher profitability. Enter DataOps: a streamlined automated process providing real-time data whilst promoting collaboration, integration, continuous improvement, and data quality.
Companies who adopt a data-driven approach are four times more likely than their peers to see growth beyond shareholder expectations. Due to its automated nature, DataOps increases the efficiency of staff while reducing human error. In addition, this approach quickly processes and analyses data streams to provide insights into consumer patterns, market trends, and price fluctuations. And finally, DataOps can provide an aggregated view of data sets and then deliver a macro analysis with greater accuracy and precision than manual processes.
With all the advantages DataOps brings, this is a trend that’s going to gather momentum in the second half of 2020 and into 2021.
Are cloud data lakes the answer? An answer, yes. The answer, no.
By 2025, it is expected that each person will have at least one data interaction every 18 seconds. This growing volume necessitates a platform that is capable of handling billions of data in fractions of a second.
Open-source software ecosystems such as Hadoop and NoSQL used to be the key to processing vast amounts of data. These technologies, however, showed themselves as prone to instability and poor security. Besides that, their manual configuration needs make the (much simpler) cloud much more alluring. Moving data warehouses to the cloud (on platforms such as AWS, Microsoft Azure, and Google Cloud) allows for better data storytelling and visualisation using the cloud-based integration tools and platforms. Subsequent insights can easily become actions. The cloud provides flexibility, disaster recovery, increased collaboration, and the ability – most crucial in the past few months – to work remotely.
Inevitably, the cloud can’t solve every problem. It is, after all, a third-party service provider, bringing along the associated security concerns, especially where sensitive information is involved. As a result, some companies have chosen to store this data on premises instead and adopt a hybrid storage approach to boost both agility and security. Others have opted for a multi-cloud environment where they are able to store data across a combination of both public and private clouds. This final option provides additional backup and recovery capabilities, enhanced risk management and, importantly, avoids vendor lock-in.
Many businesses have already adopted a hybrid approach, but as the advantages become increasingly more widespread, we can expect to see an increase in the adoption of hybrid and multi-cloud methodologies in businesses’ data ecosystem strategies.
Increased natural language processing (NLP)
As technology continues to improve, big data, AI, IoT, and ML (machine learning) are being combined to better understand the human language in both spoken and written forms.
Many businesses feature chatbots on their websites to help guide customers or answer queries. Given language is full of complexities, conventions and idioms, however, this can prove difficult for machines. Well, perhaps not for much longer. ML is being developed to include FM (fundamental meaning) to develop NLP – a branch of AI which enables computers to understand, process and generate language just as humans do.
These more human conversations provide businesses with access to sentiment analysis whilst also improving operational efficiency, reducing costs and improving customer satisfaction. By 2021, Gartner predicts that NLP and conversational analytics will boost business intelligence adoption from 35% of employees to over 50%. This will allow companies greater insight into their customers’ data such as income and education demographics which could be key to gaining an advantage over competitors.
Real-time Analytics
The increasing speed at which data can be analysed is beneficial on several fronts. Most importantly, though, it allows informed business decisions to be made much more quickly which leads to improved operations and customer experience and, overall, greater value for businesses.
Increasing numbers within a variety of different industries have already adopted this technology. Financial services, for example, are able to analyse huge data sets to almost immediately combat financial fraud. Sports teams are utilising real-time analytics in combination with statistical algorithms and software to provide match stats and improve training and performance.
The use of real-time analytics can be applied to almost every sector, from healthcare via wearable technologies and connected medical devices, through to oil and gas to assist with sensing data in both upstream and downstream operations. Analytical technology can predict natural disasters and bring greater precision to response efforts. In the battle against climate change, this technology is already being used to aid scientists and conservationists by producing visualisations. These visualisations provide a whole host of terrifying statistics, updated in real time.
IDC predicts that 30% of global data will be real-time by 2025, and so we can safely bet on an increased adoption of real-time analytics.
Big data is here to stay and it’s only getting bigger. Are you ready?
Olivia Ogilvie
Martin Tripp Associates is a London-based executive search consultancy. While we are best-known for our work across the media, information, technology, communications and entertainment sectors, we have also worked with some of the world’s biggest brands on challenging senior positions. Feel free to contact us to discuss any of the issues raised in this blog.