For mid-sized companies, follow Big Data adoption curve

Posted on October 29, 2014 09:53 am

Companies who are later to the game in the adoption curve of big data cycle have opportunities to learn from those who adopted before them. Studies have shown that Big data is no longer the sole domain of big boys. The promise of improved decision making, increased operational efficiency and new revenue streams has more organisations actively engaging in data analysis projects than ever before. In the case of  mid sized companies about to embark on big data projects, that means capitalising on lessons from their enterprise forerunners.By learning from the mistakes of big companies and taking steps to avoid them, smaller firms can position themselves to enjoy greater success and smaller companies are joining on the big data bandwagon in a big way. The so called mid-market organisations are either already in flight with a big data initiative or plan to start one soonest.That’s a whole lot of companies whose big data projects are either going to sink or swim in the very near future.However, as they compete to enter the fray, non-alignment with executive stakeholders seem to derail many projects. Many have ignored the fact that it’s about business. Before someone start any big data analytics project,securing the support of the stakeholders in critical because they guarantee resources needed and also help make tangible business decisions based on the findings of big data projects.
In Europe, studies have shown that many big companies initially thought moving their archive data off legacy databases with expensive license requirements and onto the nearly free clusters of databases such as Hadoop would yield significant cost savings.

Sadly, that hasn’t been the case.While shifting data to these unstructured sources can in fact save licensing costs, the labour required to architect, deploy and manage these systems has been significant that many large companies across Europe are finding that all they’ve done is shift costs from licensing to labour.In Spain, the takeaway for mid-sized companies is labour costs into return on investment calculations, but studies show most didn’t fixate on infrastructure savings at the start. Simply put, labour requirements in the big data realm are difficult to satisfy. Though new educational programs are now being created in countries like Germany, Belgium, France and Italy with increased regularity, universities and professional training services in those countries were not initially equipped to handle the tremendous demand for so-called data scientists. The number of people needed to support the deployment of big data technologies has overwhelmed the pool of IT resources in continental Europe according to a study report i’ve managed to read. If deep-pocketed enterprise companies in Europe can’t go out and hire the talent they need, chances are you won’t be able to either.There simply aren’t enough data scientists in the Europe and even other parts of the world today, nor will there be in the foreseeable future.
Instead of focusing on finding a single data scientist, European companies should instead focus on building data science teams from within their organisation.Researchers have in the past recommended training team members in-house to manage their customised big data initiatives.In Japan, the study found that enthusiastic database administrators and business analysts are willing to learn and to take the next step and offer the on-the-job training they need to take it. In South Korea, the study concludes that having a big data cluster hasn’t been the same as running meaningful big data analytics. Many enterprise companies in Western Europe have found it difficult to access and analyse data despite the implementation of a top-end big data platform despite the fact that big data analytics is one of the hottest markets in all of Information and Technology. New providers in Northern Europe with new offerings are sprouting up on what seems like a daily basis in Estonia, Norway, Sweden, Denmark and Finland. Line of business leaders in marketing, sales and other functional departments in Western Europe were led to believe they could successfully embark on data analysis projects without extensive knowledge of Information and Technology.  As they soon realised in countries like Ireland, Netherlands and Germany, while they might be tremendous innovators, line of business leaders were not equipped to manage, govern and scale data analytics products.Luckily,companies in Northern Europe and Asia Pacific are avoiding this pitfall by making collaboration between IT and lines of business the priority. This has ensured that data is properly governed, and that systems are properly managed and can be scaled when needed like is happening in countries like Australia and New Zealand.

Contador Harrison