Data can be useful only if it's embedded into business process: C.K Tan

Data is one of the most valuable assets for businesses today. But only when that data is used to create 'actionable insights' does it give organizations a competitive edge, said C.K Tan, senior director, solutions and value engineering at Qlik, a US-based business analytics company. An expert in data analytics having nearly two decades of experience in executing data-driven practices, Tan believes that organizations that build a robust foundation and a strong analytics culture and competency by embedding data into their daily business processes are certainly able to innovate and make decisions more wisely in today’s data driven world. Edited excerpts:

What has changed in business intelligence (BI) and data analytics in recent years in the last 2-3 years?

Many companies who earlier used traditional BI (based on pre-configured, curated data sets that are intended to inform but not necessarily take actions) have now moved to active intelligence (based on real-time data that can trigger actions and engagements with the insights that it generates). Besides, as companies are looking for scale and agility, they are moving to the cloud. 

We are seeing that adoption of cloud-based data analytics and real-time analysis are driving the evolution of analytics. That said, use cases are probably not just on the dashboard. A lot more people are looking for prescriptive kinds of insights. That's where machine learning (ML) comes in, helping businesses in accelerating analytics and decision-making in real time. 

For instance, in the banking sector, for bad debt provisioning one is able to predict whether he or she should approve a loan. Also, the entire natural language processing (NLP) is getting smarter with explainable intelligence, as we are already seeing with the evolution of say, OpenAI’s ChatGPT and others citing attributes, accuracy, statistics, etc. helping businesses make real-time data-driven decisions.

What kind of challenges do you think enterprises are facing when it comes to realizing the value from their data?

Enterprises are continuously striving to respond in these constantly shifting situations to survive and create a breakthrough. That means they’re creating a modern enterprise data ecosystem so that data is up-to-date and ready to make informed decisions. As companies collect more and more data, the ultimate goal is to translate this into insights that can help them optimize business performance. 

For example, they may wonder which of their customer segments is most profitable. How can they reduce their customer acquisition costs? How can they increase sales or revenue? And so on. While the questions you can ask the data are unlimited, you need a strategy on how you question the data for meaningful insights. When it comes to analytics, some questions can be poorly constructed or misguided. They can lead to costly, time-consuming expeditions into the data that don’t yield any actionable insights. 

To make matters worse, the increasingly complex data infrastructure landscape means that it has never been more challenging to secure and govern your data. With more enterprises moving to the cloud and the deployment of complex data lakes solutions will only add to the complexity equation today. To deal with the increasing demand and complexity, it is then essential for organizations to simplify their approach to data governance and data management on the whole.

Can you mention some strategies for enterprises to overcome these challenges and make the most out of data?

Many organizations have huge volumes of data that are still on premise, including data trapped within older legacy systems. Hence, it’s not as simple moving everything to the cloud. Among the many strategies businesses are talking about to mitigate data-driven challenges, one important technique is decoupling. A method of adding new technologies on top of legacy systems to increase their functionality. it’s a simple method of updating the systems they already have in place. At the same time, it gives a scalable, flexible and resilient architecture for companies to remain agile and innovate. One of the main advantages of decoupling data (storage and compute) is the greater ease with which companies can analyse data in real-time. This helps organizations enrich their data, sort through it, and query it interactively in real-time.

How can organizations reduce the skill gap? Can generative AI replace data science professionals and help in solving the skills gap crisis?

Data skill is an essential skill; it is the foreseeable future as we continue to live in a digital world. It is therefore important to understand the fundamentals of data from the very beginning. Institutions (schools, colleges and universities) as well as organizations should substantially invest in data-centric training. 

Besides, it is vital to build a data culture within the workplace and ensure that data literacy becomes integral to bridge the data skills gap in the long term. Now, while ChatGPT and other generative AI can assist with certain tasks, such as data cleaning and pre-processing, in data science it cannot replace the need for human expertise and judgment in areas such as data analysis, model building and decision making. 

Having said that, using ChatGPT and similar tools, the roles will require humans to learn to collaborate with technology to adapt better and optimize the business.