Outdated IT infra hindering GenAI scalability, RoI

Indian enterprises are rapidly adopting Generative Artificial Intelligence (GenAI), but outdated IT infrastructure is hindering scalability and return on investment (RoI), according to a new report.

US-based cloud solutions firm Nutanix's 7th Annual Enterprise Cloud Index reveals that application containerisation and GenAI are driving significant shifts in enterprise IT strategies, though challenges remain in data security, skills, and infrastructure modernisation in various global markets including India. The study conducted with research firm Vanson Bourne surveyed 1,500 IT leaders globally (including 110 in India), suggesting that almost all Indian organisations (99%) say they face challenges when scaling GenAI workloads from development to production — with the primary issue being integration with existing IT infrastructure.

In a conversation with TechCircle, Prasanna Ranade, Sr. Director, Enterprise & Govt Sales - India, Nutanix, noted that fragmented data and siloed storage impact GenAI model performance, exacerbated by skills shortages in machine learning, data engineering, and Kubernetes management. He also emphasised privacy and security concerns with sensitive data processed by large language models (LLMs), increasing compliance risks. “Addressing these hurdles requires infrastructure modernisation, talent upskilling, and robust security protocols, specifically simplified platforms for deployment and scaling,” he said.

The study also suggests that effective data management and governance are crucial for AI success. For example, clean, well-structured data pipelines improve accuracy and reduce bias, while centralised, accessible data accelerates training and inference. Ranade believes that strong governance, including privacy controls, encryption, and compliance with regulations like GDPR and India's DPDP Act, protects against breaches and legal risks, fostering trust and streamlining data access for optimised resource utilisation and faster time to market.

Ranade further elaborated that a significant talent gap exists in India for GenAI implementation, particularly in infrastructure, security, and data engineering roles. “Demand for cloud-native and Kubernetes professionals is surging, alongside the need for data privacy/security experts and AI/ML engineers proficient in data pipelines, model deployment, and infrastructure modernisation,” he said, noting that addressing this shortage through upskilling is vital.

Ranade further suggests a hybrid approach to AI development, combining on-premises infrastructure (for sensitive data), cloud (for scalability), and edge (for real-time processing), supported by effective data management, AI/ML (Machine Learning) frameworks and robust security pipelines.

Despite these challenges, most CIOs believe GenAI transforms customer service, productivity, and security. AI-powered tools enhance support, response times, and personalisation, while also bolstering cybersecurity and operational efficiency. However, 95% of Indian CIOs feel they could improve the GenAI model and application security, highlighting data security/privacy as a continuing barrier, potentially alleviated by further IT infrastructure modernisation. Over 90% believe their current IT infrastructure needs improvement, and 61% see a need to invest in it. Scaling AI workloads and addressing privacy/security concerns with LLMs remain significant challenges, requiring infrastructure upgrades and security prioritisation.

The majority (75%) of Indian organisations expect their GenAI projects to pay off in 1-3 years, demonstrating a long-term commitment that allows for setbacks and iteration, crucial for navigating implementation hurdles and avoiding project abandonment.

Ranade concluded that maximising GenAI RoI requires a thoughtful, long-term strategy. “CIOs should focus on sustained value through a multi-year roadmap, modernise infrastructure with hybrid multi-cloud and containerisation, enhance productivity via workflow automation, and prioritise security through robust data protection,” he said. Aligning these initiatives with business goals and investing in appropriate infrastructure and security is key to achieving sustainable RoI.