The Top 3 Challenges Enterprises Face On Their GenAI Journey

Forbes

12-05-2025

Manosiz Bhattacharyya, Chief Technology Officer, Nutanix . getty
It's been about two years since generative AI (GenAI) exploded on the scene and garnered widespread interest with the launch of OpenAI's ChatGPT in late 2022. Since then, enterprises everywhere have been in a race to harness the power of GenAI within their own organizations to boost productivity, improve decision making, optimize customer experience and fuel innovation. According to a survey conducted by Bain & Company, 87% of enterprises have deployed or are piloting the technology in their organizations.
GenAI has officially transitioned from its "hype" phase to a more practical implementation phase in which enterprises are applying the technology to a variety of use cases and seeing real value. But there's still a long way to go, and as far as GenAI's potential is concerned, this is just the tip of the iceberg.
As enterprises continue to adopt and experiment with the technology, they will inevitably encounter roadblocks related to complexity, cost and control. Let's explore what each of these challenges entails and how organizations can overcome them to get the most out of their GenAI investments.
As GenAI proliferates, enterprises should abide by the mantra, "keep it simple." This is much easier said than done, of course. By nature, building, training and running GenAI models requires incredibly complex infrastructure. In addition to its substantial resource requirements from a compute, storage and bandwidth perspective, GenAI also demands specialized hardware such as graphics processing units (GPUs). Furthermore, while training often takes place in data centers with GPU capacity, inferencing may need to occur at the edge, further compounding this complexity.
If enterprises can't effectively manage their GenAI infrastructure, it will quickly become cumbersome—especially at scale—potentially canceling out some of the technology's game-changing benefits. For many enterprises, the best way to manage this complexity is by leveraging cloud platforms that offer pre-configured GenAI environments to keep infrastructure management to a minimum. For enterprises that want to keep GenAI on-premises or within hybrid environments, adopting tools that automatically optimize resource utilization and simplify data management can greatly reduce the burden of infrastructure management.
Another way enterprises can streamline GenAI infrastructure is by establishing an internal AI committee dedicated to reducing the fragmented use of the technology across the organization. In most enterprises, every line of business is trying to leverage GenAI their way, which can create operational silos and complicate infrastructure. Having centralized oversight in this area can dictate things like which types of models are permitted and which GenAI vendors can be used within the organization. This not only further simplifies infrastructure management; it keeps costs in check and bolsters security.
Based on GenAI's robust resource and infrastructure requirements, it should come as no surprise that costs can balloon if not carefully controlled. This is especially true for GenAI applications that run in the public cloud using a token-based pricing model. As organizations expand their use of GenAI and continue to refine their existing models, they'll inevitably require more tokens, leading to a dramatic spike in costs.
Whenever possible, enterprises should reduce their exposure to these token-based platforms and instead opt for solutions that allow them to use a model as many times as needed without being charged based on usage. By only incurring infrastructure-based costs, enterprises can build, train and run models in a much more cost-effective manner.
This is especially critical as GenAI evolves from human-driven to, eventually, self-driven agentic AI systems. Every application will be infused with GenAI and utilize more models, and token-based pricing will quickly become cost-prohibitive. To avoid this predicament and better support innovation, enterprises should consider adopting platforms with an infrastructure-based cost model sooner rather than later. 3. Control
While running GenAI in the public cloud certainly has its perks, it has its downsides as well—one being that it can create data control challenges from both a training and inferencing perspective. Once a model has "seen" data, there's no going back. This is of particular concern in highly regulated industries like finance and healthcare, where data privacy is of the utmost importance.
To reap the benefits of GenAI safely, enterprises need to apply strict security controls to their models. The most effective way to do this is by ensuring that the security controls the organization already has around its training data are also reflected in its models. In other words, if someone doesn't have access to the data, they should not have access to the model either.
Where this becomes difficult is in environments where model systems and data systems are disparate: It can be challenging to ensure the data systems' security controls map back to the models since they're separate entities. In these cases, enterprises should rely on infrastructure providers that offer a model inference system with built-in rules as code (RaC) policies so their security controls are consistently applied to their models. This gives enterprises access control of every model so that any application or person who uses the model can be given the right permission. Conclusion
Organizations are just beginning to scratch the surface of what's possible with GenAI. As with any newer technology, there will undoubtedly be challenges along the way. However, by minimizing infrastructure management, controlling costs and implementing security controls, enterprises can set themselves up for success and more GenAI-powered breakthroughs in the near future.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?