AI Meets The Mainframe: Inside IBM's Bold Z17 Bet

Forbes

08-04-2025

IBM z17 Mainframe
IBM recently introduced the z17, its latest generation enterprise mainframe system, powered by the new Telum II processor. The z17 offers significant performance gains over previous generations and integrates artificial intelligence with traditional mainframe capabilities.
At the core of the z17 is IBM's Telum II processor, a second-generation chip that features eight 5.5 GHz cores and 360MB of on-chip cache and an embedded AI accelerator. The new processor enables the system to perform more than 450 billion AI inferencing operations per day, achieving latency as low as one millisecond.
According to IBM, the z17 delivers 50% more AI inference capacity than its predecessor, the z16, while preserving the performance, security, and reliability for which the mainframe is known.
Beyond its traditional mainframe attributes, the z17's ability to run generative AI workloads directly on the mainframe sets it apart. The Spyre Accelerator, expected to be available in Q4 2025, will make this possible.
The Spyre Accelerator is a PCIe-based card that enables native support for large language models and other advanced AI applications. This eliminates reliance on external GPUs or cloud services.
Both the Tellum II processor and Spyre accelerator were first detailed at the 2024 Hot Chips conference.
IBM Tellum II (left) and Spyre (right) Processors
By keeping AI workloads on-premises, businesses gain greater control over sensitive data, reduce architectural complexity, and achieve faster, more secure processing. This approach is particularly valuable in industries with strict regulatory requirements and high-security standards, such as banking, healthcare, and government.
The z17 is also deeply integrated with IBM's watsonx platform, including tools like watsonx Code Assistant for Z and watsonx Assistant for Z. These AI-driven tools support the modernization of legacy COBOL applications by simplifying code maintenance, accelerating refactoring efforts, and lowering the learning curve for new developers. This increases development speed and improves consistency across enterprise systems.
Beyond its AI capabilities, the z17 maintains the fundamental qualities that have defined the mainframe for decades. It offers built-in high availability through fault-tolerant design and advanced error detection, ensuring uninterrupted operations. Security remains a cornerstone of the platform, with end-to-end encryption and support for confidential computing that protects data while it is being used, not just during storage or transit.
To complement the z17, IBM introduced z/OS 3.2, its AI-optimized operating system that supports hybrid cloud integration, NoSQL databases, and hardware-accelerated AI processing. Operational management is enhanced through IBM Z Operations Unite, which uses OpenTelemetry to unify and streamline observability and incident response.
Security also sees a major upgrade with IBM Vault, based on HashiCorp's secrets management technology, now offering unified identity-based security for secrets, certificates, and tokens across both mainframe and hybrid environments. IBM's acquisition of HashiCorp was completed earlier this year.
IBM backs the launch with expanded lifecycle support services and new AI-powered customer service tools like IBM Agent Assist, helping clients manage critical workloads with maximum uptime and resilience.
Mainframes continue to serve as the backbone of industries where uptime, throughput, and security are non-negotiable. Sectors like financial services, healthcare, insurance, and public infrastructure still rely heavily on mainframe systems for their most critical workloads.
In today's environment—characterized by rising cloud costs, heightened privacy concerns, and increasing interest in AI-enabled decision-making—many enterprises are reconsidering the value of maintaining secure, on-premises infrastructure. A recent IBM-sponsored survey found that 78% of global IT executives consider the mainframe central to their digital transformation strategies, while 88% consider application modernization a key priority.
The z17 arrives when enterprises are under growing pressure to derive more value from existing data assets while improving security posture and managing talent shortages. IBM's positioning of the z17 as an AI-native mainframe—not a relic of the past but a platform for future innovation—aligns with these emerging business needs.
By enabling AI directly on the platform, IBM reduces the need to offload sensitive data to external environments for analysis. This reduces security risks and latency, while also sidestepping the increasing difficulty of sourcing GPU capacity for enterprise AI workloads.
While companies like Nvidia dominate the GPU market and cloud providers such as AWS, Microsoft Azure, and Google Cloud lead the AI services space, IBM is carving out a different path. With the z17, IBM offers an on-premises, vertically integrated solution tailored for industries that demand regulatory compliance, data control, and uninterrupted operations. This strategy gives IBM a unique position in the evolving AI infrastructure landscape.
Despite lingering perceptions of the mainframe as outdated or complex, IBM is actively working to change that narrative. New AI-powered tools, along with low-code development environments and modern IDEs, are helping reduce complexity and make the platform more accessible to a new generation of developers.
The z17 is also central to IBM's hybrid cloud vision. Though not cloud-native in the strictest sense, it integrates with Red Hat OpenShift and supports modern DevOps pipelines. Combined with advanced observability tools and AI-assisted development environments, the z17 complements, rather than competes with, cloud-native infrastructure.
For organizations that require scalable, secure, and resilient infrastructure, the mainframe remains relevant and essential.