By: Robbie Jerrom - Senior Principal Technologist AI at Red Hat

When industry experts and insiders talk about agentic AI, they are essentially talking about taking the technology to the next level. It is the level where autonomous agents can execute complex tasks and deliver outcomes on behalf of users above and beyond the generation and delivery of information.

Rather than just drafting an email or summarising a document, agentic AI envisions agents capable of planning an entire overseas business trip, complete with flight details and travel itinerary. Agents that can select and recruit personnel to fulfil workforce requirements or streamline customer service operations.

At a time when businesses across South Africa are looking to start extracting value from their AI investments, agentic AI represents a pathway to intelligently automating workflows and, importantly, augmenting human decision-making. Already, local financial services institutions, from established banks to innovative fintechs, are exploring how agents can enhance customer engagement while freeing skilled professionals to focus on complex, high-value work.

Consider a customer contacting their bank about retirement planning. An AI agent can analyse the customer's transaction history, identify spending patterns, assess existing savings products, and surfacing tailored investment options, all within a single interaction. The agent might identify that the customer consistently has surplus funds at month-end, cross-reference this against their risk profile and retirement timeline, then present specific product recommendations with projected outcomes. Crucially, the agent doesn't replace the financial adviser; it prepares the groundwork, handling data gathering and initial analysis so that when a human specialist joins the conversation, they can focus on nuanced guidance, relationship building, and the judgment calls that require human expertise.

For this to be feasible, it requires a hybrid approach to its underlying infrastructure, as well as platforms that address model inferencing at scale and streamline agent development and management. From there, and with specific business goals in mind, local enterprises can reach the next frontier of this exciting technology.

Complexity and cost scaled up

AI agents are not the result of one single generative AI model or one large language model (LLM) calling all the shots. They are architecturally more complex, often with a planning LLM working in concert with smaller models (SLMs) that perform specific tasks, alongside APIs that gather additional information. When a user asks an agentic question, multiple models and APIs are called, multiplying the complexity of inference and the level of communication between systems.

Crucially, agents must connect reliably to enterprise data sources. Whether retrieving customer records, querying inventory systems, or accessing policy documents, the ability to ground agent responses in accurate, real-time data separates useful automation from unreliable guesswork. Retrieval-augmented generation (RAG) architectures and well-designed data pipelines become essential infrastructure for production-grade agents.

That multiplication goes for cost as well. While businesses have a token budget or a cost to run a particular model, token counts can increase exponentially based on the computational requirements that come with agentic requests. Going from asking AI questions to assigning AI jobs leads to an increase in spending, and thus organisations need to account for that.

All that's not to say that agentic AI is only within the reach of large-scale, multinational enterprises with limitless technology budgets. Open source inference engines like vLLM provide high-performance, production-ready serving for large language models, while projects like llm-d (which Red Hat contributes to) build on this foundation to enable distributed inference across Kubernetes environments. Together, they allow organisations to bring LLM infrastructure in-house, reflecting the necessity of hybrid architecture.

For South African enterprises, this hybrid approach carries additional significance. Infrastructure realities, including connectivity challenges and power availability, make the case for on-premises and edge AI capabilities even stronger. The ability to run inference workloads locally ensures business continuity regardless of external disruptions, while still leveraging cloud resources when available. This is where enterprise AI infrastructure becomes essential. Platforms like Red Hat AI provide the foundation for deploying and scaling AI workloads consistently across hybrid environments, helping enterprises minimise computation costs while maintaining the performance and reliability that agentic workflows demand.

AI agents are dependent on model resilience, capacity and scalability. They are also reliant on hybrid cloud infrastructure that enables efficient workload and data distribution, as well as centralised platforms that streamline their deployment and management. Finally, and critically, AI agents need clearly defined use cases and task functions because these drive efficiency across the planning LLM and its supporting task-specific SLMs.

Organisations can further optimise costs and accuracy by fine-tuning these specialist models for routine agent functions, reducing reliance on expensive, general-purpose LLMs for every interaction. A well-architected agent system might use a lightweight model for initial query classification, domain-specific models for particular tasks, and reserve larger models only for complex reasoning that demands their capabilities.

Built for purpose, deployed with confidence

AI agents offer a wide range of use cases across some of the world's most important industries. In addition to financial services providers who are deploying agents for customer engagement, healthcare providers can use agents for patient management, identifying dispensary locations, setting up appointments and automating billing and claim submissions. In telecommunications, agents can offer hyper-personalised customer experiences, monitor network performance and reroute traffic as needed to avoid potential disruptions.

A common concern with agentic AI is whether it will displace human workers. The reality is more nuanced, and more positive. Well-designed agent systems augment human capabilities rather than replace them. They handle repetitive data gathering, initial triage, and routine processing, freeing skilled professionals to focus on the work that genuinely requires human judgment: complex problem-solving, empathetic customer interactions, creative thinking, and ethical decision-making.

This is where human-in-the-loop design becomes essential. Rather than granting agents full autonomy, enterprises define boundaries: which decisions can agents make independently, and which require human approval? A customer service agent might autonomously handle balance enquiries and transaction disputes under a certain threshold, but escalate credit decisions or complaints to a human specialist. This approach builds trust incrementally. Agents earn greater autonomy as they demonstrate reliability, while humans retain oversight of high-stakes decisions.

Beyond workforce considerations, regulatory and jurisdictional factors also shape how enterprises deploy agentic AI. Digital sovereignty concerns are increasingly prominent as more businesses have to comply with data residency and processing obligations. In fact, according to Red Hat's 2024 Global Tech Outlook survey, sovereignty concerns were the biggest barrier to cloud adoption among EMEA enterprises, and nearly a third of enterprises believe AI sovereignty is a priority for their AI strategies for the next 18 months.

In any use case or with any organisation, agentic workflows may cross regional and international borders. For this reason, agents and their platforms require robust guardrails: content filtering to prevent harmful outputs, policy enforcement to ensure regulatory compliance, and comprehensive audit trails for accountability. In South Africa, the Protection of Personal Information Act (POPIA) adds further requirements around how agents access, process, and store personal data.

Enterprises must ensure agent architectures respect data subject rights and maintain appropriate consent mechanisms. In regulated industries like financial services, these controls are not optional; they are prerequisites for deployment. Platforms must provide the tooling to define, test, and monitor guardrail policies across agent workflows, ensuring organisations can deploy with confidence. That confidence is a critical ingredient for sustainable, long-term success.

Asking the right questions and leveraging the right solutions

AI is evolving rapidly. We're quickly moving beyond simply generating text or compiling information. But it must be said, agentic AI, the next phase of the technology, depends on infrastructure and platforms that enable and support its operation.

Agents rely on platforms that are modular and adaptive, capable of running them securely and at scale across production environments. Critically, agents must connect seamlessly to the tools and data sources they need to complete tasks. Model Context Protocol (MCP) is emerging as the open standard that makes this possible. Rather than building bespoke integrations for every database, API, or enterprise application, MCP provides a common interface that allows agents to discover and interact with external resources in a standardised way.

For enterprises, this means faster agent development, greater interoperability across vendors, and reduced lock-in. Agents built on MCP-compatible platforms can leverage a growing ecosystem of pre-built connectors rather than starting from scratch with each integration. All this represents a strategic, holistic approach to AI development, all while allowing businesses to choose any model or hardware across their architecture.

Agentic AI is on the verge of driving widespread, large-scale change in industries and heralding the next phase of enterprise AI adoption. But that adoption must have impact and deliver value. When connecting AI agents to their workflows, businesses need to understand what they are and what they can do. They should ask themselves, "How does this make my organisation more efficient, improve the user experience or make my employees more productive?"

The answer, increasingly, is that agents make organisations more capable by making their people more capable. By handling the routine and surfacing the relevant, agents give employees the space to do their best work. Answering these questions and leveraging platforms that accelerate the deployment of agents and provide the consistency needed for scaling them across environments, enterprises can unleash the full potential of agentic AI. Red Hat AI provides exactly this foundation: a consistent, scalable platform built on open source principles for building, deploying, and managing AI agents across hybrid cloud environments.