Connecting-to and building-out Generative AI processing in day-to-day business is not the easy task it may seem, given the colorful renderings of the future that ChatGPT is known for.
Sometimes the AI train seems stuck at the HyperScaler depot. Recent news shows that AI leader Nvidia tech, while firmly established in the top rank HyperScaler houses – is still edging out, and not yet hurtling forward in the wider space.
Another view of data technology’s potential role in changing that scenario is seen in a recent Diversity.net blog post by Mohan Varthakavi, Vice President of Software Development, AI and Edge, Couchbase. “Reimagining Data Architecture for Agentic AI” addresses the data challenge from an overarching perspective, and it is an interesting take.
Varthakavi said traditional data architectures are insufficient for the demands of Agentic AI, that being the medium generally expected to bring Generative AI into wide use.
Here he refers to Generative AI’s need to work on unstructured data – that is, something outside the realm of SQL and often taking the form of human speech. Diligently prepping such data for GenAI, and finding you may not have enough to feed the AI beast, is a common problem today. And, as Agentic AI is currently envisioned, it means feeding numerous agents collaborating around a host of tasks.
Still, as Varthakavi, writes: “Advanced unstructured data processing is quickly emerging as the defining differentiator between AI leaders and followers.” It’s not that different than it was in the Big Data era. One of Big Data’s hallmark “V’ traits – Variety – still looms larger than ever.
If Agentic AI is going to work, a fundamental departure from traditional data frameworks is called for, he said. Going forward, the back-and-forth music of conversations, the probabilistic nature of machine learning, and the deep complexity of human language will call the shots, and shift the approaches used by those who apply system architecture.
The shift to agentic AI … marks a migration from traditional rule-based logic toward architectures centered around language understanding. This isn’t as simple as swapping one model for another; it requires a rethinking of how systems are composed. Large language models can provide powerful general capabilities, but they are not equipped to answer every question pertaining to a company’s specific business domain.
Also noted by Varthakavi is the growing need for architects to be aware of new types of data dependency risks. These appear when agents that use LLMs may receive corrupted or insufficiently updated data. Latency pressures also confront the designers, as workloads and user-machine dialogs become more complex.
Just as important in the Brave New World are conversation storage systems that provide persistent memory, vector search engines that quickly contextualize customer queries in the data format that machine learning likes best, and dynamic memory management to support real-time context updates.
Few people arrive at work in the morning with a complete mastery of these kinds of requirements. But these underlie the elements implied in the futuristic advertisements for Future AI.