Data

Data architects will have an increasing role in AI discussions, and may offer their usual cautionary perspective. When people ask how soon GenAI will take hold, they might keep this in mind.

So far, the major gauges for GenAI progress have been the hyperscalers’ capex accounting and Nvidia’s quarterly results. The latter came in last week, and the general picture appeared lackluster to muddy. Only Nvidia’s tremendous three-year upsurge can explain the use of the ‘lackluster’ term;  its sales jumped +56% in the quarter. [See footnote to this story.]

What’s happening with the Generative AI juggernaut may  hinge a lot on results of firms like Snowflake, makers of the noted cloud-based data platform.

A quick view on that came via Mandeep Singh, Global Head for Technology Research at Bloomberg. He spoke on Bloomberg Podcasts just as both Nvidia and Snowflake numbers came in. The context was today’s general search for a defined timeline for GenAI deployment and monetization.

What’s going on now, Singh says, is that companies are plying through their own enterprise data. They do this in order to create specialized AI agents that bring the power of LLMs to desktops.

“That’s where someone like Snowflake or MongoDB benefits, because your training is getting a lot more specialized depending on the use case,” he said. “It’s not just about the compute cluster anymore, but also about the data. I think that points to why Snowflake is doing well.”

In fact, the company beat Wall Street expectations for both earnings per share (EPS) and revenue. Quarterly revenue grew 32% Y2Y, to $1.14 billion. The company cited adoption of AI tools as a growth driver.

The importance of cloud and on-premises databases and data warehouses in this regard would be emphasized this week as OpenAI bought Statsig and its  “data warehouse native” platform for, of course, something in the neighborhood of $1 Billion. -JV

~~~~~~~~~~~~~~~~~~~~

Footnote – Nvidia leader Jensen Huang and his company are caught right now between China leader Xi Ping and US leader Donald Trump, both pursuing new forms of State Capitalism or Capitalist Statism. The Era is still searching for a name.

Connecting-to and building-out Generative AI processing in day-to-day business is not the easy task it may seem, given the colorful renderings of the future that ChatGPT is known for.

Sometimes the AI train seems stuck at the HyperScaler depot. Recent news shows that AI leader Nvidia tech, while firmly established in the top rank HyperScaler houses – is still edging out, and not yet hurtling forward in the wider space.

Another view of data technology’s potential role in changing that scenario is seen in a recent Diversity.net blog post by Mohan Varthakavi, Vice President of Software Development, AI and Edge, Couchbase. “Reimagining Data Architecture for Agentic AI” addresses the data challenge from an overarching perspective, and it is an interesting take.

Varthakavi said traditional data architectures are insufficient for the demands of Agentic AI, that being the medium generally expected to bring Generative AI into wide use.

Here he refers to Generative AI’s need to work on unstructured data – that is, something outside the realm of SQL and often taking the form of human speech. Diligently prepping such data for GenAI, and finding you may not have enough to feed the AI beast, is a common problem today. And, as Agentic AI is currently envisioned, it means feeding numerous agents collaborating around a host of tasks.

Still, as Varthakavi, writes: “Advanced unstructured data processing is quickly emerging as the defining differentiator between AI leaders and followers.” It’s not that different than it was in the Big Data era. One of Big Data’s hallmark “V’ traits – Variety – still looms larger than ever.

If Agentic AI is going to work, a fundamental departure from traditional data frameworks is called for, he said. Going forward, the back-and-forth music of conversations, the probabilistic nature of machine learning, and the deep complexity of human language will call the shots, and shift the approaches used by those who apply system architecture.

The shift to agentic AI … marks a migration from traditional rule-based logic toward architectures centered around language understanding. This isn’t as simple as swapping one model for another; it requires a rethinking of how systems are composed. Large language models can provide powerful general capabilities, but they are not equipped to answer every question pertaining to a company’s specific business domain. 

Also noted by Varthakavi is the growing need for architects to be aware of new types of  data dependency risks. These appear when agents that use LLMs may receive corrupted or insufficiently updated data. Latency pressures also confront the designers, as workloads and user-machine dialogs become more complex.

Just as important in the Brave New World are conversation storage systems that provide persistent memory, vector search engines that quickly contextualize customer queries in the data format that  machine learning likes best, and dynamic memory management to support real-time context updates.

Few people arrive at work in the morning with a complete mastery of these kinds of requirements. But these underlie the elements implied in the futuristic advertisements for Future AI.