Progressive Gauge

Media and Research

Data

Large models cooling

By

Molecular Sampler – The week just passed brought news of a combined MIT/IBM team suggesting a less compute-intensive route to AI-driven materials science.  The group said it used a subset of a larger data pool to predict molecular properties. The use case has gained attention in both ML and quantum computing circles – where a drive to speed material development and drug discovery could lead to cost savings, better health outcomes and yet-to-be-imagined innovations.

Like most AI advances of late, the work gains inspiration from NLP techniques. The methods used to predict molecular properties tap into “grammar rule production,” which by now has a long lineage. There are 1 followed by 100 zeros of ways to combine atoms, which is to say grammar rule production for materials is a big job, and that style of computation is daunting and may not be immediately exploited.

Because the grammar rule production process is too difficult even for large-scale modern computing, the research team put its efforts into preparatory paring of data, a short-cut technique that goes back to the beginning of time. Some notes from the MIT information office:

“In language theory, one generates words, sentences, or paragraphs based on a set of grammar rules. You can think of a molecular grammar the same way. It is a set of production rules that dictate how to generate molecules or polymers by combining atoms and substructures.

“The MIT team created a machine-learning system that automatically learns the “language” of molecules — what is known as a molecular grammar — using only a small, domain-specific dataset. It uses this grammar to construct viable molecules and predict their properties.

As I read it, the MIT-IBM team have come up with a simulation sampler approach. The ‘smaller corpus’ approach is much explored these days as implementers try to take some of the ‘Large’ out of Large Language Models. One may always wonder if such synthesis ultimately can gain true results. I trust an army better qualified will dig into the details of the sampling technique used here over the weekend.

***  ***  ***  ***

ChatGPT damper – The signs continue to point to a welcome damper on ChatGPT (AI) boosterism – now that each deadline journalist in the world has asked the bot to write up a global heatwave story or Met red-carpet opening story in the style of Hemingway or Mailer or another.

Among the signals of cooling:

*There’s investor Adam Coons. The Chief Portfolio Manager at Winthrop Capital Management said AI on Wall Street will continue but then fade as a hot button.

For a stock market that has endorsed Mega cap growth stocks for their ChatGPT chops, it has become a FOMO trade. “In the near term that trade will continue to work. There’s enough investors still willing to chase that narrative,” he told Reuters. On the other hand, Coons and Winthrop Capital are cautious on it, as the hyperbole has obscured the true potential. He said:

“We are moving away from the AI narrative. We think that there’s still too much to be shown. Particularly [with] Nvidia, we think the growth figures that are being priced into that stock just don’t make sense. And there’s just really not enough proof statements from a monetization standpoint behind what AI can really do within the tech sector.”

*There’s Pincecone COO Bob Wiederhold speaking at VB Transform – Pinecone is in the forefront of up-surging Vector Databases that appear to have a special place in formative LLM applications. Still, Wiederhold sees need for a realistic approach to commercializing the phenomenon.

His comments as described by Matt Marshall on VentureBeat:

Wiederhold acknowledged that the generative AI market is going through a hype cycle and that it will soon hit a “trough of reality” as developers move on from prototyping applications that have no ability to go into production. He said this is a good thing for the industry as it will separate the real production-ready, impactful applications from the “fluff” of prototyped applications that currently make up the majority of experimentation.

*There’s Rob Hirschfeld commentary “Are LLMs Leading DevOps Into a Tech Debt Trap?” on DevOps.com – Hirschfeld is concerned with the technical debt generative AI LLMs could heap onto today’s DevOps crews, which are already awash in quickly built, inefficiently engineered Patch Hell. Code generation is often the second-cited LLM use case (after direct-mail and press releases).

Figuring out an original developer’s intent has always been the cursed task of those who maintain our innovations – but LLM’s has the potential to bring on a new mass of mute code fragments contrived from LLM web whacks. Things could go from worse to worser, all the rosy pictures of no-code LLM case studies notwithstanding. Hirschfeld, who is CEO at infrastructure consultancy  RackN, writes:

Since they are unbounded, they will cheerfully use the knowledge to churn out terabytes of functionally correct but bespoke code…It’s easy to imagine a future where LLMs crank out DevOps scripts 10x faster. We will be supercharging our ability to produce complex, untested automation at a pace never seen before! On the surface, this seems like a huge productivity boost because we (mistakenly) see our job as focused on producing scripts instead of working systems…But we already have an overabundance of duplicated and difficult-to-support automation. This ever-expanding surface of technical debt is one of the major reasons that ITOps teams are mired in complexity and are forever underwater.

News is about sudden change. Generative AI, ChatGPT and LLMs brought that in spades. It is all a breathless rush right now, and analysis can wait. But, the limelight on generative AI is slightly dimmed. That is good because what is real will be easier to see. Importantly, reporters and others are now asking those probing follow-up questions like: “How much how soon?”

It’s almost enough to draw an old-time skeptical examiner into the fray. – Jack Vaughan

 

Adage

“Future users of large data banks must be protected from having to know how the data is organized in the machine….” E.F. Codd in A Relational Model of Data for Large Shared Data Banks

The March of the Language Models

By

[April 17, 2023] – Had the opportunity to speak with Forrester Analyst Ronan Curran recently for a VentureBeat article. Of course, the topic was ChatGPT, generative AI, and Large Language Models.

His counsel was both optimistic and cautionary – a good summation of the bearings IT decision makers should set as they begin yet another tango with a new technology meme.

A handy summarizer-paraphraser tells me that Curran told VentureBeat that it would be a mistake to underestimate the technology, although it is still difficult to critically examine the many potential use cases for generative AI.

Yes, such applies to each technical challenge – every day. And it bears repeating as each new technology whispers or yells that the fundamental rules no longer apply – and yet they do.

Looking back on my conversation with Curran, I find insight in what some would say is obvious. The large language models are … large! And, as Curran told me, because they are large, they cost a lot to compute and train. This reminds us, as others have, that the LLM should be viewed like polo or horse racing – as a game for the rich.

Why do we say game for the rich? On one level, the LLM era stacks up as megacloud builders’ battle, albeit with aspects of the playground grudge match. Microsoft leader Satya Nadella, who had the thankless task of competing with Google on the search front, almost seems to chortle: “This new Bing will make Google come out and dance, and I want people to know that we made them dance.”

For the cloud giants, the business already had aspects of a war of attrition, as they staked data center regions across the globe. The folks at Semianalysis.com have taken a hard stab at estimating a day in the life of an LLM bean counter, and they suggest a “model indicating that ChatGPT costs $694,444 per day to operate in compute hardware costs.” Of course, these are back of the envelope estimates – and the titans that host LLMs will look to engineer savings.

The new LLM morning summons to mind a technology that  consumed  much attention not so long ago: Big Data. The magic of Hadoop had a difficult time jumping from the likes of Google, Facebook and Netflix to the broader market. Maybe Big Data should have been named ‘Prodigious Data’ – because that would have offered fairer warning to organizations that had to gather such data, administer it, and come up with clever and profitable use cases.

“What is Big Data good for?” was a common question, even in its heyday. Eventually the answer was “machine learning”.

Much of Big Data remained in the realm of the prototype. In the end, it was a step forward for enterprise analytics. Successes and failures alike came under the banner of prototyping. Clearly, experimentation is where we are now with ChatGPT.

The more interesting future for more people may lie in outcomes with small language models, Forrester’s Curran told me. These will succeed or fail on a use case by use case basis.

As industry observer Benedict Evans writes in “ChatGPT and the Imagenet moment,” ChatGPT feels like a step-change forward in the evolution of machine learning. It falls something short of sentience. There is potential but there are plenty of questions to answer before its arc can be well gauged.  [eof]

Read “Forrester: Question generative AI uses before experimentation” – VentureBeat Feb 24, 2023
https://venturebeat.com/ai/forrester-question-generative-ai-uses-before-experimentation/

Read “ChatGPT and the Imagenet moment” – ben-evans.com Dec 14, 2022
https://www.ben-evans.com/benedictevans/2022/12/14/ChatGPT-imagenet

The Inference Cost Of Search Disruption – Large Language Model Cost Analysis – Semianalysis.com Feb 9, 2023
https://www.semianalysis.com/p/the-inference-cost-of-search-disruption

Oracle gets the memo

By

Conf
Ellison updates Oracle Cloud World crowd.

THE SKEPTICAL ENQUIRER – Pride bordering on arrogation is a typical trait of successful enterprise tech companies – it’s definitely been the case with Oracle Corp. Many times it’s met the challenge of a changing computing paradigm with brio, taking its lead from irascible founder Larry Ellison, but the sum substance of its claims would be argued.

I may be a tea leaf reader here, but I see a subtle shift in Ellison’s irascibility in the minutes of last week’s Oracle Cloud World 2022 event.

Ellison’s lively quotes have long been a reporter’s friend. But it’s hard to relay his cloud computing pitches without adding footnotes. The consensus big cloud players are AWS, Google and Microsoft – while enterprise incumbents like Oracle, Teradata, and IBM are cited as tertiary at best.

Overall, Oracle’s financial growth has been modest in the last decade, while its cloud claims have been bold. It is by no means alone in the creative work it has done to define cloud, which is a promise of future growth, on its accounting ledger.

But, in terms of pitching everyday database automation advances of Oracle Autonomous Database [It’s enterprise blocking and tackling, right?] as the one-true self-driving database for the cloud of the future – well, Oracle has no equal there.

Its latest tactic is to boost its acquisition of health system giant Cerner as a great opportunity to rapidly modernize Cerner’s systems, move them to the Oracle Cloud, and count them as such on the financial report.

As I said, the company still experiences gradual overall gains, so I may be talking style rather than substance when I say they missed some big new tech boats, although they revisit these product line gaps from time to time. I’m talking about databases, development tools and persistence stratagems that all called for less spend and new thinking:

 

  • Oracle missed opportunities to expand its MySQL brand at a time when competitive PostgreSQL database versions were becoming go-to players in AWS and Microsoft stables for open-source distributed SQL. In September, the company came out with a bigger/better MySQL known as MySQL the Heatwave Database Service.
  • In fact, Oracle played down, ignored or glossed over inexpensive clustered open-source databases – both SQL and NoSQL — despite having the original Berkeley NoSQL stalwart in its hand.
  • JSON-oriented document database development led by MongoDB is a particular thorn. Oracle, like others, found ways to bring JSON to its RDBMS, but Mongo is still on the up stroke. Oracle last week addressed what it called a mismatch between JSON and the Oracle SQL database in the form of “JSON Relational Duality” – a new developer view supported in Oracle Database 23c.
  • Also, Oracle was slow to support cloud-friendly S3-style object storage. Not surprising in that the great goal is to place data into an Oracle database. But maybe it doesn’t have to be Oracle Autonomous Database. Last week, Oracle described MySQL Heatwave Lakehouse which may be a step in a broader direction of support.

This latter trait, object storage, seems to be getting a bit more of Oracle’s attention, as Snowflake Inc. rises on the back of its 3-in-1 distributed cloud data warehouse, data lake and data lake house. At Oracle’s yearly confab, Snowflake seemed to have garnered leader Ellison’s grudging admiration.

No small feat, that!

It’s a little hard to directly discern Ellison-speak-on-the-page. But his presentation moves salesfolks – and customers too. This recalls Curt Monash once calling him “one of the great public speakers and showmen of the era.”

What did Ellison tell the conference crowd? Well, besides a lot about how Oracle technology helped deliver Covid-19 vaccines, he spoke about the cloud market. People are moving from single-cloud to multi-cloud architectures, Ellison told the crowd at Oracle Cloud World 2022.

“The fact that this is happening is changing the behavior of technology providers…So the first thing that happened as people use multiple clouds is that service providers started deploying new services in multiple clouds, maybe most famously its Snowflake,” he said. Then, in a nod to the new highflier, he added, “And Oracle got the memo.”

“You know, we noticed, and we actually thought that was a good idea,” he said.

Of course, evolution to multi-cloud is a chance for Oracle to take another at bat in the cloud game. The bad news, as in the past, is that so much of the company’s effort is toward moving everything into the Oracle database. That is why any shifts to emphasize MySQL Heatwave would be notable.

Jack Vaughan is a writer and high-tech industry observer.

~~~~~~~~~~~~~~~~~~

My Back Pages
Data Lake Houses Join Data Lake in Big Data Analytics Race – IoT World Today – 2021
Purveyors of data lake houses and cloud data warehouses are many, and opinions on qualifications vary.

Oracle Cloud IaaS security gets superhero status from Ellison – TechTarget – 2018 [mp3 – Podcast]
It’s incremental!

Updated Oracle Gen 2 Cloud aims to challenge cloud leaders – TechTarget – 2018
The central product for Oracle is the database, and all the related tools that support the continued dependence of customers on the database.

On top of the RDB mountain – ADTmag 2001
Never lose the database sale: It’s tattooed on their foreheads.

How well can Nvidia tread the Agglomerverse?

By

Nvidia has worked hard to emerge from the worlds of graphic cards, gaming, and bitcoin mining to become a potent presence in enterprise AI considerations. It also is poised to play as a key vendor in the Metaverse, an AR-imbued but ill-defined repository for the next version of the Web.

More work is in store now as the GPU company – like most companies of any sort – navigates a more difficult economic environment – one where macro winds auger a possible enterprise spending slowdown. Already, Nvidia CEO Jensen Huang has led his crew into spaces others could not imagine.

Graphic Processing Units (GPUs) support ultrahigh memory bandwidth applications. They can churn through neural networks and sundry matrix multiplications like banshees. Huang and company pursue all their possible uses, and created a large portfolio of use cases, even as would-be competitors nip at their heels with more specialized offerings.

Visionary Huang, who we heard last week in keynotes and press conferences related to Nvidia’s GTC 2022 event, calls Nvidia an “Accelerated Computing Company.” And, he has set out to exploit “the Full Accelerated Computing Stack.”

These ambitions take form in a true slew of new offerings – ranging from the Nvidia DLSS3 deep learning sampler to GeForce RTX Series GPUs for neural rendering to Omniverse Cloud Services for Building and Operating Industrial Metaverse Applications, the Omniverse Replicator for synthetic data production and the 2,000-TFLOPS Thor SOC. The latter is probably well-described as “a super chip of epic proportions.”

Nvidia was early to see the possibility that AR/VR technology could drive a more interactive world-wide computing environment. The company coined it “the Omniverse” but now it’s joined others in the “metaverse” quest. For now, the metaverse is a loose agglomeration (the ‘Agglomerverse’?) of such elements as physics simulation, digital twins, and, of course, AI modeling. This puts Nvidia in competition or what Sam Alpert called coopetition with a host of other vendors. Hype vastly surpasses reality in today’s metaverse and the pay-off is both unclear and distant.

Meanwhile, Enterprise AI has found a place in data centers, and Nvidia has established a genuine foothold there. Obscured in the rush of GTC 2022 product announcements were less-than-flashy Apache Spark accelerator technology and AI inference announcements that may show up in revenue reports sooner than metaverse cases. Huang, for his part, sees the two technical domains playing off one another.

Be that as it may, in the metaverse and enterprise AI alike, Huang needs boots on the ground. These undertakings need great advances in skilling around big data.

It remains to be proved that corporations are anymore ready now to take on enterprise AI and the metaverse with imagination and execution. Can they imagine and execute on par or better than they did with Big Data Hadoop beginning ten years ago?

It’s worth noting that GTC 2022 software tools announcement were as proliferate as hardware news, showing the company is seeking ways to simplify the way to such advancements. Nvidia will likely need to take on greater headcount, and forge more mega-partnerships like one announced with Deloitte last week, if it going to successfully seed enterprise AI and metaverse apps.

Like most, Nvidia’s stock has been in free fall. But some of its challenges are unique. When US Government policy looked to slow down or block the transfer of advanced AI to China, Nvidia felt the brunt of it.

Meanwhile, the general rout of crypto currency impedes chip sales to crypto miners – and, as some news reports have it, a recent 2.0 update to the Ethereum blockchain takes a new proof-of-stake approach to processing and reduces the general call for GPUs for mining.

At the same time, the gaming card market has gone from famine to glut in the 24-month-plus period following the start of the global COVID pandemic. Moreover, the cost of these ever-bigger and more functional chips goes up-up-up, emptying gamer’s’ coffers.

Successes in these areas gave Nvidia wiggle room as it pursued enterprise AI. The wiggle room gets smaller just as the metaverse and enterprise AI to-do list gets taller. Among this week’s slew of portfolio additions there are some parts that will find users more quickly than others, and its up to Nvidia to suss those out and ensure they prosper. – Jack Vaughan

InfluxDB Time-Series Cloud Launches on Google Cloud

By

We spoke recently with open-source time-series database maker InfluxData Inc. The occasion was the formal launch of InfluxDB Cloud — a database as a service — on Google Cloud. AWS support precedes this, and Microsoft Azure support is next up.
Launches like this are closely watched as the cloud database has become a proving ground for the future of the open source database. Highly visible wrangling between MongoDB and AWS, particularly, have placed the issue in bas relief.
The questions arise:

Will powerful cloud providers exploit open source loop holes to co-opt small startups database innovations?

Are the innovators, in the words of the old Sonny Boy Williamson song, simply “fattening frogs for snakes?”

Let’s put blues philosophy aside for the moment. Let’s look at the record.

For its part, Google has put increasing effort into closing the wide gap between itself and cloud leaders AWS and Microsoft. This mercantile motive has led to altruism that sees Google taking an interest in effectively partnering with open-source database wunderkinds.
This was clear last year as Google seized on AWS’s wranglings (with Elastic, MongoDB, principally) when it very visibly announced strategic partnerships with select open source database makers including InfluxData, MongoDB, Neo4j, Redis Labs and others. That is some of the background behind the February 4 InfluxDB Cloud on Google Cloud roll out.

For InfluxData CEO Evan Kaplan there is no question that the future of the database, including the InfluxDB time-series database, is in the cloud.
“The market has voted on open source databases,” he said. “We all recognize that the next generation of database applications are going to be founded on open source, and they will be offered on the cloud.”
Clearly, Kaplan and other opensource database company leaders walk the delicate but familiar line of “co-opitition” – working with big cloud providers to reach an audience, while also working to best the big players in ease-of-use, features and performance.
“We have to be able to compete with our technology on the cloud,” he told us.

Databases on the cloud are now somewhere in a multiyear evolution that is far from complete. But, with more and more data accruing to the cloud and the complexity of managing massive infrastructure growing, it makes sense when Kaplan says that the future of databases is in the cloud.
That could be especially true for time series data and analytics in the Industrial Internet of Things (IIoT). That is one of the sweet spots for InfluxData’s technology, Kaplan adjudges. Time series analytics have special currency in IIoT because the trends of blips of data over time disclose important information on how systems and their environments are changing. Such is part and parcel of the Industrial Internet of Things (IIoT).
While IIoT is a perhaps fancy term, it does stand for something. That is, the next generation of industrial systems that adjust their activity in real-time based on immediate analysis of data points.
Today industry looks at data through a new lens, with the purpose of improving operations. Some of these industries have been doing time series data analysis for quite a while. The speed of processing, the amount of data, the ease of programming are all elements that make this a promising technology area, and these are features vendors will focus on going forward.
As with the move from the mainframe to client-server architectures in the 1990s, this could be a telling moment for industrial systems. That is because the increasingly massive scale of processing on the cloud is accompanied by an architectural shift to serverless architecture.
Whether you select a big cloud provider or a smaller database player, the new architecture sees the advent of serverlessness. Database systems run as instances for customers – they run as instances on demand, like utilities (which, after all, was one of the first bits of nomenclature that rose up to describe what later became known as ‘the cloud’).
“InfluxDB Cloud is not tied to a server,” explains Kaplan. “It’s a serverless time-series platform built to collect, store and query processes, and to visualize rapidly ingested raw, high-precision time-stamped data.” The performance thereto is the secret sauce InfluxDB hopes will keep it a few steps ahead of the influential cloud players that dominate areas of computing today.

Other time-series specialists such as Graphite, TimeScale and others are in that hunt as well, while database stalwarts Oracle and IBM continue to tune their top offerings to bring time-series tech to a broader group of their customers. – Jack Vaughan