Progressive Gauge

Media and Research

Jack Vaughan

Kreps, Dorsey, riff on ChatGPT

By

Stagg Field Nuclear Pile [fragment]

[Boston — May 2023] — “It’s a law that any conversation around technology has to come back to AI within five minutes.”

Well put, Jay Kreps, co-founder and CEO for real-time streaming juggernaut Confluent. Speaking at J.P. Morgan’s Boston Tech Investor event, Kreps knew this was coming. ChatGPT rules the news these days.

Given the daily pounding of 1,000 reporters’ laptops, given Nvidia’s vault into the highest clouds of valuation, it is no surprise that ChatGPT generative AI is the recurring topic. It will impede all other discussion, just as expected by tech stalwarts at J.P. Morgan’s and others’ tech events.

It’s the 600-lb. ChatBot in the room, and it is bigger than big.

Confluent chief on Chatbot interaction

Back in the nascent days of social media, the founders of Confluent, then working at LinkedIn, created a distributed commit log that stored streams of records. They called it Kafka and grew it out into a fuller data stream processing system. It’s intent is to bring to the broader enterprise real-time messaging capabilities akin to that of the Flash Boys of Wall Street.

The company is still in “spend a buck to make a buck” mode. For the quarter ending March 31, Confluent revenues increased 38% to $174.3M, while net jumped 35% to $152.6M. Customers include Dominos, Humana, Lowes, Michelin and others. In January it purchased would-be competitor, Immerok, a leading contributor to the Apache Flink stream processing project.

What’s the significance of real-time streaming in “the age of AI,” Kreps is asked at the Boston event. He says:

It’s really about how a company can take something like a large language model that has a very general model of the world and combine it with information about that company, and about customers, and be able to put those things together to do something for the business.

He gives an example: A large travel company wants to have an interactive chatbot for customers. Seems the barrier ChatGPT faces there for improvements is not so high. As Kreps said: “The chatbots were always pretty bad. It’s like interacting with like the stupidest person that you’ve ever talked to.”

Improvements needed for chatbots include a real-time view of all the information the company holds about customers and operations.

What do you need to make that work? Well, you need to have the real-time view of all the information about them, their flights, their bookings, their hotel, are they going to make their connection, etcetera. And you need a large language model which can take that information and answer arbitrary questions that the customer might ask. So the architecture for them is actually very simple. They need to put together this real time view of their customers, what’s happening, where the flights are, what’s delayed what’s going on. And then they need to be able to call out to a service for the generative AI stuff, feed it this data, feed it the questions from customers, and … integrate that into their service, which is very significant. This is a whole new way of interacting with their customers. And I think that that pattern is very generalizable.

Popping the question: Dorsey

For Jack Dorsey, the question “What about ChatGTP?” is raw meat. He melded SMS and the Web to create Twitter, and now with a nod to bitcoin and block chain has built Block, nee Square. The financial services and digital payments company posted revenue results for the three months ended April 1 that increased 26% to $4.99B, while net loss decreased a significant 92% to $16.8M. The good news was based on increased use of its Cash App product.

At the J.P. Morgan tech investor conference, Dorsey told the people, while hype obviously abounds, true progress rides on use cases.

There’s a ton of hype right now. And I think there’s a lot of companies being started that are going to fail because of that hype. I think the technology industry is very trendy, and very fashionable and jumps from one thing to the next, to the next, to the next. It wasn’t so long ago that we were only talking about Bored Apes and Crypto and NFTs and now we’re talking only about AI and how it’s going to kill us.

There’s always some truth in all these things. I just would caution any company that’s approaching it from a technology perspective, [to] instead use a use case perspective. What is the use case you’re trying to solve? And what technologies can you use to solve it more creatively?

THAT’S THE WAY IT IS — Clearly, panelists and podiumists are preparing to take on ChatGPT questions. At the same time, the clamor of the now will shift to prioritizing generative AI strategically within a host of technology initiatives. ChatGPT may be generalizable — but the proof will not appear overnight. The proof is in the business use case.

Big embedded player Infineon snags Tiny MLer Imagimob

By

Infineon Technologies AG  last week acquired Stockholm-based Imagimob AB, one of the most active players bringing AI to edge devices.

 

[Published May 24, 2023] – Germany-based chip maker Infineon Technologies AG  last week acquired Stockholm-based Imagimob AB, one of the most active players among a slew of startups seeking to bring AI-based machine learning to embedded devices on the edge of the Internet of Things (IoT). Terms were not disclosed.

 

Imagimob provides end-to-end development tools and cloud-based services intended to bring the much-vaunted capabilities of neural machine learning (ML) models from the cloud data center to edge devices. These devices have small footprints, rigorous memory limits, and strict constraints on power consumption. The aspiration to do a lot with little is summed up in the umbrella term “TinyML.”

 

The edge AI devices that Imagimob seeks to support also must cope with a wide variety of sensor types, including sensors that measure and analyze vision, movement, pressure, heat, velocity and other data formats. Business uses are broad, ranging from surveillance cameras and refrigerator monitors in retail settings to actuators and anomaly detectors in oil industry field equipment, and beyond.

 

Infineon’s Thomas Rostech said the purchase is based on his company’s contention that artificial Intelligence and machine learning are about to enter every embedded application, enabling new functionalities. In a statement, Rosteck, who is president of Infineon’s Connected Secure Systems division, boosted Imagimob’s platform and expertise in developing machine learning for edge devices.

 

In recent years, Infineon has worked to build out a portfolio of advanced sensors and IoT solutions. This is an area in which software is expected to play a key role. For example, the market for edge AI software is set to grow to €10.0B in 2032, from €738.5M in 2022, for a CAGR of 29.8% over the forecast period, according to Global Edge AI Software Market Research.

 

Founded in 2013, Imagimob tech- and business-side leaders came out of the mobile applications market. Since inception, its teams have worked on a wide variety of edge AI use cases. These include gunshot and other audio event detection, fall detection, condition monitoring, signal classifiers, safety vests, and more.

 

Imagimob has been highly active within the TinyML community, centered in part around the TinyML Foundation, which is dedicated to nurturing ultra-low power machine learning. Imagimob software has been demoed in showcases with Synaptics, Syntiant, Texas Instruments, and other edge AI hardware concerns.

 

Responding to request for comment, an Infineon spokesperson said the company plans to integrate Imagimob into its organizational structure and that customer relationships with Imagimob’s customers will continue, including partners working on competitor’s hardware, “in alignment with the compliance regulations.”

 

To date, IoT growth has been fitful in the enterprise, as businesses look to move past proof-of-concept projects and achieve return-on-investment. Potential enterprise application areas that include retail, healthcare, supply chain and other operations are places where processing data on the edge translates into cost savings versus processing data in the cloud. The need to off-load processing to the edge becomes more acute as data intensive AI and machine learning capabilities come into play. Imagimob efforts to enable AI’s march from data center to the Internet’s edge are expected to fill out more fully with the backing of the larger chip maker Infineon.

 

Enterprise IoT has lost some luster in recent years as vendors grapple with a very extensive array of use cases. Intelligence in the form of machine learning makes sense, and so does the rise of TinyML as a next stage in delivering on the wide promise of IoT. But deep resources and breakthroughs on the software development side are required. That is at the same time that the venture capital markets have become less benign. So, more matches such as Infineon’s and Imagimob can be anticipated. – Jack Vaughan

Reporter’s Notebook – At MIT Tech Review Future Compute 2023: Navigating the straits of semis

By

[May 9, 2023 ] – When the US last year announced new export rules on advanced chips,  the role of semiconductors in modern foreign affairs reached a new zenith. The chips have assumed the stature of oil in today’s geopolitics and depriving China of the chips now seems a strategic objective.

Unease has only grown with the appearance of the ChatGPT AI Large Language Model, which is a chip-hungry, power-guzzling presence ready to take over the world, to hear networks of experts and Cassandras tell it. Just as unsettling are Chinese maneuvers around Taiwan, a crucial center of global chip production.

Such activity formed a partial backdrop for the MIT Technology Review’s recent Future Compute 2023 conference at the Cambridge, Mass. Campus. Semiconductor issues were probed in a Q&A session featuring Chris Miller, Tufts University lecturer and author.

Miller said the semiconductor has taken on an outsized role in strategizing on China, and that the focus now is both on economics and defense.

”China spends as much money importing chips each year as importing oil,” he said. “You can’t understand the structure of the world economy without putting semiconductors at the center of your analysis.”

This is increasingly true for economic issues, Miller continued. Semiconductors that drive computers and embedded systems are top of mind when defense ministries and intelligence agencies think about future procurements.

“What they know is that over the past half century one of the key forces that’s transformed the way militaries fight has been computing power,” according to Miller, who traced the developments leading to the present predicament in “Chip War: The Fight for the World’s Most Critical Technology,” a recent noteworthy [Financial Times Book of the Year 2022] look at semiconductor industry history and its ever-shifting role in the larger body politic.

“Chip War” is described by a New York Times reviewer as something of a nonfiction thriller in which ‘pocket-protector men’ at Fairchild Semiconductor and Intel  tamed the raw transistor, fashioned the Integrated Circuit, outdid the Soviet Union, and left a war weary Europe in the dust as they formed what’s now Silicon Valley. Many of those developments bear review as governments’ and companies’ take on present complexities.

The complexities include more seemingly modest products than high-end processors, Miller indicated. Simpler chips that complement the hot processers grow in importance as well.

“The entire electronics supply chain is actually beginning to shift. It’s not only at the chip level, it’s also electronics assembly and simpler components,” Miller said, adding that a reduction in China’s level of server assembly has led to a major increase in Mexico’s market share in that field.

The also point emergence of new market dynamics as large companies take on design of their own chips, which could be spurred for a wider range of companies as US Chips Act R&D funding addresses the need for less expensive chip design processes.

A qubit for your thoughts

Infant quantum computing looms as an adjacent technology where geopolitical ambitions may play out.

China, the US the EU, and countries such as Australia, Singapore, and Canada now devote research monies to pursue such quantum efforts. They stir this new ground at the same time they test the limits of Moore’s Law – the perceived dead end for further large-scale silicon chip integration, which Tuft’s Miller cites as a fundamental challenge facing the chip industry.

However, quantum technology is still-raw technology – the quantum researchers on the main are still found toiling at the qubit level with lab rigs and signal scopes – that is, the quantum equivalent of the lone transistor work that preceded development of the Integrated Circuit.

A high-point of the Future Compute 2023 agenda for me was a visit to MIT’s Engineering Quantum Systems Group’s labs. Smart people are working hard on this frontier technology. And, with notable exceptions, there is knowledge sharing going on.

But, in a conference panel on quantum at the event, the impression emerged that quantum computing needed a large-scale working version of a quantum computer before the international competition for quantum computing would reach a less-sanguine stage akin to that the advanced CPU, GPU, NPU and network processing chips now experience.

For his part, at Future Compute, Chris Miller hesitated somewhat in responding to an audience question on quantum computing.

“I struggle to say anything that intelligent on quantum computing, both because I’m really not an expert in computing, but also because there’s a chip industry that I can study and I know how to talk about, whereas quantum computing is still a prospective industry,” he said. “We all hope it will materialize but it hasn’t materialized in a practical form.”

My take

Global chip wars must be viewed in the context of a real war underway in Ukraine. It has exposed the pivotal role of new technology in the exercise of war, as well as the vulnerability of the supply chains that feed modern commerce. It’s also pushed diplomacy to the sidelines, narrowing the opportunity for maneuver in the semiconductor straits.

Reporter’s Notebook: At Quantum.Tech 2023 in Boston

By

[Boston/May 2, 2023] — Its mainstream profile is a bit more modest than in recent years, but quantum computing today continues to inch forward, promising future breakthroughs based on the stranger properties of electrons, atoms, ions and photons.

Quantum computing advocates project a time when the binary-state bits of classical computing are surpassed – or complemented, in the case of the more forgiving futurists — by quantum qubits capable of handling multiple states. And venture funding is active. McKinsey estimates in 2022 investors put $2.35 billion into quantum tech start-ups. Recent weeks have seen investments on several continents. These include:

*Dell Technologies Capital announcing funding a $12M seed extension investment in Quantum Source, an Israel-based company pursing photonic quantum technologies;

*Molten Ventures, Altair, the National Security Strategic Investment Fund and others announcing a $16.5 million round for UK-based Riverlane, which creates software and hardware to address the problem of galloping error rates as machines scale up their qubit counts – a basic but incomplete measure of quantum processing; and

*Funding by ParticleX for Quantier, an off-shoot of Hong Kong University of Science and Technology, said to fashion laser lights to control atoms at room temperature.

The activity of governments in such funding is common and understandable. For them the risk of misspent money is outweighed by the risk of other countries gaining a potentially key technological edge.

The startups need funding, but also increasingly need to find immediate use cases, as the difficulties facing some quantum pioneers highlights. D-Wave and Rigetti Computing, both of which turned to SPAC vehicles to ply the public markets, have found themselves with somewhat curtailed money ramps, as disclosed in recent financial reports.

But enthusiasm is still strong, as witnessed at last week’s Quantum.Tech 2023 conference here. Emergent themes held that this is a global phenomenon important to nation states of all sizes, that qubit counts matter less than qubit quality, and that vendors and users alike are looking to move past the present era of noisy intermediate-scale quantum (NISQ) computing – that is to create more capable qubits in greater numbers.

Quantum computing scorecards are more than ever a jumble of types, as small labs come forward with completely different approaches to basic quantum computing processes. Where not that long ago superconducting and trapped-ion approaches were the main game, processing modalities  based on photonics, neutral atoms, and silicon-based spin methods now vie for attention.

Marathona est, non sprint

Manfred Rieck, VP, Individual Software Development, Deutsche Bahn, sees quantum efforts feeding other advanced computing efforts. He spoke at the Boston event on quantum computing, picturing it as the next part of multinationals’ tech stacks that may also include AI, IoT, big data and digital twins.

He suggested, for example, that quantum computers could help in building fine-grain simulations for digital twins that design and operate transportation in the future, speaking with us in an interview.

Rieck marks progress of late, as quantum efforts combining the work of both physicists and computer scientists move out of the university, and into industry.

Moreover, he sees the struggles of some quantum players as a natural thing that happens with quantum-tech startups.

“I think companies are coming in and going out of the game and sometimes it works and sometimes not. At the same time, you see more and more interest in what happens in large industries,” he said. For vendors and buyers alike, he advised, the focus should be less on ‘quantum’ and more on business benefits.

As part of that, customers will look to companies that have long-term management skills to complement their belief in quantum computing’s radical promise.

One notion Rieck disavows is that quantum computing is entering a down phase, one some liken to the season of “winter.” He prefers the runner’s analogy – the quantum journey is a marathon, not a sprint, he told Quantum Tech attendees.

Hybridization erit belli

For Florian Neukart, who was deeply involved in some early Volkswagen AG quantum work, hybridization is the next step. Hybridization combines quantum computing along with classical high-performance computing, according to Neukart, who now serves as chief product officer at Terra Quantum, which provides “Quantum as a Service” in the form of algorithmic, hardware and security offerings.

“When we say hybrid, it means combining these two efficiently using an operating system using hybrid algorithms that sit on top,” said Neukart, who took part in a panel discussion on quantum’s path to commercialization at Quantum Tech 2023.

The growth in approaches based on hybrids can be drowned out by the quantum technology community’s natural preoccupation with quantum processing chips. But they bear watching.

“For the current era, where the quantum computing devices are error-prone and not perfect, or as big as we want them to be, hybridization will be a strategy, and it will carry on into the future,” he  told us. Examples of hybridization in action are recently developed quantum algorithms St. Gallen, Switzerland-based Terra Quantum undertook with German specialty chemical firm Evonik Industries to improve fluid mixing simulations and shape optimization for production parts of mixers.

Neukart agrees with the contention that quantum computing’s near future will be more influenced by computer science community members than in the past.

“People in classical high-performance computing really know how to scale systems, how to operate these systems cost effectively,” he said. That same community also is familiar with the work involved in integrating new algorithms into an existing environment – which is also among the many hurdles the quantum computing crew is now facing.

Hortus curae tuae

A software platform intended to put quantum processing in the hands of a broad group of users is the objective of Austin, Texas-based Strangeworks. At the Boston event, the five-year-old company highlighted updates to its platform said to allow  subject matter experts to  access “a marketplace of pre-packaged applications for everything from optimization and kernel alignment to variational quantum eigensolvers and neural networks.”

This comes after recent completion of a $24-million funding round led by Hitachi Ventures, with investments from IBM, Raytheon, and others.

“We’ve added abstraction layers to make it easier for subject matter experts – and everybody else – to take advantage of these tools. Below them, developers can open up pre-packaged applications and change the parameters,” Strangeworks founder and CEO William ‘whurley’ Hurley told us at Quantum Tech 2023.

We asked how does Hurly gauge the progress or lack of progress with quantum computing today.

“The fact is, is these machines are useful today,” he responded. “They work for material design, a lot of chemistry problems. Finance is starting to really get its legs. But they don’t match anywhere near the hype the industry has put out about them.”

And, he’s not averse to the term ‘quantum winter.’ Hurley said a challenging economic climate can strengthen an industry now desperately in search of skilled talent.

“We are entering –  whether anybody wants to say or not – a quantum winter, and this is a good thing. Everybody associates a quantum winter, or a downturn in funding, or whatever,  as this big, dramatic event,” he said. “In reality, it’s kind of like pruning your rosebush, or taking care of your garden.” – Jack Vaughan

The March of the Language Models

By

[April 17, 2023] – Had the opportunity to speak with Forrester Analyst Ronan Curran recently for a VentureBeat article. Of course, the topic was ChatGPT, generative AI, and Large Language Models.

His counsel was both optimistic and cautionary – a good summation of the bearings IT decision makers should set as they begin yet another tango with a new technology meme.

A handy summarizer-paraphraser tells me that Curran told VentureBeat that it would be a mistake to underestimate the technology, although it is still difficult to critically examine the many potential use cases for generative AI.

Yes, such applies to each technical challenge – every day. And it bears repeating as each new technology whispers or yells that the fundamental rules no longer apply – and yet they do.

Looking back on my conversation with Curran, I find insight in what some would say is obvious. The large language models are … large! And, as Curran told me, because they are large, they cost a lot to compute and train. This reminds us, as others have, that the LLM should be viewed like polo or horse racing – as a game for the rich.

Why do we say game for the rich? On one level, the LLM era stacks up as megacloud builders’ battle, albeit with aspects of the playground grudge match. Microsoft leader Satya Nadella, who had the thankless task of competing with Google on the search front, almost seems to chortle: “This new Bing will make Google come out and dance, and I want people to know that we made them dance.”

For the cloud giants, the business already had aspects of a war of attrition, as they staked data center regions across the globe. The folks at Semianalysis.com have taken a hard stab at estimating a day in the life of an LLM bean counter, and they suggest a “model indicating that ChatGPT costs $694,444 per day to operate in compute hardware costs.” Of course, these are back of the envelope estimates – and the titans that host LLMs will look to engineer savings.

The new LLM morning summons to mind a technology that  consumed  much attention not so long ago: Big Data. The magic of Hadoop had a difficult time jumping from the likes of Google, Facebook and Netflix to the broader market. Maybe Big Data should have been named ‘Prodigious Data’ – because that would have offered fairer warning to organizations that had to gather such data, administer it, and come up with clever and profitable use cases.

“What is Big Data good for?” was a common question, even in its heyday. Eventually the answer was “machine learning”.

Much of Big Data remained in the realm of the prototype. In the end, it was a step forward for enterprise analytics. Successes and failures alike came under the banner of prototyping. Clearly, experimentation is where we are now with ChatGPT.

The more interesting future for more people may lie in outcomes with small language models, Forrester’s Curran told me. These will succeed or fail on a use case by use case basis.

As industry observer Benedict Evans writes in “ChatGPT and the Imagenet moment,” ChatGPT feels like a step-change forward in the evolution of machine learning. It falls something short of sentience. There is potential but there are plenty of questions to answer before its arc can be well gauged.  [eof]

Read “Forrester: Question generative AI uses before experimentation” – VentureBeat Feb 24, 2023
https://venturebeat.com/ai/forrester-question-generative-ai-uses-before-experimentation/

Read “ChatGPT and the Imagenet moment” – ben-evans.com Dec 14, 2022
https://www.ben-evans.com/benedictevans/2022/12/14/ChatGPT-imagenet

The Inference Cost Of Search Disruption – Large Language Model Cost Analysis – Semianalysis.com Feb 9, 2023
https://www.semianalysis.com/p/the-inference-cost-of-search-disruption