Cloudflare Powers Hyper-local AI inference with Nvidia – The triumph that is Nvidia these days can’t be overstated – although the wolves on Wall St. have sometimes tried. Still, Nvidia is a hardware company. Ok, let’s say Nvidia is still arguably a hardware company. Its chops are considerable, but essentially it’s all about the GPU.
Nvidia is ready to take on the very top ranks in computing. But, to do so, it needs more feet on the street. So, it is on the trail of such, as seen in a steady stream of alliances, partners and showcase customers.
That’s a backdrop to this week’s announcement that Cloudflare Inc will deploy Nvidia GPUs along with Nvidia Ethernet switch ASICs at the Internet’s edge. The purpose is to enable AI inferencing, which is the runtime task that follows AI model training.
“AI inference on a network is going to be the sweet spot for many businesses,” Matthew Prince, CEO and co-founder, Cloudflare, said in a company release concerning the Cloudflare/Nvidia deal. Cloudflare said NVIDIA GPUs will be available for inference tasks in over 100 cities (or hyper-localities) by the end of 2023, and “nearly everywhere Cloudflare’s network extends by the end of 2024.”
CloudFlare has found expansive use in Web application acceleration, and could help Nvidia in its efforts to capitalize on GPU technology’s use in the amber fields of next-wave generative AI applications.
With such alliances, all Nvidia has to do is keep punching out those GPUs – and development tools for model building.
*** *** *** ***
NYT on Human Feedback – The dirty little secret in the rise of machine learning was labeling. Labeling can be human-labor intensive, time-consuming and expensive. It harkens back to the days when ‘computer’ was a human job title, and filing index cards was a way of life.
Amazon’s Mechanical Turk – a crowdsourcing marketplace amusedly named after the 18th Century chess-playing machine “automaton” that was actually powered by a chess master hidden inside the apparatus — is still a very common way to label machine learning data.
Labeling doesn’t go away as Generative AI happens. As the world delves into what Generative AI is, it turns out that human labelers are a pretty significant part.
That was borne out by some of the research I did in the summer for “LLMs, generative AI loom large for MLOps practices” for SDxCentral.com. Sources for the story also discussed how “reinforcement learning through human feedback” was needed for the Large Language Models underpinning Generative AI.
The cost of reinforcement learning, which makes sure things are working, is more than a small part of the sticker shock C-suite execs are experiencing with Generative AI.
Like everything, improvement may come to the process. Sources suggest retrieval augmented generation (RAG) is generally less labor intensive than data labeling. RAG retrieves info from an external database and provides it to the model “as-is.”
RAG is meant to address one of ChatGPT’s and Generative AI’s most disturbing traits: It can make a false claim with amazingly smug confidence. Humans have to keep a check on it.
But the build out of RAG requires some super smarts. As we have come to see, many of today’s headline AI acquisitions are as much about gaining personnel with advanced know-how as they are about gaining some software code or tool. This type of smarts comes at a high price, just as the world’s most powerful GPUs do.
This train of thought is impelled by a recent piece by Cade Metz for the New York Times. “The Secret Ingredient of ChatGPT Is Human Advice” considers reinforcement learning from human feedback, which is said to drive much of the development of artificial intelligence across the industry. “More than any other advance, it has transformed chatbots from a curiosity into mainstream technology,” Metz writes of human feedback aligned with Generative AI.
Metz’s capable piece discusses the role that expert humans are playing in making Generative AI work, with some implication that we should get non-experts involved, too. In response to the story, one Twitter wag suggested that Expert Systems are making a comeback. If so, guess we will have to make-do with more expert humans until “The Real AI” comes along! – Jack Vaughan