AI: It changes everything

By Gina Roos, editor-in-chief

Artificial intelligence is making its way across many industry segments, including consumer electronics, industrial, health care, and automotive. Now layer the internet of things on top of requirements for these industries, and designers have a big challenge on their hands. The good news is that chipmakers are making some big strides in innovation — new architectures and new techniques to deliver faster processing and lower power consumption.

The global market for memory and processing semiconductors used in AI applications is forecast to reach $128.9 billion in 2025, up from $42.8 billion in 2019, according to market research firm Omdia. The processor segment will grow faster, from $22.2 billion in 2019 to $68.5 billion in 2025, according to the firm.

“AI is already propelling massive demand growth for microchips,” said Luca De Ambroggi, senior research director for AI at Omdia. “However, the technology is also changing the shape of the chip market, redefining traditional processor architectures and memory interfaces to suit new performance demands.”

There is a variety of processors springing up to accelerate almost any neural network workload, said contributing writer Sally Ward-Foxton. She reports that these new processors all offer something different, whether they’re targeting different vertical markets, application areas, power budgets, or price points.

And these new processors aren’t just coming from processor giants; there are plenty of new startups offering big innovations.

She also found that specialized processing power for AI and machine-learning workloads is available for almost every application, from machine vision to voice interfaces.

Contributing writer Majeed Ahmad looks at the use of development kits to help developers with their AI designs . He said that toolkits can streamline the prepara­tion of trained neural networks for edge and low-latency data-center deployments, with production-grade AI develop­ment kits becoming crucial in overcoming the software support limitations and barriers to AI adoption.

This is especially true when accelerating edge inferences in endpoint devices limited by size and power usage. “That’s where the AI development kits bridge the gap between popular training frame­works such as TensorFlow and highly constrained edge and IoT deployments,” he said.

But first, you should understand the difference between edge AI and endpoint AI , as well as how much smartness is needed in an edge AI device. “Vendors vary from saying that every­thing that is not in the data center is the edge (which includes the gateways, edges of networks, and cars) to defining endpoints as a subset of edge,” said contributing writer Nitin Dahad. He found that there is a very clear distinction between edge AI and endpoint AI, but the way the edge is defined is very elastic.

“It comes down to the application and how much intelligence one can practically place at the endpoint, or at the edge,” he added. “This comes to the tradeoff between memory availability, performance needs, cost, and energy consumption. This will determine how much inferencing and analysis can be done at the edge, how many neural net­work accelerators are needed, whether this is part of an SoC, or whether it sits with a CPU, GPU, or DSP.”

Ahmad also looked at hardware accelerators for AI applications . He believes there are four factors that AI designers should think about when they incorporate hardware accelerators into custom chips for training and inference applications. These specialized devices help create tightly integrated custom processors that offer lower power, lower latency, data reuse, and data locality.

“AI accelerators are specifically designed to enable faster processing of AI tasks; they perform particular tasks in a way that’s not feasible with traditional processors,” he said.

This issue also highlights some of the latest processors developed for IoT and AI applications . These include some of the newest chips from Arm, Microchip, NXP, Renesas, STMicroelectronics, and Eta Compute.