Embedding AI into tiny devices opens a world of possibilities

By Jean-Jacques DeLisle, contributing writer

It is no surprise that the increase of complex data-rich cloud features accessible to portable devices and PCs are dramatically increasing the amount of information coming from end-user devices and applications, otherwise known as the “Edge.” Traditional cloud computing systems rely on often outdated, costly data infrastructure paths to relay this vital user and application data from the Edge to data centers. Though more data centers are being built, the flood of traffic promises to overwhelm current technological capabilities in the next few years, especially as enhanced 4G and 5G applications come to life.

Attempting to leap-frog this problem, research teams with Microsoft from both the U.S. and India have been embedding AI into tiny devices, including extremely small microcontrollers that are capable of implementing complex machine-learning algorithms. These minute machine-learning platforms would not only lower the latency of so-called artificial intelligence “AI” applications but may also herald a world of new automated and intelligent Edge applications — the Intelligent Edge.

Nearly microscopic microcontrollers are capable of running complex machine-learning algorithms. Photo by Dan DeLong.

“We’re moving from what is today’s mobile-first, cloud-first world to a new world that is going to be made up of an intelligent cloud and intelligent edge,” said Microsoft CEO, Satya Nadella, during a keynote address at the company’s Build 2017 conference in Seattle. This comment includes the efforts of a multidisciplinary research team of roughly 30 scientists, software engineers, and interns with Microsoft in both Redmond and Bangalore, India. This team is devoted to developing machine-learning software, tools, and resources that can enable embedded AI applications in extremely small microcontroller units.

“The dominant paradigm is that these devices are dumb,” said co-leader of the project, Manik Varma, a senior researcher with Microsoft Research India, when speaking about current technologies. “They sense their environment and transmit their sensor readings to the cloud, where all of the machine learning happens. Unfortunately, this paradigm does not address a number of critical scenarios that we think can transform the world.” Varma detailed an example: “If you’re driving on a highway and there isn’t connectivity there, you don’t want the implant to stop working, In fact, that’s where you really need it the most.”

There are many current restrictions in realizing an Intelligent Edge, some of which include the cost of current systems, the complexity and lack of resources to easily customize machine-learning systems to new applications, and hardware platforms that can handle resource-intensive AI tasks efficiently and in a small platform. Using a variety of algorithms, software, and hardware techniques, the Microsoft team is striving to ease the process of including intelligence in virtually any application. For instance, the team has demonstrated the use of weight-quantization techniques applied to deep neural networks, and in a computer vision application deployed on Raspberry Pi 3s, the team’s weight-quantization technique proved to be comparably accurate with about 20-times-greater computational speed.

“If we have the technology to put the smarts on the tiny devices, but the only people who can use it are the machine-learning experts, then where have we gotten?” said Saleema Amershi, a human-computer-interaction researcher in Microsoft’s Redmond lab. “Giving these powerful machine-learning tools to everyday people is the democratization of AI.”

The research team has made their preliminary work available for download on GitHub, which includes early previews of the compression and training algorithms. Microsoft’s more expansive vision includes many projects like this with the aim to explore technologies that may be commonplace a decade from now. Though not commercially viable yet, these technologies are serving the maker community of technology and solution enthusiasts, who may help bring these technologies to our everyday reality.