By Heather Hamilton, contributing writer
Microsoft unveiled a deep-learning acceleration platform, called Project Brainwave, at Hot Chips 2017, held last week in Cupertino, California. The system will allow developers to use machine-learning models on programmable silicon beyond what they might expect from a CPU or GPU.
At the symposium, Microsoft showed a Gated Recurrent Unit running on Intel’s Stratix 10 FPGA chip at a speed of 39.5 teraflops without batching operations, reports Venture Beat. This is important because it means that the hardware is capable of managing requests in real time, providing insights as they happen for machine-learning systems. For consumers, this is important because it means that they aren’t waiting for applications to respond.
Microsoft’s model is larger than neural networks Alexnet and Resnet-50, which have been used to benchmark the hardware of other companies.
Currently, Brainwave is used in FPGAs installed in Microsoft data centers, which Doug Burger, a Microsoft Research engineer, says will allow Microsoft services to more rapidly support AI features. Brainwave may also soon be available to third-party consumers via the Azure cloud platform.
Brainwave works by loading a trained machine-learning model into the memory of FPGA hardware, where it remains. The hardware is then utilized to configure the insights that the model is asked to generate. If a model is too large for a single FPGA, it utilizes multiple hardware boards upon deployment of software.
In a press release, Burger explains that the system is built in three main layers: first a high-performance, distributed system architecture, then a hardware DNN engine synthesized onto FPGAs, and finally a compiler and run time for low-friction deployment of trained models.
Brainwave leverages Microsoft’s FPGA infrastructure and, by attaching FPGAs directly to the data center, serves DNNs as hardware microservices in which DNN can be mapped to remote FPGAs and called by a server without software in the loop. Burger says that this system requires latency because the CPU doesn’t need to process incoming requests, allowing for high throughput.
By using a powerful soft DPU, in contrast to the hardened DPU that many companies use, Brainwave offers flexibility by offering a design that scales across data types. By combining the ASIC digital signal processing blocks on the FPGAs and synthesizable logic, Brainwave is more optimized to offer customizable narrow-precision data types that offer higher performance without a loss of accuracy and the incorporation of research innovations into hardware platforms within a few weeks.
Finally, Burger writes that Brainwave incorporates a software stack to support a range of deep-learning frameworks, with more to come. Currently, Brainwave supports Microsoft’s CNTK and Google’s TensorFlow framework.
Burger believes that Brainwave will demonstrate that FPGAs are just as efficient as chips made for larger operations and invites Intel and Microsoft to optimize the hardware’s performance and how Brainwave uses it. He expects that, with improvements, Microsoft will hit 90 teraflops with Intel Stratix 10.
Sources: Venture Beat, Microsoft
Image Source: Microsoft
Advertisement
Learn more about Electronic Products Magazine
