AI Comes to ASICs in Data Centers

Three years ago, when AI chip startup Nervana ventured into the uncharted territory of designing custom AI accelerators, the company’s move was less perilous than it might have been, thanks to an ASIC expert that Nervana — now owned by Intel — sought for help.

That ASIC expert was eSilicon.

Two industry sources independently told EE Times that eSilicon worked on Nervana’s AI ASIC and delivered it to Intel after the startup was sold. eSilicon, however, declined to comment on its customer.

Nervana’s first-generation AI ASIC, called Lake Crest, was one of the most-watched custom designs for AI accelerators.

Leveraging its own cumulative work with the customer on the design of AI/2.5D systems, Santa Clara, California-based eSilicon rolled out this week a machine-learning AI platform called “neuASIC.” The platform includes “a library of AI-targeted functions that can be quickly combined and configured to create custom AI algorithm accelerators,” explained eSilicon.

A novice AI chip designer could be advised to lean on someone like eSilicon just as Nervana did. After all, the ASIC expert could give some comfort and solid technology foothold in AI ASICs backed by its own real-world experience.

With many companies planning AI chips optimized for certain AI workloads, their biggest roadblock is the constant state of change in AI algorithms. As Patrick Soheili, vice president of business and corporate development at eSilicon, noted, everyone knows that ASICs provide the best power and performance for AI acceleration. But an incautious designer could end up with a static ASIC that’s obsolete on arrival.

That’s where eSilicon hopes to come in.

By using such tools as a Design Profiler and AI Engine Explorer, a variety of IP — some developed by eSilicon and others from third parties — available on the neuASIC platform can be configured as “AI tiles,” explained the company. By turning a knob on eSilicon’s tools, customers can do early power, performance, and area analysis of various candidate architectures, noted Soheili.

Knowing the functions that need to be added and customized, Soheili said that eSilicon can add them as AI titles to the neuASIC platform. This will offer eSilicon’s customers much-needed flexibility.

In short, the neuASIC platform provides a scalable, configurable ASIC chassis with swappable AI tiles, according to eSilicon.

eSilicon’s ASIC chassis offers scalable ASIC architecture 

One industry source who spoke on the condition of anonymity pointed out that “eSilicon isn’t known for its own ‘inherent’ AI skills (like a breakthrough AI processor architecture, for example).” But he added that packaging a collection of IPs, including some off the shelf, is “not a bad thing.” It gives customers “a starting point for custom ASICs.”

There’s one caveat, though. He said, “It’s still left up to the customer to know how to architect something novel from those building blocks.”

Memory blocks

Asked which IP offered on eSilicon’s neuASIC platform is particularly unique, Richard Wawrzyniak, principal analyst for ASIC & SoC at Semico Research Corp., told EE Times that it would be memory blocks.

“They have created memory blocks that incorporate some logic functions, so they are more efficient in operation. These are called Mega Cells: hardware-optimized AI primitives and functions. These blocks include MAC blocks, transpose memories, and multi-port memories.”

He added, “The idea of some ‘computing-in-memory functionality is starting to catch on with a few academic papers recently being discussed at ISSCC. But so far as I know, eSilicon is the only IP vendor to offer a version of this concept commercially to the market.”

Furthermore, Wawrzyniak noted, “Beyond discrete IP blocks, eSilicon has created what they call Giga Cells: full AI IP subsystems. These subsystems include swappable AI tiles, a convolution engine, and other AI functions.”

AI ASICs for data centers

So who would be developing these new AI ASICs?

Purpose-built chips for running artificial-intelligence tasks in data centers are all the rage among the Super 7 (Amazon, Facebook, Google, Microsoft, Alibaba, Baidu, and Tencent). Big internet companies such as Amazon and Facebook are looking for their own AI chips, explained Soheili. They follow in the footsteps of Google, which first unveiled a TPU, its own AI accelerator ASIC specifically designed for neural-network machine learning.

GPUs and FPGAs are key engines that have driven — and still drive — AI in data centers. But when Google announced a TPU in 2016, it explained that it chose to build its own ASIC rather than doubling its data center footprint. The real issue that internet giants confront is the cost of maintaining big data centers in so many different places.

Soheili said that the Super 7 want AI ASICs optimized for certain AI workloads, “because they need to keep the cost of ownership down for their data centers.” As data centers proliferate, maintenance costs and capital expenses for hardware, software, infrastructure, and especially power consumption are getting out of hand, he explained.

Soheili sees eSilicon’s potential customers not only among the Super 7, but also some 50 AI chip startups developing accelerators to be installed in data centers.

Expect more competition in IP subsystems for AI
If the industry’s appetite for AI ASICs is indeed soaring, who else will compete with eSilicon to help companies design them?

Semico’s Wawrzyniak told us, “I believe Broadcom and GlobalFoundries also offer a platform and services to customers to allow them to do a design for AI silicon.”

He added, “However, Broadcom is only currently working with TSMC, and GlobalFoundries will want to produce the silicon once the design using their platform is completed.” That makes eSilicon “the only tier-one ASIC provider that does offer their customers a choice of TSMC or Samsung as the foundry partner for AI designs currently.”

eSilicon styles itself as a key service provider with a platform that offers “customized, targeted IP offered in 7-nm FinFET technology and a modular design methodology.”

Meanwhile, there are plenty of IP vendors who claim to offer IP blocks that are geared toward providing or facilitating AI functionality, added Wawrzyniak.

“Some notable examples are ARM, Cadence Tensilica, Synopsys, Ceva, Imagination, Videantis, MIPS, and others. The parade of these types of AI-assist IP blocks is just starting, and I expect many others to enter the market offering these types of capabilities to designers.”

Furthermore, Wawrzyniak noted, “eSilicon is the first company I am aware of that is creating IP subsystems specific to AI functionality, and I also expect many others to follow them into this area.”

Where 2.5D matters

When Intel announced its Nervana neural network processor (NNP), it dwelt outspokenly on the speed necessary in training deep-learning networks. Intel found its answer in “high-capacity, high-speed high-bandwidth memory (HBM) to provide the maximum level of on-chip storage and blazingly fast memory access” while using separate pipelines for computation and data management.

To help Nervana, eSilicon appears to have provided “custom pseudo two-port memories designed by eSilicon, TSMC’s Chip-on-Wafer-on-Substrate (CoWoS) technology, 28G SerDes, and four second-generation high-bandwidth memory stacks (HBM2).” Furthermore, eSilicon offered 2.5D ecosystem management, silicon-proven HBM2 PHY, ASIC physical design, 2.5D package design, manufacturing, assembly, and testing.

All of these key elements for AI acceleration — including the 2.5D integration of HBM2 memories — are now part of the neuASIC platform’s offering. They are designed to provide vastly higher bandwidth, highly parallel connections to memory stacks, and significant reduction in power consumption.

The first clue that Nervana might be an eSilicon customer, according to one industry source, was that “the eSilicon block diagram looks a lot like the [early] Nervana chip.”

Intel’s Nervana is a large linear algebra accelerator on a silicon interposer next to four 8-Gbyte HBM2 memory stacks. (Source: Hennessy and Patterson, “Computer Architecture: A Quantitative Approach”)

Without naming names, eSilicon claimed in its announcement that the company built the industry’s first AI ASIC.

Soheili told us that eSilicon is currently engaged with several tier-one system providers and high-profile startups to deploy the neuASIC platform and its associated IP. Initial applications, according to eSilicon, will focus on the data center and information optimization, human/machine interaction, and autonomous vehicles.