Read more about the article Fractile Licenses Andes Technology’s RISC-V Vector Processor as It Builds Radical New Chip to Accelerate AI Inference

Fractile Licenses Andes Technology’s RISC-V Vector Processor as It Builds Radical New Chip to Accelerate AI Inference

San Jose, CA — Oct. 22, 2024 — Andes Technology, a leading supplier of high-efficiency, low-power 32/64-bit RISC-V processor cores and Founding Premier member of RISC-V International, are proud to announce a partnership with Fractile, the company building the chips and systems needed to reach the next frontier of AI performance. Fractile is developing AI inference accelerators based on in-memory compute, and aim to be able to run frontier AI models – large language, vision and audio models – two orders of magnitude faster than existing hardware, at a tenfold reduction in cost.

Large language models and other foundation models have become the driving force behind the skyrocketing scale of data center AI compute requirements. From ChatGPT to the open-source Llama model series, LLMs and other foundation models are finding widespread application. Model inference – the process of serving these trained models –  is coming to be the dominant portion of compute costs, exceeding the cost of model training.  Fractile has licensed the powerful Andes AX45MPV RISC-V vector processor, combined with ACE (Andes Automated Custom Extension™) and Andes Domain Library, and plans to incorporate the vector processing unit into their first generation data center AI inference accelerator.

Fractile’s uses novel circuits to execute 99.99% of the operations needed to run model inference in on-chip memory. This removes the need to shuttle model parameters to and from processor chips, instead baking computational operations into memory directly.  This architecture drives both much higher energy efficiency (TOPS/W) as well as dramatically improved latency on inference tasks (tokens per second per user in an LLM context, for instance). The company has been betting on inference scaling – leveraging more inference time-compute to improve AI performance – as the next frontier of AI scaling. The AI world seems to agree, with OpenAI recently releasing their latest LLM, o1, which requires orders of magnitude more inference compute than previous LLMs. Fractile’s hardware and software stack is built to take models that can still take many seconds to produce an answer on current hardware, and make this instantaneous.

As part of the collaboration, Fractile will integrate Andes Technology’s high-performance RISC-V vector processor with its own groundbreaking in-memory computing architecture via ACE. Fractile’s architecture leverages the strengths of both companies, aiming to deliver an exceptionally fast and cost-effective AI inference system that overcomes the limitations of conventional computing methods – blasting through the memory bottleneck.

Dr. Charlie Su, President and CTO of Andes Technology, expressed his enthusiasm for the partnership, “AX45MPV, with strong compute capabilities, high memory bandwidth and the flexible ACE tool, has been chosen by innovative AI companies large and small since its debut in 2023. Andes RISC-V vector processors have enabled many AI SoCs to break free from architecture limitation and achieve new levels of performance and efficiency. We are confident that the synergy between Fractile’s In-Memory Computing technologies and Andes’ award-winning RISC-V vector processing will lead to yet another success.”

Dr. Walter Goodwin, CEO and founder of Fractile, added: “The limitations of existing hardware present the biggest barrier to AI performance and adoption. Andes Technology has unmatched technical and commercial leadership on RISC-V vector processors and is a natural partner for us as we build Fractile’s accelerator systems. Building hardware for AI acceleration is intrinsically hard – the world’s leading models can change overnight, while chips take time to bring to market. Software-programmable vector processors like Andes’ are a key part of staying robust to these changes. We’re delighted to announce this collaboration as Fractile furthers its mission to supercharge inference.

For more information about Andes Technology and Fractile, please visit their respective websites at www.andestech.com and www.fractile.ai.

About Andes Technology
Nineteen years in business and a Founding Premier member of RISC-V International, Andes is a publicly-listed company (TWSE: 6533; SIN: US03420C2089; ISIN: US03420C1099) and a leading supplier of high-performance/low-power 32/64-bit embedded processor IP solutions, and the driving force in taking RISC-V mainstream. Its V5 RISC-V CPU families range from tiny 32-bit cores to advanced 64-bit Out-of-Order processors with DSP, FPU, Vector, Linux, superscalar, and/or multi/many-core capabilities. By the end of 2023, the cumulative volume of Andes-Embedded™ SoCs has surpassed 14 billion. For more information, please visit https://www.andestech.com. Follow Andes on LinkedIn, Twitter, Bilibili and YouTube! ! 

About Fractile
Fractile is an AI hardware company that is building its first groundbreaking new AI chip, capable of running state-of-the-art AI models up to 100x faster and 10x cheaper than existing hardware. Founded in 2022 in London by 28-year-old artificial intelligence PhD Walter Goodwin, Fractile’s transformative computing technology will enhance collective AI capabilities by enabling the largest and most capable neural networks of today and tomorrow to run faster, more efficiently and more sustainably. The company has raised $17.5m (£14m) in funding from investors including the NATO Innovation Fund, Kindred Capital, Oxford Science Enterprises, Cocoa and Inovia Capital, as well as angel investors including Hermann Hauser (founder, Acorn, Amadeus), Stan Boland (ex-Acorn, Icera, NVIDIA and Five AI) and Amar Shah (co-founder, Wayve).

Continue ReadingFractile Licenses Andes Technology’s RISC-V Vector Processor as It Builds Radical New Chip to Accelerate AI Inference
Read more about the article TetraMem Integrates Energy-Efficient In-Memory Computing with Andes RISC-V Vector Processor

TetraMem Integrates Energy-Efficient In-Memory Computing with Andes RISC-V Vector Processor

By Wenbo Yin, Vice President of IC Design, TetraMem Inc.

Introduction
The rapid proliferation of artificial intelligence (AI) across a growing number of hardware applications has driven an unprecedented demand for specialized compute acceleration not met by conventional von Neumann architectures. Among the competing alternatives, one showing the greatest promise is analog in-memory computing (IMC). Unleashing the potential of multi-level Resistive RAM (RRAM) is making the promise more real today than in the past. Leading this development, TetraMem, Inc., a Silicon Valley based startup, is addressing the fundamental challenges holding this solution back. The company’s unique IMC that employs multi-level RRAM technology provides more efficient, low-latency AI processing that meets the growing needs of modern applications in AR/VR, mobile, IoT, and beyond.

Background on the Semiconductor Industry
The semiconductor industry has seen significant advancements over the past few decades, particularly in response to the burgeoning needs of AI and machine learning (ML). Innovations in chip design have pushed the boundaries of performance and efficiency. However, several intrinsic persistent challenges remain, such as the von Neumann bottleneck and memory wall, which limits data transfer rates between the CPU and memory, and the escalating power consumption and thermal management issues associated with advanced node technologies.

In-memory computing (IMC) represents a ground-breaking computing paradigm shift in how data processing is accomplished. Traditional computing architectures separate memory and processing units, resulting in significant data transfer overheads, especially for the data centric AI applications. On the other hand, IMC integrates memory and processing within the same physical location, enabling faster and more efficient data computations with a crossbar array architecture to further eliminate the large quantity of intermediate data from those matrix operations. This approach is particularly beneficial for AI and ML applications, where large-scale data processing and real-time analytics are critical.

Selecting a suitable memory device for IMC is crucial. Traditional memory technologies like SRAM and DRAM are not optimized for in-memory operations due to their device and cell constraints and their volatility idiosyncrasies. RRAM, with its high density, multilevel capability and non-volatility with superior retention, overcomes these challenges with no refresh needed. The working principle of RRAM involves adjusting the resistance level of the memory cell through controlled voltage or current, mimicking the behavior of synapses in the human brain. This capability makes RRAM particularly suited for analog in-memory computing.

TetraMem has focused its efforts on multi-level RRAM (memristor) technology, which offers several advantages over traditional single level cell memory technologies. RRAM’s ability to store multiple bits per cell and perform efficient matrix multiplications in situ makes it an ideal candidate for IMC. This technology addresses many of the limitations of conventional digital computing, such as bandwidth constraints and power inefficiency.

The RRAM programmable circuit element remembers its last stable resistance level. This resistance level can be adjusted by applying voltage or current. Changes in magnitude and direction of voltage and current applied to the element alters its conductance, thus changing its resistivity. Akin to how a human neuron functions, this mechanism has diverse applications: memory, analog neuron, and, at TetraMem, in-memory computing. The operation of an RRAM is driven by ions. With control of the conductive filament size, ion concentration and height, different multi-levels for cell resistance can be precisely achieved.

Data processed in the same physical location as it is stored with minimum intermediate data movement and storage results in low power consumption. Massive parallel computing by crossbar array architecture with device-level grain cores yields high throughput. And computing by physical laws in this way (Ohm’s law and Kirchhoff’s current law) produces low latency. TetraMem’s nonvolatile compute in-memory cell reduces power consumption by orders of magnitude over a conventional digital von Neumann architecture.

Notable Achievements
TetraMem has achieved significant milestones in the development of RRAM technology. Notably, the company has demonstrated an unprecedented device with 11 bits per cell, achieving over 2,000 levels in a single element. This level of precision represents a major breakthrough in memory compute technology.

Recent publications in prestigious journals such as Nature1 and Science2 highlight TetraMem’s innovative approaches. Techniques to improve cell noise performance and to enhance multi-level IMC have been key areas of advancement. For example, TetraMem has developed proprietary algorithms to suppress random telegraph noise, resulting in superior memory retention and endurance characteristics for RRAM cells.

Operation of IMC
TetraMem’s IMC technology utilizes a crossbar architecture, where each cross-point in the array corresponds to a programmable RRAM memory cell. This configuration allows for highly parallel operations, which are essential for neural network computations. During a Vector-Matrix Multiplication (VMM) operation, input activations are applied to the crossbar array, and the resulting computations are collected on the bit lines. This method significantly reduces the need to transfer data between memory and processing units, thereby enhancing computational efficiency.

Real-World Applications
TetraMem’s first evaluation SoC through the commercial fab process, the MX100 chip (see figure) exemplifies the practical applications of its IMC technology. The chip has been demonstrated in various on-chip demos, showcasing its capabilities in real-world scenarios. One notable demo, the Pupil Center Net (PCN), illustrates the chip’s application in AR/VR for face tracking and authentication monitoring in autonomous vehicles.

To facilitate the adoption of its technology, TetraMem provides a comprehensive Software Development Kit (SDK). This SDK enables developers to define edge AI models seamlessly. Furthermore, the integration with Andes Technology Inc.’s NX27V RISC-V CPU with Vector extensions streamlines operations, making it easier for customers to deploy TetraMem’s solutions in their products.

The TetraMem IMC design is great for matrix multiplication but not as efficient in other functions such as vector or scalar operations. These operations are used frequently in neural networks.  For these functions, Andes provides the flexibility of a CPU plus a vector engine as well as an existing SoC reference design and a mature compiler and library to accelerate our time to market.

TetraMem collaborated with Andes Technology to integrate its IMC technology with Andes’ RISC-V CPU with Vector Extensions. This partnership enhances the overall system performance, providing a robust platform for a variety of AI tasks. The combined solution leverages the strengths of both companies, offering a flexible and high-performance architecture.

Looking ahead, TetraMem is poised to introduce the MX200 chip based on 22nm, which promises even greater performance and efficiency. This chip is designed for edge inference applications, offering low-power, low-latency AI processing. The MX200 is expected to open new market opportunities, particularly in battery-powered AI devices where energy efficiency is paramount.

Conclusion
TetraMem’s advancements in in-memory computing represent a significant leap forward in the field of AI hardware. By addressing the fundamental challenges of conventional computing, TetraMem is paving the way for more efficient and scalable AI solutions. As the company continues to innovate and collaborate with industry leaders like Andes Technology, the future of AI processing looks promising. TetraMem’s solution not only enhances performance but also lowers the barriers to entry for adopting cutting-edge AI technologies.

  1. “Thousands of conductance levels in memristors monolithically integrated on CMOS”, Nature, Mar 2023 https://rdcu.be/c8GWo
  2. “Programming memristor arrays with arbitrarily high precision for analog computing”, Science, Feb 2024 https://www.science.org/doi/10.1126/science.adi9405
Continue ReadingTetraMem Integrates Energy-Efficient In-Memory Computing with Andes RISC-V Vector Processor
Read more about the article Rivos Selects Andes NX45 for Control Functions in Upcoming High-Performance RISC-V SoC

Rivos Selects Andes NX45 for Control Functions in Upcoming High-Performance RISC-V SoC

NX45 Becomes the Only RISC-V Core to Pass Rivos’ Rigorous Verification Process After Extensive Evaluation of Leading RISC-V Cores

San Jose, CA – Sep. 11, 2024— Rivos Inc., a RISC-V Premier member company focused on accelerating data analytics and Generative AI workloads and Andes Technology, a leading supplier of 32/64-bit RISC-V processor cores and a RISC-V Founding Premier member , announced that Rivos Inc. has licensed the Andes NX45 RISC-V Processor for key control functions in their products.

Rivos was founded in 2021 by industry veterans from Google, Intel, Apple, and PA-Semi and has assembled a world class team of silicon, software, and platform engineers to build industry-leading power efficient, high performance, secure server solutions based on a high-end internally developed RISC-V CPU. 

To run control and scheduling for several key functions in Rivos’ SoC, the highly configurable and extensible Andes NX45 RISC-V processor was chosen because it allows the best tradeoffs in performance and efficiency, while meeting the highest quality standards.

“We are excited to welcome Rivos Inc. to the RISC-V community and wish them tremendous success,” said Dr. Charlie Su, President & CTO of Andes Technology. “We are proud that Rivos chose the NX45 for their project. Rivos’ selection of Andes is a testament to our flexibility, development rigor, and dedication to quality.”

“The growth of the RISC-V ecosystem and customer traction has been remarkable, and we are thrilled to be part of this movement,” said Belli Kuttanna, Co-Founder and CTO at Rivos Inc. “After evaluating several leading RISC-V cores, the Andes NX45 stood out as the only core that passed our proprietary verification process with zero bugs. Its robust configuration options and ease of integration made it the clear choice as our 64-bit control core.”

Rivos recently raised over $250M in an oversubscribed series A-3 funding round to enable the company to tape out its first silicon product, expand manufacturing operations, and scale platform hardware and software engineering efforts.

Andes Technology has been delivering a full range of processing solutions for over 19 years.  Launched in 2019, the AndesCore™ 45-series includes in-order 8-stage dual-issue RISC-V processors with options to support multicore, Linux, and vector processing to meet the demands of many high-end applications.  Andes’ customers benefit from a full-product offering including AndeSight™ IDE, Andes Custom Extension™ (ACE) and related software, and modeling, debug, and trace tools to accelerate their SoC development.

About Rivos Inc.
Rivos has assembled a world class team of silicon, software and platform designers implementing the long term vision of building industry-leading power efficient, high performance, secure server solutions, based on RISC-V, using workload-defined hardware. Rivos supports the intense requirements of the large language models and data analytics through a full solution of optimized chips; combining RISC-V CPUs, a Data Parallel Accelerator, and a reference multi-chip OCP modular server along with a full firmware-to-application open software stack. Rivos is hiring engineering talent across multiple disciplines.

About Andes Technology
Nineteen years in business and a Founding Premier member of RISC-V International, Andes is publicly-listed company (TWSE: 6533; SIN: US03420C2089; ISIN: US03420C1099) and a leading supplier of high-performance/low-power 32/64-bit embedded processor IP solutions, and the driving force in taking RISC-V mainstream. Its V5 RISC-V CPU families range from tiny 32-bit cores to advanced 64-bit Out-of-Order processors with DSP, FPU, Vector, Linux, superscalar, functional safety and/or multi/many-core capabilities. By the end of 2023, the cumulative volume of Andes-Embedded™ SoCs has surpassed 14 billion. For more information, please visit https://www.andestech.com. Follow Andes on LinkedIn、Facebook、X、Bilibili  and YouTube! 

 

Continue ReadingRivos Selects Andes NX45 for Control Functions in Upcoming High-Performance RISC-V SoC
Read more about the article Resiltech and Andes Technology Announce Collaboration to Deliver Advanced STL Solutions for Andes Automotive-Grade RISC-V Processor IP

Resiltech and Andes Technology Announce Collaboration to Deliver Advanced STL Solutions for Andes Automotive-Grade RISC-V Processor IP

Pontedera, Italy and Hsinchu, Taiwan – Aug 29th, 2024 – Resiltech, a renowned provider of comprehensive security and safety solutions and services, and Andes Technology, a leading supplier of high-performance, low-power RISC-V processor IP are pleased to announce a strategic collaboration to deliver advanced Software Test Library (STL) solutions for Andes’ automotive-grade RISC-V processor IP.

This partnership combines Andes Technology’s expertise in delivering cutting-edge RISC-V processor IP with Resiltech’s proven track record in developing robust STL solutions. Together, they aim to enhance the safety and reliability of automotive electronic systems.

The collaboration will focus on enabling Resiltech to develop advanced STL that can perform safety diagnostic analysis against Andes automotive-grade processor IP line-up. The combined offerings from both companies ensure rigorous fault detection and mitigation, providing automotive OEMs and Tier-1 suppliers with reliable and safe processor solutions.

Resiltech’s STLs are designed to streamline the safety certification process of the target system providing a pre-certified product specifically tailored for the target silicon without the need of any additional activities, an easy and fast SW integration strategy and additional ad-hoc support for the system safety integration.

Andes is committed to delivering functionally safe automotive RISC-V IP, having achieved company-wide ISO-26262 ASIL-D compliance for systematic development process in 2020. Since then, Andes has released the 25-SE series processors, including the N25F-SE and D25F-SE, which have achieved ASIL-B full compliance and gained over a dozen customer projects.  Some customers have already entered mass production and also achieved SoC level ISO-26262 compliance leveraging Andes’ work products. Furthermore, the company plans to release the D45-SE and D23-SE processors including ISO-26262 certification in the coming months, targeting ASIL-D full compliance.

“We are thrilled to partner with Resiltech to bring enhanced safety features to our automotive-grade RISC-V IP,” said Samuel Chiang, marketing director of Andes Technology. “This collaboration underscores our commitment to providing our customers with the most advanced and reliable solutions for automotive applications.”

Francesco Rossi, Safety Solution Director of Resiltech, added, “Our expertise in STL development complements Andes Technology’s innovative processor IP. Together, we are set to provide the automotive industry with a comprehensive solution that not only meets but exceeds the stringent safety requirements of modern vehicles.”

This collaboration marks a significant milestone in advancing the safety of automotive electronic systems, paving the way for the next generation of smart and safe vehicles.

About Andes Technology
Nineteen years in business and a Founding Premier member of RISC-V International, Andes is a publicly-listed company (TWSE: 6533; SIN: US03420C2089; ISIN: US03420C1099) and a leading supplier of high-performance/low-power 32/64-bit embedded processor IP solutions. Its V5 RISC-V CPU families range from tiny 32-bit cores to advanced 64-bit Out-of-Order processors with DSP, FPU, Vector, Linux, superscalar, automotive and/or multi/many-core capabilities. By the end of 2023, the cumulative volume of Andes-Embedded™ SoCs has surpassed 14 billion. For more information, please visit https://www.andestech.com . Follow Andes on LinkedIn, Twitter, Bilibili and YouTube!

About Resiltech
ResilTech is a company providing state-of-the-art solutions and services in safety and security with its 15+ years of experience gained supporting customers operating in Critical Systems. In addition, the company integrates industrial expertise with research and development skills developed while constantly joining, since its foundation, national and international R&D projects. The company is a worldwide leading provider of Software Test Libraries (STLs) for a variety of processing nodes and it is now positioning itself as the reference provider of STLs for the RISC-V ecosystem. For more information, please visit http://www.resiltech.com.

Continue ReadingResiltech and Andes Technology Announce Collaboration to Deliver Advanced STL Solutions for Andes Automotive-Grade RISC-V Processor IP