Joinwin Electronics HK Limited518031About Join WinJoinwin Electronics HK Limited FLAT/RM 29, 22/F, YAN’S TOWER, 25-27 WONG CHUK HANG ROAD,ABERDEEN HONGKONG Shenzhen ZhongYiYingTong Technology Limited Room 1610, Block A, Overseas Decorative Building, Zhenhua Road,Huaqiang North Street, Shenzhen, 518031Join-Win will be your one-stop purchasing assistant. Join together, achieve win-win!Joinwin Electronics HK Limited518031About Join WinJoinwin Electronics HK Limited FLAT/RM 29, 22/F, YAN’S TOWER, 25-27 WONG CHUK HANG ROAD,ABERDEEN HONGKONG Shenzhen ZhongYiYingTong Technology Limited Room 1610, Block A, Overseas Decorative Building, Zhenhua Road,Huaqiang North Street, Shenzhen, 518031Join-Win will be your one-stop purchasing assistant. Join together, achieve win-win!Joinwin Electronics HK LimitedJoinwin Electronics HK LimitedJoinwin Electronics HK LimitedJoinwin Electronics HK LimitedJoinwin Electronics HK LimitedJoinwin Electronics HK LimitedJoinwin Electronics HK LimitedJoinwin Electronics HK LimitedJoinwin Electronics HK LimitedJoinwin Electronics HK LimitedJoinwin Electronics HK LimitedJoinwin Electronics

Home
News
Arm's new NPU quadruples performance, and the Transformer floor edge is again strongly supported

Arm's new NPU quadruples performance, and the Transformer floor edge is again strongly supported

April 17 , 2024 0

The rise of the Internet of Things has greatly boosted the popularity of edge computing devices, and with the development of machine learning (ML) and artificial intelligence (AI) technologies, these systems are not only equipped with greater processing power and connectivity, but also capable of self-learning and adaptation. In recent years, the development of Transformer and large models has made qualitative breakthroughs in the universality, multi-modal support, and model fine-tuning efficiency of AI models, coupled with low-power AI accelerators and dedicated chips being integrated into terminal devices, and edge intelligence is becoming more and more autonomous and powerful.
In order to cater to the above trends and address the "slow product design", "inefficient software development" and "lack of scale" that are common in the iot space, As early as October 2021, Arm launched the Arm Total Solutions for IoT, which consists of Arm Corstone, Arm Virtual Hardware and Project Centauri. The goal is to achieve "co-design of software and hardware at the system level".
Its first solution combines the Arm Corstone-300 subsystem, the Cortex® -M55 processor, and the Arm Ethos™-U55 neural network processor for machine learning-based keyword recognition in general purpose computing and machine learning applications. Half a year later, in April 2022, Arm successively launched a comprehensive solution for cloud-native edge devices based on Corstone-1000, and a comprehensive solution for speech recognition based on the Corstone-310 subsystem. In 2023, Arm integrated the existing subsystem IP with third-party IP for the first time, and was pre-integrated and pre-verified by Arm Technology to launch ARM intelligent visual reference design for the Chinese market.
Ma Jian, vice president of business development of Arm's Internet of Things Division, pointed out in an interview with the media recently that the more powerful the system, the higher the complexity, and the software and hardware must work together to release the maximum potential of AI processing. Moreover, the landing of the large model on the edge AI side is not so simple as imagined, and the maturity of the model, the development of the edge AI platform, the entire ecological chain, and the tool chain support for the quantification of the large model and deployment at the edge are the main challenges it faces.
To this end, Arm today announced the introduction of Arm's highest performance and most energy efficient neural network processor (NPU), EthO ™-U85, and a new iot reference design platform, Arm Corstone™-320, to accelerate the deployment of voice, audio and vision systems. This also expands Arm's portfolio of comprehensive iot solutions.

Ethos™-U85:4x Performance Improvement With Full Support For Transformer Architecture

Unlike Ethos-U55, which is deployed in Cortex-M based heterogeneous systems, and Ethos-U65, which extends the Ethos-U family to Cortex-A based systems, the new Ethos-U85 delivers four times better performance and 20 percent better energy efficiency. At the same time, its MAC units can be expanded from 128 to 2048 (4TOPs at 1GHz), which can provide strong support for higher performance applications such as factory automation, commercial or smart home cameras.

Native hardware support for the Transformer architecture and DeeplabV3 semantic segmentation network is one of the biggest highlights of Ethos-U85. The Transformer architecture, introduced in 2017, revolutionized generative AI and is becoming the architecture of choice for many new neural networks. Models based on the Transformer architecture can leverage attention mechanisms to process sequence data and perform well in AI tasks such as machine translation, natural language understanding, speech recognition, segmentation, and image captioning generation. These models can be adjusted and compressed to run efficiently on edge devices without compromising accuracy and to take the lead in many edge-side and end-side use cases.

Ethos-U85 also supports element-level operator chain. Combining element-level operations with previous operations by linking, SRAM does not have to write and then read the intermediate tensor. This improves the efficiency of the NPU by reducing the amount of data transferred between the NPU and the memory. Compared to Ethos-U65, chain is one of Ethos-U85's new efficiency improvements, along with a fast weight encoder, optimized MAC array energy efficiency, and improved element efficiency.

To ensure partners can leverage their existing investments for a seamless developer experience, Ethos-/U55/U65/U85 all offer a unified tool chain that simplifies development and supports common ML neural network operations, including convolutional neural networks (CNNS) and recurrent neural networks (RNNS).

Corstone-320: Accelerate The Deployment OF Voice, Audio And Visual Iot Systems

At the heart of Arm's comprehensive iot solution, Arm Corstone is a proven and pre-integrated IP subsystem that allows developers to focus on what really matters: innovation and differentiation across applications and devices.

The latest Corstone-320 integrates Arm's highest performing Cortex-M cpus - Cortex-M85, Mali™-C55 ISP, and the new Ethos-U85 NPU to deliver the performance needed for a wide range of edge AI applications, including voice, audio, and vision. Examples include real-time image classification and object recognition, or enabling voice assistants with natural language translation capabilities on smart speakers.

The Arm Cortex-M85, which uses the Arm8.1-M architecture, is the most powerful Cortex-M processor to date. Compared to the Cortex-M7, its scalar performance is improved by 30%; Performance is up to 85% compared to the Cortex-M55, and the Arm Helium Vector scaling technology supports performance gains for terminal ML and DSP workloads. It is particularly noteworthy that it is equipped with Pointer Authentication and Branch Target Identification (PACBTI) functions, with enhanced software attack threat mitigation capabilities. PSA Certified Level 2 certification for a secure baseline for iot deployments.

The Mali-C55 integrates high-resolution image processing capabilities (up to 48 megapixels in image resolution), energy efficiency, configurability and unmatched image quality for a wide range of iot vision applications. At the same time, the highly configurable direct memory access controller DMA-350 enables efficient data movement for improved system performance and energy efficiency, and supports Arm TrustZone technology.

Consistent with previous iot reference designs, the Corstone-320 software suite includes firmware, drivers for all IP, middleware, real-time operating system (RTOS) and cloud integration, ML models, and reference applications. This means that software developers are able to easily select the components needed for their particular market segment and build an iot stack for that device using the chosen development tools.

At the same time, the prototype platform included in the Corstone-320 enables software development to start in parallel with SoC design, the open source application demonstrates keyword recognition, speech recognition and object recognition use cases, and the fixed virtual platform (FVP) is used to model the peripherals that make up the complete FPGA system. By using FVP, software developers can start developing applications without hardware, thus speeding up development.

The new reference design was developed for real-world workloads, and reference use cases include battery-powered camera systems deployed in smart homes and low-frame rate webcams in industrial and retail systems, Ma said. At the same time, the Corstone-320 reference design provides a secure combination of hardware and software, enabling partners developing on the basis of this reference design to successfully achieve PSA Certified Level 2 certification and achieve compliance with regional and global security standards. With the Corstone-320's pre-integrated, pre-validated reference design template, Arm hopes to help partners reduce the cost and time of edge smart chip development.

Build AN AI Software Ecosystem Based On Arm Platform

In Ma Jian's view, in the future, AI models are like a "brain" that can integrate various types of sensors, cameras, as well as external weather, consumer preferences, natural language commands and other inputs to create personalized application scenarios more safely and energy-saving.

For example, through personalized shopping experiences, intelligent inventory management, dynamic pricing strategies, seamless online and offline integration, and automated operations, AI and big models will make retail more intelligent, personalized, and automated, resulting in greater efficiency and superior customer experiences. In the industrial field, AI and large models are also expected to promote the transformation of industry from 4.0 to 5.0, realizing intelligent production lines, accurate quality control, personalized customized production, supply chain optimization, self-maintenance and remote monitoring, man-machine collaboration, energy saving and emission reduction, as well as the development of new materials and processes, etc., bringing a profound change to the manufacturing industry.

But edge AI brings challenges as well as opportunities. When designing edge AI chips and systems, not only do we need to find the right balance between computing power, energy efficiency and cost, but we also need excellent encryption and security features. At the same time, in order to better unify the diversified application requirements and achieve scale benefits, software definition and standards suitable for software transplantation are indispensable.

In addition, as large models continue to shrink optimization models through quantization, pruning, and clustering techniques to make them more suitable for deployment on edge and hyperterminal devices, the combination of large and small models at the cloud edge is becoming an important development trend for future AI products. How to face the performance and efficiency limits from the combination of iot and large model, multi-modal AI? How to create a consistent toolchain and development platform? How to make the upstream and downstream chip and system suppliers, algorithm software developers and integrators of the Internet of Things ecological chain more and more converge on the Arm computing platform? Is the focus that Arm is thinking and laying out.

"Only the Arm computing platform can deliver the features and capabilities required for cloud-to-end AI, modern agile development and deployment processes, a consistent architecture based on mass production validation, and AI transformation with a unified tool chain." Ma Jian stressed that from Arm's point of view, as long as you do three things: focus on building the best products, make products more easy to use, and continue to build a strong ecosystem, the entire industry will be able to cooperate with each other to move forward together, driven by this force, Arm will achieve greater success.

Today, the considerable AI research and development community not only continues to benefit from the rich information and knowledge provided by the Arm ecosystem and partners, but also builds a growing ecosystem of software and tools around the Arm computing platform, as well as open source software libraries and AI frameworks. The PyTorch Foundation, for example, invests in edge AI and publishes the ExecuTorch inference toolkit for mobile and edge devices, which provides a lightweight Runtime and operator registry covering various models in the PyTorch ecosystem.

In addition, due to Arm's unique IP licensing model and open ecology, Oems and ODMs can have a variety of chips and modules based on Arm architecture and computing platforms to choose from, and more flexible development of system solutions suitable for final applications.