Arm unveils image processor for driver assistance and automation

Arm has introduced a design for an automotive image signal processor to enhance driver assistance and automation technologies.

The Arm Mali-C78AE image signal processor (ISP) is part of Arm’s AE line of safety-capable intellectual property suitable for advanced drivers assistance systems (ADAS) and human vision applications. It’s the first product announcement since Nvidia called off the $80 billion acquisition of Arm last week.

The first licensee for the tech is Intel’s Mobileye, which is licenses the Mali-C78AE and the next-generation EyeQ technology.

ADAS tech has grown from a premium vehicle feature to a capability consumers now expect as standard in new vehicles, as the systems have helped with driver safety. In parallel, the global chip shortage is making it clear to the automotive industry the criticality of silicon and electronics to the development and competitive positioning of its products.

Drivers increasingly depend on ADAS applications such as collision avoidance, lane departure warnings and automated emergency braking, and vehicles increasingly rely on cameras positioned around the car to enable many of these features.

According to a recent report from Strategy Analytics, the value of the automotive camera market is expected to grow to greater than 19% from 2020 to 2025, making it the most important sensor type in providing data needed for the vehicle to make decisions about its surroundings.

As the number and sophistication of vehicle cameras increases, so does the compute power needed to translate the high throughput of image data – efficiently and safely – into outputs that meet the varying requirements for machine and human vision, Arm said.

To enable new capabilities in ADAS and autonomous driving, the industry will need a new approach to image processing, and to address this, Arm added the Mali-C78AE ISP to its portfolio of IP specifically developed to meet the performance and safety needs of automotive applications.

ADAS features use multiple cameras to enable a variety of human and machine vision applications. For example, surround view systems use data from cameras around the vehicle to visually display to the driver information to help them make decisions while parking.

Adaptive cruise control, on the other hand, directly uses camera data to interpret the environment and make decisions independent of the driver about vehicle control, such as applying the throttle or brake. Mali-C78AE is designed specifically to address both human and machine vision safety applications, and is able to process data from up to four
real-time or 16 virtual cameras.

“We know safety is paramount in ADAS, and I have spoken about this previously – a fault or failure in operation of an ADAS system could be dangerous, threatening the wellbeing of the driver, passengers, and other road users,” said Chet Babla, vice president of auto at Arm, in a blog post. “Mali-C78AE was developed from the ground up with hardware safety mechanisms and diagnostic software features enabling system designers to meet ISO 26262 ASIL B functional safety requirements.”

He said Mali-C78AE aims to prevent or detect faults in a single camera frame that may result in incorrectly processed frame data. To do this, the ISP features over 380 fault detection circuits, continuous built in self-test, and can detect sensor and hardware faults of connected cameras.

It should take 150 milliseconds to acquire an image at the sensor, process it through the ISP then GPU, and display it on a screen for the driver; anything longer is noticeable to the driver when using parking assist, for example. In a machine vision application, a vehicle should not travel more than 250 millimeters between a camera image being acquired and it being presented to the decision-making processing and anything longer means the machine vision system is too slow to react in driving situations where accurate and timely decisions are critical.

To enable drivers and machines to make the best-possible decision, ADAS cameras must collect the most relevant information possible from each frame. Mali-C78AE employs advanced noise reduction technology and dynamic range management to ensure each frame is clear and properly exposed by adjusting overly dark or bright areas of a frame.

Mali-C78AE is able to perform real-time processing of camera data from up to four high-resolution-high-frame rate cameras, significantly reducing the memory, communications, and processing requirements, making for a more efficient system.

To reduce the cost of implementing multiple ADAS functions, Mali-C78AE enables camera sensors to be dual-purpose by downscaling and color-translating the outputs of sensors optimized for machine vision to create images adapted to the human eye.

By avoiding duplication in cameras and their associated electronics and wiring, car makers save on cost and complexity and therefore, enable wider deployment of camera-based ADAS features across a diverse range of car models providing a safer, better user experience for drivers.