CMOS Image Sensors Provide Visual Perception For Autonomous Vehicles

In applications where measurement accuracy is critical, sensor resolution is a key metric because it determines the number of pixels on the rows and columns of the light-sensitive surface on which the image is captured. The minimum number of pixels required depends on the smallest features in the image that need to be detected. While in theory only 2 pixels per dimension can be used to parse a single feature of an object, the lack of contrast and image noise mean that in practice at least 4 or 5 overlapping pixels are needed to fully parse a feature.
Sensitivity measures how efficiently a sensor converts photons into electrons and is important in almost all applications. It measures the time and illuminance (illumination) required for the sensor to recognize the available image. Sensitivity is also affected by sensor noise, so the signal-to-noise ratio (SNR) is an important indicator. Sensors with a high signal-to-noise ratio provide higher quality images in low light conditions. Sensitivity is particularly important in monitoring and medical applications, which require high-quality images even in low-light conditions. Although the image sensor data book provides sensitivity indicators, it is sometimes difficult to compare the sensitivity of sensors produced by different manufacturers. Autonomous vehicles require a high level of detail even in low light (near darkness) conditions, so the two key metrics of resolution and sensitivity must be carefully considered when selecting an image sensor.

Dynamic range refers to the ratio between a pixel's minimum (noisy) signal and its maximum trap capacity, and it defines how many different brightness levels are present in the image. This is a particularly important feature in applications with extreme light conditions. The dynamic range indicator is based on the European Machine Vision Association (EMVA)1288 standard and is measured in decibels (dB), so it can be easily compared according to most sensor data books.
Frame rate is the speed indicator of the sensor, measured in the number of images (frames) read per second (fps). In applications that capture fast-moving objects, high frame rates are important because short exposure times are needed to prevent blurring and reduce motion artifacts. As the number of sensor pixels increases, the maximum achievable frame rate decreases. For example, higher frame rates can be achieved with a low-quality Video Graphics Array (VGA) image sensor, while the same results cannot be achieved with a full-resolution 20 megapixel sensor. Sensors that support "region of interest" (ROI) can combine high frame rates with high resolution. This type of sensor identifies one or more areas of the image to be processed while ignoring all other areas of the image. This effectively reduces the overall image resolution and thus enables higher frame rates. Like standard cars, autonomous vehicles encounter a variety of lighting conditions at high speeds, so high dynamic range and high frame rates are crucial features when choosing an image sensor.

The two readout methods used by CMOS image sensors are roll-shutter and global shutter. The roller-shutter method reads out the pixels of the sensor line by line as it is exposed, and is therefore a very fast technique. It uses fewer transistors per pixel, is less noisy, more sensitive, and costs less than sensors that use a global shutter. Roller shutter sensors are recommended for applications requiring high dynamic range. In the global shutter readout method, each pixel is exposed at the same time, so there is no capture delay between pixel rows. However, this approach is expensive and difficult to implement.
In applications such as measurement recording and presence detection, it is acceptable to use black and white image sensors, but many applications now require color images. However, black and white sensors also have some advantages. In order for the sensor to provide a color image, the RGB filters need to be arranged in Bayer mode at the pixel level. However, Bayer color interpolation results in lower detail and overall measurement accuracy. Therefore, the use of color sensors is only necessary when the application requires color information. Obviously, self-driving car applications need to capture color images and use roll-shutter sensors.

There is a misconception that the bigger the pixels, the better the image quality. While larger pixels have more area available to collect light, that doesn't mean you get a higher quality image. It is important to note that factors such as resolution and pixel noise indicators also play an important role in determining image quality. Smaller pixels tend to have lower dark signal inhomogeneity (DSNU); At higher temperatures, dark signal inhomogeneity limits low-light performance. In some cases, sensors with smaller pixels will perform better than those with larger pixels. When designing camera systems, a good balance between speed, sensitivity and image quality characteristics must be considered to achieve excellent performance.

The Hyperlux sensor family is the second generation of ON's super exposure pixel technology platform that performs well in automotive imaging applications from 3 megapixels to 8 megapixels and beyond. With a flicker-free (FF) single exposure of more than 120 dB and ultra-high dynamic range (HDR) and LED flicker cancellation (LFM) technology of 150 dB, the sensors provide stable image quality and dynamic range over the temperature range of automotive applications without changing Settings in different lighting conditions. This reduces latency and improves security. This series of 2.1 µm image sensors includes flexible features such as intelligent ROI, binning, windowing and dual output. These functions can simultaneously output flexible data formats such as data with different resolutions (ROI) and different integration times. The sensor is designed for ASIL-D systems and its sophisticated real-time safety mechanisms and fault detection capabilities help enable more advanced advanced driver assistance systems (ADAS).

This paper describes the key characteristics to consider when selecting a CMOS image sensor for autonomous vehicle applications, including resolution, sensitivity, speed, dynamic range, motion, and color. On's Hyperlux image sensors offer an excellent combination of these features, making them ideal for a wide range of automotive applications.