OmniVision Technologies, Inc., Ambarella, Inc., and Smart Eye have announced a new and innovative solution for dual-mode camera applications. As described by the companies, this joint solution monitors drivers while simultaneously capturing vehicle occupants for one-way video conferencing. The announcement comes ahead of AutoSens Detroit 2020, where the companies are expected to give a virtual demonstration on Wednesday, November 18th.
As designed, the solution is expected to open up an array of driver and in-cabin monitoring implementations, from safety applications to deep neural networks and AI. The one-way video conferencing allows remote conference participants to see those in the cabin, while the vehicle receives audio only to reduce driver distraction.
Multifaceted Approach
Each company plays a unique role in what is the industry’s first combined driver monitoring and video conferencing camera solution. It’s important to understand what each company brings to the table in terms of expertise and experience. As a brief overview:
- OmniVision is a leading and respected developer of advanced digital imaging solutions.
- Smart Eye develops AI-powered eye-tracking technology that understands, assists, and predicts human intentions and actions.
- Ambarella products are found in various human and computer vision applications, including video security, robotics, ADAS, and other autonomous driving applications.
How It Works
The combined driver monitoring and video conferencing camera solution employs OmniVision’s OV2312 image sensor, with a dual-mode global shutter that captures both RGB color images and IR images under low-light conditions. The OV2312 is the latest member of OmniVision’s automotive global shutter image sensor family and is pin-to-pin compatible with the OV2310 and OV2311, according to the company. (Earlier in 2020, OmniVision also announced the OX03C10 ASIL-C automotive image sensor and the OVM9284 CameraCubeChip module).
“Our OV2312 is the only automotive image sensor that offers the combination of dual-mode RGB-IR capture and a global shutter,” said Mario Heid, Vice President of OmniVision Europe. “Without this combined functionality, the camera system designers would have to use two separate image sensors, which is too expensive for most vehicles.”
The dual captures performed by the OV2312 image sensor are then processed simultaneously by Ambarella’s CV22AQ CVflow computer vision processor, which runs Smart Eye’s algorithms to analyze the driver’s state. If necessary, an alert can be issued if a driver is drowsy or not fully engaged in the task of driving.
“Our AI software makes it possible for automotive OEMs and Tier 1s to deploy a new generation of driver and in-cabin monitoring systems with advanced AI features, increasing safety and convenience for drivers and passengers,” explained Martin Krantz, CEO of Smart Eye. “This dual-mode joint solution has the unique ability to monitor the driver while simultaneously capturing images for video conferencing, regardless of lighting conditions.”
“Our CV22AQ offers best-in-class image processing and high performance AI computing at low power consumption, typically below 2.5 watts,” said Fermi Wang, President and CEO of Ambarella. “While we have worked together to pre-tune this joint solution, designers still have the flexibility to further customize it to the needs of their specific implementations.”
Reference Design
The solution is available as a complete reference design. According to the companies, this will allow automotive designers to focus on differentiating their final applications while simplifying the overall design effort. All three companies are in discussions with automakers regarding the implementation of this reference design for future vehicles.
“At the same time, the European Union, through Euro NCAP, is requiring that all new cars sold in Europe have a driver monitoring system camera by 2022,” Heid added. “Our joint solution gives OEMs the flexibility to differentiate their vehicle features while meeting this mandate.”
