For developers building AI models, one of the key challenges is optimizing, maintaining, and further improving them over time. In some cases, it might make sense to partner with an AI model optimization specialist to get the job done. As a basic definition, AI models are mathematical algorithms trained using data and human input to replicate a decision an expert would make.
AI Model Optimization Platforms
Nota AI, based in Daejeon, South Korea, with offices opening soon in California, was at the 2023 Embedded Vision Summit to showcase their hardware-aware AI model optimization platform, which makes it possible for developers to build and deploy lightweight AI models on Rz/V2M/ and RZ/V2L microprocessors, delivering, they claim, 10x faster AI inference at lower power consumption than traditional approaches. The company was founded in 2015 and has been making a name for itself in driver monitoring and intelligent transport.
Nota provides professional services to AI solutions companies and hopes to offer a subscription service in the future. They also license their software development kits (SDKs) for completely packaged driver monitoring system (DMS) solutions. Their core solution is known as NetsPresso, which is a hardware-aware AI model optimization platform.
“Using our NetsPresso pipeline,” Won Eui Hong, a Nota research engineer, said, “it’s possible to build, optimize, and deploy AI models faster – in some cases in three or fewer days – than traditional scenarios that typically take six or more weeks. As models are updated, Nota works with clients to optimize them over time.”
AI Model Optimization In Action
A handful of Nota partners offered joint demonstrations during the summit, including Renesas for an AI accelerator, Sony for an AI sensing platform, Arm for a virtual hardware platform, Nextchip for a driver monitoring system, and AimFuture for an AI compression model. Nota has also partnered with AI accelerator chip manufacturer Nvidia.
Hong was joined by YooChan Kim, Product Marketing Manager at Nota America, for a session called Maximizing Efficiency of Edge AI Models with Minimum Effort. The presentation focused on collaborative problem-solving for AI model optimization, offering attendees a hands-on opportunity to understand the intricate aspects of hardware-aware AI model optimization.
“AI model optimization is improving the performance of machine learning models to maximize the efficiency of AI models for certain hardware,” Kim said during the session. “Traditional AI models are typically large, slow models that require high-cost cloud servers and an internet connection.” As described by Kim, this frees up space on the hardware, allowing developers to do other things, like use multiple AI models to address more complex tasks.
Hong explained how an in-cabin AI model could be optimized to faster process driver conditions and behaviors like drowsiness and distraction, improving the ability of the system to alert the driver and prevent an accident. According to the Centers for Disease Control and Prevention (CDC), an estimated one in 25 adults reported having fallen asleep while driving in the past month. In that same CDC survey, adults who snore or sleep six or less hours were more likely to report falling asleep while driving.
The National Safety Council notes that crash avoidance technologies, like in-vehicle drowsiness alerts, can reduce instances of drowsy driving. Based on my observations at the 2023 Embedded Vision Summit, a more responsive in-cabin monitoring system – by way of AI model optimization in this case – could help warn drivers before a brief but dangerous case of microsleep sets in.
NetsPresso Gains Traction
Following Hong and Kim’s presentation, Renesas, during a demonstration at their display, said Nota’s NetsPresso improved the inference speed of its latest AI models by 10x. Arm, at their booth, demonstrated an optimized AI model built for Arm Virtual Hardware with Nota’s NetsPresso. They explained how NetsPresso makes it possible for developers to build, optimize, and deploy lightweight AI models on the Arm Corstone-300/310 with Arm Virtual Hardware for faster AI inference with a lower power consumption. Meanwhile, Sony demonstrated a real-time traffic counting application, a joint solution from Nota and Sony.
“Traditionally, for traffic counting, you need to have a separate edge processing unit (i.e., a GPU) connected to the camera,” said Yu Kitamura, Business Development Manager for Sony Semiconductor Solutions. “These solutions are challenging to deploy and are not scalable, so many of the projects are ended with only a proof of concept.”
Sony Semiconductor Solutions and Nota demonstrated Sony’s new AITRIOS-enabled camera, featuring an AI processor-embedded image sensor. Users only need to install one camera to capture image data and analyze it, Kitamura explained. The AITRIOS cloud platform manages devices and AI model life cycles remotely in a centralized location. Sony said installation is more straightforward, maintenance is easier, and the cost is lower. In addition, AITRIOS provides privacy protection by capturing images and processing them at the edge, ensuring images or Personal Identifiable Information is not transmitted.
ADAS Applications
Nota AI’s partner Nextchip, who exhibited at the 2023 Embedded Vision Summit, provides vision processing solutions for autonomous driving and advanced driver assistance systems. They also offer ASICS and are preparing ISP IP for embedded systems, targeting sensors that, in the future, will replace functions traditionally associated with vision SoCs. They are collaborating with Nota to run Nota’s software on their APACHE5 NPU edge processor, an ADAS SoC, for DMS. Together they are targeting camera solutions for in-cabin monitoring.
“The APACHE5 processor is an uncommon concept in the automotive industry, the current trend is having a central processor,” said Julie Kim, head of the global business development group at NextChip. “In contrast, Nota’s DMS algorithm is compact, and pairing that with the APACHE5 9×9 mm SoC, which is light and low cost, enables affordable DMS solutions.”
Optimizing AI models is not for the faint of heart. It’s a complex task that comes with constant maintenance. For those who envision this process being done by a third party, Nota’s offerings are a compelling option.
