Alibaba Cloud has integrated its large multimodal model (LMM) solution with the automotive platform by Nvidia to enhance smart mobility experiences for China-based automakers.
The collaboration, which also involves Alibaba’s intelligent cockpit solution provider Banma, aims to revolutionise in-car interactions and autonomous driving capabilities.
The collaboration has been followed by the unveiling of a large multimodal model (LMM) solution by Alibaba Group Holding’s cloud computing services unit. The LLM solution is tailored for automotive applications that Alibaba Cloud co-developed with Nvidia and start-up Banma Network Technology, Alibaba’s intelligent cockpit solution provider.
The partnership also sees Alibaba Cloud’s proprietary large language model Qwen, including Qwen2-7B and Qwen2-VL vision language model, incorporated into the NVIDIA DRIVE AGX Orin platform for autonomous vehicles. This integration, coupled with NVIDIA’s model acceleration technology, promises to reduce computational costs and minimise latency in the real-time processing of complex tasks.
This marks the first integration of Alibaba Cloud’s large AI models into the US-based semiconductor powerhouse’s Drive automotive platform. Primary Chinese electric vehicle(EV) makers—including Li Auto, Great Wall Motors, Geely Auto’s Zeekr Intelligent Technology, and Xiaomi’s EV unit—use this platform to power their next-generation fleets.
Leveraging Qwen’s advanced capabilities allows in-car voice assistance to engage in dynamic, multi-turn conversations and offer recommendations. These range from providing information about nearby landmarks to suggesting when to turn on headlights in adverse weather conditions.
The LMM solution also includes Alibaba Cloud’s Mobile Agent, a multimodal AI agent for edge devices. This technology is set to expand the capabilities of in-car voice assistants, allowing users to execute complex voice commands, such as ordering food through delivery apps, while on the move.
NVIDIA’s involvement brings substantial benefits to the partnership. The company’s expertise in AI and graphics processing has been instrumental in optimising Alibaba Cloud’s models’ performance on automotive-grade hardware. This optimisation ensures that the AI-driven features can operate efficiently within the constraints of in-vehicle computing systems.
Overall, the collaboration between Alibaba Cloud and Nvidia is expected to have implications for the broader automotive industry in China. Making this solution available to China-based automakers, Alibaba Cloud and NVIDIA are potentially accelerating the adoption of AI-driven technologies in vehicles across one of the world’s largest automotive markets.
Looking ahead, Alibaba Cloud has plans to extend its collaboration with NVIDIA. The companies aim to adapt Qwen models for NVIDIA DRIVE Thor, NVIDIA’s next-generation centralised car computing platform. This platform combines advanced driver assistance, autonomous driving, and AI cockpit capabilities into a secure system.
Separately, Nvidia is also working with Alibaba Cloud on infrastructure “to meet people’s extreme demand for computing power”, according to Nvidia global VP He Tao, at a sub-forum hosted by the US company at the Apsara Conference.
According to Nvidia, Nvidia and Alibaba Cloud are pursuing a number of LLM-related initiatives for accelerated computing. These initiatives use parallel processing to speed up work and increase energy efficiency across demanding applications. Both companies are also working to create customised solutions to help “traditional enterprises” move their LLM operations onto the cloud.
Explore other upcoming enterprise technology events and webinars powered by TechForge here.