On Monday, Google Cloud introduced its newest Automotive AI Agent, selecting the Mercedes-Benz CLA to feature it first by the year’s end. This Agent is designed to enhance the MBUX Virtual Assistant of Mercedes, expanding the communication options available to passengers.

Our initial encounter with Mercedes’ future version of the virtual assistant occurred one year prior, at CES 2024. However, the firm did not specify at that moment which substantial language model was the basis of its functionalities. Unlike the ongoing MBUX capable of responding to approximately twenty-four vehicle instructions and providing details from both ChatGPT and Bing, this updated assistant offers more. Although the existing assistant is triggered by saying “Hey, Mercedes,” it mirrors performance akin to Siri or Google Assistant, giving static replies instead of a fluid conversational exchange like ChatGPT’s Advanced Voice Mode.

The foundation of Google’s Agent lies within the Gemini LLM and incorporates Vertex AI, intended “for automakers to develop highly customized and intuitive in-car agents surpassing current vehicle voice controls,” as detailed in their announcement. The Agent can manage both multimodal and multilingual inputs and can even tackle follow-up inquiries. For instance, the AI could inform drivers about Italian eateries nearby, provide reviews, and reveal the most popular dish. The architecture is reportedly capable of maintaining multi-turn dialogues and recalling details from past discussions.

The upgraded MBUX assistant is expected to use timely and verified data from Google Maps in almost real-time, offering detailed, personalized insights on over 250 million global points of interest and prevailing traffic conditions. “Offering our customers a standout digital experience is a priority at Mercedes-Benz,” remarked Ola Källenius, CEO of Mercedes-Benz Group AG. “This collaboration with Google Cloud will enhance in-car navigation, fusing advanced location data with generative AI,” he elaborated.

Looking ahead, Mercedes aims to integrate this latest MBUX assistant into further models down the line. However, the specific models that will incorporate this feature have not been disclosed yet.