The BMW Group is launching a combined voice command technology, gesture control and gaze recognition offering connectivity that can let drivers control their sun roof, ask about their location and even book restaurants.
The AI-based technology called BMW Natural Interaction was revealed at the Mobile World Congress in Barcelona and will be available in the BMW iNEXT in 2021, according to the company.
The firm says the system allows the driver to use their voice, gestures and gaze at the same time and in various combinations to interact with their vehicle. The driver can select their preferred mode of operation intuitively, according to the situation and context.
The carmaker says the multimodal interaction is made possible by speech recognition, optimised sensor technology and context-sensitive analysis of gestures.
It also involves precise detection of hand and finger movements, gesture direction – in addition to gesture type.
Spoken instructions are registered and processed using ‘Natural Language Understanding’ technology.
The driver decides how they want to interact, based on their own personal preferences, habits or the current situation.
The firm suggests that when the driver is engaged in conversation, they would probably choose gesture and gaze control; when their eyes are on the road, it is better to rely on speech and gestures.
In this way, the car windows or the sunroof can be opened or closed, air vents adjusted or a selection made on the Control Display. If the driver wants to learn more about vehicle functions, they can also point to buttons and ask what they do.
The firm also says that occupants will be able to interact with their direct surroundings, such as buildings or parking spaces.
Even complex queries can be answered quickly and easily by pointing a finger and issuing a voice command.
It gives the examples of what a driver may ask including
“What’s this building?”
“How long is that business open?”
What is this restaurant called?”
“Can I park here and what does it cost?”
Christoph Grote, senior vice president, BMW Group Electronics says: “Customers should be able to communicate with their intelligent connected vehicle in a totally natural way. People shouldn’t have to think about which operating strategy to use to get what they want.
“They should always be able to decide freely – and the car should still understand them. BMW Natural Interaction is also an important step for the future of autonomous vehicles, when interior concepts will no longer be geared solely towards the driver’s position and occupants will have more freedom.”
The firm says that thanks to the vehicle’s depth of connectivity, extensive environmental data and artificial intelligence enable BMW Natural Interaction to transform the vehicle into a well-informed, helpful passenger as it explains in a corporate blog site.
It adds: “By connecting digital services, it will be possible to expand the scope of interaction in the future. For example, when the driver spots a parking space, they will easily be able to find out whether they are allowed to park there and what it costs, and then reserve and pay for it directly without ever pushing a button.”