At Qualcomm Technologies‘ Snapdragon Summit 2024 in Maui, HI, last month, the big automotive reveals were the Cockpit and Ride Elite platforms (see Futurride coverage here), but there were many other interesting stories Futurride followed during the three-day event including the company’s work with innovative customers. One of those stories was highlighted in a “fireside chat” with Rivian and Epic Games execs that revealed more details on a high-profile collaboration for the in-vehicle experience and how it is evolving with the latest technology.

Amanda Belcastro, Senior Director of Finance at Qualcomm, led the discussion with Wassym Bensaid, Rivian’s Chief Software Officer, and Bill Clifford, Vice President and General Manager for Unreal Engine at Epic Games on the history and tech behind the most extensive update to the Rivian UI (user interface) since its launch, which was revealed earlier this year on the updated all-electric R1S SUV and R1T pickup.

With the updated R1 (see Futurride coverage here), Rivian has expanded its use of Unreal Engine for the driver’s display and full infotainment center backed up by Qualcomm’s Snapdragon Cockpit system-on-chip.

“The new technology platform gives us the foundations to take our refined and connected vehicle experience to the next level with our in-house Rivian software stack,” said Bensaid.

 

Bespoke experiences

Connectivity is a key element of Rivian’s E/E architecture, and it goes both ways with paths from vehicle to cloud and cloud to vehicle—the latter enabling over-the-air updates.

“Every single computer in Rivian vehicles can be updated through over-the-air updates, and that provides a very different ownership experience for our customers,” said Bensaid. “The vehicle gets better and better over time. Our customers are really excited to see new features.”

Unreal sits at the heart of the graphics and rich 3D experience that enable not only updates but the ability to customize Rivian models.

“There are so many configurations for the vehicles,” continued Bensaid. “Last we counted, there’s more than 40,000. With Unreal, we can have a very accurate and personalized representation of your vehicle on the screen. That provides really a unique and bespoke experience for our customers. We can also have lots of playful moments where there are live interactions and subtle animations, which really increase that engagement [for] our customers.”

The new Rivian experience comes in nine different modes, from the mountains to the desert—and even features an updated camp mode complete with the Rivian tent and softly flickering campfire.

“Rivian has been able to push entirely new artistic boundaries with Unreal Engine to redesign this user experience in a bespoke illustration style,” explained Jeff Hammoud, Chief Design Officer at Rivian. “The level of craftsmanship for the look and feel, running in real-time in the vehicle, was in large part thanks to the tools and performance targets that only Unreal Engine can achieve today.”

 

Inspired by visual storytelling

Inspired by visual storytelling that represents the best of what’s possible in the entertainment world, Rivian designers have been using Unreal Engine for human-machine interface displays since their first vehicle, the R1T, was released in October 2021. They say that the latest update delivers a first-of-its-kind look with scenes that are alive, enabling drivers to interact with them via the touchscreen interface.

“Every physical part of the vehicle must be represented in the correct way, from the tires to wheels, cladding, paint, windows, and trim,” said Andreas Poser, Senior Manager of 3D Visualization at Rivian. “Lighting became a unique challenge, as it’s not lit in the traditional sense, but we still need to produce reflections on the vehicle because after all, it is still covered in paint and glass. We’re using a lot of custom shaders to bring this to life, and we owe the final look to the rapid iteration we were able to achieve with Unreal Engine materials.”

With both design and engineering teams working in Unreal Engine, the new reimagined UI experience was brought to life in just six months. The teams put a lot of effort into the micro-animations in each scene. From fireflies to dust clouds, everything went through several design and optimization cycles.

“Unreal Engine not only helped us create the appearance of the drive modes in a very agile and iterative way, but it also allowed us to make each mode more dynamic,” explained Eddy Reyes, Senior Manager of Embedded Software In-Vehicle Experiences at Rivian.

For example, mode transitions are smooth, and when the mode settles, the vehicle and environment around it remain dynamic, Reyes added. The viewer will see clouds, grass, flowers, and trees moving. They can even touch the vehicle with two fingers and pivot it around, and the environment moves with it in three dimensions.

The new UI design in Unreal Engine opens the opportunity to easily refresh the in-cabin experience and features over time for Rivian owners.

 

Unreal growth

The technology behind Unreal Engine, best known for making top video games like Fortnite, has evolved into a tool for many applications that require real-time 3D design visualization or simulation of immersive consumer experiences. Beginning in the 2010s, Clifford said that auto manufacturers began getting interested in that potential. They wanted to build apps for their design center reviews and marketing experiences for consumers, needed a real-time rendering engine, and started experimenting with technology. When real-time ray tracing took off, so did Unreal Engine’s momentum in the auto space.

Today, Clifford estimates that 80% of the top 20 OEMs use Unreal Engine. It is being employed by vehicle design centers to create digital twins, by engineering teams to train AI models for autonomous driving using synthetic data and simulated driving experiences, by sales and marketing teams for car configurator apps and digital experiences in the physical world, and increasingly in the cockpit to create a better UX (user experience) through HMIs (human-machine interfaces).

In October, Epic and Qualcomm announced at Unreal Fest the deployment of Unreal Engine in Snapdragon Cockpit to give automakers like Rivian access to pre-integrated and optimized features to deliver immersive user experiences. Through the technology collaboration, automakers who use Snapdragon solutions will have exclusive access to Unreal Engine tools to provide fully customizable 2D and 3D visuals.

Serving as a creative foundation for design, UX/UI, and simulation, Unreal Engine supports unified assets across applications, offering fast and flexible workflows that result in cost savings while enabling real-time collaboration and iteration.

The Snapdragon Cockpit platform enhances user interfaces, connectivity, and in-cabin environments. Leveraging high-performance computing, immersive graphics, and AI, it improves user experiences and drives the shift toward connected and intelligent mobility.

“By seamlessly integrating Unreal Engine with Snapdragon Cockpit Platforms, we’re empowering automakers and the ecosystem to innovate and elevate in-cabin experiences to new heights,” said Laxmi Rayapudi, Vice President of Product Management at Qualcomm.

“Partnering with HMI pioneer Qualcomm Technologies and leveraging the capabilities of Unreal Engine and the Snapdragon Digital Chassis, we’re excited to see how automakers usher in the next frontier in in-vehicle experiences,” said Clifford.

The new solution is already available for developers.

 

Next up: merging navigation, mapping, and ADAS

In closing the fireside chat, Belcastro asked about the most requested features that customers are asking for right now.

“3D mapping is really interesting,” said Clifford. “We previewed at our Unreal Fest event a couple of weeks ago (in Seattle) the ability to take existing mapping data and render it using Unreal in 3D. That’s an exciting breakthrough because, generally, your mapping data comes with an out-of-the-box renderer, but it’s kind of like this creative black hole within the cockpit. Now we’re excited about the opportunity to integrate mapping into the whole experience, make it customizable, personalized, and more playful and useful.”

Bensaid echoed the excitement over mapping more broadly in the context of the Epic partnership.

“The feature that we’re very excited about as we upgrade to the next generation of Unreal Engine is merging together the navigation, mapping, and ADAS (advanced driving assistance system) worlds to provide much higher fidelity in the scene around the driver,” said Bensaid. “Then add to that some of the artistic elements, which I believe would help us on our path to provide really the best driver experience.”

Bensaid and his Rivian crew are also itching to get their hands on the latest Snapdragon Cockpit Elite revealed in Maui.

“We’re super excited to also start working on the next-generation platforms,” he said. “I can’t wait to get the next SoCs with better graphics, better compute, and a lot of AI features to refine the experience for our customers.”

Clifford summed up his excitement about the automotive opportunity.

“This space is so dynamic,” he said, referring to the power of AI, thinking about the car as a supercomputer on wheels, and moving toward fully autonomous driving and software-defined vehicle. “The car becomes the new mancave, where I don’t want to get out of the car. It’s totally tricked out with great screens and impressive audio. We think it really can be a place where people do play games.”