Simulate to fabricate: vehicle plants keep VR and AR in their sights

Major OEMs are continuing to employ simulation technology to underpin their manufacturing development work, using both virtual reality (VR) and augmented reality (AR) systems in a number of different contexts. Among them are Audi, BMW, Mercedes-Benz and Volvo…

Audi is one of the early adopters of this technology – one example currently being tested is a VR ‘holodeck’ for assessing the design of new vehicle models. 

As Jens Angerer, VR specialist at the Audi Production Lab, explained, the holodeck – created alongside Stuttgart-based company Lightshape – consists of a very large walkable, virtual environment containing a three-dimensional image of a car, which allows development engineers and production experts to “gain a realistic overall impression of a new model and its proportions at an early stage”. Angerer observed that using this system Audi can reduce the number of complex physical test models, saving both development time and costs. 

“In order to ensure that all Class-A surfaces match our design and quality requirements while being producible, the construction data of the car is repeatedly evaluated during the development process. In the early phase, VR helps us to get a proportional impression of the car with small cost and high disposability,” he said.

“Later in the process, today’s technology still lacks the visual quality for detailed evaluation, but VR allows us to look at the car from a customer view and thus to ensure the customer relevance of our decisions,” he added.

More on Audi digital manufacturing: Exoskeleton trials underway at Ingolstadt

Audi is also using VR to test assembly processes for its e-tron GT model – due to begin rolling off the line alongside the Audi R8 at the Böllinger Höfe this year. As part of a pilot project, logistics planners at the company’s Neckarsulm site are currently testing how special containers can be planned and tested entirely in a virtual space and without any physical prototypes. 

“VR has already proven its potential to reduce the time needed in the real factory for planning and, above all, integrating new production structures”

Matthias Schindler, BMW

According to Angerer, this is achieved by using VR glasses and controllers alongside specialist in-house VR software that relies on 360° scans to provide a three-dimensional indoor map of the virtual space. 

“These containers are used for particularly sensitive parts such as electrics, headlights, or the windshield. They are custom-made for this task and their planning takes a corresponding amount of effort. Developing the special containers with VR is less expensive and is also better for the environment,” he explained.

Although Angerer was also keen to point out that VR is ‘not a new technology for Audi’ – and that the company has used CAVEs in a number of different areas to support engineering, manufacturing and quality assurance of its products over the past two decades or so – he revealed that, in recent years, there had been a shift towards HMDs (head-mounted-displays), which ‘provide more functionality at a fraction of the cost.’ 

“Today, HTC Vive Pro serves as the primary HMD for the VW Group. This ensures compatibility during development and reliability during operation and use. This standard can be enhanced or adapted based on the individual project requirements, “he said.

Virtual assembly station

Mercedes-Benz has applied the technology to create a ‘virtual assembly station.’ As Tobias Brandstetter, spokesperson – Global Business Communications at Mercedes-Benz Cars, explained, the station, which works in a similar way to a video game console with motion control, enables virtual assembly by avatar and “can be used to test subsequent assembly processes”. 

For this process, an employee stands on a digital floor with two digital walls on which he or she sees themselves as an avatar. Using virtual reality glasses, which project what Brandstetter described as a “strikingly realistic version of the vehicle to be produced in front of the employee,” they can test out assembly situations.

“Should a component be better installed from the left or from the right? Is this position ergonomically comfortable for the assembly station staff or too taxing over a long period? Through test runs with the avatar, experienced employees can assess how everyday work processes can be configured more efficiently,” he said.

In such a way, the company is able to use VR to check important assembly steps two years in advance of series production. According to Brandstetter, using the virtual assembly station in combination with the first designs of a vehicle also makes it possible to “simultaneously plan and optimise production efficiently and take ergonomic considerations into account.”

“Before the first hardware is built, necessary adaptations to the planning have already been identified. But it’s not just assembly processes that can be checked beforehand using VR, it also makes it possible to do a virtual check of the entire setup of each assembly station including all shelves and available tools,” he noted.

“We are currently also working on other applications, such as the integration of augmented reality and mixed reality,” he added.

3D digital factory planning

Another early pioneer in the use of VR and AR in the automotive manufacturing environment is BMW. Among other things, the German OEM uses VR technology for virtual planning activities, such as Virtual Cardboard Engineering and factory planning – including layout planning for brownfield and greenfield sites – and 3D digitalisation as a tool for gathering an accurate 3D model of existing factory structures. 

As is the case in other fields of ‘digitalisation,’ Matthias Schindler, Cluster Manager – Virtual Planning & Commissioning at BMW Group, revealed that the company often uses standard hardware and software solutions from well-known suppliers in order to “make sure its applications are cost-efficient, reliable and adaptable for a large number of use-cases.” 

Examples of the technologies currently used include the HTC Vive – which is used alongside high performance computers for VR applications – and the Microsoft HoloLens and ‘other smart glasses’ for AR and logistics applications.

“A big motivation for new digital solutions is to reduce downtime of factories and any planned or unplanned stand-still of our manufacturing resources. VR for example has already proven its potential to reduce the time needed in the real factory for planning and, above all, integrating new production structures,” said Schindler.

“It’s not just assembly processes that can be checked beforehand using VR, it also makes it possible to do a virtual check of the entire setup of each assembly station including all shelves and available tools” 

Tobias Brandstetter, Mercedes-Benz 

A recent illustration of this potential has been via the company’s use of 3D digitalisation using Virtual Cardboard engineering, which Schindler estimates takes 30 minutes, instead of “working three or four days inside the production site and blocking the specific area.” 

“New product features – especially electrical drivetrains and components – also have to be analysed early in the product development process and virtualisation helps to ‘bring’ the factory to designers and production engineers. 3D visualisation also helps to improve the maturity of production concepts before implementing them, as possible errors become obvious in a very early stage,” he observed.

More on BMW digital manufacturing: Quick learners

“An interactive 3D view is always more powerful than 2D layouts since everybody can interact intuitively. This way we can integrate many more user groups in concept discussions and in the planning process – meaning we can include, right from the start, production workers, logistics personnel, maintenance staff, IT colleagues and even suppliers into planning discussions,” he added.

Last year, the company also began to use AR technology for all incoming goods inspection of delivered tools at its Toolmaking and Plant Engineering unit in Munich. 

As part of this inspection process, staff at the unit capture an image of each tool on a standard tripod-mounted tablet – before using an AR application to overlay the image with the related CAD construction data. Technicians then refer to around 50 criteria relating to aspects such as drill holes and other clear surface features, to assess whether all the necessary production specifications have been implemented. 

A key benefit of this approach is that the early detection of minor differences leaves staff with sufficient time to rework the tool on site before it must be sent to the assembly area for completion with other components.

Mixed reality headsets

Elsewhere, Volvo Cars has been using novel mixed-reality headsets – created by Helsinki-based technology company Varjo – in its workflow since 2018. The company uses the XR-1 prototypes to add photorealistic virtual elements to the interior of cars, enabling designers to perform design-studies of future cars before they are even built – and even test-drive new designs on a test track. 

“Their immersive design and user research use cases include testing new Heads Up Display systems as well as infotainment and active safety systems. Also, they’ve been able to extend their use to the visualization of what the car can see around the driver, for example a virtual moose,” explained Marcus Olsson, product manager, Mixed Reality at Varjo.

“Volvo’s designers no longer have to wait for feedback on their ideas until an actual concept interior has been mocked up. Instead, they can get input at a much earlier stage and make any adjustments quickly, and see how a new design looks in different environments – for example, in sunny weather, under the trees or on a cloudy day. They can keep as much as possible of reality – the real road, nature, road signs and more – and only exchange the things they want to evaluate, for example a new display or interior,” he added.

Immersive technologies

In Olsson’s view, a key benefit of this approach is that it allows Volvo Cars to “iterate design and UX concepts quickly and avoid tedious tool and software changes, saving a significant amount of money along the way.” 

“Essentially, the system allows them to do things in a day that would normally take weeks or longer. Especially building a prototype for a Heads-Up Display is extremely costly as it involves advanced projectors and optics. Doing this virtually saves a lot of money,” he noted.

“Moreover, Volvo engineers can now drive a real car while wearing a mixed reality headset, and seamlessly add virtual elements or features that seem real to both the driver and the car’s sensors. This enables Volvo’s safety experts to use the technology to develop and evaluate active safety features. They can drive a real car at a test track while wearing the headset and try out virtual active safety systems and scenarios that are imposed on the real-life environment,” he added.

Olsson revealed that “a number of automotive manufacturers” are currently using the company’s human-eye resolution VR headsets – with close partners including Volkswagen Group, Audi and BMW ‘among others.’ 

He observed that virtual prototyping of both existing and new car features can significantly reduce costs for OEMs, because any changes to physical prototypes, like clay models, are “very expensive and slow down iteration cycles”.