Cruden - Ready for real

Written by

Cruden

Published on

December 20, 2019

Article 4 of 7 in our Content Series: Cruden’s Dennis Marcus explains the crucial compromise between image detail, resolution and frame rate that sits at the heart of an immersive driving simulator.

Toy Story 4 was one of the movie hits of the summer. The technology behind computer-animated movies (CGI) has evolved dramatically since the first installment of this popular franchise was released back in 1995, but the detail and complexity that’s now possible in digital animation means that it can still take the Pixar team dozens of hours to render a single frame of the movie.

That sort of time is not a luxury we enjoy in the world of driving simulators, where the images must be rendered in real time. Each image must be calculated while the previous image is being displayed. And unlike those Pixar animators, we never know what the next image is going to look like. It depends entirely on what the driver chooses to do on the digital road or track, and on how the vehicle model responds to the commands it is given.

In a driving simulator, fast panning objects are the norm as the scenery flashes by in the driver’s peripheral view. To make that view realistic, it must be rendered at the highest frame rate available from the image generator PCs and that the projectors or display panels can handle.

For best results, providing sharp, fast, moving images, 120 frames per second (fps) is highly desirable. By comparison, most movies are shot at 24 fps. All but the latest high-end televisions have refresh rates of 50-60 fps (even though many televisions perform interpolation to claim higher rates, but the native refresh rate often stays limited to 60Hz).

That’s not enough for a real-time driving simulator, however, which must also do without the smart optimization algorithms available to televisions and in movie theatres. Image enhancement technology, as commonly used in television systems, is not an option in a simulator as it would introduce significant additional latency between input and output – although it may still be possible to maintain a high framerate. This would not be an issue if watching a football match, but it would be a big problem in interactive applications like a driving simulator. Another difficulty arises if algorithms consume more time than available between frames: The rendering engine will struggle to provide 120 fps and will eventually switch to a lower frame rate, where an image that stutters, flickers or breaks up can quickly result in distracting the driver or getting tired or sick in the simulator.

Most of all, lower frame rates are bad for immersion and inherently result higher latency. Drivers should exhibit natural behavior in the driving simulator and for that to happen, the driving experience must be as immersive as possible. With frame rate variation, the driver soon realizes that they’re part of a computer experiment and may change their behavior accordingly. Keeping 120 fps may be possible when reducing the level of detail in the images dynamically to make it less demanding for the computer to render. This can result in grass that’s all one color or trees that are suddenly no longer rendered (or in a more basic form) when looked at from a further distance. A skillful compromise is required to ensure that the available computing power is sufficient to maintain 120 fps but can still render the nicest looking images during the available computing time.

Cruden has built an image generator using the Unity rendering engine with bespoke optimization features to assist in getting the 3D driver simulator content rendering at 120 fps. Unity is renowned for its advanced, cutting edge image generation techniques, like physically based rendering (PBR) for accurate material representation. A future article will cover Cruden’s use of Unity.

Every object in a 3D world consists of polygons, with surfaces to which materials are attached that interact with virtual light sources. The number of polygons – the number of surface sections in an object and the material properties – determines roughly how long it takes for a rendering engine to create an image based on that 3D object. The overall frame rate also depends on image resolution and image quality features, but the graphical detail is determined by the number of polygons used to define the 3D objects.

Similarly to a movie set, geometry not visible to the driver, such as this city view, does not need to be created or rendered. Therefore, fewer polygons are used.[/caption]

If you want to show detail in a non flat object that you are rendering then you need a lot of polygons, but the more you have, the longer it will take to render an image of the 3D world through which you are driving. Cruden’s 3D artists in combination with Cruden’s image generator ensure that the right level of detail goes to the right place in the image.

To achieve that, 3D models of objects shown on-screen are designed in several different levels of detail. A low-resolution model of a traffic car, for example, will be used when the simulator driver is approaching it from afar. A low-resolution traffic car with fewer polygons is much easier to render than a high-resolution version that from far away, looks no different on screen compared to a high-resolution version. As the driver approaches the traffic car, more detail is required and so a different level of detail version of the same object is used.

In addition, Cruden reuses many objects multiple times within the same digital environment. The traffic lights within a city probably all look the same, so a single 3D object can be reused to help to keep the 3D graphics files small. That’s important to ensure that the 3D world can be loaded faster onto the graphics card in the computer. All of the 3D geometry visible from the virtual eye point must be loaded onto the graphics card so that the rendering can be done in real time. That can be a challenge, especially for the large 3D worlds used in automotive engineering, which might incorporate 20-30km of road rather than simply 5km of racetrack.

Different Level Of Detail (LOD) stages can be employed as needed. For this wheel example, LOD 0 is for distances up to 20 m; LOD 1 is for distances up to 80 m; and LOD 2 is for distances over 80 m.[/caption]

Variation in the on-screen objects is important for immersion, however. Driving through a forest will be unrealistic if all the trees are of the same shape and size. Cruden’s 3D artists instead provide enough variation in the designs of the trees to keep it interesting for the driver and maintain the illusion of natural vegetation. But at the same time, they minimize the number of different objects in the 3D scenery so that the resulting, smaller and faster file can be loaded onto the graphics card.

We can’t promise that our content is as entertaining as the latest Pixar masterpiece, but Cruden’s 3D worlds can be guaranteed to provide smooth, detailed images that fully immerse the driver in the simulated environment. In the next article, we’ll explore how to connect the driving simulator to other engineering tools and sprinkle some of Cruden’s own movie magic on the 3D environments that come with them. Until next time!

To speak to an expert about resolution & frame rate for your driving simulation, please contact Dennis Marcus via d.marcus@cruden.com or on +31 20 707 4646.