NVIDIA has long been a real “engine of progress” in the development of graphics solutions. Its graphics cards are still among the best solutions on the market. But to infinity “to increase gigabytes of video memory” it won’t turn out and the industry needs fresh ideas. And this is what NVIDIA is ready to give the world with its new technology, thanks to which the processing of 3D-graphics can be carried out not by a graphics processor, but by an advanced artificial intelligence system.
According to the press service of the company, NVIDIA has created an AI algorithm that creates virtual objects based on real videos, but unlike most similar solutions, it does not work with individual frames, but with the entire video stream at once. According to the company,
“AI will allow developers and artists to create new interactive three-dimensional virtual worlds for the automotive, gaming and virtual reality projects, simply learning from real-world video. This will reduce the cost and development time. Neural networks, and in particular generative models, will change the way graphics are created.»
To demonstrate the technology, the research team developed a simple racing game on the Unreal Engine 4, which allows players to navigate the interactive environment generated by artificial intelligence. The neural network operates with high-level descriptions of scenes, such as segmentation maps, which describe the characteristics of objects, as well as where they are located. Based on this AI “understands” whether a certain part of the image contains a car or a building, as well as their boundaries and properties. The network then fills the scene with details that it has learned from the source data and a number of real-life videos.
For AI training, developers used NVIDIA Tesla V100 GPUs with the cuDNN deep learning algorithm, as well as several thousand videos of urban landscapes to train the system. In addition to generating virtual worlds, AI can also create and animate 3D models of people.
“The ability to model and recreate the dynamics of our world in virtual space is important. In addition to purely scientific research, training and continuous visual series generation process has a wide range of applications in robotics, computer graphics and so on.”