Interested in learning what's next for the gaming industry? Join gaming executives to discuss emerging parts of the industry this October at GamesBeat Summit Next. Learn more.
Nvidia spans both gaming graphics and artificial intelligence, and it is showing that with its announcements this week at the Siggraph computer graphics event in Los Angeles.
Those announcements range from providing external graphics processing for content creators to testing AI robotics technology inside a virtual environment known as the Holodeck, named after the virtual reality simulator in the Star Trek series. In fact, Nvidia’s researchers have created a way for AI to create realistic human facial animations in a fraction of the time it takes human artists to do the same thing.
“We are bringing artificial intelligence to computer graphics,” said Greg Estes, vice president of developer marketing at Nvidia, in an interview with GamesBeat. “It’s bringing things full circle. If you look at our history in graphics, we took that into high-performance computing and took that into a dominant position in deep learning and AI. Now we are closing that loop and bringing AI into graphics.”
“Our strategy is to lead with research and break new ground,” he said. “Then we take that lead in research and take it into software development kits for developers.”
Event
MetaBeat 2022
MetaBeat will bring together metaverse thought leaders to give guidance on how metaverse technology will transform the way all industries communicate and do business on October 3-4 in San Francisco, CA.
Above: Nvidia’s Optix 5.0 can “de-noise” images by removing graininess.
Nvidia has 10 research papers this year at the Siggraph event, Estes said. And some of that will be relevant to Nvidia’s developers, which number about 550,000 now. About half of those developers are in games, while the rest are in high-performance computing, robotics, and AI.”
Among the announcements, one is particularly cool. Estes said that Nvidial will show off its Isaac robots in a new environment. These robots, which are being used to vet AI algorithms, will be brought inside the virtual environment that Nvidia calls Project Holodeck. Project Holodeck is a virtual space for collaboration, where full simulations of things like cars and robots are possible. By putting the Isaac robots inside that world, they can learn how to behave, without causing havoc in the real world.
Above: The Project Holodeck demo
“A robot will be able to learn things in VR,” Estes said. “We can train it in a simulated environment.”
Nvidia is providing external Titan X or Quadro graphics cards through an external graphics processing unit (eGPU) chassis. That will boost workflows for people who use their laptop computers for video editing, interactive rendering, VR content creation, AI development
and more, Estes said.
To ensure professionals can enjoy great performance with applications such as Autodesk Maya and Adobe Premier Pro, Nvidia is releasing a new performance driver for Titan X hardware to make it faster. The Quadro eGPU solutions will be available in September through partners such as Bizon, Sonnet, and One Stop Systems/Magma.
Nvidia also said it was launching its Optix 5.0 SDK on the Nvidia DGX AI workstation. That will give designers, artists, and other content-creation professionals the rendering capability of 150 standard central processing unit (CPU) servers.
The tech could be used by millions of people, Estes said. And that kind of system would cost $75,000 over three years, compared to $4 million for a CPU-based system, the company said.
OptiX 5.0’s new ray tracing capabilities will speed up the process required to visualize designs or characters, thereby increasing a creative professional’s ability to interact with their content. It features new AI “de-noising” capability to accelerate the removal of graininess from images, and brings GPU-accelerated motion blur for realistic animation effects. It will be available for free in November.
By running Nvidia Optix 5.0 on a DGX Station, content creators can significantly accelerate training, inference and rendering (meaning both AI and graphics tasks).
“AI is transforming industries everywhere,” said Steve May, vice president and chief technology officer of Pixar, in a statement. “We’re excited to see how Nvidia’s new AI technologies will improve the filmmaking process.”
On the research side, Nvidia is showing how it can animate realistic human faces and simulate how light interacts with surfaces. It will tap AI technology to improve the realism of the facial animations. Right now, it takes human artists hundreds of hours to create digital faces that more closely match the faces of human actors.
Nvidia Research partnered with Remedy Entertainment, maker of games such as Quantum Break, Max Payne and Alan Wake, to help game makers produce more realistic faces with less effort and at lower cost.
Above: Nvidia is using AI to create human facial animations.
The parties combined Remedy’s animation data and Nvidia’s deep learning technology to train a neural network to produce facial animations directly from actor videos. The research was done by Samuli Laine, Tero Karras, Timo Aila, and Jaakko Lehtinen. Nvidia’s solution requires only five minutes of training data to generate all the facial animation needed for an entire game from a simple video stream.
Antti Herva, lead character technical artist at Remedy, said that over time, the new methods will let the studio build larger, richer game worlds with more characters than are now possible.
Already, the studio is creating high-quality facial animation in much less time than in the past.
“Based on the Nvidia research work we’ve seen in AI-driven facial animation, we’re convinced AI will revolutionize content creation,” said Herva, in a statement. “Complex facial animation for digital doubles like that in Quantum Break can take several man-years to create. After working with Nvidia to build video- and audio-driven deep neural networks for facial animation, we can reduce that time by 80 percent in large scale projects and free our artists to focus on other tasks.”
In another research project, Nvidia trained a system to generate realistic facial animation using only audio. With this tool, game studios will be able to add more supporting game characters, create live animated avatars, and more easily produce games in multiple languages.
Above: AI can smooth out the “jaggies,” or rough edges in 3D graphics.
AI also holds promise for rendering 3D graphics, the process that turns digital worlds into the life-like images you see on the screen. Film makers and designers use a technique called “ray tracing” to simulate light reflecting from surfaces in the virtual scene. Nvidia is using AI to improve both ray tracing and rasterization, a less costly rendering technique used in computer games.
In a related project, Nvidia researchers used AI to tackle a problem in computer game rendering known as anti-aliasing. Like the de-noising problem, anti-aliasing removes artifacts from partially-computed images, with this artifact looking like stair-stepped “jaggies.” Nvidia researchers Marco Salvi and Anjul Patney trained a neural network to recognize jaggy artifacts and replace those pixels with smooth anti-aliased pixels. The AI-based solution produces images that are sharper (less blurry) than existing algorithms.
Nvidia is also developing more efficient methods to trace virtual light rays. Computers sample the paths of many light rays to generate a photorealistic image. The problem is that not all of those light paths contribute to the final image.
Researchers Ken Daum and Alex Keller trained a neural network to guide the choice of light paths. They accomplished this by connecting the math of tracing light rays to the AI concept of reinforcement learning. Their solution taught the neural network to distinguish the paths most likely to connect lights with virtual cameras, from the paths that don’t contribute to the image.
Above: Nvidia uses AI to figure out light sources in 3D graphics.
Lastly, Nvidia said it taking immersive VR to more people by releasing the VRWorks 360 Video SDK to enable production houses to livestream high-quality, 360-degree, stereo video to their audiences.
Normally, it takes a lot of computation time to stitch together images for 360-degree videos. By doing live 360-degree stereo stitching, Nvidia is making life a lot easier for the live-production and live-event industries, said Zvi Greenstein, vice president at Nvidia.
The VRWorks SDK enables production studios, camera makers and app developers to integrate 360 degree, stereo stitching SDK into their existing workflow for live and post production. The Z Cam V1 Pro (made by VR camera firm Z Cam) is the first professional 360 degree VR camera that will fully integrate the VRWorks SDK.
“We have clients across a wide range of industries, from travel through sports, who want high quality, 360 degree video,” said Chris Grainger, CEO of Grainger VR, in a statement. “This allows filmmakers to push the boundaries of live storytelling.”
GamesBeat's creed when covering the game industry is "where passion meets business." What does this mean? We want to tell you how the news matters to you -- not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it. Learn more about membership.
