Tobii Spotlight’s foveated rendering can cut VR graphics load by 57%

Tobii’s gaze-tracking hardware for VR headsets is getting some serious software enhancements, the company announced today, in the form of full support for foveated rendering — with “even more foveated-related capabilities and benefits moving forward.” The features are being offered as Tobii Spotlight Technology, with significant initial rendering performance benefits.

Foveated rendering is an advanced technology that uses your eye’s position to create a particularly sharp focus point for a GPU to concentrate more of its resources upon, while areas outside that point can be rendered at lower detail levels. In order for the feature to work, the eye’s position needs to be tracked — Tobii’s specialty — and the software needs to have both designated “in focus” and “out of focus” areas, with a rendering pipeline to differentially tackle those areas.

Tobii has worked with Nvidia to achieve variable rate shading foveation in Tobii Spotlight, and says that it’s already seeing GPU rendering load reductions of 57%, bringing the average shading rate using dynamic foveated rendering down to 16% from around 24%. This frees up GPU resources for other purposes, such as boosting frame rates or conserving power, and Tobii expects that it will be used in the future to transfer and stream data that’s been dynamically optimized for the user’s current visual focus, including over low-latency 5G networks.

“Tobii Spotlight Technology is advanced eye tracking specialized for foveation,” said Tobii CEO Henrik Eskilsson. “Tobii Spotlight Technology directly addresses the ever-increasing demand for computational efficiency. Working with our partners to enable foveated rendering is just the beginning, and Tobii Spotlight Technology reflects our ongoing commitment to innovation in this area for years to come.”

The higher the resolution of the headset, the better the dynamic rendering works compared with fixed rendering, which would otherwise need to fill every pixel of a high-res frame with full detail. Initial results are coming from a Vive Pro Eye headset with Tobii gaze tracking hardware, plus an Nvidia RTX 2070, but the company expects that the dynamic foveated rendering will be able to keep shading manageable even when headset pixel counts reach or exceed 15K (compared with current 2K, 3K, 5K, and 8K models).