How high refresh and frame rates will change phone screens and cameras

Though 2020 will be remembered for many things, one of the year’s noteworthy smartphone device trends — broad availability of “120Hz” screens — could slip under the mainstream radar, mostly because average consumers have no idea what Hz (“Hertz”) means. Displays targeted at gamers and creative professionals adopted 120Hz technology several years ago, sometimes referring to “120Hz refresh rates,” and in 2020, even budget smartphones will include “120Hz display support.”

What that means is that smartphone users at $300-$400 price points are about to have access to the imperceptibly fast display technologies that were once restricted to high-end computers and tablets, right at the same time higher frame rate cameras are becoming increasingly popular on phones. Today, I’m going to simplify this complex topic, and explain why it actually matters when a screen moves from 30Hz to 60Hz or 120Hz refresh rates — as well as when it’s overkill.

Hertz and frames per second

Hertz (generally abbreviated Hz) is the international unit of frequency per second — how many times something happens in a second — and is used across many different technologies. When we talk about screens such as TVs or monitors, a completely still photograph is “1Hz,” such that it doesn’t change at all every second. Most videos today move at 30Hz, which is to say a video is really a collection of still pictures that you see flashing before your eyes 30 times per second.

The screen on your phone may be capable of updating 30 or 60 times each second, which is known as its “refresh rate” — how often it can completely refresh the whole display in a second. But the content it displays may not change as often as the screen is capable of showing. The rate the content plays back at is measured in “frames per second.”

Human eyes and brains collectively know something’s not quite right when a video plays back at fewer than 24 frames per second. That’s the minimum number of changes that are needed each second for something to look “lifelike” to a human viewer, and scientific tests have shown that most people can’t perceive motion faster than 60 frames per second, or under most circumstances quantify the differences between 30 and 60 frames per second. Consequently, movie studios settled on 24 frames per second for theatrical films, and most prior-generation TVs were built to display most content at 30 or fewer frames per second, with narrow exceptions. Filmmakers commonly described 24FPS as “more cinematic” than the faster-refreshing 30FPS, but both rates peacefully co-existed.

The 60Hz-60FPS shock

For many decades, 24FPS and 30FPS were considered so ideal for video playback that little traction developed for faster displays. Movies could distinguish themselves from more common TV shows, video games, and user-created content. While adding more pixels (resolution) could dramatically improve the detail of every frame, there wasn’t an obvious need for a faster display.

Moreover, digital video creators knew that switching from 30 to 60 frames per second meant having twice as many images to store per second, bloating file sizes in ways that might not let movies fit on optical discs or quickly transfer over the internet. As smartphones continued to become more popular, 60FPS videos would have required twice the limited space on a pocket-sized device, and their screens might not be able to show video at that higher rate, arguably making the extra data wasteful.

There wasn’t a specific single event that changed perceptions about the “right” speed for a display, but rather scattered changes from multiple directions. As 60Hz and 120Hz screens became easier to make, some TV makers saw an opportunity to sell them, even though 60FPS TV and movie content wasn’t available. So they began to offer “motion smoothing” features that added extra, approximated frames each second, making 24FPS movies (and other content) run artificially at 60FPS, an effect that made cinematic productions look like overly fast TV shows. This profoundly upset Hollywood filmmakers, who felt like their movies were being heavily adulterated on some TVs, and some TV show creators as well.

Game companies weren’t as shy about higher frame rates. Computer graphics chip designers and some video game developers began to push for 60FPS as a floor, with the prospect of even higher display refresh rates and content frame rates under some circumstances. Today, achieving a guaranteed 60FPS frame rate is considered to be ideal for games, but isn’t universally guaranteed. Many gamers still struggle to precisely explain why 60FPS or faster speeds are better than 30FPS, beyond to say that games look smoother at higher frame rates.

Even so, gamers (and some others) have pushed for video recordings and streams of games to be offered at 60FPS, forcing YouTube, Vimeo, Twitch, and other platforms to embrace the change and deal with the added bandwidth/storage demands. While Hollywood has largely stuck to 24FPS content, Chinese filmmakers and film financiers have embraced higher frame rates with the same verve as game designers, and some Chinese theaters currently display Hollywood films that were officially converted to 60FPS.

Why 120Hz displays matter

The simplest way to explain the difference between 30Hz, 60Hz, and 120Hz displays is that the higher the number, the more frames per second the screen can display, and the smoother content will look. As suggested above, however, most of today’s videos remain at 24-30FPS, so why does someone need even a 60Hz display, let alone a 120Hz version?

One key advantage to a 120Hz screen is that it can be easily divided by both 24 and 30, so it can just keep each frame of a 24FPS movie on screen for 5 of its 120 frames in a second, while 30FPS TV shows can display each frame for 4 of the screen’s 120 frames. That means older movies and TV shows are displayed as they were originally intended, without using artificial frame smoothers like the ones that upset Hollywood. Additionally, newer videos (including ones recorded with smartphone cameras) can be played back at 60 or even 120FPS.

Videos aren’t the only things that play back on smartphone screens; people do a lot of web browsing, scrolling through social media feeds, and other things with their phones that can also benefit from smoother movement. Apple introduced “ProMotion” to its iPad Pro tablets three years ago, and the primary benefit has been that scrolling through anything is smooth enough that you can continue to read text even while it’s moving up or down. On non-ProMotion screens, text scrolling is jittery enough that you unconsciously only really look at it while it’s stopped.

That having been said, gamers are arguably the biggest potential beneficiaries of faster displays. In January, Epic Games updated the iPad version of Fortnite to support 120FPS frame rates on iPad Pro models with ProMotion screens — a feature that lets gamers (with $799+ tablets) see up to twice the frames of the PlayStation 4 and Xbox One, or four times the frames of the Nintendo Switch, an advantage one YouTuber described as “so good it’s unfair.” The reasoning, which hardcore computer gamers will attest to, is that the smoother frame rates enable faster split-second reads on how you and your opponents are moving, as well as more accurate responses.

Relatively few iOS/iPadOS game developers support 120FPS at this point, but Fortnite is a heavy hitter, and the ProMotion update is a sign of what’s coming elsewhere. Roughly 100 titles have been updated to support the “UltraMotion” 120Hz displays on Razer’s niche smartphones, including several high-profile fighting games, some popular MMORPGs, and the perennially successful Pokémon Go. Since Qualcomm’s latest high-end Snapdragon 865 chipsets recently added support for up to 144Hz displays, it’s clear that 120Hz isn’t the upper limit for smartphones, and could follow PC monitors to reach ridiculously high refresh rates.

Why 120FPS (and faster) cameras matter

While screens were climbing to 60Hz and 120Hz speeds, high frame rate cameras began to take off for action sports and “prosumer” videographers interested in using Hertz in a different way. Since some cameras can take one picture every 120th or 240th of a second, what if the resulting videos could be played back at 30 frames per second? Every real world second would take 4 or 8 seconds to watch — “slow motion” — and viewers could pick up on details that would be impossible to see at regular speeds.

Current-generation iPhones and iPads have the option to record “Slo-mo” videos at 120 or 240FPS, maxed out at 1080p resolution. iOS 14 betas suggest that the next iPhones will be able to record higher-resolution 4K videos at these speeds. Qualcomm’s Snapdragon 865 has the memory speed and bandwidth to create 960FPS videos at 720p resolutions — yes, that’s 960 individual images per second — for mind-bogglingly slow motion, such that every real-world second would take 32 seconds to watch at 30fps.

Now 120FPS cameras can also be paired with 120Hz screens to deliver crazy-smooth real-time videos. While there isn’t much of an application for this yet with traditional 2D screens, VR and AR headsets can use higher frame rate videos and displays to mitigate nausea.

One drawback to bear in mind is that high framerate cameras consume a ton of storage and wireless bandwidth. iOS and iPadOS warn users that 4K 60FPS video recording requires 400 megabytes per minute, which could fill a base model 64GB iPhone 11 in around two hours, assuming relatively few apps or other files are also installed. If 4K 120FPS and 240FPS recordings are added this year, you can expect the storage requirements to roughly double or quadruple — though most people won’t create two-hour-long (or even 30-minute-long) videos at these speeds.

Do you really need a 120Hz (or faster) phone display?

For most people, the short answer is currently “no.” A 120Hz screen is a “nice but not mandatory to have” feature that many people won’t even notice 99% of the time. Smoother scrolling through a device’s UI and apps is certainly better, but in a way that people have struggled to quantify, and smoother videos still have some iffy bandwidth tradeoffs, particularly when viewed on tiny screens and streamed over spotty cellular networks. While smoother gaming is nice, only serious computer gamers (and the companies that supply them) care at this point about higher than 60FPS titles.

Over time, this will clearly change. 120Hz screens are about to become ubiquitous due to a growing supply of inexpensive components, and device chips such as Qualcomm’s, Apple’s, Samsung’s, and MediaTek’s will support high refresh rates because it’s increasingly trivial to do so. Right now, 120Hz technology is at best a frill for average people, but as chip speeds, memory capacities, and overall system bandwidth increase, companies will simultaneously support high-refresh screens and high-FPS cameras in even the most affordable devices. Hopefully, you’ll be able to both notice and explain the improvements for yourself.