Get ready for serious games that improve your judgment

For years, video games have provided useful imitations of real-world scenarios. From flight simulations to military training, video games offer a low-risk environment to develop necessary experience. Now, a recent government intelligence program has taken that a step further, creating video games to improve cognitive skills.

The recurrent errors in our decision-making

In psychology, heuristics are the simple, efficient rules that people use to form judgments and make decisions. These classifiers are the effortless, intuitive thought processes that we have encoded into every facet of our daily lives for acts like forming sentences and recognizing faces. And they serve a crucial purpose. Unlike newborn children, we aren’t paralyzed in thought from every stimulus we encounter. We instead utilize millions of split-second mental shortcuts, accumulated over a lifetime, that allow us to function at a high level.

But our heuristics sometimes lead us astray. Here’s a quick thought exercise: a baseball bat and a baseball cost a combined $1.10. The bat costs a dollar more than the ball. How much does the bat cost? If you’re like most people, your first thought was “the bat costs $1.00!” In fact, over 50 percent of Harvard, Princeton and MIT students gave the same answer. But that answer is actually incorrect. The bat costs $1.05, 1 dollar more than the 5-cent ball.

Heuristic errors comes in a variety of forms, and academics like Nobel Prize winner Daniel Kahneman have devoted a substantial amount of work to identifying and categorizing these cognitive biases. The basic mechanics inherent in all cognitive biases, however, are mostly similar: we observe a challenging problem, substitute in a much easier problem, and then answer the easier problem. For the bat and ball problem, we know that the combined cost is $1.10, and that $1 + $.10 = $1.10. However, we ignore the more challenging criteria — the bat has to be exactly $1 more than the ball.

IARPA and the search for good judgment

In 2006, the Office of the Director of National Intelligence created IARPA, the Intelligence Advanced Research Projects Activity. IARPA was modeled after DARPA (Defense Advanced Research Projects Agency), an organization responsible for the invention of revolutionary technologies like GPS, voice recognition software, and most notably, the internet. IARPA is structurally similar, and provides research contracts to projects at the cutting edge of science, technology and engineering, with a singular goal–generating revolutionary capabilities for the U.S. Intelligence Community.

IARPA soon recognized the opportunity for cognitive skill training. Intelligence analysts have to interpret ambiguous information on a daily basis, and then make recommendations with significant downstream consequences. If the judgment of intelligence agencies is clouded by cognitive biases, such as groupthink, confirmation bias, or anchoring, the ramifications could be severe. IARPA sought a solution, and in 2011, they funded a program that offered a novel approach: ‘serious’ video games.

‘Serious’ video games helped analysts fast-track the experience curve by offering a training environment that was immersive, interactive, and without the fail risk. Skills like information assessment and cognitive bias mitigation could be practiced and developed in short training sessions, instead of real-life trial-and-error.

IARPA’s five-year game contest

To create the games, IARPA’s SIRIUS program developed a five-year contest. They invited six academic and industry teams to develop debias training video games, and challenged them to target six well-known biases (see table below). These six biases were chosen because of their impact on intelligence analysis, but they also impair decision-making in a variety of other fields, including business, policy, medicine, and education.

Once the games were complete, IARPA scored the teams based on a series of controlled experiments involving hundreds of participants. Some participants played the video games, while a passive-learning control group watched instructional videos. They tested participants on their bias mitigation skills before, immediately after, and eight-12 weeks after the experience. The winning team was determined based on the percentage point improvement in bias mitigation score.

The results were profound. Measured based upon percentage point improvement over their base score, the video game players showed significantly stronger persistent bias reduction effect on the test population, as much as three times larger than the effect produced by the control group. Moreover, when tested for information retention eight-12 weeks later, the video-watching control group reverted close to their pre-experiment levels, while the game players retained their skill. In fact, in some studies, the gamers’ bias mitigation skills actually improved during the interim period! The video games had provided a decision-making skill set which was then reinforced by real-world experiences.

One of the top performing teams was led by Tomek Strzalkowski, a computer scientist at the University of Albany, and also included researchers from Syracuse University, Colorado State University, University of Arizona, and Temple University, as well as the game development company 1st Playable Productions. Commenting on the success of his team’s strategy, Professor Strzalkowski remarked, “A key takeaway was that we did not need a game that also teaches bias mitigation; instead, we needed a teaching strategy that was a game.” This is a major insight for any serious game developers. First, find the best teaching mechanism. Then, support it with gameplay.

In recognition of their accomplishment, Professor Strzalkowski’s team earned a gold medal at the International Serious Play Awards competition. The team has since partnered with Correlation One, my talent solutions technology company in New York City, to repurpose the games for commercial use.

Applications beyond the intelligence community

Beyond the intelligence community, the applications for cognitive games are vast. Cognitive biases impact decision-making in a variety of disciplines and industries, and so technology that trains employees to improve their judgment could create an enormous value opportunity. Consider a financial trader whose job requires her to digest a constant stream of information, and use that information to make impactful, time-sensitive decisions. Or a hiring manager, who wants to ensure that each candidate is given a fair, objective screening process. Even in the emerging world of artificial intelligence, where algorithms increasingly make decisions in place of humans, bias mitigation is extremely important. An algorithm that isn’t well-calibrated can lead to sub-optimal decisions across the board, so the AI developer must be especially aware of potential biases.

The SIRIUS program also strengthens the argument for wider adoption of ‘serious’ video games. U.S. companies spent over $70B on training last year, but only 16% looked to games or simulations for their solutions. Yet video games have a proven efficacy for training purposes. They provide a low-risk, high-repetition environment to gain necessary skills and experience, and are being deployed in an increasing variety of use cases. With a younger generation that is already familiar with gaming technology and rapidly entering the workforce, one should expect ‘serious’ training games to become more prevalent in the future.

Andrew Strong is Head of Decision Solutions at Correlation One, a NYC-based technology startup.