The industrial metaverse: Where simulation and reality meet

No other topic took 2021 by storm quite like the metaverse. As we all experienced yet another year of living through a pandemic, the idea of a new, immersive reality captured the interests and imaginations of many.

As with any new concept, it’s helpful to level set on what the metaverse is — or will be. I like how my Unity colleague and one of the early pioneers of 3D media and virtual reality, Tony Parisi, put it in his excellent article on the metaverse: “The metaverse is the next evolution of the internet … enhanced and upgraded to consistently deliver 3D content, spatially organized information and experiences, and real-time synchronous communication.”

Much of the attention around the metaverse to date has centered on social experiences where people can meet up, but I’m most excited by the potential of the “industrial metaverse” where the goal doesn’t have anything to do with social interaction; rather, it’s about simulating experiences in the virtual world before moving into the physical world. (Note: There is only one metaverse, as Tony points out in his piece, so I’m using “industrial metaverse” to demonstrate how companies in industry can benefit from the metaverse.)

The industrial metaverse can transform the way every physical asset — buildings, planes, robots, cars, etc. — on the planet is created, built, and operated.

Boeing, which wants to build its next airplane in the metaverse, is one of many companies embracing this shift. Across industries such as architecture, engineering, construction, automotive, transportation, manufacturing, and beyond, the imperative to bring to market increasingly complex, intelligent, and connected projects or products — and then operate and maintain them — requires a new approach.

A key part of Boeing’s strategy is to “build and link virtual three-dimensional ‘digital twin’ replicas of the jet and the production system able to run simulations.” Digital twins are poised to play a pivotal role in the industrial metaverse, providing virtual, behaviorally accurate representations of physical assets. When teams use digital twins to visualize and simulate complex operations before taking real-world action, they can make informed decisions far more easily and cost-effectively.

Better decisions during product development and operations result in better outcomes for businesses and their customers. The costs of poor decision-making are high: Boeing says that more than 70% of its quality issues are tied to design issues. As teams across different disciplines and geographies come together to create complex products, migrating these workflows into virtual environments will improve understanding and collaboration.

The convergence of simulation and reality

The power of the industrial metaverse is that it is where simulation and reality meet. Data created in simulation leads to design and implementation in the real world. Let me illustrate this concept using another example from Boeing, which recently worked with Unity (where I work), on a project to explore the future of aircraft inspection and maintenance.

Boeing is researching how to use augmented reality (AR) devices such as tablets and headsets to compare a plane’s present-day and previous states, using current and historic maintenance data. They developed a machine learning algorithm to help power this experience, but came up against the limitations of reality while gathering data to train the algorithm. Even though they took thousands of photographs of the plane to feed into the algorithm, it didn’t work.

Boeing’s experience is not uncommon, and these challenges have helped spark the data-centric AI movement. As algorithms mature and become general purpose, it is the training data — not the code — that becomes the difference between success and failure.

In this case, Boeing worked with Unity’s computer vision experts to build a digital twin of the plane and generate over 100,000 machine-created, curated images at a fraction of the time and cost. Adding these synthetic datasets to the real-world data (i.e., photographs) enabled the newly trained model to successfully anchor historical records onto a 3D AR model of the aircraft. This paves the way for more efficient aircraft inspections, carried out by mechanics and engineers every day.

Better simulations make the real world better

Synthetic data is just one example of data created in simulation that delivers real-world impact. Across industries, groundbreaking work is happening in multi-agent deep reinforcement learning and massively scaled simulated environments. We’ve seen machine learning agents develop collaborative and sometimes superhuman strategic skills — it’s a hint of what is to come.

Coupling these simulation and training environments with the real world using 5G and Internet of Things (IoT) data creates a virtuous feedback loop between the virtual and the physical world. This in turn creates a whole set of scenarios for systems with AI as the flywheel, from big-picture observations to what-if studies. Increasingly accurate simulations lead to improvements in the real world, from testing autonomous robots to optimizing retail store layouts.

Looking ahead

While the metaverse will take shape over the years to come, it will be a transformational medium for the companies that embrace it. Forward-thinking organizations are already getting a head start by investing in key building blocks such as AI and machine learning, cloud and edge computing, 5G and connectivity, IoT, extended reality (virtual, augmented and mixed reality), and more. Because it is rooted in reality, the industrial metaverse will offer immense opportunity to help companies better understand and improve the physical world in a more scalable, sustainable and safer way.

Danny Lange is the Senior Vice president of Artificial Intelligence and Machine Learning at Unity. Follow him on Twitter and LinkedIn.