A team from the University of Washington has developed a way to bring soccer matches to the world of augmented reality. (Photo: Konstantinos Rematas / YouTube)
Future games to decide who hoists the World Cup of soccer may one day play out not on your smartphone or television, but on any flat surface in your home.
A team of researchers from the University of Washington have created a machine-learning algorithm that can convert 2D YouTube clips into 3D reconstructions. Experienced through an augmented reality headset like the Microsoft HoloLens or the HTC Vive Pro, the system places a virtual representation of the match on any real-world flat surface. Those viewing the simulation can then walk around or move in close to see key pieces of the action.
You can see a video of the virtual playing field in the clip below.
To create a 3D replication of a 2D video, the researchers explained in a published study how they first trained their machine on data gathered from FIFA 2017, a wildly popular video game that accurately simulates soccer matches using real players.
“I had to play FIFA 2017 for, I don’t know, a couple of days to get all the data for training a neural network,” team member Konstantinos Rematas told New Scientist. Once the machine had an idea of how the matches unfolded, they then transitioned it to recreating matches uploaded to YouTube. The result, as shown in the screenshot below, is at first glance a near-perfect re-creation.
A screenshot of an augmented reality soccer match created from a 2D video on YouTube. The reference source is located in the bottom right. (Photo: Konstantinos Rematas / YouTube)
While the team members admit that the technology still needs refinements, the potential to bring augmented reality sports events into the home is an exciting step forward for interactive entertainment. Ira Kemelmacher-Shlizerman, an Israeli-born computer engineer who helped lead the study and also founded UW's Reality Lab, says that creating 3D experiences from 2D images and videos will open new doors for a wide variety of industries
“Our photos and videos tell a ton about ourselves, our histories, how people grow, age, learn to walk, and change over time, Kemelmacher-Shlizerman, who graduated from the Weizmann Institute of Science in Israel, told GeekWire. "Exploring and learning from that data will enable magical applications in telepresence, health, sports, entertainment and many other unexpected ones. Enabling all this in 3D will create novel augmented and virtual reality content. This will be a breakthrough in how we communicate, and enable a much more connected world.”
Companies like Magic Leap on working on creating lighter and more advanced augmented reality headsets for the home consumer.
Fortunately, Kemelmacher-Shlizerman and her team still have plenty of time to perfect their intelligent conversion system. Mass adoption of augmented and virtual reality tech is still a few years away, with startup companies like Magic Leap promising both more affordable, lightweight and advanced headsets. The team's ultimate goal is a live experience that can seamlessly be streamed from the field to your dining room table in real-time.
"We need both a real-time reconstruction method and a method for efficient data compression and streaming," they explained in the study.
Something tells us they'll have that bit of jargon figured out in time for the next World Cup in 2022.
