“Mixed Reality” States Explore Link Between Real and Virtual Worlds

A University of Illinois physicist has built a system that explores the connection between the real and virtual worlds by linking a mechanical pendulum to its virtual twin. It is the first real/virtual physics experiment, and could help clarify the influence that virtual communities exert on the real world, and vice versa. For instance, the experiment could help us understand how the economies of online games such as Second Life could affect real economies.

According to UI physics professor Alfred Hubler, his latest experiment is an example of a “mixed reality” state where there is no clear boundary between the real system and the virtual system: “The line blurs between what's real and what isn’t.”

At the APS March Meeting, Hubler reported on a recent experiment that he believes supports the existence of mixed reality states. He used a standard mechanical pendulum coupled with a virtual pendulum programmed to follow the well‑known equations of motion. He and his colleagues sent data about the real pendulum to the virtual one, while sending information about the virtual pendulum to a motor that influenced the motion of the real pendulum. They found that when the two pendulums were of different lengths, they remained in a “dual reality state” in which their motion was uncorrelated, and thus not synchronized.

They also discovered that when the pendulum lengths were similar, they reached a critical transition point and became correlated. “They suddenly noticed each other, synchronized their motions, and danced together indefinitely,” said Hubler. He compared it to a phase transition: the critical temperature/pressure point wherein matter moves from one state (gas) to another (liquid). In this case, the “phase transition” occurs when the boundary between reality and virtual reality disappears.

This is the “mixed reality” state, where a real pendulum and a virtual pendulum move together as one. The trick is real‑time feedback. Scientists have coupled mechanical pendulums with springs to create correlated motion, but without the staggering computational speed now achievable, coupling pendulums with a virtual system simply hadn’t been possible. “Computers are now fast enough that we can detect the position of the real pendulum, compute the dynamics of the virtual pendulum, and compute appropriate feedback to the real pendulum, all in real time,” said Hubler.

As flight simulations, immersive VR, and online virtual games and worlds become increasingly accurate in their depictions of the real world, Hubler believes such “mixed reality” states will become more common. He thinks his lab‑induced mixed reality states could be used to better understand real complex systems with a large number of parameters, by coupling a real system to a virtual one until their constant interactions result in a mixed reality state–for instance, modeling neurons by coupling a real neuron with a virtual one.

Instantaneous interaction is a critical requirement and while Hubler has shown that we can manage this in the lab with real and virtual pendulums, expanding that to an entire virtual world will require even faster computers, as well as far better probes and actuators and other supporting device technologies. Future generations of Second Life and other online games could become very exciting indeed, and almost indistinguishable from “reality.”