Putting reality into Virtual Reality

Canon USA introduced its immersive video calling platform at this year’s CES. Kokomo transports you into a virtual world, just the way you are.

Humans need to be together.

When we are isolated from each other, we feel the impact on our mental and physical health. This is why, from the earliest moments of the pandemic, we got creative in our ways to reach out to each other. We shouted through glass, hugged through plastic, Zoomed, Teamsed, WhatsApped and Housepartied our way through lockdowns.

It’s also why Virtual Reality is gaining so much traction these days. The pandemic may well be long gone, but that desire to be together and interact even when apart lives on – and Virtual Reality is a great way to achieve it.

The only issue is – there’s still not a lot of reality in it. Most VR solutions you’ll find on the market use avatars or representations of how people wish to be seen, rather than how they actually are. That makes it difficult for any interaction to feel natural and takes away the pleasure of face-to-face conversations.

Canon USA’s Kokomo software is aiming to change that.

So long avatars, hello lifelike VR

Showcased at the Consumer Electronics Show (CES) in Las Vegas, Kokomo is a software platform that brings lifelike video calling into the Virtual Reality space.

As Jon Lorentz, Senior Innovation Designer at Canon USA, explains, it aims to “create memorable experiences in a new way with technology so realistic that users forget they’re wearing a headset”.

“You won’t find any kooky digital avatars here,” he says. “But rather your own live appearance, including facial expressions, in a photoreal environment.”

You can carry out face-to-face and person-to-person video calling, in real time, just using a headset. Raise an eyebrow in real life and it’s right there in your VR. Those facial cues that get missed through videoconferencing lag are no more. Just natural faces, moving in all their expressive glory.

And because ‘immersive’ is much more than just sight, the Kokomo software and these visual subtleties are enhanced by spatial audio that mimics the way we take in sound in the real world.

Real-time face-to-face communication for anyone

One of the best features of the Kokomo platform is that it doesn’t need complex kits or extra equipment to work. A Canon camera, a VR headset and your smartphone is all it takes to transport you to meet your friend on the beach. Or a mountaintop. Or a Parisian café.

That’s another great advantage – the locations currently available have all been captured in every exquisite detail using Canon EOS R5 cameras and there are plenty more to come as the software develops. “Our experience is not about multiple cameras, complex systems or professional set ups,” explained Jon. “It’s about real-time face-to-face communication that can be made accessible to anyone.”

And while the immediate potential for developing the Kokomo platform is clear, the underlying technology could also have implications for Virtual Reality creators around the world. Especially when you consider that the increasing demand for high-quality 3D 180° VR content led to the recent launch of our dual fisheye lens for capturing VR.

Having our own state-of-the-art hardware to work with as they designed Kokomo’s ‘ImmersiveCall’ technology clearly enhanced the process for Canon USA’s Development Team and their goal to create a photoreal, immersive experience.

The desire to bring people, not avatars, together has placed Kokomo front and centre in truly putting reality into Virtual Reality.