Gold Sponsors
Array Telepresence Logo   Human Productivity Lab Logo   Ashton Bentley Logo
Silver Sponsors
Bronze Sponsors
Telepresence Options Magazine

Latest Telepresence and Visual Collaboration News:
Full Article:

The Trillion Dollar 3D Telepresence Gold Mine

November 20, 2017 | Telepresence Options


A still taken from the 2016 video of the demonstration of Telepresence, or Holoportation, with the Microsoft HoloLens.

Story and images by Forbes

We decided to find out if what Microsoft says is true: remote volumetric telepresence and collaboration can and will be done, sooner than people think and--despite obvious technical hurdles--it will be the killer app of Augmented and Virtual Reality.

Rewind. It took the personal computer roughly 15 years to hit an inflection point and become a consumer product everyone had to have. At first its killer app, email, which most people first got at work, didn't seem so revolutionary. Hardly anyone outside the company was using it. The network effect, a phenomenon whereby a service becomes more valuable when more people use it, hadn't kicked in. New technology always penetrates the enterprise before the home. Once people started getting Internet online services with a personal email address, it made the PC something everyone had to have at home. The telephone is another great example. The more people who got one, the more people had to have one.

Similarly, messaging and social media are the killer apps of smartphones. Our need to connect with other people follows us, no matter where technology takes us. New technology succeeds when it makes what we are already doing better, cheaper, and faster. It naturally follows that Telepresence should likewise be one of the killer apps for both AR and VR. A video of Microsoft Research's 2016 Holoportation experiment suggests Microsoft must have been working on this internally for some time, maybe even before the launch of the HoloLens itself.

Telepresence, meaning to be electronically present elsewhere, is not a new idea. As a result, the term describes a broad range of approaches to virtual presence. It breaks down into six main types:

1) 2D video conference systems. These have gotten incredibly sophisticated and include eye tracking to help create presence for colleagues who are still seen on a monitor. Cisco's Spark System dominates the billion-dollar teleconferencing industry.

2) Robotic telepresence. Describes any remotely operated vehicle with a driver's view such as Remote Underwater Vehicles (ROVs) and Unmanned Aerial Vehicles (UAVs), or RPAs (Remotely Piloted Aircraft). NASA has long dreamed of true, real-time robotic telepresence, which was, in fact, one of the initial purposes of their VR research in the 80s. However, due to the time-delay lag of signals to travel from Earth to Mars and back, NASA scientists can't directly tele-operate a robotic explorer like the Curiosity Mars rover. However, it's possible astronauts aboard a spacecraft orbiting Mars may be able to.


CREECH AIR FORCE BASE, NV - AUGUST 08: United States Air Force Senior

3) Remote experts. They use AR to see what you're seeing, although they cannot see you. They can even draw on the live feed you are sharing with them, interacting with real objects in your field of view in real time. Remote experts turn low-skilled employees into higher skilled ones.

4) VR telepresence. This allows us to share a virtual world like Oculus Rooms or AltSpace VR where we are represented by an avatar. Today most avatars are cartoon-like, but they will soon be able to use 3D volumetric captures taken on a cellphone to skin avatars that are eerily accurate. Lip sync (more precisely real-time lip animation) and eye contact introduced by Sansar and High Fidelity, already make you can feel very, very present.

5) AR telepresence. This allows two or more remote people to have volumetric presence in the same room, which Microsoft calls Holoportation, because it uses their HoloLens. This has been convincingly demonstrated, and now companies are seeking to bring that technology to business conferencing. However, not all the technical and practical issues around this have been solved. Several companies are working on solutions that could disrupt the teleconferencing business Cisco dominates. Not surprisingly, Cisco is also working on this.

6) True holographic (visible to the unaided eye) telepresence, is illustrated by Star War's Jedi Council, pictured below. This unaided volumetric holographic presence can be done today with holographic projection, mirrors, and an invisible projection surface. This works well under very specific circumstances. In no way would the participants perceive each other, but to people outside the simulation, it is completely real. They'd see two (or more) people in remote locations in real life, interacting, on stage, without headsets, in a shared 3-D space. However, the players could not see each other, they'd be looking past the reflections at a monitor. From the audience, you'd never know.


The next best thing to being there.

Like Star Wars, the Stephen Speilberg movie "Minority Report," also features augmented reality in the scene where data floats in front of Tom Cruise without a projection surface, visible to the naked eye, and he manipulates it with his hands. This would only be possible if Cruise's character had either contacts or some sort of neural input that could send images directly to his brain. Otherwise, projected holograms can only visible to the naked eye if there is a transparent projection surface.

The HoloLens and other AR HMDs are equipped with inside-out cameras. In order to create a telepresence app, however, an outside-in camera that can face you and take videos of you is necessary.

I visited Steve McNelley, founder, and CEO of DVE Telepresence, in his workshop. DVE has been working for the Department of Energy and some of the largest companies in the world to provide what he calls the "only true telepresence". This requires three things, he explained, "absolute photorealism, perfect camera alignment for eye contact, and augmented reality images (holograms) appearing in space with no glasses required." DVE has a podium based system called "the 4Dp Telepresence Podium" which accomplishes all these things in a portable solution. The speaker behind the podium is captured in a remote location (such as a classroom room or a personal office) and projected in real time onto an invisible translucent surface and seen in the middle of the room by an audience. The speaker is projected onto the surface, and the camera is positioned to maintain eye contact with the audience.


Which one is the hologram?

DVE has demonstrated and patented many different technologies to create this holographic experience from OLED, LED, direct projection and a variant of an illusion enabling natural telepresence called "Pepper's Ghost, first demonstrated by stage artist John Henry Pepper in 1862. This method creates a "ghost" by reflecting an object onto a translucent surface, like a pane of glass, so the image seems to float in front of us. Today DVE has advanced this to create bright solid looking people that look like they are really in the room, as can be seen in the above image where I appear to be in the same room with Zach McNelley, DVE's 3D content creator, appearing as a hologram. The two requirements are a perfectly black background and a translucent projection surface.

Pepper's Ghost was most famously deployed in Disneyland's "Haunted Mansion" to create the illusions of spectral dinner parties and hitchhiking ghosts. In fact, the Star Wars Jedi Challenge VR Headset from Lenovo uses a similar method of bouncing an image off a mirror onto a transparent projection surface to create the illusion of 3D characters floating in space before us.


How to make a ghost--Pepper's Ghost

Microsoft has been promoting another vision of telepresence and remote collaboration for the HoloLens that they call HoloPortation. It was first demonstrated in this video from Microsoft Research, which allowed participants in remote locations (they were actually down the hall) to be present in each other's physical reality. Multiple 3D cameras were placed in each room. These inputs were fed into local computers which broadcast the compressed 3D image to the user's HoloLens. This video was posted on November 2016, which means that MS engineers must have already been working on Holoportation when the HoloLens was released in March 2016.

Add New Comment

Telepresence Options welcomes your comments! You may comment using your name and email (which will not be displayed), or you may connect with your Twitter, Facebook, Google+, or DISQUS account.