Last week, the Institute for Creative Technologies (ICT) at the University of Southern California demonstrated their second edition 3D teleconferencing telepresence device at the Army Science Conference in Florida.  Dubbed "Live 3D Teleconferencing," ICT's setup captured a single participant in 3D and then transmitted that image to a viewer about 10 feet away for an interactive experience between the two participants.

The demonstration brought together many familiar pieces of technology - like mirrors, cameras and monitors - but in a novel way.  3D participants sit before a simple array of equipment, comprised of a single light projector flanked by only two cameras, one on each side.  The light projector shines the 3D participant's face in a particular pattern that allows each camera to best capture the contours of the human face.  



The twist is that these two cameras are angled such that they are capturing slightly different angles of the participant.  Both camera feeds are then shuttled into a PC equipped with an nVidia graphics card whose job it is to dynamically texture-map a composite of the camera feeds onto a singular geometric model.

This computer generated image composition is projected through a DLP processor onto a double-sided mirror found in the middle of the public 3D display (although it is hidden from view itself).  The mirror is moving at 15 fps so when you add both sides together, you get a 30fps feed - equivalent to that of regular video.  The mirror makes a pass of 180 degrees in front of the viewer, creating the full 3D space required to make the image 'float'.

All of this is happening within microseconds.  The PC + nvidia card and the DLP processor are rendering so quickly that they are creating 72 different facial views per second, enough different images so that each pass of the mirror shows one of the viewer's eyes a slightly different image.  It is this optical illusion that creates the 3D effect you see in the video.


Meeting the Masters

The Lab had a chance to talk with Andrew Jones, one of the architects behind the latest generation of the Live 3D teleconference system.  

Overall he said the group is trying to create enveloping 3D environments where the screen becomes invisible to the user, basically having a 3D experience without the glasses.  Although the 3D technologies used in this demonstration are generally new to the lab, it is something ICT is moving into as time goes on.  

Jones noted that this 3D conferencing model really helps to address one of the major sticking points related to more traditional 2D conferencing environments - the lack of eye contact.   By using the 24 bit mirrors found in DLP projectors, ICT was able to massively multiply the available number of display images.

  This breakthrough is the key to device because it allows the participant's head to float in space and address any number of people in his viewing space while still remaining in eye contact with all of them.

Jones laid out some plans for the future which include continued work on 2nd generation of this platform and some beginning developments on a 3rd generation of this device that will use an even larger mirror to capture more of the 3D participant and deliver color images vs. the black and white images of V2.  We can look forward to an improved second edition as soon as January.


Thoughts and Analysis...

What makes this exhibit especially impressive is that ICT is taking equipment - like an nVidia graphics card, a PC, some cameras and a DLP projector - hardware that all can be bought almost anywhere today - and combining it with their own sophisticated software to create an impressive 3D experience.  Still, the display technology seems limited to the realities of displaying images on a spinning mirror. The 3-D capture and texture mapping could be interesting as other projection solutions become available.  In reviewing the Lab's website it seems like the ICT is working in a variety of areas that show promise for commercial telepresence conferencing so we look forward to future developments.