[home] | [publications] | [diary] | [AIG home]

Interactive Calibration And Reconstruction from Image Sequences

Simon Gibson, Jon Cook, Toby Howard, Roger Hubbold, Dan Oram


Many virtual reality applications require the construction of complex and accurate three-dimensional models which represent real-world scenes. As part of the REVEAL project, we have developed new semi-automatic modelling techniques that allow such environments to be quickly and easily built from image sequences taken with uncalibrated digital video and stills cameras.

We are interested in applying these techniques to the real-world problem of scene-of-crime reconstruction (see the REVEAL home page for further information).

A description of our suite of calibration and reconstruction software, called ICARUS is given below. The camera calibration component can also be used to provide match-moving capabilities for off-line Augmented Reality applications.

ICARUS can also be downloaded from here (free for non-commercial use).

Video Sequence Calibration

The first stage of the calibration process involves estimating and removing the effects of lens distortion in the video sequence. This is done semi-automatically by having the user identify straight lines in the images.

Once lens distortion has been removed, a sparse set of features are automatically identified and tracked throughout the video sequence. Here is an MPEG movie showing the results of our automatic feature detection and tracking algorithm.

The tracked feature positions are then used to automatically estimate the extrinsic and intrinsic camera parameters for each frame. This MPEG shows the final result, where a grid representing the ground-plane has been overlayed on the calibrated video sequence.

Model Reconstruction

The user may quickly construct a model of the scene, using the previously calibrated camera data to guide placement of simple primitives. A non-linear optimization algorithms runs in real-time to select the appropriate position, orientation, and size of objects to match features the user identifies in the video frames.

Once a model has been reconstructed, textures may be automatically extracted from the video footage. Here is an MPEG sequence showing a reconstructed wire-frame model, visualized using the calibrated camera path, and a VRML2 model showing the final texture mapped reconstruction.

In addition to reconstructing a virtual representation of a scene, the user can also use the calibrated camera data to augment a video sequence, or set of images, with synthetic objects. This MPEG animation shows a sequence augmented with a number of synthetic objects. (Note that these are old examples, and the slight jittering seen here has been removed by significant improvements to the matchmoving algorithms).

Additional Examples

Reconstruction can also be performed using one or more images captured using a digital stills camera. These two examples show reconstructions from single images (total reconstruction time for each scene was less than 30 minutes). On are screenshots showing the wireframe model overlayed on the original image. On the right are texture-mapped renderings of the reconstruction from different viewpoints. Texture-mapped VRML models are available here and here.

Finally, calibration and reconstruction can also be performed for pan/tilt/zoom camera motions. This example shows two views of a reconstruction made from a video sequence captured by a camera mounted on a tripod.

All text, images, and models are Copyright (C) Advanced Interfaces Group, University of Manchester

For further information please contact Simon Gibson at sg@cs.man.ac.uk.

This page was last modified on February 13th 2002 by Simon Gibson.