[home] |
[publications] |
[diary] |
[AIG home]
ICARUS
Interactive Calibration And Reconstruction from Image Sequences
Simon Gibson, Jon Cook, Toby Howard, Roger Hubbold,
Dan Oram
Introduction
Many virtual reality applications require the construction of complex
and accurate three-dimensional models which represent real-world
scenes. As part of the REVEAL project, we have developed new
semi-automatic modelling techniques that allow such environments to be
quickly and easily built from image sequences taken with uncalibrated
digital video and stills cameras.
We are interested in applying these techniques to the real-world
problem of scene-of-crime reconstruction (see the REVEAL home page
for further information).
A description of our suite of calibration and reconstruction software,
called ICARUS is given below. The camera calibration component can
also be used to provide match-moving capabilities for off-line
Augmented Reality applications.
ICARUS can also be downloaded from here (free for non-commercial use).
Video Sequence Calibration
The first stage of the calibration process involves estimating and
removing the effects of lens distortion in the video sequence. This is
done semi-automatically by having the user identify straight lines in
the images.
Once lens distortion has been removed, a sparse set of features are
automatically identified and tracked throughout the video sequence. Here is an MPEG movie showing the results of
our automatic feature detection and tracking algorithm.
The tracked feature positions are then used to automatically estimate
the extrinsic and intrinsic camera parameters for each frame. This MPEG shows the final result, where a
grid representing the ground-plane has been overlayed on the
calibrated video sequence.
Model Reconstruction
The user may quickly construct a model of the scene, using the
previously calibrated camera data to guide placement of simple
primitives. A non-linear optimization algorithms runs in real-time to
select the appropriate position, orientation, and size of objects to
match features the user identifies in the video frames.
Once a model has been reconstructed, textures may be automatically
extracted from the video footage. Here is an
MPEG sequence showing a reconstructed wire-frame model, visualized
using the calibrated camera path, and a VRML2
model showing the final texture mapped reconstruction.
In addition to reconstructing a virtual representation of a scene, the
user can also use the calibrated camera data to augment a video
sequence, or set of images, with synthetic objects. This MPEG animation shows a sequence
augmented with a number of synthetic objects. (Note that these are old examples, and the slight jittering seen here has been removed by significant improvements to the matchmoving algorithms).
Additional Examples
Reconstruction can also be performed using one or more images captured
using a digital stills camera. These two examples show reconstructions
from single images (total reconstruction time for each scene was less
than 30 minutes). On are screenshots showing the wireframe model
overlayed on the original image. On the right are texture-mapped
renderings of the reconstruction from different
viewpoints. Texture-mapped VRML models are available here and here.
Finally, calibration and reconstruction can also be performed for
pan/tilt/zoom camera motions. This example shows two views of a
reconstruction made from a video sequence captured by a camera mounted
on a tripod.
All text, images, and models are Copyright (C) Advanced Interfaces
Group, University of Manchester
For further information please contact Simon Gibson at sg@cs.man.ac.uk.
This page was last modified
on February 13th 2002 by Simon Gibson.
