The Daedalus Project

Investigators: Roger Hubbold (PI), and Toby Howard (CI)

Research Co-investigator: Mashhuda Glencross (RF)

Collaborators: Greg Ward (Dolby Canada), Rob Rhodes (Napper Architects), and Alan Chalmers (The Warwick Digital Laboratory).

Students: Jun Liu (PhD), and Francho Melendez (PhD)

The Advanced Interfaces Group has been working in the field of reconstructing real-world scenes from photographs and video sequences since 1995. The REVEAL project (1998-2001), funded by the UK-EPSRC and in partnership with Greater Manchester Police, investigated construction of fully interactive virtual environments which faithfully represent real-world scenes. The target application area for this early work was crime scene reconstruction, investigation and training, and forensic analysis. Fundamental results from this were subsequently applied and developed in the European Commission funded ARIS project (2001-2004), a collaboration with Fraunhofer Institute (Germany), Intracom S.A. (Greece), Inria-Loria (France), University of Bristol (UK), Athens Technology Centre (Greece), and IKEA (Greece). During this time, the focus of the AIG contribution was real-time augmentation of images and video sequences with photorealistically-rendered synthetic 3D objects (gallery). An important outcome of these projects was the ICARUS semi-automatic calibration and reconstruction system, which was later incorporated into commercial match moving software by The Pixel Farm.

In this current project we build upon the group's prior expertise in 3D reconstruction, to advance state-of-the-art methods for automatically constructing and re-lighting visually plausible 3D models of real-world scenes from multiple digital images. Limitations of most current 3D reconstruction methods are that they can be labour intensive, work only in specific contexts, produce geometry that contains noise, and require additional complicated procedures and special equipment to measure material reflectance. While important advances in 3D reconstuction algorithms have been made by the computer vision community, for the listed reasons their adoption in real applications has been slow. We focus our work on finding novel practical solutions to some of these problems in order to apply 3D reconstruction techniques in architectural site context visualisation, design, planning and public consultation processes. Our research considers two main aspects:

• 1. Automatic reconstruction of 3D geometry of existing sites from high-resolution images of sites captured from widely separated viewpoints (known as wide-baseline images). We employ robust computer vision methods to achieve this.
• 2. Relighting the scene so that its appearance under different lighting conditions can be assessed.

Wide-baseline Reconstruction

Our 3D reconstruction method is predominantly automatic and based on a feature matching approach. We employ our own efficient implementation of Lowe's SIFT algorithm as a starting point, and then use a novel oriented, normalised cross-correlation method for detailed matching of features. This gives us a dense point cloud of 3D points representing the 3D geometry captured in the photographs. The point clouds can be visualised using point-based rendering, with colour information derived from the original images, or meshed and rendered as textured triangles. In the latter case, we use Poisson surface reconstruction to recover a 3D surface which can be edited and simplified.

Relighting 3D Geometry

In order to relight the recovered geometry, we first need to estimate materials reflectance characteristics from images and under uncontrolled lighting conditions. In the general case, this is a difficult task. We have developed a completely new, simple, and efficient method called surface depth hallucination for capturing local surface detail of textured surfaces from a single view. Our method allows us to reproduce subtle changes in appearance, for surfaces such as stone and brickwork. In particular self-shadowing from fine surface detail, that depends on the direction of illumination, is plausibly reproduced.

This work is being widely reported by the media. The following links are just a few of the stories that have turned up on the web: [New Scientist...] [Youtube video...] [Slashdot...] [Pressetext...]

Currently for final render visualisations we use the Radiance physically based renderer. Towards the end of the project, we expect to substitute this for Warwick Digital Laboratory's interactive renderer to produce high quality real-time visualisations.

The Daedalus project is funded by the UK-EPSRC under grant EP/D069734/1 (May 2006 - April 2009).