Multiple cameras model tracking for Augmented Reality

A. Sanson, A. Fusiello and U. Castellani Department of Computer Science
University of Verona, Verona - Italy

Overview

Augmented Reality deals with the problem of enhancing the real world with computer generated data; to this end it is important to determine the pose of a given object with respect to the camera(s). In this paper we present a technique for tracking complex objects in video sequences with multiple cameras. Our method uses information derived from image gradient by comparing them with the edges of tracked object, whose 3D model is known. A score function is defined, depending on the amount of image gradient ``seen'' by the model edges. The sought pose parameters are obtained by maximizing this function using a non deterministic algorithm which proved to be optimal for this problem. In order to provide at each step a better starting guess and to smooth out noise, a recursive Kalman filter has been also implemented. We also propose a technique to cope with cameras with variable focal length. Results with real sequences have shown small errors in pose estimations and a good behavior in augmented reality applications.

Method

Our tracking algorithm uses image gradient space compared to model edges. A score function is defined, depending on the amount of image gradient "seen" by the model edges. Main features are:

no image features extraction are necessary;
it is robust to occlusions;
it uses multiple cameras;
it copes with zooming cameras;
a temporal evolution system (Kalman filter) has been implemented;
the optimization algorithm has been carefully chosen;
it implements a vectorial graphic engine with hidden line removal;
it can track every object type (of arbitrary complexity) which can be rendered by the graphic library OpenGL.

Results

Planar motion Forward motion Rotation Occlusions

Demo movie

Note: all videos are encoded using XviD codec.

Reference paper

A. Sanson, U. Castellani, A. Fusiello, and V. Murino. Fast model tracking with multiple cameras for augmented reality. In ACM Symposium on Virtual Reality Software and Technology (VRST'04), pages 170 -- 173, Hong Kong, November 10-12 2004. (PDF)

[Previous work with A. Valinetti]