(work while done at U.C. Berkeley with Todd Templeton, Marci Meingast and Shankar Sastry; funded by Boeing Phantom Works and AFRL's SEC Program)
A fundamental problem in computer vision is the development of robust vision systems capable of supporting safe and predictable interactions with the world. In this project we developed a system for automatic terrain recovery for determining safe landing areas for rotorcraft in unknown environments.
- design of vision landing system [ICRA '07,CDC '04]:
vision landing system in progress (over 70,000 lines of
code today) integrating sparse feature tracking,
recursive structure-from-motion, and multi-frame planar
parallax in a multi-threaded asynchronous architecture
with MMX optimized image processing routines;
- models of rolling-shutter cameras [OMNIVIS '05]:
model of rolling-shutter cameras for use in motion
estimation, equivalence in certain cases with
crossed-slit cameras.
- recursive multi-frame planar parallax algorithm
[3DPVT '06]: recursive, dense & direct method for
performing 3D reconstructions using the multi-frame
planar parallax framework
Stay tuned for videos!
Harmonic Analysis of Structure-from-motion
Structure-from-motion without Correspondences
(work with Ameesh Makadia and Kostas Daniilidis)
The expression of the space of essential matrices as a homogeneous space allows one to study harmonic analysis of functions on this constraint space, allowing for the analysis of two-view geometry from a signal processing viewpoint. For one of the many drawbacks of the traditional structure-from-motion framework is its dependence on correspondences between features, with the often false assumption that feature matches are sufficient statistics; the error in matching, though, rarely has normal Gaussian noise characteristics. In the proposed signal processing framework, we use a non-parametric representations of match likelihood, and therefore do not make the Gaussian noise assumption. Some of our progress:
- convolution of asymmetric functions on the sphere
[OMNIVIS '04]: a convolution theorem for two asymmetric
functions on the sphere, yielding a function on SO(3);
- fast epipolar Radon transform [CVPR '05,OMNIVIS '04]:
a comparatively fast Radon transform for estimating
motion from two views;
- estimation without correspondences [CVPR '05]:
estimation and tracking of motion using the epipolar
Radon transform.
A long-term goals of this research is to investigate statistical properties of multiple images, where there is a lack of temporal coherence typically present in movies obtained from continuously moving cameras, for the purposes of representation, compression, and analysis.
Omnidirectional Structure from Motion
(work with my former advisor Kostas Daniilidis)
The goal of this work has been to create a theory of structure-from-motion for catadioptric cameras. Structure-from-motion (SFM) is the problem of simultaneoulsy recovering structure and motion from multiple images or a moving camera. We have developed a complete theory, from calibration to motion and structure estimation, to rectification, for cameras combining a parabolic mirror and an orthographic lens. We also introduced a differential geometric treatment of general SFM constraints, expressing spaces of essential or fundamental matrices as a homogeneous space, thereby enabling a study of harmonic analysis discussed below. The results are enumerated below; my ICCV 2003 tutorial on omnidirectional SFM is available here.
- calibration of parabolic cameras [ICCV '99]:
algorithm to calibrate parabolic cameras from 3 lines in
a single view, equivalent to the finding the intersection
of three spheres;
- unifying model [ECCV '00]: model unifying all central
catadioptric cameras in common intuitive framework,
demonstrating equivalence of parabolic-mirror cameras
with stereographic projection;
- parabolic fundamental matrix [CVPR '01]: used to
express the epipolar constraint for parabolic cameras as
a linear constraint, and where intrinsic parameters
(calibration) is encoded by its nullspace;
- multiple view constraints [ECCV '02]: multiple view
constraints for parabolic cameras;
- constraints on parabolic fundamental matrices [ECCV
'02]: necessary and sufficient conditions on parabolic
fundamental matrices;
- primitives & transformations of parabolic
projections [CVPR '01,ECCV '02]: Lorentz transformations
of parabolic images are natural transformations of
parabolic images, representations of line images and the
image of the absolute conic;
- conformal rectification of pairs of parabolic images
[OMNIVIS '03]: unique conformal rectifications of
parabolic imagery for disparity estimation;
- algebraic characterization of spaces of fundamental
or essential matrices [ICCV '03]: the
identification of the space of essential (fundamental)
matrices with the quotient space SO(3)×SO(3)/SO(2).
My continuing interests in this subject are the study of non-central cameras, estimation problems, and applications to robotics through visual servoing.
Other interests
- Q: Is the reconstruction ambiguity of two parabolic projections projective?
- Computational geometry; on finding bathrooms and computing Voronoi diagrams on the sphere
- Q: What are the roots of the derivative of a polynomial?
- Attitude control (a survey of topological problems
with attitude control for the PhD program at Penn)