Nonlinear Rectification
Image rectification is the process of resampling a pair of images so that the epipolar constraint becomes simplified. The conventional epipolar constraint states that given a point in one image it's matching point in another image must lie on a particular line (the epipolar line) defined by the movement of the cameras (and vice versa). The pair of images below illustrate this line constraint for three different point matches.
After rectification the constraint changes to restrict a matching point to lie on collinear horizontal lines. The pair of images below illustrate the same point matches but after rectification.
As can be seen the upshot of this resampling is to greatly simplify the point matching problem since given a point in one image we know the matching point in the other image must lie on the line with the same y co-ordinate.
The earliest methods of rectification relied on applying simple and invertible linear transformations to both images to produce the desired effect. However, this can result in problems of large or unbounded rectified images if there is a lot of forward displacement between the camera(s) used to take the pictures. Later methods attempt to overcome this problem by applying different nonlinear transformations to each image.
The new technique presented here further improves these nonlinear methods by using a two stage process:
- Firstly the images are partially rectified with a linear technique that makes epipolar lines collinear whilst simultaneously trying to minimise image distortion. The advantage of this is that it uses a principalled approach to selecting the zero disparity plane, that is the plane in the scene whose matching points are at exactly the same position in both images. This makes subsequent dense correspondance much easier because maximum and minimum disparities will be roughly centred on 0.
- Finally epipolar lines are made parallel to the x axis by applying thenonlinear transformations to both images. Unlike other general methods, because epipolar lines are now colinear it is possible to use the same nonlinear transformation for both images. This produces less distorted images which are easier to match.
An implementation of the method in C++ can be downloaded here.