OpenCV  3.2.0
Open Source Computer Vision
Affine Transformations

.2.0+dfsg_doc_tutorials_imgproc_imgtrans_warp_affine_warp_affine

Goal

In this tutorial you will learn how to:

Theory

What is an Affine Transformation?

  1. It is any transformation that can be expressed in the form of a matrix multiplication (linear transformation) followed by a vector addition (translation).
  2. From the above, We can use an Affine Transformation to express:

    1. Rotations (linear transformation)
    2. Translations (vector addition)
    3. Scale operations (linear transformation)

    you can see that, in essence, an Affine Transformation represents a relation between two images.

  3. The usual way to represent an Affine Transform is by using a \(2 \times 3\) matrix.

    \[ A = \begin{bmatrix} a_{00} & a_{01} \\ a_{10} & a_{11} \end{bmatrix}_{2 \times 2} B = \begin{bmatrix} b_{00} \\ b_{10} \end{bmatrix}_{2 \times 1} \]

    \[ M = \begin{bmatrix} A & B \end{bmatrix} = \begin{bmatrix} a_{00} & a_{01} & b_{00} \\ a_{10} & a_{11} & b_{10} \end{bmatrix}_{2 \times 3} \]

    Considering that we want to transform a 2D vector \(X = \begin{bmatrix}x \\ y\end{bmatrix}\) by using \(A\) and \(B\), we can do it equivalently with:

    \(T = A \cdot \begin{bmatrix}x \\ y\end{bmatrix} + B\) or \(T = M \cdot [x, y, 1]^{T}\)

    \[T = \begin{bmatrix} a_{00}x + a_{01}y + b_{00} \\ a_{10}x + a_{11}y + b_{10} \end{bmatrix}\]

How do we get an Affine Transformation?

  1. Excellent question. We mentioned that an Affine Transformation is basically a relation between two images. The information about this relation can come, roughly, in two ways:
    1. We know both \(X\) and T and we also know that they are related. Then our job is to find \(M\)
    2. We know \(M\) and \(X\). To obtain \(T\) we only need to apply \(T = M \cdot X\). Our information for \(M\) may be explicit (i.e. have the 2-by-3 matrix) or it can come as a geometric relation between points.
  2. Let's explain a little bit better (b). Since \(M\) relates 02 images, we can analyze the simplest case in which it relates three points in both images. Look at the figure below:

    the points 1, 2 and 3 (forming a triangle in image 1) are mapped into image 2, still forming a triangle, but now they have changed notoriously. If we find the Affine Transformation with these 3 points (you can choose them as you like), then we can apply this found relation to the whole pixels in the image.

Code

  1. What does this program do?
    • Loads an image
    • Applies an Affine Transform to the image. This Transform is obtained from the relation between three points. We use the function cv::warpAffine for that purpose.
    • Applies a Rotation to the image after being transformed. This rotation is with respect to the image center
    • Waits until the user exits the program
  2. The tutorial code's is shown lines below. You can also download it from here
    #include <iostream>
    using namespace cv;
    using namespace std;
    const char* source_window = "Source image";
    const char* warp_window = "Warp";
    const char* warp_rotate_window = "Warp + Rotate";
    int main( int, char** argv )
    {
    Point2f srcTri[3];
    Point2f dstTri[3];
    Mat rot_mat( 2, 3, CV_32FC1 );
    Mat warp_mat( 2, 3, CV_32FC1 );
    Mat src, warp_dst, warp_rotate_dst;
    src = imread( argv[1], IMREAD_COLOR );
    warp_dst = Mat::zeros( src.rows, src.cols, src.type() );
    srcTri[0] = Point2f( 0,0 );
    srcTri[1] = Point2f( src.cols - 1.f, 0 );
    srcTri[2] = Point2f( 0, src.rows - 1.f );
    dstTri[0] = Point2f( src.cols*0.0f, src.rows*0.33f );
    dstTri[1] = Point2f( src.cols*0.85f, src.rows*0.25f );
    dstTri[2] = Point2f( src.cols*0.15f, src.rows*0.7f );
    warp_mat = getAffineTransform( srcTri, dstTri );
    warpAffine( src, warp_dst, warp_mat, warp_dst.size() );
    Point center = Point( warp_dst.cols/2, warp_dst.rows/2 );
    double angle = -50.0;
    double scale = 0.6;
    rot_mat = getRotationMatrix2D( center, angle, scale );
    warpAffine( warp_dst, warp_rotate_dst, rot_mat, warp_dst.size() );
    namedWindow( source_window, WINDOW_AUTOSIZE );
    imshow( source_window, src );
    namedWindow( warp_window, WINDOW_AUTOSIZE );
    imshow( warp_window, warp_dst );
    namedWindow( warp_rotate_window, WINDOW_AUTOSIZE );
    imshow( warp_rotate_window, warp_rotate_dst );
    waitKey(0);
    return 0;
    }

Explanation

  1. Declare some variables we will use, such as the matrices to store our results and 2 arrays of points to store the 2D points that define our Affine Transform.
    Point2f srcTri[3];
    Point2f dstTri[3];
    Mat rot_mat( 2, 3, CV_32FC1 );
    Mat warp_mat( 2, 3, CV_32FC1 );
    Mat src, warp_dst, warp_rotate_dst;
  2. Load an image:
    src = imread( argv[1], 1 );
  3. Initialize the destination image as having the same size and type as the source:
    warp_dst = Mat::zeros( src.rows, src.cols, src.type() );
  4. Affine Transform: As we explained lines above, we need two sets of 3 points to derive the affine transform relation. Take a look:
    srcTri[0] = Point2f( 0,0 );
    srcTri[1] = Point2f( src.cols - 1, 0 );
    srcTri[2] = Point2f( 0, src.rows - 1 );
    dstTri[0] = Point2f( src.cols*0.0, src.rows*0.33 );
    dstTri[1] = Point2f( src.cols*0.85, src.rows*0.25 );
    dstTri[2] = Point2f( src.cols*0.15, src.rows*0.7 );
    You may want to draw the points to make a better idea of how they change. Their locations are approximately the same as the ones depicted in the example figure (in the Theory section). You may note that the size and orientation of the triangle defined by the 3 points change.
  5. Armed with both sets of points, we calculate the Affine Transform by using OpenCV function cv::getAffineTransform :
    warp_mat = getAffineTransform( srcTri, dstTri );
    We get as an output a \(2 \times 3\) matrix (in this case warp_mat)
  6. We apply the Affine Transform just found to the src image

    warpAffine( src, warp_dst, warp_mat, warp_dst.size() );

    with the following arguments:

    • src: Input image
    • warp_dst: Output image
    • warp_mat: Affine transform
    • warp_dst.size(): The desired size of the output image

    We just got our first transformed image! We will display it in one bit. Before that, we also want to rotate it...

  7. Rotate: To rotate an image, we need to know two things:

    1. The center with respect to which the image will rotate
    2. The angle to be rotated. In OpenCV a positive angle is counter-clockwise
    3. Optional: A scale factor

    We define these parameters with the following snippet:

    Point center = Point( warp_dst.cols/2, warp_dst.rows/2 );
    double angle = -50.0;
    double scale = 0.6;
  8. We generate the rotation matrix with the OpenCV function cv::getRotationMatrix2D , which returns a \(2 \times 3\) matrix (in this case rot_mat)
    rot_mat = getRotationMatrix2D( center, angle, scale );
  9. We now apply the found rotation to the output of our previous Transformation.
    warpAffine( warp_dst, warp_rotate_dst, rot_mat, warp_dst.size() );
  10. Finally, we display our results in two windows plus the original image for good measure:
    namedWindow( source_window, WINDOW_AUTOSIZE );
    imshow( source_window, src );
    namedWindow( warp_window, WINDOW_AUTOSIZE );
    imshow( warp_window, warp_dst );
    namedWindow( warp_rotate_window, WINDOW_AUTOSIZE );
    imshow( warp_rotate_window, warp_rotate_dst );
  11. We just have to wait until the user exits the program

Result

  1. After compiling the code above, we can give it the path of an image as argument. For instance, for a picture like:

    after applying the first Affine Transform we obtain:

    and finally, after applying a negative rotation (remember negative means clockwise) and a scale factor, we get:

cv::Mat::rows
int rows
the number of rows and columns or (-1, -1) when the matrix has more than 2 dimensions
Definition: mat.hpp:1959
cv::Point_< float >
imgproc.hpp
cv::IMREAD_COLOR
@ IMREAD_COLOR
If set, always convert image to the 3 channel BGR color image.
Definition: imgcodecs.hpp:67
cv::getAffineTransform
Mat getAffineTransform(const Point2f src[], const Point2f dst[])
Calculates an affine transform from three pairs of the corresponding points.
cv::Mat::zeros
static MatExpr zeros(int rows, int cols, int type)
Returns a zero array of the specified size and type.
cv::hal::warpAffine
void warpAffine(int src_type, const uchar *src_data, size_t src_step, int src_width, int src_height, uchar *dst_data, size_t dst_step, int dst_width, int dst_height, const double M[6], int interpolation, int borderType, const double borderValue[4])
cv::warpAffine
void warpAffine(InputArray src, OutputArray dst, InputArray M, Size dsize, int flags=INTER_LINEAR, int borderMode=BORDER_CONSTANT, const Scalar &borderValue=Scalar())
Applies an affine transformation to an image.
cv::Point2f
Point_< float > Point2f
Definition: types.hpp:181
cv::waitKey
int waitKey(int delay=0)
Waits for a pressed key.
highgui.hpp
cv::namedWindow
void namedWindow(const String &winname, int flags=WINDOW_AUTOSIZE)
Creates a window.
cv::imread
Mat imread(const String &filename, int flags=IMREAD_COLOR)
Loads an image from a file.
cv::Mat::cols
int cols
Definition: mat.hpp:1959
cv::getRotationMatrix2D
Mat getRotationMatrix2D(Point2f center, double angle, double scale)
Calculates an affine matrix of 2D rotation.
cv::Mat::size
MatSize size
Definition: mat.hpp:1978
imgcodecs.hpp
cv::imshow
void imshow(const String &winname, InputArray mat)
Displays an image in the specified window.
cv::Point
Point2i Point
Definition: types.hpp:183
cv::Mat
n-dimensional dense array class
Definition: mat.hpp:741
cv::Mat::type
int type() const
Returns the type of a matrix element.
cv
Definition: affine.hpp:52
cv::WINDOW_AUTOSIZE
@ WINDOW_AUTOSIZE
the user cannot resize the window, the size is constrainted by the image displayed.
Definition: highgui.hpp:184
CV_32FC1
#define CV_32FC1
Definition: interface.h:112