Category Archives: Computer Vision

Open source Structure-from-Motion and Multi-View Stereo tools with C++

Structure-from-Motion (SFM) is genuinely an interesting topic in computer vision, Basically making 3D structure from something 2D is absolutely mesmerizing 🙂

There two open source yet very robust tools for SFM, which sometimes compiling them might be complicated, here I will share my experience with you.




Download the glew from SF at Do NOT get it from Github as to seems to have some problems.



Install DevIl Image library

open makefile and enable siftgpu_enable_cuda

now go to bin directory and to your vsfm bin directory


you can see the some of my results here:

Note: if you don’t have cuda or Nvidia driver you can use your CPU but then you have to limit your file image size and sift binary should be available in your vsfm/bin directory.

Download and make it and copy the sift binary to vsfm/bin.


The installation fo this one is almost straightforward except you need the latest version of Ceres Solver (do not use the one binary package from Ubuntu they are old and they won’t work). So download and make and install the Ceres Solver using the following:

Now in the colmap CMakeList.txt add the following line:

just before “find_package(Ceres REQUIRED)”

and now you can make and install it. You can see some of my results here:

In the next post, I will show you how can you do that in OpenCV.

Computing OpticalFlow with OpenCV

There are two main approaches for computing optical flow, computing optical flow for all pixel or computing optical flow for corners.

1. Computing optical flow for all pixel

2. Computing optical flow for corners



Finding Homography Matrix using Singular-value Decomposition and RANSAC in OpenCV and Matlab


Solving a Homography problem leads to solving a set of homogeneous linear equations such below:

-x1 & -y1 & -1 & 0 & 0 & 0 & x1*xp1 & y1*xp1 & xp1\\
0 & 0 & 0 & -x1 & -y1 & -1 & x1*yp1 & y1*yp1 &  yp1\\
-x2 & -y2 & -1 & 0 & 0 & 0 & x2*xp2 & y2*xp2 & xp2\\
0 & 0 & 0 & -x2 & -y2 & -1 & x2*yp2 & y2*yp2 & yp2\\
-x3 & -y3 & -1 & 0 & 0 & 0 & x3*xp3 & y3*xp3 & xp3\\
0 & 0 & 0 & -x3 & -y3 & -1 & x3*yp3 & y3*yp3 & yp3\\
-x4 & -y4 & -1 & 0 & 0 & 0 & x4*xp4 & y4*xp4 & xp4\\
0 & 0 & 0 & -x4 & -y4 & -1 & x4*yp4 & y4*yp4 & yp4\\
\right) *H=0

\( H^{*} \underset{H}{\mathrm{argmin}}= \|AH\|^{2}  \) subject to \( \|H\|=1 \)

We can’t use least square since it’s a homogeneous linear equations (the other side of equation is 0 therfore we can’t just multyly it by the psudo inverse). To solve this problem we use Singular-value Decomposition (SVD).

For any given matrix \( A_{M{\times}N} \)

\underbrace{\mathbf{A}}_{M \times N} = \underbrace{\mathbf{U}}_{M \times M} \times \underbrace{\mathbf{\Sigma}}_{M\times N} \times \underbrace{\mathbf{V}^{\text{T}}}_{N \times N}

\(U\) is an \( {M\times M} \) matrix with orthogonal matrix (columns are eigen vectors of A).
\(\Sigma\) is an \( {M\times N} \) matrix with non-negative entries, termed the singular values  (diagonal entries are eigen values of A).
\(V \) is an \( {N\times N} \) orthogonal matrix.

\( H^{*} \) is the last column of \(V \)

Finding homography matrix in Matlab between 4 pairs of points

Finding homography matrix in OpenCV between 4 pairs of points


Finding homography matrix in OpenCV between N pairs of points

Applying homography matrix in OpenCV


Finding Affine Transform with Linear Least Squares

\( \)

linear least squares is a method of fitting a model to data in which the relation between data and unknown paramere can be expressed in a linear form.
\( Ax=b\)
\( X^{*}= \underset{x}{\mathrm{argmin}}= \|Ax-b\|^{2} =(A^{T}A)^{-1}A^{T}b \)

And testing the code:


Peak Signal-to-Noise Ratio (PSNR) in Image using OpenCV and Matlab


Peak signal-to-noise ratio (PSNR) shows the ratio between the maximum possible power of a signal and the power of the same image with noise. PSNR is usually expressed in logarithmic decibel scale.

\( MSE =1/m*n \sum_{i=0}^{m-1} \sum_{j=0}^{n-1}  [   Image( i,j)  -NoisyImage( i,j)  ]  ^2   \)
\( PSNR =10* \log_{10} \Bigg(MAXI^2/MSE\Bigg)  \)

MSE is Mean Square Error and MAXI is the maximum possible pixel value of the image. For instance if the image is uint8, it will be 255.


Ref [1],[2], [3], 4

Seam Carving Algorithm for Content-Aware Image Resizing with Matlab Code

Seam carving is an algorithm for resizing images while keeping the most prominent and conspicuous pixels in the image. The important pixels in an image are usually those who are located over horizontal or vertical edges, so to throw away some pixels we first find horizontal and vertical edges and store their magnitude as pixel energy. Then we use dynamic programming to find a path which contains the minimum energy level and we drop those pixels. We iteratively do this until we got our image in the desired size.

our image is 720×480,

let’s drop some rows and resize it to 640×480:

we can drop some columns and resize it to 720×320:


HARRIS Corner detector explained


Eigenvectors of PCL pointcloud (moment of inertia)

The snippet below shows how to computer PCA eigenvectors and eigenvalues (in this case could be interpreted as the moment of inertia).


A GUI ROS-package for cropping pcl pointcloud with dynamic reconfigure

This ROS package enables you to crop the scene from Kinect (input topic type: PCL pointcloud). You can even enable fitting a plane to remove the ground from the scene and by adjusting correct parameter you can get the desired object from the scene. code available on my Github.

White dots are original scene and rgb dots are from the cropped cloud. Values for the volume of cuboid are coming from sliders.

List of OpenCv matrix types and mapping numbers

ref: Special thanks to this post.