Category Archives: Image Processing

Human detection and Pose Estimation with Deep Learning for Sport Analysis

Pose estimation and tracking human is one the key step in sports analysis. Here is in this work I used openpose for analysis of player in a Bundesliga game HSV Hamburg vs Bayer München. Warning: the video might be disturbing for HSV fans 🙂

 

Original Video

Analyzed Video

Original Video

Analyzed Video

Original Video

Analyzed Video

Original Video

Analyzed Video

Vaganova_Ballet_Academy from Behnam Asadi on Vimeo.

 

Original Video

Analyzed Video

 

 

Thiem_Zverev from Behnam Asadi on Vimeo.

RGBD PCL point cloud from Stereo vision with ROS and OpenCV

In my other tutorial, I showed you how to calibrate you stereo camera. After Calibration, we can get disparity map and  RGBD PCL point cloud from our stereo camera cool huh 🙂

1)Save the following text under “stereo_usb_cam_stream_publisher.launch

2) Then run the following node to publish both cameras and camera info (calibration matrix)

3) Run the following to rectify image and compute the disparity map:

Super important: If you have USB cam with some delays you should add the following “_approximate_sync:=true”

4) Let’s view everything:

Super important: If you have USB cam with some delays you should add the following “_approximate_sync:=True _queue_size:=10”

5) Running rqt graph should give you the following:

6) Run the to configure the matching algorithm parameter:

7) PCL pointcloud in RVIZ

Stereo Camera Calibration with ROS and OpenCV

In this tutorial, I’m gonna show you stereo camera calibration with ROS and OpenCV. So you need a pair of cameras, I bought a pair of this USB webcam which is okay for this task.

1)Save the following text under “stereo_usb_cam_stream_publisher.launch

2)Then run the following node to publish both cameras.

3)Now call the calibration node:

Super important:

If you have USB cam with some delays you should add the following “–no-service-check –approximate=0.1”

4)Pose the chess board in different position, and then click on the calibrate and save button.

5) The result gonna be store at /tmp/calibrationdata.tar.gz. Unzip the file and save it under “/home/<username>/.ros/stereo_camera_info

Open source Structure-from-Motion and Multi-View Stereo tools with C++

Structure-from-Motion (SFM) is genuinely an interesting topic in computer vision, Basically making 3D structure from something 2D is absolutely mesmerizing 🙂

There two open source yet very robust tools for SFM, which sometimes compiling them might be complicated, here I will share my experience with you.

1)VisualSFM

Prerequisite:

1)Glew

Download the glew from SF at http://glew.sourceforge.net/. Do NOT get it from Github as to seems to have some problems.

2)SiftGPU

Prerequisite:

Install DevIl Image library

open makefile and enable siftgpu_enable_cuda

now go to bin directory and libsiftgpu.so to your vsfm bin directory

VSFM

you can see the some of my results here:

Note: if you don’t have cuda or Nvidia driver you can use your CPU but then you have to limit your file image size and sift binary should be available in your vsfm/bin directory.

Download http://www.cs.ubc.ca/~lowe/keypoints/siftDemoV4.zip and make it and copy the sift binary to vsfm/bin.

2)Colmap

The installation fo this one is almost straightforward except you need the latest version of Ceres Solver (do not use the one binary package from Ubuntu they are old and they won’t work). So download and make and install the Ceres Solver using the following:

Now in the colmap CMakeList.txt add the following line:

just before “find_package(Ceres REQUIRED)”

and now you can make and install it. You can see some of my results here:

In the next post, I will show you how can you do that in OpenCV.

Finding Homography Matrix using Singular-value Decomposition and RANSAC in OpenCV and Matlab

\(\)

Solving a Homography problem leads to solving a set of homogeneous linear equations such below:

\begin{equation}
\left(
\begin{array}{ccccccccc}
-x1 & -y1 & -1 & 0 & 0 & 0 & x1*xp1 & y1*xp1 & xp1\\
0 & 0 & 0 & -x1 & -y1 & -1 & x1*yp1 & y1*yp1 &  yp1\\
-x2 & -y2 & -1 & 0 & 0 & 0 & x2*xp2 & y2*xp2 & xp2\\
0 & 0 & 0 & -x2 & -y2 & -1 & x2*yp2 & y2*yp2 & yp2\\
-x3 & -y3 & -1 & 0 & 0 & 0 & x3*xp3 & y3*xp3 & xp3\\
0 & 0 & 0 & -x3 & -y3 & -1 & x3*yp3 & y3*yp3 & yp3\\
-x4 & -y4 & -1 & 0 & 0 & 0 & x4*xp4 & y4*xp4 & xp4\\
0 & 0 & 0 & -x4 & -y4 & -1 & x4*yp4 & y4*yp4 & yp4\\
\end{array}
\right) *H=0
\end{equation}

\( H^{*} \underset{H}{\mathrm{argmin}}= \|AH\|^{2}  \) subject to \( \|H\|=1 \)

We can’t use least square since it’s a homogeneous linear equations (the other side of equation is 0 therfore we can’t just multyly it by the psudo inverse). To solve this problem we use Singular-value Decomposition (SVD).

For any given matrix \( A_{M{\times}N} \)

\begin{equation}
\underbrace{\mathbf{A}}_{M \times N} = \underbrace{\mathbf{U}}_{M \times M} \times \underbrace{\mathbf{\Sigma}}_{M\times N} \times \underbrace{\mathbf{V}^{\text{T}}}_{N \times N}
\end{equation}

\(U\) is an \( {M\times M} \) matrix with orthogonal matrix (columns are eigen vectors of A).
\(\Sigma\) is an \( {M\times N} \) matrix with non-negative entries, termed the singular values  (diagonal entries are eigen values of A).
\(V \) is an \( {N\times N} \) orthogonal matrix.

\( H^{*} \) is the last column of \(V \)

Finding homography matrix in Matlab between 4 pairs of points

Finding homography matrix in OpenCV between 4 pairs of points

 

Finding homography matrix in OpenCV between N pairs of points

Applying homography matrix in OpenCV

 

Finding Affine Transform with Linear Least Squares

\( \)

linear least squares is a method of fitting a model to data in which the relation between data and unknown paramere can be expressed in a linear form.
\( Ax=b\)
\( X^{*}= \underset{x}{\mathrm{argmin}}= \|Ax-b\|^{2} =(A^{T}A)^{-1}A^{T}b \)

And testing the code:

 

Peak Signal-to-Noise Ratio (PSNR) in Image using OpenCV and Matlab

\(\)\(\)

Peak signal-to-noise ratio (PSNR) shows the ratio between the maximum possible power of a signal and the power of the same image with noise. PSNR is usually expressed in logarithmic decibel scale.

\( MSE =1/m*n \sum_{i=0}^{m-1} \sum_{j=0}^{n-1}  [   Image( i,j)  -NoisyImage( i,j)  ]  ^2   \)
\( PSNR =10* \log_{10} \Bigg(MAXI^2/MSE\Bigg)  \)

MSE is Mean Square Error and MAXI is the maximum possible pixel value of the image. For instance if the image is uint8, it will be 255.

 

Ref [1],[2], [3], 4

Seam Carving Algorithm for Content-Aware Image Resizing with Matlab Code

Seam carving is an algorithm for resizing images while keeping the most prominent and conspicuous pixels in the image. The important pixels in an image are usually those who are located over horizontal or vertical edges, so to throw away some pixels we first find horizontal and vertical edges and store their magnitude as pixel energy. Then we use dynamic programming to find a path which contains the minimum energy level and we drop those pixels. We iteratively do this until we got our image in the desired size.

our image is 720×480,

let’s drop some rows and resize it to 640×480:

we can drop some columns and resize it to 720×320:

 

HARRIS Corner detector explained

 

A GUI ROS-package for cropping pcl pointcloud with dynamic reconfigure

This ROS package enables you to crop the scene from Kinect (input topic type: PCL pointcloud). You can even enable fitting a plane to remove the ground from the scene and by adjusting correct parameter you can get the desired object from the scene. code available on my Github.

White dots are original scene and rgb dots are from the cropped cloud. Values for the volume of cuboid are coming from sliders.