Tutorials Archives - Page 7 of 13

How to find Essential and Fundamental Matrices

RGBD PCL point cloud from Stereo vision with ROS and OpenCV

Leave a Comment / Computer Vision, Image Processing, ROS, Tutorials / admin

In my other tutorial, I showed you how to calibrate you stereo camera. After Calibration, we can get disparity map and RGBD PCL point cloud from our stereo camera cool huh 🙂 1)Save the following text under “stereo_usb_cam_stream_publisher.launch”

<launch>

<arg name="video_device_right" default="/dev/video1" />
<arg name="video_device_left" default="/dev/video2" />

<arg name="image_width" default="640" />
<arg name="image_height" default="480" />

<arg name="right_image_namespace" default="stereo" />
<arg name="left_image_namespace" default="stereo" />

<arg name="right_image_node" default="right" />
<arg name="left_image_node" default="left" />

<arg name="image_topic_name_right" default="image_raw" />
<arg name="camera_info_topic_name_right" default="camera_info" />

<arg name="image_topic_name_left" default="image_raw" />
<arg name="camera_info_topic_name_left" default="camera_info" />



<arg name="left_camera_name" default="left" />
<arg name="right_camera_name" default="right" />

<arg name="left_camera_frame_id" default="left_camera" />
<arg name="right_camera_frame_id" default="right_camera" />

<arg name="framerate" default="30" />

<arg name="camera_info" default="false" />
<arg name="camera_info_path" default="stereo_camera_info" />
	  
<node ns="$(arg right_image_namespace)" name="$(arg right_image_node)" pkg="usb_cam" type="usb_cam_node" output="screen" >
	<param name="video_device" value="$(arg video_device_right)" />
	<param name="image_width" value="$(arg image_width)" />
	<param name="image_height" value="$(arg image_height)"/>
	<param name="pixel_format" value="yuyv" />
	<param name="io_method" value="mmap"/>
	<remap from="/usb_cam/image_raw" to="$(arg image_topic_name_right)" />
	<param name="framerate" value="$(arg framerate)"/> 
	<!-- if camera_info is available, we will use it-->
	<param name="camera_frame_id" value="$(arg right_camera_frame_id)" if="$(arg camera_info)" /> 
	<param name="camera_info_url" value="file://${ROS_HOME}/$(arg camera_info_path)/right.yaml" if="$(arg camera_info)"/>
	<param name="camera_name" value="narrow_stereo/$(arg right_camera_name)" if="$(arg camera_info)"/> 
	<remap from="/usb_cam/camera_info" to="$(arg camera_info_topic_name_right)" if="$(arg camera_info)"/>

</node>

<node ns="$(arg left_image_namespace)" name="$(arg left_image_node)" pkg="usb_cam" type="usb_cam_node" output="screen" >
	<param name="video_device" value="$(arg video_device_left)" />
	<param name="image_width" value="$(arg image_width)" />
	<param name="image_height" value="$(arg image_height)"/>
	<param name="pixel_format" value="yuyv" />
	<param name="io_method" value="mmap"/>
	<param name="framerate" value="$(arg framerate)"/> 
	<remap from="/usb_cam/image_raw" to="$(arg image_topic_name_left)"/>
	<!-- if camera_info is available, we will use it-->
	<param name="camera_frame_id" value="$(arg left_camera_frame_id)" if="$(arg camera_info)"/> 
	<param name="camera_name" value="narrow_stereo/$(arg left_camera_name)" if="$(arg camera_info)"/> 
	<param name="camera_info_url" value="file://${ROS_HOME}/$(arg camera_info_path)/left.yaml" if="$(arg camera_info)"/> 
	<remap from="/usb_cam/camera_info" to="$(arg camera_info_topic_name_left)" if="$(arg camera_info)"/>
</node>
</launch>

</node>

</node>

</launch>

2) Then run the following node to publish both cameras and camera info (calibration matrix)

roslaunch stereo_usb_cam_stream_publisher.launch video_device_right:=/dev/video1 video_device_left:=/dev/video2 image_width:=640 image_height:=480 left_camera_name:=left right_camera_name:=right camera_info:=true camera_info_path:=stereo_camera_info

1	roslaunch stereo_usb_cam_stream_publisher.launch video_device_right:=/dev/video1 video_device_left:=/dev/video2 image_width:=640 image_height:=480 left_camera_name:=left right_camera_name:=right camera_info:=true camera_info_path:=stereo_camera_info

3) Run the

RGBD PCL point cloud from Stereo vision with ROS and OpenCV Read More »

Stereo Camera Calibration with ROS and OpenCV

5 Comments / Computer Vision, Image Processing, ROS, Tutorials / admin

In this tutorial, I’m gonna show you stereo camera calibration with ROS and OpenCV. So you need a pair of cameras, I bought a pair of this USB webcam which is okay for this task. 1)Save the following text under “stereo_usb_cam_stream_publisher.launch”

<launch>

<arg name="video_device_right" default="/dev/video1" />
<arg name="video_device_left" default="/dev/video2" />

<arg name="image_width" default="640" />
<arg name="image_height" default="480" />

<arg name="right_image_namespace" default="stereo" />
<arg name="left_image_namespace" default="stereo" />

<arg name="right_image_node" default="right" />
<arg name="left_image_node" default="left" />

<arg name="image_topic_name_right" default="image_raw" />
<arg name="camera_info_topic_name_right" default="camera_info" />

<arg name="image_topic_name_left" default="image_raw" />
<arg name="camera_info_topic_name_left" default="camera_info" />



<arg name="left_camera_name" default="left" />
<arg name="right_camera_name" default="right" />

<arg name="left_camera_frame_id" default="left_camera" />
<arg name="right_camera_frame_id" default="right_camera" />

<arg name="framerate" default="30" />

<arg name="camera_info" default="false" />
<arg name="camera_info_path" default="stereo_camera_info" />
	  
<node ns="$(arg right_image_namespace)" name="$(arg right_image_node)" pkg="usb_cam" type="usb_cam_node" output="screen" >
	<param name="video_device" value="$(arg video_device_right)" />
	<param name="image_width" value="$(arg image_width)" />
	<param name="image_height" value="$(arg image_height)"/>
	<param name="pixel_format" value="yuyv" />
	<param name="io_method" value="mmap"/>
	<remap from="/usb_cam/image_raw" to="$(arg image_topic_name_right)" />
	<param name="framerate" value="$(arg framerate)"/> 
	<!-- if camera_info is available, we will use it-->
	<param name="camera_frame_id" value="$(arg right_camera_frame_id)" if="$(arg camera_info)" /> 
	<param name="camera_info_url" value="file://${ROS_HOME}/$(arg camera_info_path)/right.yaml" if="$(arg camera_info)"/>
	<param name="camera_name" value="narrow_stereo/$(arg right_camera_name)" if="$(arg camera_info)"/> 
	<remap from="/usb_cam/camera_info" to="$(arg camera_info_topic_name_right)" if="$(arg camera_info)"/>

</node>

<node ns="$(arg left_image_namespace)" name="$(arg left_image_node)" pkg="usb_cam" type="usb_cam_node" output="screen" >
	<param name="video_device" value="$(arg video_device_left)" />
	<param name="image_width" value="$(arg image_width)" />
	<param name="image_height" value="$(arg image_height)"/>
	<param name="pixel_format" value="yuyv" />
	<param name="io_method" value="mmap"/>
	<param name="framerate" value="$(arg framerate)"/> 
	<remap from="/usb_cam/image_raw" to="$(arg image_topic_name_left)"/>
	<!-- if camera_info is available, we will use it-->
	<param name="camera_frame_id" value="$(arg left_camera_frame_id)" if="$(arg camera_info)"/> 
	<param name="camera_name" value="narrow_stereo/$(arg left_camera_name)" if="$(arg camera_info)"/> 
	<param name="camera_info_url" value="file://${ROS_HOME}/$(arg camera_info_path)/left.yaml" if="$(arg camera_info)"/> 
	<remap from="/usb_cam/camera_info" to="$(arg camera_info_topic_name_left)" if="$(arg camera_info)"/>
</node>
</launch>

</node>

</node>

</launch>

2)Then run the following node to publish both cameras.

roslaunch stereo_usb_cam_stream_publisher.launch video_device_right:=/dev/video1 video_device_left:=/dev/video2 image_width:=640 image_height:=480 left_camera_name:=left right_camera_name:=right

1	roslaunch stereo_usb_cam_stream_publisher.launch video_device_right:=/dev/video1 video_device_left:=/dev/video2 image_width:=640 image_height:=480 left_camera_name:=left right_camera_name:=right

3)Now call the calibration node:

Stereo Camera Calibration with ROS and OpenCV Read More »

How to use OpenCV 3 Contribution Modules

Leave a Comment / C++, Tutorials / admin

OpenCV Contribution Modules are developed code from the community of OpenCV and are not very stable but after they got stable they will be part f OpenCV. Anyhow, there are a lot of cool modules there (fuzzy logic, sfm, cnn, …) that made me have look at them and try them. gflags

git clone https://github.com/gflags/gflags
cd gflags
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX:PATH=~/usr 
cmake --build build -j8
cmake --install build

git clone https://github.com/gflags/gflags

cd gflags

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX:PATH=~/usr

cmake --build build -j8

cmake --install build

2) glog

How to use OpenCV 3 Contribution Modules Read More »

Open source Structure-from-Motion and Multi-View Stereo tools with C++

Leave a Comment / C++, Computer Vision, Image Processing, Linear Algebra, Tutorials / admin

Structure-from-Motion (SFM) is genuinely an interesting topic in computer vision, Basically making 3D structure from something 2D is absolutely mesmerizing 🙂 There two open source yet very robust tools for SFM, which sometimes compiling them might be complicated, here I will share my experience with you. 1)VisualSFM Prerequisite: 1)Glew Download the glew from SF at http://glew.sourceforge.net/. Do NOT get it from Github as

Open source Structure-from-Motion and Multi-View Stereo tools with C++ Read More »

Computing OpticalFlow with OpenCV

1 Comment / C++, Computer Vision, Tutorials / admin

There are two main approaches for computing optical flow, computing optical flow for all pixel or computing optical flow for corners. 1. Computing optical flow for all pixel 2. Computing optical flow for corners

Computing OpticalFlow with OpenCV Read More »

Finding Relative Between Frames

RANSAC Algorithm parameter explained

2 Comments / Machine Learning, Matlab, Tutorials / admin

In this tutorial I explain the RANSAC algorithm, their corresponding parameters and how to choose the number of samples: N = number of samples e = probability that a point is an outlier s = number of points in a sample p = desired probability that we get a good sample N =log(1-p) /log(1- (1-

RANSAC Algorithm parameter explained Read More »

Finding Homography Matrix using Singular-value Decomposition and RANSAC in OpenCV and Matlab

3 Comments / C++, Computer Vision, Image Processing, Linear Algebra, Matlab, Tutorials / admin

Solving a Homography problem leads to solving a set of homogeneous linear equations such below: \begin{equation} \left( \begin{array}{ccccccccc} -x1 & -y1 & -1 & 0 & 0 & 0 & x1*xp1 & y1*xp1 & xp1\\ 0 & 0 & 0 & -x1 & -y1 & -1 & x1*yp1 & y1*yp1 & yp1\\ -x2 & -y2 &

Finding Homography Matrix using Singular-value Decomposition and RANSAC in OpenCV and Matlab Read More »

Finding Affine Transform with Linear Least Squares

3 Comments / Computer Vision, Image Processing, Linear Algebra, Tutorials / admin

linear least squares is a method of fitting a model to data in which the relation between data and unknown paramere can be expressed in a linear form. $ Ax=b$ $ X^{*}= \underset{x}{\mathrm{argmin}}= \|Ax-b\|^{2} =(A^{T}A)^{-1}A^{T}b $

function M= affine_least_square(x0,y0, x1,y1, x2,y2, xp0,yp0, xp1,yp1, xp2,yp2)
% This function finds the affine transform between three points

% an affine transformation with a 3x3 affine transformation matrix: 
% 
% [M11 M12 M13]
% [M21 M22 M23]
% [M31 M32 M33]

%Since the third row is always [0 0 1] we can skip that.

% [x0 y0 1  0  0  0 ]     [M11]   [xp0]
% [0  0  0  x0 y0 1 ]     [M12]   [yp0] 
% [x1 y1 1  0  0  0 ]     [M13]   [xp1]
% [0  0  0  x1 y1 1 ]  *  [M21] = [yp1] 
% [x2 y2 1  0  0  0 ]     [M22]   [xp2] 
% [0  0  0  x2 y2 1 ]     [M23]   [yp2]
                          
%         A            *    X   =  B
%A -> 6x6
%X -> 6x1 in fact 
%X=A\B


A=[x0 y0 1  0  0  0 ; 0  0  0  x0 y0 1 ; x1 y1 1  0  0  0 ; 0  0  0  x1 y1 1 ; x2 y2 1  0  0  0;0  0  0  x2 y2 1];
B=[xp0; yp0; xp1;  yp1; xp2 ; yp2 ];


% X=A\B; 
% this is what we need but accroding to https://www.mathworks.com/matlabcentral/newsreader/view_thread/170201
%The following is better:
X=pinv(A)*B;

M11 =X(1,1);
M12 =X(2,1);  
M13 =X(3,1);
M21 =X(4,1);
M22 =X(5,1);
M23 =X(6,1);
M=[ M11 M12 M13; M21 M22 M23; 0 0 1];
end

function M= affine_least_square(x0,y0, x1,y1, x2,y2, xp0,yp0, xp1,yp1, xp2,yp2)

% This function finds the affine transform between three points

% an affine transformation with a 3x3 affine transformation matrix:

% [M11 M12 M13]

% [M21 M22 M23]

% [M31 M32 M33]

%Since the third row is always [0 0 1] we can skip that.

% [x0 y0 1 0 0 0 ] [M11] [xp0]

% [0 0 0 x0 y0 1 ] [M12] [yp0]

% [x1 y1 1 0 0 0 ] [M13] [xp1]

% [0 0 0 x1 y1 1 ] * [M21] = [yp1]

% [x2 y2 1 0 0 0 ] [M22] [xp2]

% [0 0 0 x2 y2 1 ] [M23] [yp2]

% A * X = B

%A -> 6x6

%X -> 6x1 in fact

%X=A\B

A=[x0 y0 1 0 0 0 ; 0 0 0 x0 y0 1 ; x1 y1 1 0 0 0 ; 0 0 0 x1 y1 1 ; x2 y2 1 0 0 0;0 0 0 x2 y2 1];

B=[xp0; yp0; xp1; yp1; xp2 ; yp2 ];

% X=A\B;

% this is what we need but accroding to https://www.mathworks.com/matlabcentral/newsreader/view_thread/170201

%The following is better:

X=pinv(A)*B;

M11 =X(1,1);

M12 =X(2,1);

M13 =X(3,1);

M21 =X(4,1);

M22 =X(5,1);

M23 =X(6,1);

M=[ M11 M12 M13; M21 M22 M23; 0 0 1];

end

And testing the code:

clc;
clear all;
%This is a simple test that project 3 point with an affine trasnform and
%then tries to find the transformation:
% tform.T and M' should be equal

%Points are located at:
x0=0;
y0=0; 
x1=1;
y1=0; 
x2=0;
y2=1;

p1=[x0,y0; x1,y1; x2,y2];

%We rotate them (a roll-rotation around Z axes) theta degree, first
%building the rotation matrix
theta=24;
A11=cosd(theta);
A12=-sind(theta);
A21=sind(theta);
A22=cosd(theta);
%Translation part
Tx=1;
Ty=2;

tform = affine2d([A11 A12 0; A21 A22 0; Tx Ty 1]);
fprintf('3x3 transformation matrix:')
tform.T

[x,y]=transformPointsForward(tform,p1(:,1),p1(:,2));
fprintf('transformed points:')
[x,y]


xp0=x(1,1);
yp0=y(1,1);
xp1=x(2,1);
yp1=y(2,1);
xp2=x(3,1);
yp2=y(3,1);



%finding the tranformation matrix from set of points and their respective
%transformed points:
M=affine_least_square(x0,y0, x1,y1, x2,y2, xp0,yp0, xp1,yp1, xp2,yp2);
fprintf('3x3 affine transformation matrix from least_squares:')
M'

tform = affine2d(M');
[x,y]=transformPointsForward(tform,p1(:,1),p1(:,2));
[x,y];

clc;

clear all;

%This is a simple test that project 3 point with an affine trasnform and

%then tries to find the transformation:

% tform.T and M' should be equal

%Points are located at:

x0=0;

y0=0;

x1=1;

y1=0;

x2=0;

y2=1;

p1=[x0,y0; x1,y1; x2,y2];

%We rotate them (a roll-rotation around Z axes) theta degree, first

%building the rotation matrix

theta=24;

A11=cosd(theta);

A12=-sind(theta);

A21=sind(theta);

A22=cosd(theta);

%Translation part

Tx=1;

Ty=2;

tform = affine2d([A11 A12 0; A21 A22 0; Tx Ty 1]);

fprintf('3x3 transformation matrix:')

tform.T

[x,y]=transformPointsForward(tform,p1(:,1),p1(:,2));

fprintf('transformed points:')

[x,y]

xp0=x(1,1);

yp0=y(1,1);

xp1=x(2,1);

yp1=y(2,1);

xp2=x(3,1);

yp2=y(3,1);

%finding the tranformation matrix from set of points and their respective

%transformed points:

M=affine_least_square(x0,y0, x1,y1, x2,y2, xp0,yp0, xp1,yp1, xp2,yp2);

fprintf('3x3 affine transformation matrix from least_squares:')

tform = affine2d(M');

[x,y]=transformPointsForward(tform,p1(:,1),p1(:,2));

[x,y];

3x3 transformation matrix:
ans =

    0.9135   -0.4067         0
    0.4067    0.9135         0
    1.0000    2.0000    1.0000

transformed points:
ans =

    1.0000    2.0000
    1.9135    1.5933
    1.4067    2.9135

3x3 affine transformation matrix from least_squares:
ans =

    0.9135   -0.4067         0
    0.4067    0.9135         0
    1.0000    2.0000    1.0000

3x3 transformation matrix:

ans =

0.9135 -0.4067 0

0.4067 0.9135 0

1.0000 2.0000 1.0000

transformed points:

ans =

1.0000 2.0000

1.9135 1.5933

1.4067 2.9135

3x3 affine transformation matrix from least_squares:

ans =

0.9135 -0.4067 0

0.4067 0.9135 0

1.0000 2.0000 1.0000

Finding Affine Transform with Linear Least Squares Read More »

Cookie	Duration	Description
cookielawinfo-checbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-necessary	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-performance	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy	11 months	The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.