Two wheels differential drive robot (with two caster wheels).
List of installed sensors:
• Velodyne VLP-16.
• Velodyne HDL-32E.
• Hokuyo Laser scanner.
• IMU.
• Microsoft Kinect/Asus Xtion Pro.
• RGB Camera.
You can manually control the robot with Joystick controller for mapping robot environment.
Autonomous navigation is possible by setting goal pose.
|
1 2 3 4 5 6 7 |
#include <sensor_msgs/point_cloud_conversion.h> sensor_msgs::PointCloud2 point_cloud2; sensor_msgs::PointCloud point_cloud; sensor_msgs::convertPointCloudToPointCloud2(point_cloud, point_cloud2); sensor_msgs::convertPointCloud2ToPointCloud(point_cloud2, point_cloud); |
|
1 |
#include <pcl/conversions.h> |
|
1 2 3 4 5 6 |
pcl::PCLPointCloud2 point_cloud2; pcl::PointCloud<pcl::PointXYZ> point_cloud; pcl::fromPCLPointCloud2( point_cloud2, point_cloud); pcl::toPCLPointCloud2(point_cloud, point_cloud2); |
more about pcl point cloud and conversion
For this work, first I loaded the RRBot in Gazebo and
launched Its joints controller, then I sent a periodic signal to the robot such that the laser scanner mounted on the robot swings.
In the following, I assembled the incoming laser scans with the transformation from tf and created PCL point cloud.
Install the necessary package:
|
1 |
sudo apt-get install ros-kinetic-laser-assembler ros-kinetic-laser-geometry ros-kinetic-rqt-controller-manager ros-kinetic-ros-controllers ros-kinetic-gazebo-ros kinetic-gazebo-ros-control |
Add the path to ROS_PACKAGE_PATH
|
1 2 |
source /opt/ros/kinetic/setup.sh export ROS_PACKAGE_PATH=~/.softwares/gazebo_ros_demos/:$ROS_PACKAGE_PATH |
|
1 2 3 4 5 |
roslaunch rrbot_gazebo rrbot_world.launch roslaunch rrbot_control rrbot_control.launch rosrun rqt_gui rqt_gui |
set the second joint value
(/rrbot/joint2_position_controller/command) into (pi/4)+(1*pi/4)*sin(i/40)*sin(i/40)
and the frequency into 50 Hz.
Laser Assembler:
|
1 2 3 |
mkdir -p catkin_ws/src && cd catkin_ws/src git clone https://github.com/behnamasadi/laser_assembler cd ../ && catkin_make |
Finally, run:
|
1 2 |
cd /home/behnam/catkin_ws/devel/lib/laser_assembler ./laser_assembler_service_caller |
Create a launch file and save the following lines to it and run it with roslaunch:
|
1 2 3 4 5 6 7 |
<launch> <node type="laser_scan_assembler" pkg="laser_assembler" name="my_assembler"> <remap from="scan" to="/rrbot/laser/scan"/> <param name="max_clouds" type="int" value="400" /> <param name="fixed_frame" type="string" value="world" /> </node> </launch> |
source code at my git hub
Real-time object recognition and 6DOF pose estimation with PCL pointcloud and ROS. This approach is based on normal coherence
In this tutorial, I’m gonna show you how to do object recognition and 6DOF pose estimation in real-time based on Linemod algorithm with ROS and PCL pointcloud. First, you need to install ork:
|
1 |
sudo apt-get install ros-indigo-object-recognition-ros-visualization ros-indigo-object-recognition-msgs ros-indigo-object-recognition-core |
Then add the model of your object for tracking to Couch DB:
|
1 |
rosrun object_recognition_core object_add.py -n coke -d "A universal can of coke" --commit |
|
1 |
rosrun object_recognition_core mesh_add.py <YOUR_OBJECT_ID_FROM_PREVIOUS_STEP> /home/behnam/ork_tutorials/data/coke.obj --commit |
You need to install Kinect driver, if you don’t know how, follow my tutorial about that.
|
1 |
roslaunch openni_launch openni.launch |
|
1 |
rosrun rviz rviz |
|
1 |
rosrun rqt_reconfigure rqt_reconfigure |
And select /camera/driver from the drop-down menu. Enable the depth_registration checkbox. Now go back to RViz, and change your PointCloud2 topic to /camera/depth_registered/points.
|
1 |
rosrun object_recognition_core detection -c `rospack find object_recognition_linemod`/conf/detection.ros.ork |
In my other tutorial, I showed you how to install your code in an arbitrary location in Unix/ Linux systems. In this tutorial, I’m gonna show you how to find them after installation. Here I have two examples: OpenCV, PCL point cloud
I can assume that you have compiled and installed them using the following command:
|
1 |
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX:PATH=~/usr .. && make -j8 all install |
1)PCL point cloud
put this line in your CMake file:
|
1 |
set(PCL_DIR "$ENV{HOME}/usr/share/pcl-1.8/") |
and check if everything is correct:
|
1 2 3 4 5 6 7 8 9 |
find_package(PCL 1.8.0 REQUIRED COMPONENTS common octree io kdtree search sample_consensus filters 2d features registration geometry visualization outofcore surface keypoints ml segmentation recognition people tracking stereo) MESSAGE("PCL_FOUND:" ${PCL_FOUND}) MESSAGE("PCL_INCLUDE_DIRS:" ${PCL_LIBRARY_DIRS}) MESSAGE("PCL_LIBRARIES:" ${PCL_LIBRARIES}) MESSAGE("PCL_LIBRARY_DIRS:" ${PCL_LIBRARY_DIRS}) MESSAGE("PCL_VERSION:" ${PCL_VERSION}) MESSAGE("PCL_COMPONENTS:" ${PCL_COMPONENTS}) MESSAGE("PCL_DEFINITIONS:" ${PCL_DEFINITIONS}) |
2)OpenCV
put this line in your CMake file:
|
1 |
set(OpenCV_DIR "$ENV{HOME}/usr/share/OpenCV/") |
|
1 2 3 4 5 6 |
FIND_PACKAGE( OpenCV REQUIRED ) MESSAGE("OpenCV_DIR: " ${OpenCV_DIR}) MESSAGE("OpenCV_VERSION: " ${OpenCV_VERSION}) MESSAGE("OpenCV_INCLUDE_DIRS: " ${OpenCV_INCLUDE_DIRS}) MESSAGE("OpenCV_LIBS: " ${OpenCV_LIBS}) |
|
1 |
cmake -DCMAKE_BUILD_TYPE=Release -DOpenCV_DIR=~/usr/share/OpenCV |
registration is aligning 3D point cloud on each other such that it gives you a complete model. To achieve this, you need to find the relative positions and orientations of each point cloud, such that you maximize the overlapping intersecting areas between them [1].
So I got the idea from here and I implemented a software based on that. In the following, you can see the main idea and the step I took and finally the results:
Main Flowchart of pairwise point cloud registration
image source: image has been taken from http://pointclouds.org/documentation/tutorials/registration_api.php
1)Importing point cloud acquired from different angles, down sampling, selecting keypoint extractor method SIFT, NARF, Harris, SUSAN and respected parameters
2)Selected keypoints are highlighted in green, for each keypoint a descriptor (PFH or FPFH) is estimated
3) Correspondences between keypoint descriptor are estimated (histogram distance) and correspondent points are connected.
4) Correspondent points are rejected via several algorithm and from the remained correspondent points a 4×4 transformation matrix is computed
5) 4×4 transformation matrix is used for initial estimation of ICP algorithm and two point clouds are merged into one
In this work at first, I recognize the object in the scene and estimate the 6 DOF pose of that. Then I track the object by using particle filter. RGB data acquired from Kinect 2 and turned into PCL pointcloud.
I demonstrate a task several times to the robot. In this case, I move an object (a detergent) over an “S” shape path to get an “S” shape trajectory.
In the following, you can see the result of 6 times repeating the job. Trajectories are very noisy and each repeat gives you a new “S” shape.
Then I compute the GMM (Gaussian mixture model) of trajectory in each dimension. Numbers of kernels can be set by the user or can be determined automatically based on BIC (Bayesian information criterion).
After that, I computed the Gaussian mixture regression to generalize the task and get the learned trajectory.
DTW (Dynamic time warping) can be used to align the trajectories to a reference trajectory to get rid of time difference problem between several trajectories.
Finally, you can see the learned trajectory in black.
All codes have been done with C++ in ROS (indigo-level).
The relation between ROS and PCL point cloud is a little bit complicated, at first, it was part of ROS, then it became a separate project and whole the time you need some serializer and deserializer to send and receive point cloud ROS messages. There are some old deprecated ways to do that and they still exist in the ROS documentation which confuses the programmers, so here is the right API to use:
converting a PCL point cloud to a ROS PCL message
first, include the corresponding headers:
|
1 2 |
#include <sensor_msgs/PointCloud2.h> #include <pcl_conversions/pcl_conversions.h> |
|
1 2 3 4 |
sensor_msgs::PointCloud2 object_msg; pcl::PointCloud::Ptr object_cloud; object_cloud.reset(new pcl::PointCloud); pcl::toROSMsg(*object_cloud.get(),object_msg ); |
|
1 2 3 4 5 |
pcl::PointCloud::Ptr received_cloud_ptr; received_cloud_ptr.reset(new pcl::PointCloud); sensor_msgs::PointCloud2ConstPtr pointcloud_msg; pcl::fromROSMsg(*pointcloud_msg.get(), *received_cloud_ptr.get()); |
