Pose estimation and tracking human is one the key step in sports analysis. Here is in this work I used openpose for analysis of player in a Bundesliga game HSV Hamburg vs Bayer München. Warning: the video might be disturbing for HSV fans 🙂
Vaganova_Ballet_Academy from Behnam Asadi on Vimeo.
Thiem_Zverev from Behnam Asadi on Vimeo.
visit https://developer.nvidia.com/cuda-downloads and download the correct deb file then:
|
1 2 3 4 |
sudo dpkg -i cuda-repo-ubuntu1604_9.1.85-1_amd64.deb sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub sudo apt-get update sudo apt-get install cuda |
|
1 |
sudo apt-get install libatlas-base-dev libatlas-dev |
|
1 |
sudo apt-get install protobuf-compiler libprotobuf-dev libprotoc-dev |
or you can install protobuf v3 it from source:
|
1 2 3 4 5 6 7 8 9 |
export PROTOBUF_ROOT=~/.softwares/protobuf git clone https://github.com/google/protobuf.git $PROTOBUF_ROOT -b '3.2.x' cd $PROTOBUF_ROOT ./autogen.sh ./configure make "-j$(nproc)" make"-j$(nproc)" install DESTDIR=~ cd python python setup.py install --cpp_implementation --user |
|
1 |
sudo apt-get install liblmdb-dev |
|
1 |
sudo apt-get install libleveldb-dev |
|
1 |
sudo apt-get install libhdf5-dev libhdf5-cpp-11 libhdf5-openmpi-dev libhdf5-mpi-dev libhdf5-10 |
|
1 2 3 |
git clone https://github.com/gflags/gflags cd gflags && mkdir build && cd build cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX:PATH=~/usr .. && make -j8 all install |
|
1 2 3 |
git clone https://github.com/google/glog cd glog && mkdir build && cd build cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX:PATH=~/usr .. && make -j8 all install |
|
1 2 3 |
git clone https://github.com/google/snappy cd snappy && mkdir build && cd build cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX:PATH=~/usr .. && make -j8 all install |
|
1 2 3 |
git clone https://github.com/BVLC/caffe cd caffe && mkdir build && cd build cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX:PATH=~/usr .. && make -j8 all install |
This work is similar to the previous work here, but this time I used Single Shot MultiBox Detector (SSD) for car detection. Installation is similar, clone the SSD Caffe:
|
1 2 3 |
git clone https://github.com/weiliu89/caffe.git ssdcaffe cd ssdcaffe git checkout 5365d0dccacd18e65f10e840eab28eb65ce0cda |
add the following lines to your Makefile.config
|
1 2 |
INCLUDE_DIRS += /usr/include/hdf5/serial LIBRARY_DIRS += /usr/lib/x86_64-linux-gnu/hdf5/serial |
and build it:
|
1 |
make && make distribute |
used video_stream_opencv to stream your video:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
<!-- Example of run: roslaunch video_file.launch video_file:=/home/behnam/Desktop/dash_cam.mp4 --> <launch> <!-- launch video stream --> <!--remaping the topic --> <remap from="/videofile/image_raw" to="/image_raw"/> <include file="$(find video_stream_opencv)/launch/camera.launch" > <!-- node name and ros graph name --> <arg name="camera_name" value="videofile" /> <!-- full path to the video file --> <arg name="video_stream_provider" value="$(arg video_file)" /> <!-- throttling the querying of frames to --> <arg name="fps" value="30" /> <!-- setting frame_id --> <arg name="frame_id" value="videofile_frame" /> </include> </launch> |
download the trained model from here and put them in the model directory.
|
1 2 3 4 |
source /opt/ros/kinetic/setup.bash source /home/<user_name>/Autoware/ros/devel/setup.sh export ROS_PACKAGE_PATH=/home/<user_name>/Autoware/:$ROS_PACKAGE_PATH roslaunch cv_tracker ssd.launch |
In my ssd.launch, I have changed my trained network into:
|
1 2 3 4 5 6 7 |
<!-- <arg name="network_definition_file" default="$(env HOME)/ssdcaffe/models/VGGNet/VOC0712/SSD_500x500/deploy.prototxt"/> <arg name="pretrained_model_file" default="$(env HOME)/ssdcaffe/models/VGGNet/VOC0712/SSD_500x500/VGG_VOC0712_SSD_500x500_iter_60000.caffemodel"/> --> <arg name="network_definition_file" default="$(env HOME)/ssdcaffe/models/VGGNet/ilsvrc15/SSD_500x500/deploy.prototxt"/> <arg name="pretrained_model_file" default="$(env HOME)/ssdcaffe/models/VGGNet/ilsvrc15/SSD_500x500/VGG_ilsvrc15_SSD_500x500_iter_480000.caffemodel"/> |
Now run the following to open rviz:
|
1 2 3 4 |
source /opt/ros/kinetic/setup.bash source /home/<user_name>/Autoware/ros/devel/setup.sh export ROS_PACKAGE_PATH=/home/<user_name>/Autoware/:$ROS_PACKAGE_PATH rosrun rviz rviz |
in the rviz, go to add a panel, and add integrated viewer>ImageViewrPlugin.
Now correct the topic in the added panel and you should see detected cars:
To run this, you need to install Fast-RCNN and Autoware. Just in case you got error regarding hd5f when making Fast-RCNN, add the following lines to your Makefile.config
|
1 2 |
INCLUDE_DIRS += /usr/include/hdf5/serial LIBRARY_DIRS += /usr/lib/x86_64-linux-gnu/hdf5/serial |
Now run the following command to start:
|
1 2 3 4 |
source /opt/ros/kinetic/setup.bash source /home/<user_name>/Autoware/ros/devel/setup.sh export ROS_PACKAGE_PATH=/home/<user_name>/Autoware/:$ROS_PACKAGE_PATH roslaunch cv_tracker rcnn.launch |
if you got an error like :
|
1 |
Check failed: error == cudaSuccess (30 vs. 0) unknown error |
That means your graphics card is not ready or accessible, in my everytime I suspend my notebook I get that error and I need a restart :/
now you should publish your video stream on the topic "image_raw", for that purpose I used video_stream_opencv. Here is my launch file:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
<!-- Example of run: roslaunch video_file.launch video_file:=/home/behnam/Desktop/dash_cam.mp4 --> <launch> <!-- launch video stream --> <!--remaping the topic --> <remap from="/videofile/image_raw" to="/image_raw"/> <include file="$(find video_stream_opencv)/launch/camera.launch" > <!-- node name and ros graph name --> <arg name="camera_name" value="videofile" /> <!-- full path to the video file --> <arg name="video_stream_provider" value="$(arg video_file)" /> <!-- throttling the querying of frames to --> <arg name="fps" value="30" /> <!-- setting frame_id --> <arg name="frame_id" value="videofile_frame" /> </include> </launch> |
Now run the following to open rviz:
|
1 2 3 4 |
source /opt/ros/kinetic/setup.bash source /home/<user_name>/Autoware/ros/devel/setup.sh export ROS_PACKAGE_PATH=/home/<user_name>/Autoware/:$ROS_PACKAGE_PATH rosrun rviz rviz |
in the rviz, go to add a panel, and add integrated viewer>ImageViewrPlugin.
Now correct the topic in the added panel and you should see detected cars:
