User:Redo

From jderobot
Jump to: navigation, search
  • Project Name: Car Classifier
  • Authors: Redouane Kachach (redo.robot [at] gmail [dot] com) and Jose María Cañas Plaza (jmplaza [at] gsyc [dot] es)
  • Academic Year: 2008-2009
  • Degree: Master
  • Jde Version: jde 4.3.0
  • SVN Repository: [1]
  • Tags: classifier, vehicle, car, camera, real time, AVC, robotics, automatic FLOSS (Free Libre Open Source Software)
  • Technology: C, jde suite
  • State: Developing

Contents

Goals 

  The main goal of this project is to develop a robust vision-based vehicle classifier able to categorize the different types of vehicles: 
  cars, motorcycles, trucks, etc. The main sensor that we will be using is a simple stationary camera set up on the road. The classifier should
  be able to track and classify the vehicle on real time, this a very important constraint in this project.

In the block diagram below we can see the different steps involved in the classification process.

Background estimation 

The background detection is an other critical stage. We have developed and implemented a Gaussian based technique that updates
the background at high frequency meeting the real-time constraint. In the following video you can see an example of its execution.
The "Play" button is used to play/stop the video (simulating objects that become static at some moment). You can see how after re-
playing the video the background is recalculated correctly and the static car disappear.

 This is an other video where we can see how the background algorithm works. In this capture the yellow truck is incorporated as
 part of the background after a while being static. Once moving, the background is updated newly and the truck disappear.

Road detection

Automatic mode

 In order to improve the algorithm performance we will be tracking just the ongoing vehicles. Generally the camera vision field may
include other non interesting lanes, our goal is to discard all this extra information so the classification and tracking algorithms
will focus just on the ongoing lane. To deal with this, we have implemented an algorithm for automatic road Detection. This improves 
the rest of processing performance since we will be just processing vehicles on the interesting lanes. Any vehicle or object moving 
in other lanes or out of the road will be discarded. The road is updated frequently to incorporate any change. This stage is very 
important in the whole algorithm since it's the input for the tracking stage.

In the next two videos we can see an example of the automatic road detection algorithm.

Manual mode

 Additionally we offer the user the possibility to choose the road manually. This mode may be used if the automatic road
 detection incorporate more lanes than necessary or when the user wants just classify the vehicles passing in a single lane
 or limited zone.

Gap filling

 The implemented road detection algorithm tries always to fill the gaps that may be produced by any color inconsistency 
 in the background or due to some noise. In the following picture we can see the effect of this process. In the left image
 we can see the result of road detection without filling the gaps meanwhile the right image shows the result when this 
 mechanism is applied.

Real time vehicle tracking (2D tracking) 

In this video we can see an example of the tracker 2D we have implemented. All the vehicles that enter the scene are captured
and followed on real-time. Each vehicle has a unique id that's maintained unique during all the tracking process. The tracking
algorithm uses the background detected in the previous state to detect movement and track the vehicle blobs.
Tracking on two lane highway
Tracking on multi lane highway
Tracking example 1
Tracking example 2
Tracking example 3
Tracking example 4

Working in 3D 

 All the work done in the previous steps was in 2D, from now on we will be working in 3D. The first step is to calibrate the camera.
 JDEROBOT platform gives us the possibility to work either with a real camera or a simulated one (from Gazebo for eg). 

 Using a simulated environment is a big advantage in since we can simulate all the scenario: road, vehicles, speeds and give us 
 the freedom when positioning the camera in the world. An other advantage is that we know already the camera parameters 
 (intrinsic and extrinsic) so we can test our algorithms in an ideal scenarios.

Camera calibration process

Intrinsic parameters estimation 
 The calibration process is the first step when working in 3d world. The calibrated camera is used to classify the vehicles using 
 predefined 3D models, those are projected on the road and compared  with the moving blobs. The camera parameters should be as 
 accurate as possible since the projection output may be used by the classifier in order to determine the vehicle category. 

  To calibrate the camera we used the JDEROBOT calibrator schema. You can find more details about this component in the 
  following link. This schema use a predefined 3D pattern to estimate the camera intrinsic parameters.

https://jderobot.org/index.php/Redo_automatic_camera_calibration

Extrinsic parameters estimation 
  To get an accurate extrinsic parameters (when using real video captures) JDEROBOT offers a schema called extrinsic that
  may help us to adjust the camera extrinsic parameters. As input it receives a 3D world in absolute coordinates then it give
  us the possibility to adjust the camera parameters and see at the same time the result -projecting the simulated world- on the
  real video. To get an accurate measurement Google Earth has been used to estimate the highway dimensions.


You can find more details about this schema in the next link. 

https://www.jderobot.org/index.php/Manual#Extrinsics

  In the next picture we can see an example of the calibration process results.

Vehicle Classification

Vehicle 3D wire modeling

Each vehicle category is represented by a simple wire 3D model. Five different vehicles categories are defined using their dimensions.

  • Motocycles
  • Cars
  • Vans
  • Trucks
  • Road-Trains

In the next picture you can see the models used in this project. Those are basically simple cuboids.

Vehicle matching 

 The 3D models defined above are used in order to represent each vehicle category. A real time matching algorithm has 
 been  implemented in order to match the moving blobs with its correspondent vehicle category. In the next picture we 
 can see an example of the matching process. The areas I and D represent the intersection and the difference respectively 
 between the 3D model projection and the vehicle blobs.

Shadow modelling 

 The shadows have been modelled using a sun model where the user should define the sun position. With this, we are able to estimate
 the light direction and get the shadow projection. This way we can incorporate the shadows when matching the different 3D models
 with its correspondent blobs. In the next picture we can see and example of the shadow estimation of a 3D model. This scenario 
 has been created using the Gazebo simulator.


Classification example using a Gazebo simulated road 

 Gazebo is a multi-robot Simulator. In this project it's used to simulate a road with different vehicles categories.
 Each vehicle category is represented by a simple wire 3D model. For each tracked vehicle we compare its resulting blob
 with the projection of the different 3D models, the assigned category is the one that gives the best matching.
 
 In the next picture we can see an example of how the 3D models are projected. Tags TR and CA means truck and car
 respectively.


 In the following video we can see the classification process working on simulated Gazebo world. The Vehicles are
 modeled using the CarChassis Gazebo model, their speed are controlled by the CarSim schema (borrowed from the CarSpeed project). 
 The original CarChassis model doesn't allow the size specification, thus minor changes have been  
 introduce to this model to allow this functionality.

Classification example on a real highway (A6 Madrid) 

 In the following videos we can see some examples of the classification process on a multi-lane (Madrid, A6 highway). The following
 colors are used to distinguish between the different classes:
  • Motorcycles: Yellow
  • Cars: Green
  • Vans: Blue
  • Trucks: Pink
  • Road-Train: Black 
 Each tracked/classified vehicle is assigned a unique number during the classification process. At the end of the
 tracking/classification zone at the right of the video you can see the final assigned class and the estimated speed 
 in km/h for this vehicle. The following abbreviations are used for the different classes:
  • Motorcycles: MC
  • Cars: CA
  • Vans: VA
  • Trucks: TR
  • Road-Train: RD



Retrieved from "http://www.jderobot.org/index.php?title=User:Redo&oldid=4927"
Personal tools