Manual-4.3
Introduction
What is this Manual for?
This manual is intended to describe the JDE software, its design and its components. It is oriented to JDE users and JDE developers. For developers it includes descriptions for each piece of software (services, drivers and schemas) in the official distribution.
This is the official most up to date documentation of JDE. It is written in the mediawiki tool because reading a mediawiki is more handy than reading a manual in paper and because videos can be included, enhancing the descriptions.
What is JDE?
jde is a software development suite for robotics applications, written in C language. It provides a programming environment where the robot control program is made up of a collection of several concurrent asynchronous threads named schemas. It is based on JDE cognitive architecture for autonomous robots. The underlying theoretical foundations can be found in the related publications.
jde simplifies the access to robotic hardware from the control program. Getting sensor measurements is as simple as reading a local variable, and ordering motor commands as easy as writing an actuator variable. The platform updates those sensor variables with fresh readings and implements such actuator variables. All of them together set a shared variable API for the robot programming. The robotic application reads and writes such variables to unfold its behavior. They can be connected to real sensors and actuators or simulated ones, both locally or remotely using the network. Several drivers have been developed to support different physical sensors, actuators and simulators. The drivers are used as plugins installed at will depending on your configuration. They are included in the official release.
On top of such variable API there may be perceptive and actuation schemas as building blocks of the robotic application. Perceptive schemas make some data processing to provide information about the world or the robot. Actuation schemas make decisions in order to reach or maintain some goal. They order motor commands or activate new schemas, because the schemas can be combined forming hierarchies. Several schemas are included in the official release.
Installing Jderobot
JDErobot-4.3 works fine with these three robotic tools: Stage is a multiple robot 2D simulator Gazebo is a multiple robot 3D simulator. Player provides a network interface to a variety of robot and sensor hardware. It has become a de facto standard in robotics. They all belongs to the same open source project, check their official web page for further details and downloads.
It may smoothly use Player-2.0.5, Stage-2.0.4 and Gazebo-0.7.
i386 Architecture
How do I download lastest stable release of JDE 4.3.0?
You can download the lastest stable release of JDE in the section Downloads.
- You can install JDE 4.3.0 by means of debian packages. Add the next line in your '/etc/apt/sources.list'.
# Ubuntu deb http://jderobot.org/apt/ hardy jderobot # Debian Sid deb http://jderobot.org/apt/ sid jderobot
Then, type:
>$ apt-get update >$ apt-get install jderobot
How do I build JDE 4.2.1?
(1) First install some libraries it depends on. Jde uses many external libraries, but it is highly modular, so maybe you will only need some of them. In order to build JDE you will need to satisfy the following dependences installing the proper software packages:
- Kernel headers, for video4linux support.
- libraw1394-13-dev and libdc1394-13-dev, for firewire support.
- Player/Stage packages for Player/Stage support
- XForms (libforms-bin libforms-dev libforms1 libformsgl1 libx11-dev x11proto-core-dev libformsgl-dev), for XForms GUI support.
- OpenGL headers and glu headers (libgl1-mesa libgl1-mesa-dev, libglu1-mesa-dev mesa-common-dev glutg3-dev), for opengl support (necessary in visualization)
- GTK (libgtk2.0-dev, libgtkglext1, libgtkglext1-dev, libglade2-dev), for GTK GUI support.
- doxygen, latex2pdf for documentation
(2) Build just the components you will need. For instance, a basic set to program the intelligence of a simulated Pioneer robot using JDE, Player and Stage is: jde executable, graphics_xforms driver, player driver, introrob schema. To generate the jde executable:
$ cd my-jde-4.2.1 $ cd base $ make
(3) Graphics_xforms is the plugin which supports schema GUIs programmed with the XForms library:
$ cd my-jde-4.2.1 $ cd drivers/graphics_xforms $ make $ cp graphics_xforms.so /usr/local/share/jde-robot/
(4) Mastergui schema
$ cd my-jde-4.2.1 $ cd schemas/mastergui $ make $ cp mastergui.so /usr/local/share/jde-robot/
(5) To build the driver in JDE that talks with Player/Stage middleware, Player must be properly installed in your system and then:
$ cd my-jde-4.2.1 $ cd drivers/player $ make $ cp player.so /usr/local/share/jde-robot/
(6) Introrob schema
$ cd my-jde-4.2.1 $ cd utils/pioneer $ make $ cd my-jde-4.2.1 $ cd schemas/introrob $ make $ cp introrob.so /usr/local/share/jde-robot/
(7) Generate old JDE pdf manual
$ cd my-jde-4.2.1 $ cd docs/manual $ make
How do I build JDEROBOT?
Once checked out the source code you have to configure it. If you haven't the configure script you have to build it first:
$> autoreconf -vfi
This should start the build system stuff. After this, just configure it:
$> ./configure --with-player=<playerpath> --with-gazebo=<gazebopath> --prefix=<myownpath>
It is possible to switch on/off some parts of the system, this way you will only build those parts you need. Some interesting options are the compilation flags that will allow you to build different configurations (release, debug,...). For example to configure the system with debug flags and show all the warning messages:
$> ./configure CFLAGS="-g -Wall" CXXFLAGS="-g -Wall" --prefix=<myownpath>
After that you can build the system with just a make, and install it in the specified path with make install (make sure you have rights to write in the specified path).
How do I build JDE on a Ubuntu 8.04?
Since JDE 4.3 debian packages are available and JDE uses autotools. When using the debian package of JDE all these dependencies should be solved by the package, pulling up the installation of those packages which it depends on.
JDE is a very modular software you can only use a subset of its modules (services, drivers, schemas, etc.). When downloading the source code from svn, follow these steps. First, install several packages JDE depends on. For instance:
- OpenGL
- XForms graphics library
- GTK graphics library and Glade
- Player and Stage
- Gazebo
- OpenCv for opencvdemo schema
- For base: libreadline5-dev
- For wiimote driver: libcwiid1 libcwiid1-dev lswm python-cwiid wmgui libbluetooth-dev libbluetooth2
- For mplayer driver: mencoder mplayer
- For calibrador schema: libgsl0-dev gsl-ref-html libgsl0ldbl libgsl0
- For docs: doxygen doxygen-doc doxygen-gui graphviz
Second, build the JDE software itself:
$> autoreconf -vfi $> ./configure --with-player=<playerpath> --with-gazebo=<gazebopath> --prefix=<myownpath> $> make $> sudo make install
You can tune which components of JDE you want to install, on which directories, etc. at configure time. The sample line says you can use JDE with player (at playerpath) and gazebo (at gazebopath), and install JDE on myownpath
Running "./configure --help" you can get much more information about which arguments you can use.
How do I install JDE on Ubuntu 9.04
0. Install libtool
sudo aptitude install libltdl7-dev
1. Install Python libs
aptitude install python2.5 python2.5-dev
2. Install Glade libs
aptitude install libwxgtk2.8-dev libglade2-dev libgtkglext1-dev glade-2
4. Install JDE with debian package: You have to add "deb http://jderobot.org/apt/ hardy jderobot" line to /etc/atp/sources.list. You can to it editing the file or running this (don't forget running apt-get update):
fichero=`mktemp` echo "deb http://jderobot.org/apt/ hardy jderobot" > $fichero cat /etc/apt/sources.list >> $fichero sudo cp $fichero /etc/apt/sources.list sudo apt-get update
Then we are able to install jderobot.
sudo apt-get install jderobot jderobot-doc jderobot-drivers jderobot-libs jderobot-schemas jderobot-services
7. Install Xforms
sudo apt-get install libforms1 libforms-dev libforms-bin libforms-doc libxpm-dev sudo apt-get install libformsgl-dev libformsgl1
8. Install OpenCV
sudo apt-get install libavformat-dev sudo aptitude install libcv-dev
x86_64 Architecture
How do I build JDE components?
Before to start the configuration of jde we make sure that we have install the followings packages:
- OpenGL
- XForms graphics library
- GTK graphics library and Glade
- Player and Stage
- Gazebo
- OpenCv for opencvdemo schema
- For base: libreadline5-dev
- For wiimote driver: libcwiid1 libcwiid1-dev lswm python-cwiid wmgui libbluetooth-dev libbluetooth2
- For mplayer driver: mencoder mplayer
- For calibrador schema: libgsl0-dev gsl-ref-html libgsl0ldbl libgsl0
- For docs: doxygen doxygen-doc doxygen-gui graphviz
How do I download lastest stable release of JDE?
For install jde in this architecture we must to go to Downloads section and:
- Download the latest version of jde
- Unpack it in home directory
- Build the components manually
$ cd jderobot-4.3.0 $ ./configure $ make $ sudo make install
If we have problems during "./configure" process with any driver, schema, library... We can execute
$ ./configure -help
Here we will learn to desactivate the problematic component.
Running Jde
How to use the Jde shell
Jde provides a text shell at run time. The user can run or stop the loaded schemas at will while the jde application is running. Command line is parsed using readline, allowing the use of tab to autocompletion and command history. Shell has two states main and schema.
The available commands in the main state are:
- help/?
- Display help screen.
- cd <dir>
- Change working directory to <dir>.
- dir
- List file at the working directory.
- pwd
- Print current working directory.
- view <file>
- View the contents of <file>.
- ls
- List loaded schemas.
- load_driver <name>
- Load driver <name>.
- load_service <name>
- Load service <name>.
- load_schema <name>
- Load schema <name>.
- ps
- Print schemas states.
- run <name>
- Run schema <name>.
- stop <name>
- Stop schema <name>.
- show <name>
- Show schema <name>.
- hide <name>
- Hide schema <name>.
- zoom <name>
- Enters schema state of <name> schema.
- show_licence
- Show JDE licence.
- quit/exit/Ctrl+D
- Quit using jdeC
On schema state the available commands are:
- help/?
- Display help screen.
- cd <dir>
- Change working directory to <dir>.
- dir
- List file at the working directory.
- pwd
- Print current working directory.
- view <file>
- View the contents of <file>.
- run
- Run schema.
- stop
- Stop schema.
- show
- Show schema.
- hide
- Hide schema.
- init [configfile]
- Init schema. If [configfile] is supplied it is used to init schema, otherwise global config file will be used.
- terminate
- Terminate schema.
- exit/Ctrl+D
- Exit schema mode.
An running example is shown here:
Configuration from ./jderobot.conf Starting cronos... Starting shell... jdeC $> [TAB] ? dir hide load_schema mastergui pwd show view cd exit load load_service opengldemo quit show_license zoom colorA help load_driver ls ps run stop jdeC $> ls colorA mastergui opengldemo jdeC $> ps mastergui: 15 ips, winner () jdeC $> zoom mastergui jdeC[mastergui] $> [TAB] ? dir guiresume help init resume show suspend view cd exit guisuspend hide pwd run stop terminate jdeC[mastergui] $> show jdeC[mastergui] $> hide jdeC[mastergui] $> jdeC $>
How to write the configuration file for a jde execution?
The general configuration file for jde is jde.conf. By default jde will search for such file. You can launch jde with any other configuration file with the command: jderobot <config.file>
Lines beginning with # are comment lines. The configuration file is divided into several sections:
1. service's configuration section 2. driver's configuration section 3. schema's configuration section 4. which services should jde load? 5. which drivers should jde load? 6. which schemas should jde load? 7. which schemas should jde start from the beginning? 8. which schemas should show their GUIs from the beginning?
All drivers, services and schemas are plugins, and must be explicitely loaded for a given execution of jde. At the beginning of the general configuration file the command path <dir1>:<dir2>:<dir3>:...:<dirn> sets the directories to look for the schema, service and driver plugins required to load. To avoid loading a given service, driver or schema just delete the load command line (or comment it). It is not required to comment out its configuration section.
The command to load a service is:
load_service <servicename> [<config_file>]
The <config_file> parameter is optional. It is the name of the file with the specific configuration for that service. Its syntax is service-specific. When loaded without any <config_file> the service can read its configuration from the general jde configuration file, from the lines between
service <servicename>
blabla
blabla
end_service
The command to load a driver is: load_driver <servicename> [<config_file>]
The <config_file> parameter is optional. It is the name of the file with the specific configuration for that driver. Its syntax is driver-specific. When loaded without any <config_file> the driver can read its configuration from the general jde configuration file, from the lines between
driver <drivername>
blabla
blabla
end_driver
The command to load a schema is: load_schema <schemaname> [<config_file>]
The <config_file> parameter is optional. It is the name of the file with the specific configuration for that schema. Its syntax is schema-specific. When loaded without any <config_file> the schema can read its configuration from the general jde configuration file, from the lines between
schema <schemaname>
blabla
blabla
end_schema
This way, services, drivers and schemas may have their own configuration files, or their configuration can be set into the general configuration file, at their own sections.
To run an schema from the very beginning just type run <schemaname> at the end of the general configuration file. To show its GUI from the beginning, just type show <schemaname> after the run line.
First robotic example
The first robotic example uses JDE, Player and Stage. Writing in schemas/introrob/navigation.c you can program the intelligence of a Pioneer robot simulated at Stage, and program it to navigate, make maps or whatever you want.
(1) Properly install JDE, Player and Stage
(2) Launch Player with a configuration file
$ player /usr/local/share/stage/worlds/simple.cfg
In such configuration file you set the sensor equipment of the robot.
(3) Export the environment variables PATH and LD_LIBRARY_PATH. They should include the directory where you have put all the plugins (mastergui.so, player.so, introrob.so, etc.).
$ export PATH=/usr/local/bin:$PATH $ export LD_LIBRARY_PATH=/usr/local/lib/:/usr/local/share/jde-robot/:$LD_LIBRARY_PATH
(4) Launch JDE with a configuration file
$ jderobot /usr/local/include/jde-robot/example-conf-jde.conf
In such configuration file you tell JDE in which socket port the Player server is running and which sensors and motors you want to use from the simulated robot. In that file you also configure which schemas, services and drivers you want to load for that JDE execution. Check here for instructions to write such configuration file. Anyway the best way to proceed is just to get a sample configuration file and tune it.
(5) Start the schemas you want. Typically, mastergui and introrob. Check here for directions about how to do it.
First computer vision example
Networkserver/networkclient configuration example
This is a real configuration for serving firewire camera images:
- jde.conf in medias.gsyc.es (server host)
driver firewire provides colorA 0 autofocus_off #provides colorB 1 autofocus_on end_driver
#networkserver driver must be always the last driver to be loaded driver networkserver socket 3001 ##images section serves colorA 0 #serves colorB 1 #serves colorC 2 #serves colorD 3 ##pantilt section #serves pantiltencoders #serves pantiltmotors ##robot section #serves laser #serves sonars #serves encoders #serves motors end_driver
### DRIVER LOAD FOR SENSOR SOURCES AND MOTOR DEVICES ------------ load_driver firewire load_driver networkserver
- jde.conf in client jdec host
driver networkclient #images section provides colorA medias.gsyc.es 3001 0 #provides colorB ordago.gsyc.es 3001 0 #provides colorC <remote_host> <port_number> <id_net_2> #provides colorD <remote_host> <port_number> <id_net_3> #pantilt section #provides pantiltencoders <remote_host> <port_number> #provides pantiltmotors <remote_host> <port_number> #robot section #initial_position <x_coordinate> <y_coordinate> <theta_orientation> #provides laser <remote_host> <port_number> #provides sonars <remote_host> <port_number> #provides encoders <remote_host> <port_number> #provides motors <remote_host> <port_number> end_driver
### DRIVER LOAD FOR SENSOR SOURCES AND MOTOR DEVICES ------------ load_driver networkclient
Configuration examples for Player+Stage and jderobot working together
One pioneer at Simple World
- Jde configuration file The red one
- [World-1]: Player+Stage configuration files. The pioneer robot has sonars, encoders, motors and laser.
Two pioneers at Simple World
- Jde configuration file The red one
- Jde configuration file The blue one
- [World-2]: Player+Stage configuration files. The pioneer robots have sonars, encoders, motors and laser.
Two pioneers at Cheste circuit
- Jde configuration file The red one
- Jde configuration file The blue one
- [World-3]: Player+Stage configuration files. The pioneer robots have sonars, encoders, motors and laser.
Configuration examples for Gazebo and jderobot working together
These are real configurations to obtain sensor readings from Gazebo simulator:
Pioneer robot in the visual follow line example
- Jde configuration file: encoders, motors, laser, colorA
- World-1: Gazebo configuration file
Two pioneer robots in the departmental building without doors
- Jde configuration file for robot 1
- Jde configuration file for robot 2
- World-2: Gazebo configuration file
Four cameras in the robotics lab's ceiling
A pioneer robot in the departmental building with colored doors
- Jde configuration file
- [World-4]: Gazebo configuration file. The pioneer robot has sonars, encoders, motors, laser, a camera and a pantilt unit.
Two pioneer robots in the obstaclesURJC world
- Jde configuration file for robot 1
- Jde configuration file for robot 2
- [World-5]: Gazebo configuration file
A pioner robot in the robocup 2009 field
- Jde configuration file
- [World-6]: Gazebo configuration file and textures. The pioneer robot has sonars, encoders, motors, laser, a camera and a pantilt unit.
A pioneer robot in a more detailed departmental building
- Jde configuration file
- [World-7]: Gazebo configuration file. The pioneer robot has encoders, motors, laser, a camera and a pantilt unit. This new world has all the extinguishers and other things like a table and the photocopier.
Programming style
JDErobot code is documented using doxygen. You can find more information here.
Programming schemas
The schema is the basic unit that works JDE with. The schema can be in sleep mode or in running mode, that means stopped or running, this states are defined in some functions in the structure of the schema.
Basic structure
All the schemas are defined by three files: myschema.c myschema.h and his Makefile.
The structure of myschema.c is this:
#include "jde.h"
#include "myschema.h"
int finish_flag=0;
int myschema_id=0;
int myschema_brothers[MAX_SCHEMAS];
arbitration myschema_callforarbitration;
int myschema_cycle=150; /* ms */
void myschema_iteration()
{
printf("-->Hello world!\n");
}
void myschema_stop()
{
pthread_mutex_lock(&(all[myschema_id].mymutex));
put_state(myschema_id,slept);
printf("myschema: off\n");
pthread_mutex_unlock(&(all[myschema_id].mymutex));
}
void myschema_run(int father, int *brothers, arbitration fn)
{
int i;
pthread_mutex_lock(&(all[myschema_id].mymutex));
/* this schema resumes its execution with no children at all */
for(i=0;i<MAX_SCHEMAS;i++) all[myschema_id].children[i]=FALSE;
all[myschema_id].father=father;
if (brothers!=NULL) {
for(i=0;i<MAX_SCHEMAS;i++) myschema_brothers[i]=-1;
i=0;
while(brothers[i]!=-1) {myschema_brothers[i]=brothers[i];i++;}
}
myschema_callforarbitration=fn;
put_state(myschema_id,notready);
printf("myschema: on\n");
pthread_cond_signal(&(all[myschema_id].condition));
pthread_mutex_unlock(&(all[myschema_id].mymutex));
}
void *myschema_thread(void *not_used)
{
struct timeval a,b;
long diff, next;
for(;finish_flag==0;) {
pthread_mutex_lock(&(all[myschema_id].mymutex));
if (all[myschema_id].state==slept) {
pthread_cond_wait(&(all[myschema_id].condition),&(all[myschema_id].mymutex));
pthread_mutex_unlock(&(all[myschema_id].mymutex));
}
else {
/* check preconditions. For now, preconditions are always satisfied*/
if (all[myschema_id].state==notready) put_state(myschema_id,ready);
/* check brothers and arbitrate. For now this is the only winner */
else if (all[myschema_id].state==ready) {
put_state(myschema_id,winner);
}
else if (all[myschema_id].state==winner);
if (all[myschema_id].state==winner) {
/* I'm the winner and must execute my iteration */
pthread_mutex_unlock(&(all[myschema_id].mymutex));
gettimeofday(&a,NULL);
myschema_iteration();
gettimeofday(&b,NULL);
diff = (b.tv_sec-a.tv_sec)*1000000+b.tv_usec-a.tv_usec;
next = myschema_cycle*1000-diff-10000;
if (next>0)
/* discounts 10ms taken by calling usleep itself */
usleep(myschema_cycle*1000-diff);
else {
/* just let this iteration go away. overhead time negligible */
printf("time interval violated: myschema\n");
usleep(myschema_cycle*1000);
}
}
else {
pthread_mutex_unlock(&(all[myschema_id].mymutex));
usleep(myschema_cycle*1000);
}
}
}
pthread_exit(0);
}
void myschema_terminate()
{
finish_flag=1;
printf ("myschema terminated\n");
}
void myschema_init(char *configfile)
{
pthread_mutex_lock(&(all[myschema_id].mymutex));
myexport("myschema","id",&myschema_id);
myexport("myschema","run",(void *)&myschema_run);
myexport("myschema","stop",(void *)&myschema_stop);
printf("myschema schema initialized\n");
put_state(myschema_id,slept);
pthread_create(&(all[myschema_id].mythread),NULL,myschema_thread,NULL);
pthread_mutex_unlock(&(all[myschema_id].mymutex));
}
void myschema_show()
{
}
void myschema_hide()
{
}
In myschema.c is the code of the functions that let to manage the schema, will been explained later.
The structure of myschema.h is this:
extern void myschema_init(char *configfile); extern void myschema_terminate();
extern void myschema_stop(); extern void myschema_run(int father, int *brothers, arbitration fn);
extern void myschema_show(); extern void myschema_hide();
The most important functions to manage the schema are defined in myshcema.h and his properties are these:
myschema_cycle: is defined in myshcema.c and is the speed wich with is run the schema.
myschema_iteration: defined in myschema.c, is the code that will be running in concurrent mode when the schema is in running mode.
myschema_init: loads the schema in a new thread of JDE.
myshcema_terminate: removes the thread of the schema from JDE's threads.
myshcema_stop: puts the schema in sleep mode.
myshcema_run: puts the schema in running mode.
myschema_show: opens the GUI of the schema if it had.
myschema_hide: closes the GUI of the schema if it had.
The Makefile is the file that define the guidelines compiled of the schema and creates a file called myschema.so that will run in JDE environment. The code for a basic scheme is this:
JDEDIR = /usr/local INC-DIR = -I../../base -I. -I/usr/include -I/usr/local/include -I$(JDEDIR)/include/jde-robot LIB-DIR = -L. -L/usr/lib -L -L/usr/local/lib GCC = gcc CFLAGS = -g -O -pedantic -Wall $(INC-DIR) LIBS = -lm myschema: myschema.o $(GCC) -shared -Wl,-soname,myschema.so $(LIB-DIR) -o myschema.so myschema.o $(LIBS) myschema.o: myschema.c myschema.h $(GCC) $(CFLAGS) -fPIC -c myschema.c clean: rm -f *.o myschema.so install: cp myschema.so ../../libraries release: tar -zcvf newrelease.tgz *c *h *fd makefile
Configuring JDE
To load our schema in JDE, we must modify the JDE configuration file usually called jde.conf adding in the SCHEMA LOAD section a line for the new schema with this text:
load myshcema
After all of this we can try the new schema running in JDE.
Trying the myschema example
The following example has been tested on a Debian/Lenny Linux box running JDE-4.3.
- Download the example: myschema.tar.gz
- uncompress it
- Change to the myschema directory
- Compile it (just type make)
- Execute jde:
$ jderobot myschema.conf jdec 4.3-svn Configuration from myschema.conf Reading configuration... graphics_gtk driver loaded (driver 0) Loading GTK support... GTK support loaded. mastergui schema loaded (id 0) mastergui schema started up myschema schema loaded (id 1) myschema schema started up Starting cronos... Starting shell... jdec$
The mastergui schema is launched and the following window appears:
When myschema is started (by clicking on the play check button or by typing myschema in the jde shell), the messages "hello world" will be printed every cycle until the schema is stoped:
myschema: on -->Hello world! -->Hello world! -->Hello world! -->Hello world! ... myschema: off
Important issues
Some problems have appeared when we want load a schema and in the same system there is another file with the same name (e.g if we have myschema.so that is a JDE schema and other file called myschema.so) because if you don't have good define the preference in the library's path of jde.conf, JDE can confuse the schema with the other file and try to load the file that isn't a schema and returns an error like this:
WARNING: Unresolved symbol myschema_startup in myschema schema WARNING: Unresolved symbol myschema_id in myschema schema WARNING: Unresolved symbol myschema_resume in myschema schema WARNING: Unresolved symbol myschema_suspend in myschema schema WARNING: Unresolved symbol myschema_stop in myschema schema WARNING: Unresolved symbol myschema_guiresume in myschema schema WARNING: Unresolved symbol myschema_guisuspend in myschema schema
So must be careful to set preferences in the path variable in configuration file of JDE
Programming Graphical Interfaces in JDE
Currently two graphics libraries are supported in JDE: GTK and XForms. They are available through the services graphics_gtk and graphics_xforms. Several schemas with GTK GUIs may be simultaneously active, several with XForms, and some subset with GTK and another subset with XForms at the same time.
Using XForms Library
First, you have to design your GUI with "fdesign". We have supposed your schema is called "firstSchema", so we have used "firstSchemagui" as the name of our first schema with GUI. When you finish your design you will obtain the files firstSchemagui.c and firstSchemagui.h. You need to add the following functions to your code (remember you mustn't modify the generated code by designer):
void firstSchema_guiresume () {
fd_firstSchemagui = create_form_firstSchemagui(); // create form
fl_set_form_position(fd_firstSchemagui->firstSchemagui,400,50); // position
fl_show_form(fd_firstSchemagui->firstSchemagui,FL_PLACE_POSITION,FL_FULLBORDER,"HERE, ADD YOUR FORM TITLE"); // title showed
firstSchema_canvas_win= FL_ObjWin(fd_firstSchemagui->mycanvas); // main window
gc_values.graphics_exposures = False;
firstSchemagui_gc = XCreateGC(display, firstSchema_canvas_win, GCGraphicsExposures, &gc_values);
/* canvas handlers */
fl_add_canvas_handler(fd_firstSchemagui->mycanvas,ButtonPress,firstSchema_button_pressed_on_mycanvas,NULL); // button pressed
fl_add_canvas_handler(fd_firstSchemagui->mycanvas,ButtonRelease,firstSchema_button_released_on_mycanvas,NULL); // button released
fl_add_canvas_handler(fd_firstSchemagui->mycanvas,MotionNotify,firstSchema_mouse_motion_on_mycanvas,NULL); // button motion
}
void firstSchema_guisuspend () {
mydelete_buttonscallback(firstSchema_guibuttons);
mydelete_displaycallback(firstSchema_guidisplay);
fl_hide_form(fd_firstSchemagui->firstSchemagui);
}
Furthermore, you must add all display behavior functions. For example, firstSchema_guidisplay() includes some code like this:
fl_rectbound(0,0,firstSchema_width,firstSchema_height,FL_WHITE); // clear the screen (paint white) fl_set_foreground(firstSchemagui_gc,FL_LEFT_BCOL); // grid at the floor fl_set_foreground(firstSchemagui_gc,FL_CYAN); // foreground color = cyan
Another important function is firstSchema_guibuttons(void *obj1). Here you control the behavior of your GUI buttons. For example:
FL_OBJECT *obj=(FL_OBJECT *)obj1;
if (obj == fd_firstSchemagui->exit) firstSchema_guisuspend(); // schema suspend button
else if (obj == fd_firstSchemagui->stop) { // pioneer robot stop button
// ...
} else if (obj == fd_firstSchemagui->laserButton) { // pioneer laser button
if (fl_get_button(obj)==PUSHED) // ...
} else if (obj == fd_firstSchemagui->selectWindow) { // window selector
fl_set_button(fd_firstSchemagui->windowSelected, 1); // windowSelected is set to TRUE
}
Using GTK Library
You can develop a GTK windows using 'glade-2' designer (Watch more in FAQ). When the interface has been developed, you have to program the following set of functions:
- yourschema_show: the first time that this function is called, the graphical interface is charged and the callback functions are assigned to the different events of each widget. Last, the function register the refresh callback of the display.
-yourschema_hide: this function is used to hide the display and eliminate the subscription of the refresh callback of the display.
-yourschema_guidisplay: this function is used every time it's wanted to refresh the interface.
To realize the registration of the yourschema_guidisplay function is necessary to import the register_displaycallback and delete_displaycallback functions of the graphics_gtk service, through the yourschema_guiinit function. The service also offers a function that makes the charge of the .glade file easier. You only have to put the .glade file in the same folder of the .so file (the plugin).
To show how these functions work, let's explain the next source code. First of all, we declare these global variables:
registerdisplay myregister_displaycallback; deletedisplay mydelete_displaycallback; GladeXML *xml; /*Xml file*/ GtkWidget *win; /*Main window*/
To use the "graphics_gtk" service we need to import the "register_displaycallback" and "delete_displaycallback" functions from the "graphics_gtk" service in the "yourschema_guiinit" function:
void yourschema_guiinit(){
if (myregister_displaycallback==NULL){
if ((myregister_displaycallback=
(registerdisplay)myimport ("graphics_gtk",
"register_displaycallback"))==NULL)
{
printf ("I can't fetch register_displaycallback from graphics_gtk\n");
jdeshutdown(1);
}
if ((mydelete_displaycallback=
(deletedisplay)myimport ("graphics_gtk", "delete_displaycallback"))
==NULL)
{
printf ("I can't fetch delete_displaycallback from graphics_gtk\n");
jdeshutdown(1);
}
}
}
Now, we are going to implement the "yourschema_show" function, loading the ".glade" file and connecting the callbacks. In this example, the "schema.glade" file contains a button called "button1" and a window called "window". When we click on "button1", the program execute the "on_button1_clicked" function:
void yourschema_show(void){
static int loaded=0;
static pthread_mutex_t schema_gui_mutex;
pthread_mutex_lock(&schema_gui_mutex);
if (!loaded){
loadglade ld_fn;
loaded=1;
pthread_mutex_unlock(&schema_gui_mutex);
/*Load the window from the .glade xml file*/
gdk_threads_enter();
if ((ld_fn=(loadglade)myimport("graphics_gtk","load_glade"))==NULL){
fprintf (stderr,"I can't fetch 'load_glade' from 'graphics_gtk'.\n");
jdeshutdown(1);
}
xml = ld_fn ("schema.glade");
if (xml==NULL){
fprintf(stderr, "Error loading graphical interface\n");
jdeshutdown(1);
}
win = glade_xml_get_widget(xml, "window");
/*Connect the callbacks*/
{
g_signal_connect(G_OBJECT(win), "delete-event",
G_CALLBACK(on_delete_window), NULL);
g_signal_connect(GTK_WIDGET(glade_xml_get_widget(xml, "button1")),
"clicked", G_CALLBACK(on_button1_clicked), NULL);
}
if (win==NULL){
fprintf(stderr, "Error loading graphical interface\n");
jdeshutdown(1);
}
else{
gtk_widget_show(win);
gtk_widget_queue_draw(GTK_WIDGET(win));
}
gdk_threads_leave();
}
else{
pthread_mutex_unlock(&schema_gui_mutex);
gdk_threads_enter();
gtk_widget_show(win);
gtk_widget_queue_draw(GTK_WIDGET(win));
gdk_threads_leave();
}
myregister_displaycallback(yourschema_guidisplay);
all[yourschema_id].guistate=on;
}
Then, we implement the "yourschema_hide" function, which hides the main window and calls the "mydelete_displaycallback" function:
void yourschema_hide(void){
if (win!=NULL){
gdk_threads_enter();
gtk_widget_hide(win);
gdk_threads_leave();
}
mydelete_displaycallback(yourschema_guidisplay);
}
Finally, the "yourschema_guidisplay" function is used to refresh the interface, in our example, the "schema.glade" file contains a text label called "label1", depending on the variable "is_activated", we change the text shown by the text label:
void yourschema_guidisplay(){
gdk_threads_enter();
if (is_activated){
gtk_label_set_text((GtkLabel *)glade_xml_get_widget(xml, "label1"), "Label1 activated");
}
else{
gtk_label_set_text((GtkLabel *)glade_xml_get_widget(xml, "label1"), "Label2 desactivated");
}
gtk_widget_queue_draw(win);
gdk_threads_leave();
}
The interface of our example could be like this:
Compiling your schema
In order to use and run your component inside jderobot you will need to (1) build a plugin file yourschema.so and (2) load it from a jderobot execution ordering that in the configuration file (load_schema yourschema). A full section of this manual describes how to configure jderobot and order the loading of several components (schemas, drivers or services). To build your schema from the source code, you will have to compile it and link it. For sure it will depend on jderobot stuff, libraries and headers, but also on other libraries and headers as well.
YOU DON'T NEED TO DOWNLOAD THE SVN SOURCE TREE IN ORDER TO COMPILE YOUR COMPONENT !!. The recommended way to work is to simply install jderobot from its debian package (see Downloads section), build your component's plugin and configure jderobot to load it at execution time. Once jderobot has been installed in your machine from the debian package, then all the jderobot libraries and headers have been properly installed at the right directories. If your component depends on other libraries then check they are properly installed on your machine. For C/C++ software check both the libraries and the header files.
Here you have a sample Makefile (myMakefile) to use with your schema, together with a standard jderobot installation from debian package. Just copy it at the directory with the source code of your component, tune it to include the right dependencies and search directories, and type "make -f myMakefile". You can also rename myMakefile as Makefile and just type "make".
JDEDIR =
#directories to look for the header files
INC-DIR = -I. -I/usr/include -I/usr/include/opencv -I/usr/include/jderobot -I/usr/X11R6/include -I/usr/include/X11 -I$(JDEDIR)/include/jderobot `pkg-config --cflags libglade-2.0 gtkglext-1.0 gdkglext-1.0 gdkglext-1.0 gthread-2.0`
#directories to look for the lib files
LIB-DIR = -L. -L/usr/lib -L/usr/lib/jderobot -L/usr/X11R6/lib `pkg-config --libs libglade-2.0 gtkglext-1.0 gdkglext-1.0 gdkglext-1.0 gthread-2.0`
GCC = gcc
CFLAGS = -g -pedantic -Wall
#which libraries your component depends on
LIBS = -lm -lX11 -lcv -lcxcore -lhighgui -lcvaux -lcolorspaces
#your component's name
COMPONENT= opencvdemo2
#linking step
$(COMPONENT): $(COMPONENT).o
$(GCC) -shared -Wl,-soname,$(COMPONENT).so $(LIB-DIR) -o $(COMPONENT).so $(COMPONENT).o $(LIBS)
#compilation step
$(COMPONENT).o: $(COMPONENT).c
$(GCC) $(CFLAGS) $(INC-DIR) -fPIC -c $(COMPONENT).c
clean:
rm -f *.o $(COMPONENT).so
(Watch out, at the beginning of $(GCC) and rm lines there is a TAB character, not spaces).
- The COMPONENT variable sets the component's name. In this case, opencvdemo2.
- The LIBS line states the libraries the component depends on. In this example, opencvdemo2 depends on libm (math library), libX11 (Xwindow library), libcv, libcxcore, libhighgui, libcvaux (libraries of OpenCV) and libcolorspaces (from jderobot).
- The LIB_DIR line sets the directories to look for the libraries.
- The INC_DIR line sets the directories to look for the required header files.
Programming drivers and services
Schemas
Extrinsics
Extrinsics schema is used for fine tunning of the calibration parameters of any camera, both intrinsic and extrinsic ones, but mainly intended for the last. It accepts two files (as especified in its own configuration file): first, one with the description of the world around the real camera, in terms of lines and points; second, one with the calibration parameters of the camera.
It shows on its GUI one window with the world as seen from a virtual camera, and another window with the camera images and the world superimposed over the pixel values. It also offers several widgets (sliders, positioner, etc.) to change any calibration parameter and instantly see the effect of such modification. It delivers an output file with the final fined tunned calibration parameters of the camera.
For instance in the sample image, the green cross in the image is the optical center, the green line is the horizon line. Several sliders may tune the camera position, orientation, focal distance, roll, etc.
In the configuration file of jde you should provide a source for "colorA" images and load the extrinsics schema as:
load extrinsics extrinsics_configuration_file
Mastergui
Mastergui is a schema who manage the other schemas an allow you to turn on and off the other schemas, start the GUI of the schema and shows you information about the schema like speed running in IPS (iteration per second).
Introrob
The Introrob schema is being used to learn basic behavior programming of a Pioneer robot. For instance, to implement navigation algorithms like Virtual Field Force (VFF), deliberative navigation, Gradient Path Planning (GPP) and others to move the robot. The sensor equipment of the robot typically includes: a laser scanner, encoders and a color camera. In the actuator side the Pioneer robot has two motors that provide a simple interface: rotation and translation speeds. This schema has a GUI with which you can teleoperate the robot, see the laser and other sensor readings in a window, or activate your own algorithm. It also shows the robot camera images when available.
It works with Stage and Gazebo simulators, as well as with a real Pioneer robot. Since it deals with images, it allows vision-based control algorithms like line following, follow-other-robots behavior, local vision navigation (as Polly robot from I.Horswill), etc..
Teleoperator
The Teleoperator schema permits to teleoperate a robot (for instance the pioneer robot) and visualize it in a 3D canvas. It's sensory measurements (laser, sonar, encoders...) are also shown in the canvas. This schema can capture images from platform, for example gazebo simulated images.
The GUI is divided in three parts: 3D canvas, images and controls. Through images and 3D canvas we can visualize information and controls permits to insert information in the schema. The most important are buttons below 3D canvas, camera joystick, base joystick and pantilt joystick. The buttons activate sensors, camera joystick is for handling camera position, base joystick is for changing robot position and speed and joystick pantilt is for managing pantilt position.
- Schema teleoperator with Stage simulator
- Schema teleoperator with real sensors
Hsvtuner
HSVtuner is an image filtering schema that accepts images from any driver and filters them by color.
GUI: In the left part of the gui we can see the color wheel (H and S coordinates) with a sector in pale green informing which colour is being filtered. We can drag'n'drop its corners so we can change the color being filterd. Also, below the color wheel we have sliders for the same purpose. We can use the method we are more confortable with. There are two sliders for each the Hue, Saturation and Value, for minimum and maximum thresholds. For images coming in RGB format we use the color conversions at the wikipedia.
In the right part of the GUI, we can see the original images with the filtrated ones. We can click on any pixel to select the color we want to filter in the original image.
By default, below the images the Background button in greyscale is turned on, so all pixels in the filtered image are grey but the ones which made it through the filter. If we set off this button, the image we see is all black but the filtered pixels.
Colortuner
Colortuner is an image filtering schema, base on the previous schema "HSVtuner", Colortuner perform image color filtering in the next color spaces:
- RGB - HSV - YUV
Colortuner is an application to configure tailored color filters in HSV, RGB, or Y'UV color spaces. It is use to obtain optimal values of tint and saturation, as well as lighting, in that kind of filters. To perform the different color conversions between spaces we used the conventions appear in wikipedia, HSV color conversions, and for YUV, YUV color conversions . Colortuner has a very simple way to work. We have two displays, in the first one it shows the original image, as the camera capture it, in the other, it shows the filtered image.
Build a filter is an easy task with it clever display, every single color space have the next possibilities:
- Moving the sliders, up and down to set the maximum and minimum values, for example of RED, GREEN and BLUE, to apply the filter. - From an example pixel of the camera image, just click on a colored area of the image, and HSV, RGB or Y'UV values will configure automatically.
The image filtered will be displayed at right in the interface, the pixels with the values selected for the filter, they are show in original color, the user could choose to see the pixels that did not pass the filter in grey, black or white.
Finally, for HSV and Y'UV color spaces, colortuner includes an interface representing both color spaces:
- Clicking on the vertex of the area, we could modify the color filter. - On the color spaces representation, they are display the different color tint detected by a white pixel. the "threshold slider" brings the possibility to determine how many pixels of the same color are necessary to do the representation on the color space frame.
Demonstrative video:
Calibrador
ImageRectifier
This schema transforms a image using homography, you can read more about this technique here.
To create a image you need to select 4 points in the first image in the specified order, and then push in "rectangle mode" and select two points in the second image.
Image_viewer
FollowBall
This scheme keeps track an object of a specific color, through the use of mechanical neck.
This tracking has been programmed using a control on speed, that is, continuously monitor the unit pantilt ordering rates to the 2-axis (and not ordering position). Telling the speeds at which you have to move 2 axes, and set to cero the speed when we have the objective focused. We get a fairly acceptable behavior. We use the library libcolorspaces to get a very quick process (the image is analyzed and filtered 15-18 times per second, and control the speeds of the neck about 10-12 times per second).
You can see the fluid movement that we obtaing in the next video
OpencvDemo
This schema shows some image functionalities developed with opencv and libcolorspaces library: GrayScale, Color filter, Canny filter, Sobel filter, Optical Flow, multiresolution pyramid and convolutions.
To use this schema, you need to configure VarcolorA from any driver (videoforlinux2, mplayer, firewire, etc..) and some properties of the schema itself, as you can see in the opencvdemo.conf file.
You can see some images of this schema working:
For the convolution operation you can set the kernel matrix in the configuration file of the schema. Changing that matrix several effects can be performed on the image. For instance, the sharpening is achieved with:
#Sharpening size 3 modulo 1 offset 0 row 1 0 -1 0 row 2 -1 5 -1 row 3 0 -1 0
Blurring may be achieved with:
#gaussian blur size 5 modulo 20 offset 0 row 1 0 0 1 0 0 row 2 0 1. 2. 1. 0 row 3 1 2. 4. 2. 1 row 4 0 1. 2. 1. 0 row 5 0 0 1 0 0
Embossing may be achieved with:
#Embossing effect #size 3 #modulo 1 #offset 0 #row 1 -2 -1 0 #row 2 -1 1 1 #row 3 0 1 2
For edge detection you can configure this schema with:
#Edge detection size 3 modulo 1 offset 128 row 1 0 -1 0 row 2 -1 4 -1 row 3 0 -1 0
NaoOperator
The NaoOperator schema permits to teleoperator the Nao Robot. This schema captures images from the Nao cam and shows it in the GUI. For the moment you can change the position of the robot (Nao can walk straight and truns) and moves the head (uses pantiltl). NaoOperator uses: ptencoders to control the state of the robot's head. ptmotors to control the robots's head movement. motors to control the robot's movement. varcolorX to get images from the robot's cam.
Drivers
Imagefile
This driver provides images from raw PPM files. It offers two types of interfaces: (a) colorA, colorB, colorC, colorD and (b) varcolorA, varcolorB, varcolorC and varcolorD, through up to eight virtual schemas, with the corresponding names.
It is configured as in the following example:
driver imagefile # colorA, colorB, colorC and colorD interfaces are defined as 320x240. provides colorA /usr/local/share/jde-robot/casaleft.pnm provides colorB atardecer.ppm provides colorC /usr/local/share/jde-robot/atardecer.ppm provides colorD /usr/local/share/jde-robot/casaright.pnm # varcolorA, varcolorB, varcolorC and varcolorD accept any size provides varcolorA /usr/local/share/jde-robot/casaleft.pnm 500 400 provides varcolorD /usr/local/share/jde-robot/casaright.pnm 320 240 # autosize for varcolorX interfaces provides varcolorB atardecer.ppm provides varcolorC /usr/local/share/jde-robot/atardecer.ppm end_driver
The size of the images can be setup in the configuration file or left them out. In the last case the driver finds the proper image size exploring the image file, and sets width and height properly (autosize).
The data API of each virtual schema is as follows:
- varcolorA (char *), it points to an array with the pixel values, where each pixels is stored as 3 bytes: BGR
- clock (unsigned long int) logical timestamp, its value increases with each new frame
- width (int) number of columns of the image
- height (int) number of rows of the image
For instance:
(*varcolorA)[i*3]; /* Blue Byte */ (*varcolorA)[i*3+1]; /* Green Byte */ (*varcolorA)[i*3+2]; /* Red Byte */
Player
This driver connects Jde with the Player server, which provides several sensors and actuators. It is mostly used for applications for the Pioneer robot (both real Pioneer or one simulated in Stage). It provides laser readings, encoders readings and sonar readings.
The output of laser measures is a vector called laser[] with 180 float values that represent the distance of an object.
The output encoder is a vector of float called robot[] with this values:
- robot[0]: x value of the robot position.
- robot[1]: y value of the robot position.
- robot[3]: angle that represent the difference between map's x axis and robot's x axis.
- robot[4]: cos() of the angle defined in robot[3].
- robot[5]: sin() of the angle defined in robot[3].
To load this driver you must specify this parameters in the configuration file:
driver player hostname <pc_name> port <port_number> initial_position <coord_x> <coord_y> <angle_theta> provides laser 180 1 provides sonars provides encoders provides motors end_driver
hostname and port are the IP and the port of the PC where Player server is running, if are running in local this line will be like this : hostname localhost port 6665.
initial_position gives the robot an initial point and an angle to start, coord_x and coord_y are in mm and angle_theta is in grad.
provides load the functionalities of the robot like: encoders, sonars, motors, laser. You only have to write the functionalities that you will use in your code. In line of laser you must specified number of grads and values per grad for laser (180 grads 1 value per grad).
Pantilt
This driver connects Jde with the Directed Perception's pan-tilt unit. It allows to transmit commands and to receive real-time information about the motors. The evi camera has an angle of view of 65º without zoom and 6.6º with the full zoom.
It can be loaded with the next configuration:
driver pantilt serial <serial_port> provides pantiltencoders provides pantiltmotors end_driver
serial is the serial port used.
provides it provides the virtual schemas which control the encoders and the motors of the pan-tilt unit.
We can load just one virtual schema or both. The pantiltencoders virtual schema permits us to control the state of the device. The available variables are:
- id: The id of the schema.
- pan_angle: The actual angle in the horizontal axis in degrees.
- tilt_angle: The actual angle in the vertical axis in degrees.
- clock: It increase it value each time there are new datas available.
- resume: The function to activate the driver.
- suspend: The function to deactivate the driver.
The pantiltmotors virtual schema permit us to control the device. The available variables are:
- id: The id of the schema.
- cycle: Is the time between two writes in the port of the driver.
- resume: The function to activate the driver.
- suspend: The function to deactivate the driver.
- longitude: The horizontal coordinate to go to. The coordinate is in absolute terms and in degrees.
- latitude: The vertical coordinate to go to. The coordinate is in absolute terms and in degrees.
- max_longitude: The maximum value of longitude. It is provided by the device.
- max_latitude: The maximum value of latitude. It is provided by the device.
- min_longitude: The minimum value of longitude. It is provided by the device.
- min_latitude: The minimum value of latitude. It is provided by the device.
- longitude_speed: The speed in the horizontal axis. It is measure in degrees/sec.
- latitude_speed: The speed in the vertical axis. It is measure in degrees/sec.
- max_longitude_speed: The maximum speed in the horizontal axis. It is measure in degrees/sec and provided by the device.
- max_latitude_speed: The maximum speed in the vertical axis. It is measure in degrees/sec and provided by the device.
Evi
This driver connects Jde with the "Sony EVI-D100" camera. It allows to transmit commands and to receive real-time information about the motors. This device is much more faster than the Directed Perception's pan-tilt unit. It also provides virtual schemas to control the zoom. It has 40x levels of zoom (10x optical + 4x digital).
It can be loaded with the next configuration:
driver evi
serial <serial_port>
provides pantiltencoders
provides pantiltmotors
provides zoomencoders
provides zoommotors
end_driver
serial: Is the serial port used.
provides: It provides the virtual schemas which controls the encoders and the motors of the pan-tilt unit.
For the pantiltencoders and pantiltmotors virtual schemas it uses the same interface as the pantilt driver. The pantiltencoders virtual schema permit us to control the state of the device. The avaible variables are:
- id: The id of the schema.
- pan_angle: The actual angle in the horizontal axis in degrees.
- tilt_angle: The actual angle in the vertical axis in degrees.
- clock: It increase it value each time there are new datas available.
- resume: The function to activate the driver.
- suspend: The function to deactivate the driver.
The pantiltmotors virtual schema permit us to control the device. The available variables are:
- id: The id of the schema.
- cycle: Is the time between two writes in the port of the driver.
- resume: The function to activate the driver.
- suspend: The function to deactivate the driver.
- longitude: The horizontal coordinate to go to. The coordinate is in absolute terms and in degrees.
- latitude: The vertical coordinate to go to. The coordinate is in absolute terms and in degrees.
- max_longitude: The maximum value of longitude. It is provided by the device.
- max_latitude: The maximum value of latitude. It is provided by the device.
- min_longitude: The minimum value of longitude. It is provided by the device.
- min_latitude: The minimum value of latitude. It is provided by the device.
- longitude_speed: The speed in the horizontal axis. It is measure in degrees/sec.
- latitude_speed: The speed in the vertical axis. It is measure in degrees/sec.
- max_longitude_speed: The maximum speed in the horizontal axis. It is measure in degrees/sec and provided by the device.
- max_latitude_speed: The maximum speed in the vertical axis. It is measure in degrees/sec and provided by the device.
To monitorized the zoom we have the zoomencoders virtual schemas. The available variables are:
- id: The id of the schema.
- zoom_position: The actual level of zoom.
- clock: It increase it value each time there are new datas available.
- resume: The function to activate the driver.
- suspend: The function to deactivate the driver.
To control the zoom we have the zoommotors virtual schemas. The available variables are:
- id: The id of the schema.
- cycle: Is the time between two writes in the port of the driver.
- resume: The function to activate the driver.
- suspend: The function to deactivate the driver.
- zoom: The zoom level.
- max_zoom: The maximum value. It is provided by the device.
- min_zoom: The minimum value. It is provided by the device.
- zoom_speed: The zoom's speed.
- max_zoom_speed: The zoom's maximum speed.
Video4Linux
This drivers allows JDE to capture digital images from cameras that use the Video4Linux video capture API.
To load this driver you must specify these parameters in the configuration file:
driver video4linux provides varcolorA /dev/video0 width height provides colorA /dev/video1 end_driver
The first and last line tell JDE to load this driver.
The first 'provide' line is telling JDE to store the captured images from file /dev/video0 in the 'varcolorA' buffer. You can use the parameters 'width' and 'height' to set the size of the image stored in the 'varcolorA' variable.
The second 'provide' line is setting a second camera (attached to /dev/video1) and JDE will store the images in 'colorA' buffer. As no resolution has been set, JDE will asume a default resolution of 320x240 for the captured images, which is the standard for 'colorX' buffers.
Video4Linux2
This driver supports the second version of the Video4Linux API, which fixes a number of design bugs of the first version. The main difference at the configuration file is that you must indicate the particular input that you want from the device. In most cases where cameras provide only one channel this parameter is '0'.
driver video4linux2 provides colorA /dev/video1 input_id provides varcolorA /dev/video0 input_id width height end_driver
Example: Logitech Quickcam Pro 9000
Here is a configuration that currently works:
driver video4linux2 provides varcolorA /dev/video0 0 176 144 end_driver
(the word /dev/video0 may vary depending on the other video devices connected to your PC).
Firewire
This drivers allows JDE to capture digital images from cameras that use the firewire (ieee1394) plug in interface. Currently this driver is able to provide images from two firewire cameras simultaneously.
This driver is able to capture images at 320x240 resolution with a maximum speed of 30 frames per second (fps). It also allows capturing 640x480 video images with a lower speed of 15 fps. Both firewire 4-pins and 6-pins interfaces are supported (usually the 4-pin interface cameras require extra power using an external AC adaptor).
The driver is also able to activate or deactivate the "autofocus" option if the camera supports it. For example, the Apple iSight firewire camera is able to use this option.
To load this driver you must specify these parameters in the configuration file:
driver firewire provides varcolorA 0 640 480 autofocus_off provides colorB 1 autofocus_off end_driver
The first and last line tells JDE to load this driver.
The first 'provide' line is telling JDE to store the captured images in the 'varcolorA' buffer. The next value (in this case 0) specifies the node of the camera in the firewire bus. The capture resolution will be 640x480 in this example (only 'varcolor' buffers are able to store 640x480 resolution at the moment). The 'autofocus_off' option is setting off the 'autofocus' option of this camera.
The second 'provide' line is setting a second camera (conected to node 1) and JDE will store the images in 'colorB' buffer. The 'autofocus_on' is setting the 'autofocus' feature off. As no resolution has been set, JDE will assume a default resolution of 320x240 for the captured images.
Networkclient
This driver permits to Jdec to connect to other Jdec that is using networkserver. The configuration for this driver is shown here.
driver networkclient #images section provides colorA <remote_host> <port_number> <id_net_0> provides colorB <remote_host> <port_number> <id_net_1> provides colorC <remote_host> <port_number> <id_net_2> provides colorD <remote_host> <port_number> <id_net_3> #pantilt section provides pantiltencoders <remote_host> <port_number> provides pantiltmotors <remote_host> <port_number> #robot section initial_position <x_coordinate> <y_coordinate> <theta_orientation> provides laser <remote_host> <port_number> provides sonars <remote_host> <port_number> provides encoders <remote_host> <port_number> provides motors <remote_host> <port_number> end_driver
- provides: indicates which devices will be used. It's important to put remote host and remote port correctly.
- initial position: indicates robot's initial position in relative coordinates world. This option isn't obligatory
Gazebo
This driver connects Jde with the Gazebo simulator, which provides several sensors and actuators. It can simulate different robots (pioneer robot, for example) and also several cameras from a configured world. For robot simulation provides laser readings, encoders readings, sonar readings and visual readings. The actuators are motors and pantilt, which permits to change robot base and pantilt.
A configuration snip for a pioneer robot with this driver is shown here.
- jde.conf
driver gazebo provides laser laser1 provides motors robot1 provides encoders robot1 provides colorA camera1 end_driver
### DRIVER LOAD FOR SENSOR SOURCES AND MOTOR DEVICES ----- load_driver gazebo
- Gazebo configuration file (.world)
<model:Pioneer2DX>
<id>robot1</id>
<xyz>-14.713 -8.808 0.234</xyz>
<rpy>-0 -0 12</rpy>
<model:SickLMS200>
<id>laser1</id>
<xyz>0.0 0 0</xyz>
<rayCount>180</rayCount>
<rangeCount>180</rangeCount>
</model:SickLMS200>
<model:SonyVID30>
<id>camera1</id>
<xyz>0 0 0.30</xyz>
<rpy>0 0 0</rpy>
</model:SonyVID30>
</model:Pioneer2DX>
A provides line indicates to the driver a kind of measurement that is desired. There is the possibility to activate more than one kind of sensor. Make sure of the number checking the gazebo configuration file.
A configuration snip for four cameras with this driver is shown here.
- jde.conf
driver gazebo provides colorA camera1 provides colorB camera2 provides colorC camera3 provides colorD camera4 end_driver
### DRIVER LOAD FOR SENSOR SOURCES AND MOTOR DEVICES ----- load_driver gazebo
- Gazebo configuration file (.world)
<model:MonoCam> <id>camera1</id> <xyz>0.3 0.3 3.0</xyz> <rpy>0. +45. 45. </rpy> <gravity>0. 0. +9.8</gravity> </model:MonoCam> <model:MonoCam> <id>camera2</id> <xyz>0.3 4.289 3.0</xyz> <rpy>0. +45. -45. </rpy> <gravity>0. 0. +9.8</gravity> </model:MonoCam> <model:MonoCam> <id>camera3</id> <xyz>7.625 4.479 3.0</xyz> <rpy>0. +45. -135. </rpy> <gravity>0. 0. +9.8</gravity> </model:MonoCam> <model:MonoCam> <id>camera4</id> <xyz>7.625 -0.254 3.0</xyz> <rpy>0. +45. -225 </rpy> <gravity>0. 0. +9.8</gravity> </model:MonoCam>
A configuration snip for get big images from gazebo cameras is shown here.
- jde.conf
driver gazebo provides varcolorA camera1 provides varcolorB camera2 end_driver
### DRIVER LOAD FOR SENSOR SOURCES AND MOTOR DEVICES ----- load_driver gazebo
- Gazebo configuration file (.world)
<model:SonyVID30>
<id>camera1</id>
<xyz>0 0.15 0.30</xyz>
<rpy>0 0 0</rpy>
<imageSize>640 480</imageSize>
</model:SonyVID30>
With option "imageSize" we shall determine the size of the image given by gazebo
- Gazebo configuration file (.world)
<imageSize>640 480</imageSize>
See how to install Gazebo simulator here.
See real configuration examples here.
NaoBody
This driver connect JDE with the Naoqi middleware. With this driver you can control the Nao Robot thought Naoqi using JDE. A provides line indicates to the driver a kind of measurement that is desired. There is the possibility to activate more than one kind of sensor.
NaoBody can be loaded with the next configuration:
driver naobody provides ptencoders provides ptmotors provides motors provides varcolorA 0 320 240 fps n end_driver
The motors virtual schema is to control the movement of the robot, ptmotors and ptencoders controls de head movement.
motors gives us two values v (speed) and w (angular velocity).
Ptencoders and ptmotors are the same as evi driver and pantilt driver.
Simulated3D
This driver provides 2D images from 3D world. The world and camera parameters are indicated through configuration files. This files are compatible with extrinsics schema.
- World file example:
#units:milimeters #room #worldline X1 Y1 Z1 H1 X2 Y2 Z2 H2 worldline 0.000000 0.000000 3000.000000 1.000000 0.000000 4589.000000 3000.000000 1.000000 worldline 0.000000 0.000000 3000.000000 1.000000 1211.000000 0.000000 3000.000000 1.000000 ...
In world file each line defines a world line through two 3D points.
- Camera file example:
#position positionX 7875.000000 positionY 4749.000000 positionZ 3000.000000 positionH 1.000000 #extrinsics, orientation FOApositionX 6264.000000 FOApositionY 3725.000000 FOApositionZ 1875.000000 FOApositionH 1.000000 roll 3.007028 #intrinsics fx 405.399994 fy 405.399994 skew 0.000000 u0 142.600006 v0 150.399994 columns 320 rows 240
- jderobot.conf:
driver simulated3D #provides colorXX <FileWithCamParameters> #provides varcolorXX <FileWithCamParameters> worldfile ./robotics-lab provides varcolorA cameraAfile provides varcolorB cameraBfile provides varcolorC cameraCfile provides varcolorD cameraDfile end_driver
load_driver simulated3D
This driver supports varcolor interface. Image width and height is configurable from camera configuration file.
Opencv
This driver provides static, video file and camera images. It uses Opencv library to access devices. Configuration example for this driver is shown here.
driver opencv #provides varcolorA <image|video|videofile> <path> <width> <height> #provides varcolorC <image|video|videofile> <path> autosize #provides colorB <image|video|videofile> <path> provides varcolorA video /dev/video1394/0 620 480 provides colorC videofile /home/salons/jde-videos/camC.ogg provides colorD video /dev/video0 provides varcolorB image /usr/local/share/jderobot/casaleft.pnm 620 480 provides varcolorD video /dev/video0 autosize end_driver
### DRIVER LOAD FOR SENSOR SOURCES AND MOTOR DEVICES ----- load_driver opencv
- autosize property: you only can use it with varcolor interface and takes original resolution from image, video file or camera. It does not work for firewire cameras.
Gazebo08
This driver has the same duties than Gazebo driver but this driver connects to Gazebo simulator version 0.8-pre3. It uses libgazebo C++ API and integrate it in Jderobot platform with C API.
A configuration snip for a pioneer robot with this driver is shown here.
- jderobot.conf
driver gazebo08 provides laser laser_iface_0 provides motors position_iface_0 provides encoders position_iface_0 #provides pantilt ptz_iface_1 #provides varcolorA stereo_iface_0 left /* disparity left image */ #provides varcolorB stereo_iface_0 right /* disparity right image */ #provides varcolorD camera_iface_0 /* image can be from mono camera or stereo camera */ #provides varcolorC camera_iface_1 end_driver
### DRIVER LOAD FOR SENSOR SOURCES AND MOTOR DEVICES ----- load_driver gazebo08
- Snip Gazebo configuration file (.world)
<model:physical name="pioneer2dx_model1"> <xyz>0 0 0.145</xyz> <rpy>0.0 0.0 0.0</rpy>
<controller:differential_position2d name="controller1">
<leftJoint>left_wheel_hinge</leftJoint>
<rightJoint>right_wheel_hinge</rightJoint>
<wheelSeparation>0.39</wheelSeparation>
<wheelDiameter>0.15</wheelDiameter>
<torque>5</torque>
<interface:position name="position_iface_0"/>
</controller:differential_position2d>
<model:physical name="laser">
<xyz>0.15 0 0.18</xyz>
<attach>
<parentBody>chassis_body</parentBody>
<myBody>laser_body</myBody>
</attach>
<include embedded="true">
<xi:include href="models/sicklms200.model" />
</include>
</model:physical>
<include embedded="true">
<xi:include href="models/pioneer2dx.model" />
</include>
</model:physical>
Firewire22
This is a reimplemented firewire driver in order to use 22 version of libdc1394 library, that has changed its API.
Configuration example for this driver is shown here.
driver firewire22 #provides <color> <idcam> <fps> <autofocus> provides colorA 0 30 autofocus_on provides colorB 1 15 autofocus_on provides varcolorA 0 30 640 480 autofocus_on provides varcolorA 2 15 640 480 autofocus_on end_driver
### DRIVER LOAD FOR SENSOR SOURCES AND MOTOR DEVICES ----- load_driver firewire22
- camera id: it indicates which camera you choose
- fps: it admits 15 or 30 fps
- resolution: it admits 320x240 or 640x480. This last only for varcolor interface.
- autofocus: autofocus_on or autofocus_off
Services
Networkserver
Jdec can be used to serve devices that are conected through TCP/IP conexion. It can serve static or camera images and robot or pantilt devices. Therefore, Jdec has also a service role. Access to devices could be done through a local net or internet.
The following lines are the configuration that appears in jde.conf file:
driver networkserver socket <port_number> #images section serves colorA <id_net_0> serves colorB <id_net_red_1> serves colorC <id_net_red_2> serves colorD <id_net_red_3> #pantilt section serves pantiltencoders serves pantiltmotors #robot section serves laser serves sonars serves encoders serves motors end_driver
- socket: port number where server will be binded and will recive request from clients.
- serves: means the devices you want to serve. Each device must be included in a new line but it isn't necessary to serve all of them.
Images are a special case. You have to indicate image id (colorA, colorB, colorC or colorD) and net id (0, 1, 2, or 3).
Finally, all served devices has to be initialized before this service begin to work. Don't forget to enable the device drivers in the same configuration file. For instance, if you want to service robot devices simulated by player, you have to enable player driver and the networkserver service.
See a configuration example Manual#Networkserver.2Fnetworkclient_configuration_example.
Graphics_xforms
This service offers the support for Xforms interface, connecting with X server to show Xforms windows. It generates a thread that collects user request through pussing or interacting with widgets. Applications must provide functions to refresh, to attend to widgets, and to show and hide the display. This functions will be called from service thread. This thread will process schema requests and will communicate with X server, avoiding concurrency problems. If each schema made request to X server, there would be concurrent accesses and it doesn't work well. Through this method all request are made by the same thread.
The following lines explain how to use Xforms service.
Programming with graphics_xforms
- guishow_aux: initializes the display and register buttons and display callbacks. Also this function shows the form.
- guihide: imports from xforms service a function that permits the service execute indicated suspend function.
- guihide_aux: it is the indicated suspend function to cancel callbacks registry and hide the form.
void hsvtuner_hide_aux(void){
/*Switch off display refresh and button control*/
mydelete_buttonscallback(myschema_guibuttons);
mydelete_displaycallback(myschema_guidisplay);
/*Hide form*/
fl_hide_form(fd_myschemagui->myschemagui);
}
void hsvtuner_hide(){
callback fn;
if((fn=(callback)myimport("graphics_xforms","suspend_callback"))!=NULL){
fn ((gui_function)myschema_hide_aux);
}
}
void hsvtuner_show_aux(void)
{
static int k=0;
if (k==0) /* not initialized */
{
k++;
fd_hsvtunergui = create_form_hsvtunergui();
fl_set_form_position(fd_hsvtunergui->hsvtunergui,400,50);
fl_show_form(fd_hsvtunergui->hsvtunergui,FL_PLACE_POSITION,FL_FULLBORDER,HSVtunerVer);
hsvtuner_win= FL_ObjWin(fd_hsvtunergui->oculo_orig);
hsvtunergui_setupDisplay();
}
else
{
fl_show_form(fd_hsvtunergui->hsvtunergui,FL_PLACE_POSITION,FL_FULLBORDER,HSVtunerVer);
hsvtuner_win= FL_ObjWin(fd_hsvtunergui->oculo_orig);
}
myregister_buttonscallback(hsvtuner_guibuttons);
myregister_displaycallback(hsvtuner_guidisplay);
fl_set_slider_bounds(fd_hsvtunergui->w_slider,100,1);
fl_set_slider_value(fd_hsvtunergui->w_slider,hsimap_threshold);
fl_set_slider_bounds(fd_hsvtunergui->Vmin,255,0);
fl_set_slider_value(fd_hsvtunergui->Vmin,v_min);
fl_set_slider_bounds(fd_hsvtunergui->Vmax,255,0);
fl_set_slider_value(fd_hsvtunergui->Vmax,v_max);
fl_set_slider_bounds(fd_hsvtunergui->Hmin,2.*PI,0);
fl_set_slider_value(fd_hsvtunergui->Hmin,h_min);
fl_set_slider_bounds(fd_hsvtunergui->Hmax,2.*PI,0);
fl_set_slider_value(fd_hsvtunergui->Hmax,h_max);
fl_set_slider_bounds(fd_hsvtunergui->Smin,1,0);
fl_set_slider_value(fd_hsvtunergui->Smin,s_min);
fl_set_slider_bounds(fd_hsvtunergui->Smax,1,0);
fl_set_slider_value(fd_hsvtunergui->Smax,s_max);
}
void hsvtuner_show(){
callback fn;
if((fn=(callback)myimport("graphics_xforms","resume_callback"))!=NULL){
fn ((gui_function)myschema_show_aux);
}
}
Graphic interface of hsvtuner schema that uses graphics_xforms.
At initialization time, some functions must be imported from graphics_xforms service. They permit to register functions guidisplay and guibuttons.
registerbuttons myregister_buttonscallback; registerdisplay myregister_displaycallback; deletebuttons mydelete_buttonscallback; deletedisplay mydelete_displaycallback;
void hsvtuner_guiinit(){
if (myregister_buttonscallback==NULL){
if ((myregister_buttonscallback=
(registerbuttons)myimport ("graphics_xforms",
"register_buttonscallback"))==NULL){
printf("I can't fetch register_buttonscallback from graphics_xforms\n");
jdeshutdown(1);
}
if ((mydelete_buttonscallback=
(deletebuttons)myimport ("graphics_xforms",
"delete_buttonscallback"))==NULL){
printf("I can't fetch delete_buttonscallback from graphics_xforms\n");
jdeshutdown(1);
}
if ((myregister_displaycallback=
(registerdisplay)myimport ("graphics_xforms",
"register_displaycallback"))==NULL){
printf("I can't fetch register_displaycallback from graphics_xforms\n");
jdeshutdown(1);
}
if ((mydelete_displaycallback=
(deletedisplay)myimport ("graphics_xforms",
"delete_displaycallback"))==NULL){
printf("I can't fetch delete_displaycallback from graphics_xforms\n");
jdeshutdown(1);
}
}
}
Finally, you have to fill this functions.
void hsvtuner_guibuttons(void *obj){
/* This function check if a button or widget has been used and implements expected actions. This is a handler example for a slider and a button. */
if (obj ==fd_hsvtunergui->w_slider)
{
hsimap_threshold=(int)fl_get_slider_value(fd_hsvtunergui->w_slider);
/*printf("hsimap_threshold %d\n",hsimap_threshold);*/
}
if (obj ==fd_hsvtunergui->toblack){
if (fl_get_button(fd_hsvtunergui->toblack)==RELEASED){
toblack=0;
}
else
toblack=1;
}
}
void hsvtuner_guidisplay(){
/* This function implements display refesh */
drawcheese(histograma_buf,centro_x,centro_y,h_max,h_min,s_max,s_min,FL_PALEGREEN);
XPutImage(mydisplay,hsvtuner_win,hsvtuner_gc,imagenOrig,0,0,fd_hsvtunergui->oculo_orig->x, fd_hsvtunergui->oculo_orig->y, SIFNTSC_COLUMNS, SIFNTSC_ROWS);
XPutImage(mydisplay,hsvtuner_win,hsvtuner_gc,hsifiltrada,0,0,fd_hsvtunergui->oculo_modif->x, fd_hsvtunergui->oculo_modif->y, SIFNTSC_COLUMNS, SIFNTSC_ROWS);
XPutImage(mydisplay,hsvtuner_win,hsvtuner_gc,histograma,0,0,fd_hsvtunergui->histograma->x, fd_hsvtunergui->histograma->y, SMAX, SMAX);
}
Graphics_gtk
This service supports GTK library for schemas whose graphic interface has been implemented with GTK. It initializes the library and creates two threads: GTK's main thread (gtk_main) and the other one refresh all displays.
Programming with graphics_gtk The first step is create the interface with glade application (available in ubuntu repositories). Glade stores the interface information as XML document with .glade extension. This file is required for loading the interface in execution time. Some functions have to be implemented:
- show: this function loads the interface XML document and assign callbacks functions to each widget. This function must be implemented too. Also this function shows the display on screen.
- hide: imports from xforms service a function that permits the service execute indicated hide function.
- guidisplay: this function is called to refresh the interface.
This function is called to initialize the graphic interface.
void esquema_show(void){
static int cargado=0;
static pthread_mutex_t esquema_gui_mutex;
pthread_mutex_lock(&esquema_gui_mutex);
if (!cargado){
cargado=1;
pthread_mutex_unlock(&esquema_gui_mutex);
/*Load the window from xml .glade*/
gdk_threads_enter();
if ((ld_fn=(loadglade)myimport("graphics_gtk","load_glade"))==NULL){
fprintf (stderr,"I can't fetch 'load_glade' from 'graphics_gtk'.\n");
jdeshutdown(1);
}
xml = ld_fn ("interface.glade");
if (xml==NULL){
fprintf(stderr, "Error al cargar la interfaz gráfica\n");
jdeshutdown(1);
}
win = glade_xml_get_widget(xml, "window1");
/*Connecting callbacks*/
{
/* Buttons */
g_signal_connect (G_OBJECT (glade_xml_get_widget(xml,"rectificar")),
"clicked", G_CALLBACK (on_rectificar_clicked), NULL);
g_signal_connect (G_OBJECT (glade_xml_get_widget(xml,"descartar_rectificada")),
"clicked", G_CALLBACK (on_descartar_rectificada_clicked), NULL);
g_signal_connect (G_OBJECT (glade_xml_get_widget(xml,"undo")),
"clicked", G_CALLBACK (on_undo_clicked), NULL);
g_signal_connect (G_OBJECT (glade_xml_get_widget(xml,"modo_rectangulo")),
"clicked", G_CALLBACK (on_modo_rectangulo_clicked), NULL);
/* Image */
GdkPixbuf *imgBuff1;
img1=(GtkImage *)glade_xml_get_widget(xml, "image1");
imgBuff1 = gdk_pixbuf_new_from_data((unsigned char *)image1->imageData,
GDK_COLORSPACE_RGB,0,8,
mycolorAwidth,mycolorAheight,
mycolorAwidth*3,NULL,NULL);
gtk_image_clear(img1);
gtk_image_set_from_pixbuf(img1, imgBuff1);
gtk_widget_queue_draw(GTK_WIDGET(img1));
}
if (win==NULL){
fprintf(stderr, "Error al cargar la interfaz gráfica\n");
jdeshutdown(1);
}
else{
gtk_widget_show(win);
gtk_widget_queue_draw(GTK_WIDGET(win));
}
gdk_threads_leave();
}
else{
pthread_mutex_unlock(&esquema_gui_mutex);
gdk_threads_enter();
gtk_widget_show(win);
gtk_widget_queue_draw(GTK_WIDGET(win));
gdk_threads_leave();
}
myregister_displaycallback(esquema_guidisplay);
}
Graphic interface of schema that uses graphics_gtk.
This function is called to stop showing the display.
void esquema_hide(void){
if (win!=NULL){
gdk_threads_enter();
gtk_widget_hide(win);
gdk_threads_leave();
}
mydelete_displaycallback(esquema_guidisplay);
}
This function refresh the display.
registerdisplay myregister_displaycallback;
deletedisplay mydelete_displaycallback;
void esquema_guidisplay(){
if (display_imageA){
/*Obtener el widget de la imagen desde el archivo xml*/
GtkImage *image = GTK_IMAGE(glade_xml_get_widget(xml, "image1"));
gdk_threads_enter();
/*Refrescar la imagen en el display*/
gtk_widget_queue_draw(GTK_WIDGET(image));
gdk_threads_leave();
}
}
Some functions must be imported from graphics_gtk service. They permits to regiter functions guidisplay and guibuttons.
void esquema_guiinit(){
if (myregister_displaycallback==NULL){
if ((myregister_displaycallback=
(registerdisplay)myimport ("graphics_gtk",
"register_displaycallback"))==NULL){
printf ("I can't fetch register_displaycallback from graphics_gtk\n");
jdeshutdown(1);
}
if ((mydelete_displaycallback=
(deletedisplay)myimport ("graphics_gtk",
"delete_displaycallback"))==NULL){
jdeshutdown(1);
printf ("I can't fetch delete_displaycallback from graphics_gtk\n");
}
}
}
The service also provides a function for loading .glade file. It's only necessary to put the .glade file in the same directory as plugin (.so file) in order to program locates it. The function is exported with 'load_glade' name and its declaration is in service's header file.
/** * Search in the path the .glade file and loads it * @param file_name The name of the .glade file * @returns the newly created GladeXML object, or NULL on failure. */ typedef GladeXML* (*loadglade) (char * file_name);
Graphics_glut
This service has been created to interact with openGL motor by using the free implementation toolkit FREEGLUT.
Programming with graphics_glut
To create interfaces using graphics_glut we will define some different functions in the code of the schema. Firs of all we must include the library and create the main function that initializes the motor.
#include <GL/glut.h> #include "graphics_glut.h"
void display_init(int argc, char ** argv) {
glutInit(&argc, argv);
glutInitDisplayMode(GLUT_DEPTH | GLUT_DOUBLE | GLUT_RGB);
}
Once this function is defined ....
Under Construction
Libs
Progeo Library
Progeo is a projective geometry library that offers some very useful functions that relates 2D points and 3D points.
Image Coordinate System
Progeo system uses a frame of reference according to the right-hand rule. It's represented on the following image:
If you consider another different one, you should adjust the values. For example, if you use the next usual coordinate system, you realize x and y axis are rotated 90º and the origin is different respect to the Progeo system.
Compile
To compile this library you only need to do a make in the Progeo folder on your version of JDE or including the progeo files in your schema folder and include the following line in your main code:
"#include "progeo.h"
Data types
HPoint2D
This structure allows to store a 2D point that usually belongs to the image plane. The 'x' and 'y' fields represents the point's coordinates and 'h' field is a homogeneous coordinate that always must be 1.
typedef struct {
float x;
float y;
float h;
} HPoint2D;
HPoint3D
The 3D data point structure represents a point in the 3D space. It stores (x,y,z) position coordinates and 'H' field indicates the point's position with regard to image plane, which is normally 1.
typedef struct {
float X;
float Y;
float Z;
float H;
} HPoint3D;
TPinholeCamera
Represents a Pinhole camera model that is defined by its extrinsics and intrinsics parameters. Extrinsics parameters consist on camera 3D position, foa and roll. On the other hand, intrinsics parameters are fdist, u0 and v0.
typedef struct {
HPoint3D position;
HPoint3D foa;
float roll;
float fdistx;
float fdisty;
float u0,v0;
int rows;
int columns;
float skew;
float k11,k12,k13,k14,k21,k22,k23,k24,k31,k32,k33,k34;
float rt11,rt12,rt13,rt14,rt21,rt22,rt23,rt24,rt31,rt32,
rt33,rt34,rt41,rt42,rt43,rt44;
float d1,d2,d3,d4,d5,d6,dx,dy;
HPoint3D tr, bl;
char name[256];
} TPinHoleCamera;
That's the meaning of the structure fields:
(HPoint3D) position: the camera 3d position in mm. (HPoint3D) foa: the camera 3d focus of attention in mm. (float) roll: the camera roll position angle in rads. (float) fdistx: the x focus distance in mm. (float) fdisty: the y focus distance in mm. It is used as only fdist by default. (float) u0,v0: height (u0, vertical) and width (v0, horizontal) positions of the central pixel. (int) rows: camera image height in pixels (int) columns: camera image width in pixels (float) skew: angle between the x and y axes (matrix) k11..k34: camera K matrix (matrix) rt11..tr44: camera rotation and translation matrix (float) d1,d2,d3,d4,d5,d6,dx,dy: radial distortion parameters (distortion is not implemented yet by project and backproject functions). (HPoint3D) tr: 3D point that represents top right image plane point (HPoint3D) bl: 3D point that represents bottom left image plane point (char*) name: camera's name
TPinholeStereoCamera
Represents a stereo camera based on the Pinhole model. This data type is composed by two simple Pinhole camera.
typedef struct {
HPoint3D position;
HPoint3D foa;
float roll;
TPinHoleCamera camera1;
TPinHoleCamera camera2;
float baseline;
char name[256];
} TPinHoleStereocamera;
The structure has similar attributes to simple camera. The baseline field is the distance of separation between two cameras.
Calibrate
The two main functions of the Progeo library are project and backproject, but before using them, it is neccesary to calibrate your (real or virtual) camera. It is important to advertise that Progeo uses a Pinhole type camera model, a very simple camera with no lens and a single very small aperture.
The calibration can be done giving the K (intrinsic) , R (rotation) and T (translation) matrix.
In a typical PinHole camera the variables could take the following values:
camera.roll=0; camera.u0=240.0; camera.v0=320.0; camera.fdisty=480.0/(2.0*tan(DEGTORAD*45.0/2.0)); camera.fdistx=camera.fdisty; camera.skew=0.0
Once you have done this, you must call this functions to finish the camera calibration:
extern void update_camera_matrix(TPinHoleCamera *camera); or extern void update_stereocamera_matrix(TPinHoleStereocamera *stereo);
This functions allows to update camera K matrix and rotation and traslation matrix after a parameter's change.
Import cameras from Gazebo and Webots
To import cameras from Gazebo or Webots, we need to understand how a camera works in opengl. To define the intrictics parameters, opengl uses horizontal angle and aspect ratio (instead of focal distance and optical centre, used in progeo), however, the extrinsics parameters are the same in opengl and progeo, roll and focus.
We can obtain the intrinsics parameters in progeo in a easy way:
The optical centre will be perfect (since it's a simulated camera), so, if the image size is 320x240, the optical centre will be (120, 160).
The focal distance is not so easy to obtain, we need to calculate it from this formula:
Where size is either the height or the width of the image, and Φ is the horizontal angle in radians used in opengl. If we need to import our camera from Gazebo, the size will be the height of the image, however, if we are using Webots, the size will be the width of the image. The horizontal angle is 45 degrees (0.785398 radians) in Webots and 60 degrees in Gazebo.
Use
When you have calibrated the camera, you can use the main functions of the Progeo Library now:
Project: this function allows to project a 3D point of the world to the corresponding 2D pixel of the indicated camera image.
extern int project(HPoint3D in, HPoint2D *out, TPinHoleCamera camera);
Backproject: this function does the inverse action. It allows to obtain the projection line that connect the camera with the focus and the 3D ray that it projects in a pixel of the image plane. To do this you only need to join the focus with the out point obtained with this funcion.
extern int backproject(HPoint3D *out, HPoint2D in, TPinHoleCamera camera);
DisplayLine: this function allows to know if a line defined by two 2D points is visible inside a image plane with a concrete size. If it's possible, a 2D image points are returned to display the line over the image.
extern int displayline(HPoint2D p1, HPoint2D p2, HPoint2D *a, HPoint2D *b, TPinHoleCamera camera);
Display_info: This simple function shows all the information about the used camera.
extern void display_camerainfo(TPinHoleCamera camera);
Examples
The following example uses the project function to project a set of lines that represents a room. There are two cameras in this room and it displays the lines over the image planes of both cameras.
/* It is supposed that camera parameters are defined */
TPinholeCamera camera1,camera2;
HPoint3D lineList[MAX_LINES*2];
/* There are two buffers that will be visualized in the screen
and will be initialized to white color */
unsigned char buffer1[MAX_HEIGHT*MAX_WIDTH*COLOR_DEPTH];
unsigned char buffer2[MAX_HEIGHT*MAX_WIDTH*COLOR_DEPTH];
setWhiteBuffer(&buffer1);
setWhiteBuffer(&buffer2);
/* Necessary 2D points to draw lines.*/
HPoint2D auxPoint1,auxPoint2;
HPoint2D goodAuxPoint1,goodAuxPoint2;
/* Getting projections of each point and drawing them into the buffer */
for(int i=0;i<MAX_LINES;i++)
{
/* 3D LINES PROJECTIONS ON CAMERA 1*/
project(lineList[i],&auxPoint1,camera1);
project(lineList[i+1],&auxPoint2,camera1);
if(displayline(auxPoint1,auxPoint2,&goodAuxPoint1,&goodAuxPoint2,camera1))
{
/* Drawing in the image of camera 1 */
lineinimage(goodAuxPoint1,goodAuxPoint2,FL_BLACK,&buffer1);
}
/* 3D LINES PROJECTIONS ON CAMERA 2*/
project(lineList[i],&auxPoint1,camera2);
project(lineList[i+1],&auxPoint2,camera2);
if(displayline(auxPoint1,auxPoint2,&goodAuxPoint1,&goodAuxPoint2,camera2))
{
lineinimage(goodAuxPoint1,goodAuxPoint2,FL_BLACK,&buffer2);
}
}
These are the resulting images:
This code uses backproject function and shows a triangulation sample to obtain a 3D point that is projected on image plane.
/* It is supposed that camera parameters are defined */ TPinholeCamera camera1,camera2; /* Traslation of graphic coordinates to optical coordinates and vice versa */ #define GRAPHIC_TO_OPTICAL_X(x,y) (SIFNTSC_ROWS-1-y) #define GRAPHIC_TO_OPTICAL_Y(x,y) (x) #define OPTICAL_TO_GRAPHIC_X(x,y) (y) #define OPTICAL_TO_GRAPHIC_Y(x,y) (SIFNTSC_ROWS-1-x) /* It is supposed that we know the position in pixels of 3D point */ int pointxCam1,pointyCam1; int pointxCam2,pointyCam2;
/* Defining auxiliar points*/ HPoint2D auxPoint2DCam1,auxPoint2DCam2; HPoint3D auxPoint3DCam1,auxPoint3DCam2; /* Getting 3D points from optical coordinates.*/ auxPoint2DCam1.x = GRAPHIC_TO_OPTICAL_X(pointxCam1,pointyCam1); auxPoint2DCam1.y = GRAPHIC_TO_OPTICAL_Y(pointxCam1,pointyCam1); auxPoint2DCam1.h = 0; auxPoint2DCam2.x = GRAPHIC_TO_OPTICAL_X(pointxCam2,pointyCam2); auxPoint2DCam2.y = GRAPHIC_TO_OPTICAL_Y(pointxCam2,pointyCam2); auxPoint2DCam2.h = 0;
/* Getting 3D points from 2D points */ backproject(&auxPoint3DCam1,auxPoint2DCam1,camera1); backproject(&auxPoint3DCam2,auxPoint2DCam2,camera2);
/* After this, with the two 3D points calculated and focus positions of both cameras we can calculated two lines in the spaces. The intersection of these lines is the 3D point wanted. */
Camera integration from different platforms
Gazebo
This section explains how to import sintetic images generated by gazebo to progeo. It's needed to understand the relation between opengl and progeo camera model. Defining intrinsics parameters is the difference: opengl works with aspect relation and progeo works with focal distance and optical center. Gazebo uses opengl perspective projection, whose aspect relation is defined by four attributes:
fovy: horizontal view angle aspect ratio: width/height nearClip: objects that are nearer than this distance will not be showed farClip: objects that are further than this distance will not be showed
This image represents gazebo camera model.
Working with simulated camera means that we have ideal calibration, therefore optical center will be (120,160). On the other hand, focal distance must be calculated:
f= height / 2 tan (p/2) where f is focal distance, height is pixels count of image height and p is aperture angle.
ARToolKit
This is a calibration tool developed by Osaka, Washington and Canterbury universities. It obtains extrinsics and intrinsics parameters through different patterns. Image center, distorsion generated by pixel size and focal distance can be calculated using calib_dist and calib_cparam programs. Extrinsics parameters must be calculated with exview program, the result is a camera 3D position vector and a quaternion that represents focus of attention and roll.
The structure of camera progeo model has to be filled with ARToolKit calculated data. Calibration programs give us fdist and (u0,v0) fields and other data to compose K matrix:
Trhough exview we obtain camera position and rotation+traslation matrix parameters:
Observation: ARToolKit works with 640x480 image size, so if you use 320x240 image size, optical center and focal distance must be divided by 2.
Fuzzy Library
Is an implementation of fuzzy logic that brings all the functionalities to manage a fuzzy controller and use this in your C code.
Compile
To compile this library you only need to do a make in the fuzzylogic's folder on your version of JDE.
Use
Use the library is very easy, you must add the path of the library on you Makefile file and now you can use these functions:
int fc_open(char *filename): creates a fuzzy controller from a fuzzy file (*.fzz) giving you all the logic symbols and the fuzzy rules.
void fc_close(int controlador): closes a fuzzy controller.(Don't do it if you want the values that provides this controller later).
void fc_save(int controlador, char *filename): makes a fuzzy file from a fuzzy controller.
int fc_output(int controlador, char *varname, float *output): brings you the value of the "varname" variable of fuzzy logic on the "*output" variable, this will be the output values of fuzzy logic.
int fc_link(int controlador, char *varname, float *varpointer): links "*varpointer" variable to a variable of fuzzy logic called "varname" this will be the input values of the fuzzy logic.
Fuzzy file
The structure of a fuzzy file will be like this:
#File with fuzzy controller info. If line begin with # is ignored controlador fuzzy_controller #Example of a definition of fuzzy variable etiqueta fuzzyvar1 valuename1 = 8000 12000 16000 20000 etiqueta fuzzyvar1 valuename2 = 1000 2000 4000 8000 etiqueta fuzzyvar1 valuename3 = 300 500 800 1000 #Example of a definition of fuzzy variable etiqueta fuzzyvar2 valuename4 = 300 500 600 800 etiqueta fuzzyvar2 valuename5 = 200 225 275 300 etiqueta fuzzyvar2 valuename6 = 100 175 187 200 #Example of a definition of fuzzy variable etiqueta fuzzyvar3 valuename7 = -40 -35 -30 -25 etiqueta fuzzyvar3 valuename8 = -25 -20 -15 -10 etiqueta fuzzyvar3 valuename9 = 0 0 0 0 #Example of a fuzzy rules definition IF ( fuzzyvar1 = valuename1 ) AND ( fuzzyvar2 = valuename4 ) THEN ( fuzzyvar3 = valuename7 ) IF ( fuzzyvar1 = valuename3 ) AND ( fuzzyvar2 = valuename6 ) THEN ( fuzzyvar3 = valuename7 ) IF ( fuzzyvar2 = valuename5 ) AND ( fuzzyvar1 = valuename1 ) THEN ( fuzzyvar3 = valuename9 ) IF ( fuzzyvar1 = valuename1 ) THEN ( fuzzyvar3 = valuename8 ) IF ( fuzzyvar2 = valuename6 ) THEN ( fuzzyvar3 = valuename9 )
The file must end in a enter character, and all the lines also end up in the enter character. It's very important respect the structure of each line on definitions and rules because if the lines aren't equal the fuzzy controller doesn't work. Is highly recomended call the fuzzy file ends in .fzz to distinguish it from other files.
Example of an implementation with fuzzylib
int k;
float var1, var2;
float var3;
int main()
{
/*Make the fuzzy controller*/
k=fc_open("test.fzz");
fc_link(k,"fuzzyvar1",&var1);
fc_link(k,"fuzzyvar2",&var2);
fc_link(k,"fuzzyvar3",&var3);
/*The section before only executes once in the program*/
var1 = 12000;
var2 = 500;
fc_output(k,"fuzzyvar3",&var3);/*You can request the value on var3 with this function as many times as you want*/
printf("Var = %f \n",var3);
return0;
}
The result of this program will be something like:
Var = -35
You can find more examples on the examples folder in your JDE.
ColorSpaces Library
libcolorspaces is a library and thread safety programmed in C that is capable of generating traslation table between color's spaces. The current version creates a table of translation RGB-> HSV, so the calculations to convert from RGB to HSI are calculated only once (to fill the table), and the next request are looking at an array.
To create a image you need to select 4 points in the first image in the specificated order, and then push in "rectangle mode" and select two points in the second image.
The algorithm to translate RGB to HSV we have obtained the specifications of Wikipedia. The input is the component RGB between 0 and 255, and the output is the component HSV.
