the collection of tools which provide control and
visualization/monitoring of the robot's run time state. There are
three levels to the interface: local/physical, console, and GUI.
You can read more about the UI philosophy in the ControlBase
is the base class for all of the menu items you will see below.)
to see a list of the controls available by default at run time.
Level 1: Physical
With the Aibo series of robots, you need nothing more than the robot itself to interact with the control system. Any time the emergency stop is activated (look for the pulsing tail LEDs) you will
be able to access the Controller through the head buttons on an ERS-210,
or the tail buttons on an ERS-220. Robots which lack any direct human interface are unfortunately unable to use this level of the system.
||Front Head Button
||Left Tail Button
||Front Back Button
||Rear Head Button
||Right Tail Button
||Rear Back Button
||Center Tail Button
||Center Back Button
||Main Back Button
|Request Text Input Prompt:
||Press "Next" and "Previous" buttons at the same time.
The Aibo's system console has a limitation that when input is being
read, all execution in that process halts until the read has
completed. Thus, you need to request a prompt when you wish to
enter data on the system
console. The Tekkotsu console (see below) does not have
this limitation, and thus will ignore "Text Input Prompt" requests if
the Tekkotsu console is connected.
Feedback is provided through sound
and the LEDs. The 210 will display a "number",
whereas you can simply count the number of LEDs on the heads of the 220
Level 2: Console
For development, a console is
indispensable - debugging would be near impossible without it. Of
course, this requires a computer, wireless ethernet hardware, and implies physical separation from
the robot, which is why it's important to allow users to do basic
operations in Level 1 without all of those requirements. At this point, note that a stand to elevate from the ground can be very useful in development to prevent
unexpected (and costly) motions which could cause a robot to fall off of the table.
There are several different
consoles, each accessible by telneting to a different port.
- System console - direct terminal console (or port 59000 on Aibo)
- All output from system calls, printf, cout, and cerr goes here.
- Output from sout
(below) will be redirected here if it is not connected.
- Output is non-blocking, so if a crash occurs, the last
few lines of output may not be displayed.
- Reading from this console (cin) is blocking - your code
will freeze until the user hits return, and there is no way (that we
know) to check if input is waiting. (hence the need for the user to
specify when they are ready to input data using the buttons listed
- Tekkotsu standard out (sout)-
- If you connect to this port, all tekkotsu-generated
output will be sent here instead of the system console. This can help
you separate framework messages from system messages, or your own
debugging messages. (if you are using cout)
- Output from serr
is sent here if there is no other connection.
- Output is non-blocking.
- Input from this port is sent to the controller. If
there is a GUI active, anything you type will be broadcast as a TextMsgEvent.
If there is no GUI active, the input will be processed by the
controller (commands listed below). This behavior can be modified via tekkotsu.cfg
- Tekkotsu standard error (serr) - port 10002
- Output is blocking. Thus, if you send something
here, your code will stop until the message is sent. This is very
handy for debugging so you can tell what it was doing right before a
crash, but is also rather slow, so it should be used sparingly.
- NOTE: It appears this is not completely blocking.
There appears to be some system buffering, so that a few lines of
output may still be lost.
- If this port is unconnected, it will be redirected to sout, but then no claims about
blocking can be made (sout
is responsible for its transmission).
- There is no input from this port.
When sout is
being used as the main display for the Controller, (i.e. when there is
no open GUI) the following commands are available: (from the Controller
- redisplays the current control (handy on first connecting, or when
other output has scrolled it off the screen)
- return to the root control
- '!next' -
calls doNextItem() of the
- '!prev' -
calls doPrevItem() of the
- calls doSelect() of the
- calls doCancel() of the
- '!msg text' - broadcasts text as a TextMsgEvent
- '!root text' - calls takeInput(text) on the root control
- '!hello' -
responds with 'hello\ncount\n' where count is the number of
times '!hello' has been
sent. Good for detecting first connection after boot vs. a
[n1 [n2 [...]]]' -
zero, one, or more items in the menu
- '!input text' - calls takeInput(text) on the currently hilighted
- '!set section.key = value'
- will be sent to Config::setValue(section,key,value)
- any text not beginning with ! - sent to takeInput() of
the current control
By default, most controls provide a menu interface. Typing the
number of an item or a unique prefix will select it.
Level 3: GUI
To ensure platform portability,
we have written our GUI tools using Java, and have some legacy code through MATLAB which may be of interest
Both the MATLAB and Java versions
share network code. Each network connection to the Aibo
is maintained by a Java object, which is then either used by a MATLAB
interface or a Java interface. In this way, there is no code
Available Java tools are listed below.
Setting up TekkotsuMon
To use the Java tools, you will
need Sun's Java Software
higher. The machine you are running on needs to be able to access
the Aibo's IP address. Beware firewalls at NAT issues!
If you had Java when you compiled
the framework, make
should have already built
the tools. Change directory to
Type make now if
are not already built. (look for .class files in org/tekkotsu/mon subdirectory).
To launch the main GUI, type:
This should bring up the ControllerGUI window similar to the one shown in the next screenshot.
The Controller is the main
interface to edit the robot's run time state. It provides all of
the capabilities of the console - messages typed in its input field are
interpreted exactly as they are on the console.
Double clicking menu items will open them. Items with
arrowheads on the right signify submenus. The menu system
contains a number of useful tools for editing parameters and producing
various status reports and logs.
The chasing arrows in the lower right will force a reconnect -
this is handy if the robot is forceably rebooted and the Controller doesn't notice
The protocol used in communication with the ControllerGUI is
defined in Controller's
documentation. This same protocol can be used manually to send
commands to port 10001 if the GUI isn't connected.
You can stream video from the robot in real time and monitor
performance of the color segmentation. You can also record these
streams to image sequences on the desktop for later processing.
The Raw Vision window also lets you select what color space to
The network tends to be the main bottleneck - the FPS is
relative to frames received by the GUI, not necessarily the FPS the robot is
processing on its end.
You can modify compression, resolution, and channel options
from the tekkotsu.cfg
However, you can also change these options at run time using the !set command (see example here)
Using a bit of C++ overloading, we can monitor all changes to
data types without changes to existing code. You can then see the
current, real-time values of relevant variables in a window as shown
here, or have other sections of code listen for value changed events so
they can update themselves as well. Note in the screen shot the
recursive structure of embedded registries to keep related variables
together. Custom GUIs can be created to visualize more complex
Unfortunately, the screen shot isn't nearly as exciting as the
real thing, since the whole point is that the values automatically
update. To demo this, the ChaseBallBehavior (under the Mode
Switch menu) features two variables that store the last known veritcal
and horizontal position of the ball. Launch the ChaseBallBehavior
and then launch "View WMVars" under the TekkotsuMon menu. (you may need
to press "refresh" within the WMVar Viewer to see new variables)
This interface allows you to walk the Aibo around, and is overloaded to
drive a wheeled robot as well - combined
with the vision monitors, it's everything you need for basic
telepresence. Using the right mouse button in the drag area will
temporarily reverse the Rotate/Strafe setting.
Point Remote Control
This interface allows you to control the Aibo's head. It
is very similar to the interface used for the Walk GUI.
Future version may include additional HeadPointerMC
Using 3D models released by Sony, we have build a tool which
will allow you to visualize the Aibo's current posture, or grab joints
and move them around in 3D.
This tool requires java3D to be
installed. (Available for Linux and Windows)
interface allows you to turn the emergency stop on/off
remotely. This tool is also embedded in the Controller
GUI, in the lower left hand
By alt-clicking the button in ControllerGUI, a full sized window can
be brought up and made any size you please.
No roboticist should be without a big red panic button! ;)
This very simple class
from the robot to
messages up on
the screen. In the future, anything sent to serr (port 10002) will be
brought up in a console similar to this to further streamline
TekkotsuMon - MATLAB
NOTE: The MATLAB tools are currently
broken in version 1.5. There is a work around: copy all of the *Listener.java files from tools/mon/org/tekkotsu/mon/ to tools/mon (with the MATLAB .m files)
In order to use
TekkotsuMon, you need a reasonably fast computer running Matlab 6.5 or
higher. If you want to modify or recompile the network communication
classes (written in Java), Change directory to
Edit config.m to specify the Aibo's IP address (conf.ip).
Start Matlab, and at the prompt type
It is important to launch MATLAB from within the tools/mon
directory so that it can find related support files. On Windows,
you will need to make a shortcut to MATLAB and set the work path
("Start in:") to this directory (via right click -> Properties).
This should bring up a control window with buttons for various
TekkotsuMon components found such as Raw Vision, Segmented Vision,
Joint monitor, etc. Start the Aibo with Tekkotsu (wait for it to boot),
and click on any button to start that component.
Stuff - Currently Broken
An earlier MATLAB prototype for the Java WalkGUI.
However, this interface has easily customizable buttons, which are
currently set up to play sounds. (To play sounds in the Java
tools, use the Controller to open the File Access -> Play Sound
menu, which will show a listing of all available sounds.)
These graphs show the
(red) on each
of the joints. This graph was taken during a run of the
FollowHeadBehavior - look at the neck joints to see the different
directions that the head was pointed in.
A similar graph is available for PID settings.
Similar to the Java tools, this should help you quickly get
prototype MATLAB code for processing image data.