Mirage Setup and Usage

Mirage is a simulation environment which can host multiple agents in a virtual world.

Features

  • Simulated cameras render image stream for client computer vision. Video: MP4 (High-Res), YouTube
  • Collision detection and dynamics from the Bullet physics engine. Video: MP4 (High-Res), YouTube
  • Automatic camera tracking for mobile robots. Video: MP4 (High-Res), YouTube
  • Preview collision shapes and verify kinematic configuration. Screenshots
  • "Invisible" objects shown in Mirage window but not rendered by simulated cameras — provides environment markup for user feedback and debugging without interfering with client execution. Screenshots

Building Mirage From Source

  1. You will need to install the following packages on your machine.
    Please consult with the corresponding projects if you have trouble with their installation.
    • Ogre3d
      • Linux Instructions - note you will need to install from source to get the latest version; at last check none of the major distributions carry a reasonably recent build in their package managers, but feel free to double check this first. (you would want 1.7.x or newer) This also implies installation of Ogre's prerequisistes, such as OIS.
      • Mac OS X Instructions - note the copy of OIS distributed with the pre-compiled Ogre SDK (as of 1.7.3) is outdated, and awkward to use as a “naked” static library and header files. Instead, our Xcode project is configured to use a framework:
    • Bullet Physics Library
      • Mirage has been developed with Bullet 2.78. Newer versions may or may not work, but should be fine.
      • It is recommended to install as shared libraries (pass -DBUILD_SHARED_LIBS=ON -DINSTALL_LIBS=ON to cmake)
      • Mac OS X users should build as frameworks (namely, add -DFRAMEWORK=ON as well as above; see Framework Build Recipe in Bullet's installation instructions for full command)
  2. The Mirage source is distributed as part of Tekkotsu, in the tools/mirage directory. If you haven't already, you can find instructions for downloading Tekkotsu.

  3. Linux Instructions (using make):

    1. If you have not already built any Tekkotsu projects, you will need to build some helper tools first:

      cd TEKKOTSU_ROOT/tools
      make
    2. If you installed Ogre or Bullet to non-standard locations, you will need to set the OGRE_ROOT and/or BULLET_ROOT environment variables, or edit the default values (/usr/local) in TEKKOTSU_ROOT/tools/mirage/Makefile.

    3. Now we can build Mirage itself:

      cd TEKKOTSU_ROOT/tools/mirage
      make
    4. Uncomment the PluginFolder setting in plugins.cfg by removing the # character, and change the value to OGRE_ROOT/lib/OGRE. (where OGRE_ROOT is the location of your installation, e.g. if your OGRE_ROOT is /usr/local, then the plugin directory would be /usr/local/lib/OGRE).

  4. Mac OS X Instructions (using Xcode)

    1. Open TEKKOTSU_ROOT/tools/mirage/Make.xcodeproj
    2. You should have installed Ogre, Bullet, and OIS as framework bundles in /Library/Frameworks. If you didn't, you can change the library locations in the ▸FrameworksLinked Frameworks file group in the navigation panel on the left. (delete old references and drag-and-drop from current locations, or right click, "Get Info", "Choose")
    3. Choose "Build" from the "Build" menu to compile.
      • As a command line alternative, try running xcodebuild from within the mirage directory. (You don't need to be running Xcode)
    4. The application will be placed at TEKKOTSU_ROOT/tools/mirage/build/Debug/Mirage.app. You can run it from the Finder, or choose "Run" from within Xcode. (or the open command from the command line…)
  5. The Mirage WorldBuilder tool is included in the Tekkotsu/rools/mirage directory, but it depends on some Ruby packages that are not installed by default. To install these packages, do:

    sudo apt-get install ruby rubygems
    sudo gem install plist treetop

Running Mirage

    • Mac OS X Users: Run Mirage from within Xcode, or double click the application from the Finder.

    • Linux Users: Launch mirage from within the TEKKOTSU_ROOT/tools/mirage directory:

      cd TEKKOTSU_ROOT/tools/mirage
      ./mirage

    You will first see an Ogre configuration window—you will probably want to disable fullscreen mode and reduce the window resolution.

    Afterward, a window should appear, displaying the Mirage 3D environment.

  1. Camera Controls:

    W
    move forward
    S
    move backward
    A
    move left
    D
    move right
    PgUp
    move up
    PgDn
    move down
    pitch up
    pitch down
    yaw left
    yaw right
    U
    unfollow the robot (if a tekkotsu robot has connected)
    See Advanced Usage below for more controls
  2. Loading custom environments

    You can load environment files either by passing file names as arguments on the command line:

    ./mirage worlds/tictactoe.mirage

    Mac OS X users can also double-click or drag-and-drop a file on the application to launch with that environment. Mirage can load .kin kinematic files, or its own .mirage environment files. You can load multiple files to combine their effects.

    See the Configuration section below for more information on customizing Mirage's environment, as well as the World Builder tool.

  3. Connecting Tekkotsu

    We will now launch Tekkotsu and tell it to connect to Mirage. If you have not yet built and run a Tekkotsu project, see these instructions for help. Alternatively, see Advanced Usage below for other examples of clients which can connect to Mirage (e.g. mazegen).

    In a new terminal, change to your project directory and run Tekkotsu with the -c mirage.plist arguments.

    cd PROJECT
    ./tekkotsu-TARGET -c mirage.plist

    This will load the hardware abstraction layer settings from the mirage.plist file, so Tekkotsu will talk to Mirage instead of looking for "real" hardware. It is possible to run Mirage in parallel with physical hardware, or on a remote machine, by customizing the HAL settings at the Tekkotsu command prompt.

  • The robot should have appeared in the Mirage window. If you can't find it, try relaunching Mirage to reset the camera to look at the origin, where the robot should appear.
  • You can use the ControllerGUI to control the simulated robot, simply pass localhost for the robot address when launching ControllerGUI (more instructions on using ControllerGUI). Try clicking the "Raw Cam" button to bring up the rendered camera image.

Advanced Mirage Usage

  • By default, Mirage opens with two objects in the world: a single light (light1), and a ground plane with a reference frame at its origin (ground). You can create property list XML files which modify a number of parameters, documented below in “Environmental Configuration”.
  • A dynamically generated maze
    Generated by the mazegen tool
    The default Mirage server port is 19785. You can connect additional "agents" to the world to serve as dynamic targets, construct procedural objects (such as the maze to the right), or have multiple full-fledged robots. These agents can be written as Tekkotsu behaviors using Mirage for user feedback, or as separate programs which serve a specific function.
  • Press the 'H' key to toggle view of the simplified collision shapes instead of graphical models.
  • Press the 'B' key to toggle view of the link bounding boxes.
  • '1' (the number one) will set the camera to rotate-follow RobotID-1 (e.g. handy if you've lost the robot or want to automatically track a mobile robot.) Eventually, this will be extended for multiple robots, so pressing a number key will rotate the camera to track the corresponding robot.
  • The 'F' key will follow any object at center of the window.
    • Pressing 'F' by itself will simply select the object, displaying its bounding box.
    • Ctrl-F will track the object by moving the camera (but not changing its orientation).
    • Alt-F (aka Option-F) will track the object by rotating the camera (but not changing its position).
    • Ctrl-Alt-F will lock the camera to the object, so that it moves the same as the object.
    When a client disconnects, Mirage will attempt to resume its following if the client reconnects. Note Mirage will follow a specific part of the a target structure, so you can explicitly follow the gripper or other appendage. Otherwise, be sure to target the body itself.
  • 'U' will “unfollow” the object, so the camera will only move under user control. Following will not be resumed on client reconnect.
  • 'R' will reload the initial launch environment: all current clients will be disconnected, and if you double-clicked an environment file or passed a filename as a command line argument, those file(s) will be reloaded. Clients such as the Tekkotsu driver will automatically reconnect. This should be handy for both editing environment files as well as resetting trials when developing algorithms.

Robot Locomotion

Mirage supports two forms of robot locomotion: pure physics-based simulation, and “assisted” locomotion. The former is useful if you have an accurate model of your robot's mass distribution and you want to test the stability of different gaits. The latter “assisted” method uses hints from the walk engine to hold support feet immobile so that the resulting locomotion is nearly perfect, without slippage or falling over. The assisted method is also the only way to move a robot when it has incomplete physics information, such as unspecified mass or collision shapes.

Further, robots can be wheeled or legged. The effects of various combinations of wheeled, legged, with or without physics configuration, and with or without locomotion assistance is listed in the following table:

The “Mass” column refers to LinkComponents (and thus derived KinematicJoints) which have non-zero Mass specified. In this column, the value “Partial” means the root KinematicJoint (BaseFrame) of the robot DOES NOT have mass, but all of its sub-links (i.e. moving parts) DO have mass.

The “Shape” column refers to LinkComponents (and thus derived KinematicJoints) which have a CollisionModel specification. The regular Model is used for display only, and is typically too high-resolution to use for physics simulation. You must specify one or more primitive collision shapes which approximate the display model in order to allow physical simulation of object interaction. (hint: the ‘H’ key will switch to displaying collision shapes instead of the usual display models.)

The “Assisted” column refers to whether the MirageDriver's PhysicsWalk or PhysicsWheels is DISABLED. (e.g. Drivers.Mirage.PhysicsWalk=false) If appropriate physics setting is enabled, then only normal physical simulation is applied, allowing ground contacts to slip. If physics interaction is disabled in the driver, then the walk engine's intended foot contacts are sent to Mirage, allowing it to 'assist' the locomotion. As summarized in the table, robots lacking complete physics specification require this assistance in order to move.

Mass Shape Assisted Result Usage CPU
No No No Immobile, no environmental interaction Graphics visualization, vision tests (it’s a “ghost”) Low
No No Yes Ground plane mobility, no environmental interaction Low
No Yes No Immobile, one-way interaction

Simple manipulation tests: poor simulation accuracy (effector has no true momentum, collision contacts are a heuristic)

“One-way interactions” because objects cannot push back, or in other words the robot is infinitely strong.

Low
No Yes Yes Ground plane mobility, one-way interaction Low
Partial* Yes No Immobile, but can physically interact Fixed robots like HandEye: body is bolted down, but limbs can interact with environment Med
Partial* Yes Yes Obstacle-sensitive mobility, allows environmental interaction except with the body itself Almost-perfect locomotion for wheeled and legged robots: will respect obstacles and ground for joint motion, but body will move without physics; wheeled robots can ride over obstacles without risk of high-centering Med
Yes No Cannot use mass without shape (falls through ground, nothing to hold it up)
Yes Yes No Purely physical locomotion: simulates friction, momentum, support Realistic wheeled robot motion, or when developing new legged locomotion gaits (assuming an accurate mass model of the robot!) Med
Yes Yes Yes Semi-physics based locomotion: feet/wheels will not slip but otherwise move under physics control.

Navigation tests for robots when you wish to minimize locomotion error.

This is very similar to the “partial mass” configuration, but allows more realistic interaction between feet, body, and environment (i.e. slightly less ‘perfect’)

High

* Partial indicates that the robot has a zero mass body (base frame) but has positive mass listed for its other links.

The assisted locomotion for legged robots can be controlled via the Mirage driver's PhysicsWalk and PhysicsWheels setting (see 'help set Drivers.Mirage.PhysicsWalk' from the Tekkotsu HAL command line). Enabling PhysicsWalk/Wheels will prevent the Tekkotsu executable from sending foot location or wheel speed hints to Mirage.

Environment Configuration

The Mirage environment begins with a cloudy blue sky, a single light, and a ground plane with a reference frame at the origin. You can change many aspects of the configuration listed below, including adding objects, simulation speed, and graphics options.

A Mirage environment file is an XML format based on the Property List used by Apple for its configuration files. For example, to configure the user's camera starting position and orientation:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
   "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">

<dict>
<key>Observer</key> <dict>
<key>Location</key> <array>
<real>500</real>
<real>0</real>
<real>600</real>
</array>
<key>PointAt</key> <array>
<real>-200</real>
<real>200</real>
<real>0</real>
</array>
</dict>
</dict>
</plist>

Other supported value types include string, integer, true, and false.

The Environment dictionary is the root element of the property list. You should name your property list with the .mirage extension so that the application can be associated with the file. This should allow you to double-click environment files to launch Mirage with that environment.

You do not have to specify all values in a dictionary, unspecified values simply retain their current value. This also allows you to launch with multiple environment files to combine their settings.

Note specialized types:

  • Types labeled “color” below can accept either a string with an HTML style color specification (e.g. #001122) or an array of real values, 0-1 for each color channel.
  • The “…Scale” parameters which are listed as type float[3] can also accept a single real value which will be applied to all three components for uniform scaling.
  • Rotation values are based on the (x,y,z) components of a quaternion. To convert from axis/angle convention with a normalized axis of rotation:
    if |angle| ≤ 180° then q = axis · sin(angle / 2)
    if |angle| > 180°, then flip signs: q = axis · -sin(angle / 2)

Environment Parameters
NameTypeDescription
AmbientLight color Color of unlit areas, see also Shadows. (default #808080)
Background Background Parameters related to the sky coloring.
InertiaDisplayScale float Scale used for displaying mass and inertia, factor to convert inertia to spatial units.
Lights Dictionary of Lights A collection of lights. By default one light is included, 'light1'
MassDisplayScale float Scale used for displaying mass and inertia, factor to convert mass to spatial units.
Objects Dictionary of PhysicalObjects Initial objects to populate the scene, including the ground itself.
Observer Object Position and orientation of user camera.
Physics Physics Parameters related to physics simulation.
RenderFPS float Frames per second for the user window.
Shadows Shadows Parameters related to shadow generation.
TimeScale float Scales time used for physics simulation, e.g. a value of 0.5 runs in slow-motion.
Background Parameters
NameTypeDescription
Color color Solid color if no Material is specified (default black)
Curvature float Curvature used for Plane and Dome models (default -100)
Distance float The distance parameter for Box and Dome models (default 5000)
Material string An Ogre material name (default CloudySky)
Model model_t enumeration Type of background to use (default None)
Value is one of: { None | Plane | Box | Dome}
Tiling unsigned int Tiling factor for Plane and Dome models (default 5)
Object Parameters
NameTypeDescription
Location float[3] Controls position of object
Orientation float[3] The (x,y,z) components of a quaternion controlling object orientation, except if PointAt is set.
PointAt float[3] If this is a length-3 array, will override Orientation to point the z axis at the specified point.
Light Parameters
NameTypeDescription
(Object) Location float[3] Controls position of object
(Object) Orientation float[3] The (x,y,z) components of a quaternion controlling object orientation, except if PointAt is set.
(Object) PointAt float[3] If this is a length-3 array, will override Orientation to point the z axis at the specified point.
AttenuateConst float The constant attenuation factor c: light = 1/(c + l·dist + q·dist²), default 0.35
AttenuateLinear float The linear attenuation factor l: light = 1/(c + l·dist + q·dist²), default 0
AttenuateQuad float The quadratic attenuation factor q: light = 1/(c + l·dist + q·dist²), default 0
AttenuateRange float The maximum range the light will reach before being cut off, default 30000
Color color Color of the light, by default #606060.
LinkComponent Parameters
NameTypeDescription
CenterOfMass float[3] Position of average mass relative to parent frame.
CollisionModel string A Bullet primitive collision shape: { Cube | Cylinder | Sphere | Plane }
CollisionModelOffset float[3] Positions the CollisionModel, relative to parent frame.
CollisionModelRotation float[3] Rotates the CollisionModel relative to parent frame. (supply axis component of quaternion, e.g. from axis/angle: axis * sin(angle/2) )
CollisionModelScale float[3] Scales the CollisionModel, which by default is 1x1x1, so this sets the object dimensions.
Mass float Mass of the component, in kilograms. If 0, indicates static object, does not move but can still collide. (default 0)
Material string Name of an Ogre material, found in mirage/media/*.material files.
Model string An Ogre .mesh file, or "CollisionModel" to render the collision primitive. (default CollisionModel)
ModelOffset float[3] Positions the graphics mesh loaded by Model, relative to parent frame.
ModelRotation float[3] Rotates the graphics mesh loaded by Model, relative to parent frame. (supply axis component of quaternion, e.g. from axis/angle: axis * sin(angle/2) )
ModelScale float[3] Scales the graphics mesh loaded by Model.
Visible bool If true, indicates component should be rendered for simulated cameras, otherwise only appears in the Mirage user window. (default true)
PhysicalObject Parameters
NameTypeDescription
(LinkComponent) * Inherits all LinkComponent fields: for a single component, just describe directly at the root, otherwise add to Components list (or use Kinematics to load a file).
(Object) Location float[3] Controls position of object
(Object) Orientation float[3] The (x,y,z) components of a quaternion controlling object orientation, except if PointAt is set.
(Object) PointAt float[3] If this is a length-3 array, will override Orientation to point the z axis at the specified point.
Components LinkComponent array Directly define a series of LinkComponents to construct the object.
Kinematics string Pass the path of a .kin kinematics file to load for this object.
Physics Parameters
NameTypeDescription
ERP float Error resolution proportion, analgous to gradient descent step size.
Larger values can help reduce solver iterations, but can cause jitter/instability if too high
Gravity float Controls the force of gravity along z vector in meters per second. (default -9.80665)
MassScale float Controls an inertial scaling factor for simulation stability, applied to kilogram scale (default 1, i.e. internal base unit of 1 kilogram for physics)
SolverIterations unsigned int Maximum number of iterations for the constraint solver.
Increasing this will resolve 'breaking' constraints, such as servos under load
SpaceScale float Controls a spatial scaling factor for simulation stability, applied to millimeter scale used for graphics. (default 0.01, i.e. 10cm internal base unit for physics)
StepsPerFrame unsigned int Increases simulation accuracy by subdividing each graphics update into several physics updates. This can have subtle effects on the outcome of the simulation. (default 4)
Shadows Parameters
NameTypeDescription
Color color If Modulative is set, this controls the color of the shadow.
Enabled bool Must be set to true for any of the other parameters to apply (default true)
Modulative bool If true uses simpler method where shadows are subtracted from final scene, otherwise uses more accurate additive model where lit areas are rendered for each light and added together.
Stencil bool If true uses stencil shadow generation (CPU based), otherwise uses texture shadow generation (more GPU based); default true

Downloads

Last modified: 2012-03-17