FileSystemDataSource Class Reference
Manages the loading of a series of files from disk. More...
#include <FileSystemDataSource.h>
Detailed Description
Manages the loading of a series of files from disk.
Can handle an index file listing other data files and their timestamps, a directory holding a set of files, or a single explicit file.
Definition at line 17 of file FileSystemDataSource.h.
Classes | |
| struct | DataThread |
| struct | FileInfo |
| stores basic information regarding each file in the queue, including routines for loading to and from disk More... | |
Public Member Functions | |
| FileSystemDataSource (LoggedDataDriver &p, const std::string &filter) | |
| constructor | |
| ~FileSystemDataSource () | |
| destructor | |
| virtual unsigned int | nextTimestamp () |
| Returns the simulator time of the next data segment. | |
| virtual const std::string & | nextName () |
| Returns a descriptive name of the next data segment for user feedback (e.g. a filename for logged data). | |
| virtual bool | advance () |
| Called when the simulator is stepping while paused or advancing a frozen data source, return true if successful, or false if no more data is available. | |
| virtual void | registerSource () |
| User hook, called when the data source should claim which outputs it provides feedback (providingOuput()). | |
| virtual bool | isRegistered () const |
| returns registered | |
| virtual void | deregisterSource () |
| User hook, called when the data source should release its claim on outputs with feedback (ignoringOuput()). | |
| virtual void | enteringRealtime (const plist::Primitive< double > &simTimeScale) |
| User hook, called when the controller is going to be running in realtime mode, which is probably the normal mode you'd expect. | |
| virtual void | leavingRealtime (bool) |
| User hook, called when leaving realtime mode, which means you have no idea when motionCheck() is going to be called in terms of wall-clock time. | |
| virtual void | setFrame (unsigned int f, unsigned int numPreload=2) |
| sets the next frame to be sent (e.g. pass 0 to reset to the first frame) | |
| virtual void | nextFrame (unsigned int numPreload=2) |
| increments curfile to the next frame, preparing (pre-loads) numPreload frames from the new curfile onward | |
| virtual double | getLoopTime (bool actual=true) const |
| returns the total time taken to loop over the files | |
| virtual void | setLoopTime (double t) |
| adds time to the final frame to increase total sequence time to t milliseconds | |
| virtual bool | usingIndexFile () const |
| returns indexed | |
| void | clearFiles () |
| empties files | |
| virtual void | plistValueTouched (const plist::PrimitiveBase &pl) |
| virtual void | plistValueChanged (const plist::PrimitiveBase &pl) |
| virtual void | loadXML (xmlNode *node) |
| virtual const std::string & | getUsedPath () const |
| returns the target path, either path, or parent.path if path is empty | |
| virtual bool | loadFileList (bool clearCurrent=true, bool reportMissing=true) |
| call this to (re)load the list of available file names from disk | |
Public Attributes | |
| plist::Primitive< std::string > | path |
| The directory, data file, or index file from which to load via call to loadFileListFromDirectory(), loadSingleFile(), or loadFileListFromIndex(). | |
| plist::Primitive< std::string > | filenameFilter |
| a regular expression (POSIX.2 extended format) to select which files to load from path, if path is a directory or index file | |
| plist::Primitive< bool > | loop |
| controls whether to restart curfile at the beginning of files when it reaches the end | |
| plist::Primitive< float > | framerate |
| The sensors updates per second which should be loaded. Note this is limited by the global Sensors.Framerate setting. | |
| plist::Primitive< int > | verbose |
| Controls how much feedback to give on the console regarding progress. | |
Protected Types | |
| typedef std::list< FileInfo * > | files_t |
| type of files, the list of files to load | |
Protected Member Functions | |
| virtual bool | sendData ()=0 |
| Gives the data referenced by curfile to the simulator. | |
| virtual void | doFreeze () |
| user hook for when frozen is set to true; advance() will be called by simulator at user discretion. | |
| virtual void | doUnfreeze () |
| user hook for when frozen is set to false; if enteringRealtime() has been called then you should resume sending data. | |
| virtual void | resetPoller () |
| virtual void | loadSingleFile (const std::string &file) |
| load a single file | |
| virtual void | loadFileListFromDirectory () |
| load a list of files from a directory specified by path | |
| virtual bool | loadFileListFromIndex () |
| load a list of files from an index file specified by path | |
| virtual double | calcLoopTime () const |
| adds up the lifetime of all files, plus initialDelay | |
| virtual void | preprepare (const FileSystemDataSource::files_t::iterator &fi) |
| calls prepare on fi, dumping diagnostic info if verbose is high enough | |
| virtual void | enqueueFile (const std::string &name, double lifetime) |
| creates a new entry on files, virtual to allow subclasses to use a FileInfo subclass with more meta data (e.g. see FileSystemImageSource::ImageInfo) | |
Protected Attributes | |
| FileSystemDataSource::DataThread | poller |
| files_t | files |
| files_t::iterator | curfile |
| an iterator referencing files -- indicates next file to send | |
| double | initialDelay |
| milliseconds to wait before sending first frame | |
| double | nextTime |
| timestamp that curfile should be sent | |
| bool | indexed |
| true if the file list was specified by an index file | |
| unsigned int | freezeTime |
| time at which doFreeze was called | |
| const plist::Primitive< double > * | timeScale |
| simulator's timeScale parameter, so we can register for updates when it changes | |
| LoggedDataDriver & | parent |
| device driver this is a member of | |
| double | actualLoopTime |
| time it would take to run through all of the loaded frames | |
| double | naturalLoopTime |
| time it would take to run through all of the frames as set immediately after load (i.e. no setLoopTime()) | |
| Thread::Lock | lock |
| don't be in advance() while modifying file list | |
| bool | registered |
| synced to registerSource/deregisterSource so we don't start the thread if not actually in use | |
Static Protected Attributes | |
| static const unsigned int | MAX_LOAD = 4000 |
| maximum number of data elements to try to keep 'active'. If there's more than this in files, we'll only load one at time, and immediately release it afterward | |
Private Member Functions | |
| FileSystemDataSource (const FileSystemDataSource &) | |
| don't call | |
| FileSystemDataSource & | operator= (const FileSystemDataSource &) |
| don't call | |
Member Typedef Documentation
type of files, the list of files to load
Definition at line 98 of file FileSystemDataSource.h.
Constructor & Destructor Documentation
| FileSystemDataSource | ( | LoggedDataDriver & | p, | |
| const std::string & | filter | |||
| ) |
constructor
Definition at line 20 of file FileSystemDataSource.h.
| ~FileSystemDataSource | ( | ) |
destructor
Definition at line 21 of file FileSystemDataSource.cc.
| FileSystemDataSource | ( | const FileSystemDataSource & | ) | [private] |
don't call
Member Function Documentation
| bool advance | ( | ) | [virtual] |
Called when the simulator is stepping while paused or advancing a frozen data source, return true if successful, or false if no more data is available.
Implements DataSource.
Definition at line 41 of file FileSystemDataSource.cc.
Referenced by DataThread::run(), and setFrame().
| double calcLoopTime | ( | ) | const [protected, virtual] |
adds up the lifetime of all files, plus initialDelay
Definition at line 257 of file FileSystemDataSource.cc.
Referenced by loadFileList().
| void clearFiles | ( | ) |
empties files
Definition at line 277 of file FileSystemDataSource.cc.
Referenced by loadFileList(), LoggedDataDriver::plistValueChanged(), and ~FileSystemDataSource().
| void deregisterSource | ( | ) | [virtual] |
User hook, called when the data source should release its claim on outputs with feedback (ignoringOuput()).
It would be wise to call this from your destructor as well.
Reimplemented from DataSource.
Reimplemented in FileSystemSensorSource.
Definition at line 96 of file FileSystemDataSource.cc.
| void doFreeze | ( | ) | [protected, virtual] |
user hook for when frozen is set to true; advance() will be called by simulator at user discretion.
Reimplemented from DataSource.
Definition at line 120 of file FileSystemDataSource.cc.
| void doUnfreeze | ( | ) | [protected, virtual] |
user hook for when frozen is set to false; if enteringRealtime() has been called then you should resume sending data.
Reimplemented from DataSource.
Definition at line 124 of file FileSystemDataSource.cc.
| virtual void enqueueFile | ( | const std::string & | name, | |
| double | lifetime | |||
| ) | [protected, virtual] |
creates a new entry on files, virtual to allow subclasses to use a FileInfo subclass with more meta data (e.g. see FileSystemImageSource::ImageInfo)
Reimplemented in FileSystemImageSource, and FileSystemSensorSource.
Definition at line 168 of file FileSystemDataSource.h.
Referenced by loadFileListFromDirectory(), loadFileListFromIndex(), and loadSingleFile().
| void enteringRealtime | ( | const plist::Primitive< double > & | ) | [virtual] |
User hook, called when the controller is going to be running in realtime mode, which is probably the normal mode you'd expect.
You might be in realtime mode, but a debugger breakpoint will still pause things, or thread scheduling could hiccup, so try to be robust.
The argument is a reference to SharedGlobals::timeScale, so the data source can subscribe to changes in simulation speed if it can use that information. (We avoid direct dependency on the tekkotsu simulator so this code can be reused for other tools too.)
Reimplemented from DataSource.
Definition at line 108 of file FileSystemDataSource.cc.
| virtual double getLoopTime | ( | bool | actual = true |
) | const [virtual] |
returns the total time taken to loop over the files
- Parameters:
-
actual if false, will return the "natural" time of a loop, ignoring setLoopTime() effects
Definition at line 54 of file FileSystemDataSource.h.
Referenced by advance(), nextFrame(), LoggedDataDriver::plistValueChanged(), and setLoopTime().
| const std::string & getUsedPath | ( | ) | const [virtual] |
returns the target path, either path, or parent.path if path is empty
Definition at line 144 of file FileSystemDataSource.cc.
Referenced by loadFileList(), loadFileListFromDirectory(), loadFileListFromIndex(), and LoggedDataDriver::plistValueChanged().
| virtual bool isRegistered | ( | ) | const [virtual] |
returns registered
Definition at line 40 of file FileSystemDataSource.h.
Referenced by LoggedDataDriver::plistValueChanged().
| void leavingRealtime | ( | bool | ) | [virtual] |
User hook, called when leaving realtime mode, which means you have no idea when motionCheck() is going to be called in terms of wall-clock time.
Argument set to true if entering full speed mode, which indicates everything should run at full native "frame rate", and may indicate more data will be processed than normal, CPU speed permitting. However, if false, almost certainly indicates updates will be sparse, trigger by user 'step' commands. May be called multiple times if changing between full-speed mode and paused
A non-realtime mode might be triggered if the user wants to pause the simulator/controller to step through something... No guarantees though! The debugger might catch a breakpoint and stop things, and this won't be called!
Reimplemented from DataSource.
Definition at line 112 of file FileSystemDataSource.cc.
Referenced by deregisterSource(), and ~FileSystemDataSource().
call this to (re)load the list of available file names from disk
If clearCurrent is set, then the current file list will be cleared; otherwise, the loaded files will be appended to the current queue
Definition at line 146 of file FileSystemDataSource.cc.
Referenced by LoggedDataDriver::plistValueChanged(), plistValueChanged(), and plistValueTouched().
| void loadFileListFromDirectory | ( | ) | [protected, virtual] |
load a list of files from a directory specified by path
Definition at line 338 of file FileSystemDataSource.cc.
Referenced by loadFileList().
| bool loadFileListFromIndex | ( | ) | [protected, virtual] |
load a list of files from an index file specified by path
This supports either the format produced by VisionGUI, or a simplier 'filename [<tab> time]\n' format, where if time is unspecified, the frame's time is incremented by the framerate from the previously listed file. Filenames should either be either absolute paths or relative to the directory which contains the index file.
Definition at line 401 of file FileSystemDataSource.cc.
Referenced by loadFileList().
| void loadSingleFile | ( | const std::string & | file | ) | [protected, virtual] |
load a single file
Definition at line 332 of file FileSystemDataSource.cc.
Referenced by loadFileList().
| void loadXML | ( | xmlNode * | node | ) | [virtual] |
Definition at line 321 of file FileSystemDataSource.cc.
| void nextFrame | ( | unsigned int | numPreload = 2 |
) | [virtual] |
increments curfile to the next frame, preparing (pre-loads) numPreload frames from the new curfile onward
Definition at line 229 of file FileSystemDataSource.cc.
Referenced by advance().
| const std::string & nextName | ( | ) | [virtual] |
Returns a descriptive name of the next data segment for user feedback (e.g. a filename for logged data).
Just use your class name if you don't have a useful name for individual samples.
Implements DataSource.
Definition at line 32 of file FileSystemDataSource.cc.
| unsigned int nextTimestamp | ( | ) | [virtual] |
Returns the simulator time of the next data segment.
Should be in the future if nothing new since last data segment, otherwise should be the timestamp of the most recent data segment (older segments are skipped), return -1U if there is no more data
Implements DataSource.
Definition at line 28 of file FileSystemDataSource.cc.
Referenced by plistValueChanged(), resetPoller(), and DataThread::run().
| FileSystemDataSource& operator= | ( | const FileSystemDataSource & | ) | [private] |
don't call
| void plistValueChanged | ( | const plist::PrimitiveBase & | pl | ) | [virtual] |
Implements PrimitiveListener.
Definition at line 294 of file FileSystemDataSource.cc.
Referenced by loadXML(), and registerSource().
| void plistValueTouched | ( | const plist::PrimitiveBase & | pl | ) | [virtual] |
Reimplemented from PrimitiveListener.
Definition at line 288 of file FileSystemDataSource.cc.
| void preprepare | ( | const FileSystemDataSource::files_t::iterator & | fi | ) | [protected, virtual] |
calls prepare on fi, dumping diagnostic info if verbose is high enough
Definition at line 556 of file FileSystemDataSource.cc.
Referenced by advance(), loadFileList(), nextFrame(), and setFrame().
| void registerSource | ( | ) | [virtual] |
User hook, called when the data source should claim which outputs it provides feedback (providingOuput()).
Does not indicate the data source should start sending updates yet — wait for enteringRealtime() or advance() to be called
Reimplemented from DataSource.
Reimplemented in FileSystemSensorSource.
Definition at line 85 of file FileSystemDataSource.cc.
| void resetPoller | ( | ) | [protected, virtual] |
Definition at line 129 of file FileSystemDataSource.cc.
Referenced by doFreeze(), doUnfreeze(), enteringRealtime(), loadFileList(), and plistValueChanged().
| virtual bool sendData | ( | ) | [protected, pure virtual] |
Gives the data referenced by curfile to the simulator.
Implemented in FileSystemImageSource, and FileSystemSensorSource.
Referenced by advance().
| void setFrame | ( | unsigned int | f, | |
| unsigned int | numPreload = 2 | |||
| ) | [virtual] |
sets the next frame to be sent (e.g. pass 0 to reset to the first frame)
prepares (pre-loads) numPreload frames from the new curfile onward
Definition at line 210 of file FileSystemDataSource.cc.
| void setLoopTime | ( | double | t | ) | [virtual] |
adds time to the final frame to increase total sequence time to t milliseconds
Definition at line 265 of file FileSystemDataSource.cc.
Referenced by LoggedDataDriver::plistValueChanged().
| virtual bool usingIndexFile | ( | ) | const [virtual] |
returns indexed
Definition at line 57 of file FileSystemDataSource.h.
Referenced by plistValueChanged().
Member Data Documentation
double actualLoopTime [protected] |
time it would take to run through all of the loaded frames
Definition at line 178 of file FileSystemDataSource.h.
Referenced by clearFiles(), getLoopTime(), loadFileList(), and setLoopTime().
files_t::iterator curfile [protected] |
an iterator referencing files -- indicates next file to send
Definition at line 171 of file FileSystemDataSource.h.
Referenced by advance(), clearFiles(), loadFileList(), nextFrame(), nextName(), nextTimestamp(), plistValueChanged(), preprepare(), and setFrame().
| plist::Primitive<std::string> filenameFilter |
a regular expression (POSIX.2 extended format) to select which files to load from path, if path is a directory or index file
Definition at line 80 of file FileSystemDataSource.h.
Referenced by deregisterSource(), FileSystemDataSource(), loadFileList(), loadFileListFromDirectory(), loadFileListFromIndex(), plistValueChanged(), and registerSource().
Definition at line 170 of file FileSystemDataSource.h.
Referenced by advance(), calcLoopTime(), clearFiles(), FileSystemSensorSource::enqueueFile(), FileSystemImageSource::enqueueFile(), enqueueFile(), loadFileList(), loadFileListFromIndex(), nextFrame(), nextName(), nextTimestamp(), plistValueChanged(), preprepare(), setFrame(), and setLoopTime().
The sensors updates per second which should be loaded. Note this is limited by the global Sensors.Framerate setting.
Definition at line 86 of file FileSystemDataSource.h.
Referenced by deregisterSource(), FileSystemDataSource(), loadFileListFromDirectory(), loadFileListFromIndex(), loadSingleFile(), plistValueChanged(), and registerSource().
unsigned int freezeTime [protected] |
time at which doFreeze was called
Definition at line 175 of file FileSystemDataSource.h.
Referenced by doFreeze(), doUnfreeze(), loadFileList(), and setFrame().
true if the file list was specified by an index file
Definition at line 174 of file FileSystemDataSource.h.
Referenced by loadFileListFromDirectory(), loadFileListFromIndex(), loadSingleFile(), and usingIndexFile().
double initialDelay [protected] |
milliseconds to wait before sending first frame
Definition at line 172 of file FileSystemDataSource.h.
Referenced by calcLoopTime(), clearFiles(), loadFileListFromIndex(), nextFrame(), and preprepare().
Thread::Lock lock [protected] |
don't be in advance() while modifying file list
Definition at line 180 of file FileSystemDataSource.h.
Referenced by advance(), clearFiles(), loadFileList(), loadFileListFromDirectory(), loadFileListFromIndex(), loadSingleFile(), nextFrame(), resetPoller(), setFrame(), and ~FileSystemDataSource().
controls whether to restart curfile at the beginning of files when it reaches the end
Definition at line 83 of file FileSystemDataSource.h.
Referenced by deregisterSource(), FileSystemDataSource(), nextFrame(), plistValueChanged(), registerSource(), and setFrame().
const unsigned int MAX_LOAD = 4000 [static, protected] |
maximum number of data elements to try to keep 'active'. If there's more than this in files, we'll only load one at time, and immediately release it afterward
Definition at line 97 of file FileSystemDataSource.h.
Referenced by nextFrame(), and setFrame().
double naturalLoopTime [protected] |
time it would take to run through all of the frames as set immediately after load (i.e. no setLoopTime())
Definition at line 179 of file FileSystemDataSource.h.
Referenced by clearFiles(), getLoopTime(), and loadFileList().
double nextTime [protected] |
timestamp that curfile should be sent
Definition at line 173 of file FileSystemDataSource.h.
Referenced by advance(), deregisterSource(), doUnfreeze(), loadFileList(), loadFileListFromIndex(), nextFrame(), nextTimestamp(), plistValueChanged(), preprepare(), registerSource(), FileSystemImageSource::sendData(), and setFrame().
LoggedDataDriver& parent [protected] |
device driver this is a member of
Definition at line 177 of file FileSystemDataSource.h.
Referenced by getUsedPath(), nextFrame(), and plistValueChanged().
| plist::Primitive<std::string> path |
The directory, data file, or index file from which to load via call to loadFileListFromDirectory(), loadSingleFile(), or loadFileListFromIndex().
A single file can be either a single data file (e.g. sensor or camera image), or an index file as output by VisionGUI, or in the format 'filename <tab> time', where 'filename' is an absolute path or relative to the directory containing the index file, and 'time' is in milliseconds, relative to the time at which the index file is loaded.
In the future, this could also be network addresses for teleoperation and remote processing.
Definition at line 77 of file FileSystemDataSource.h.
Referenced by deregisterSource(), FileSystemDataSource(), getUsedPath(), loadXML(), LoggedDataDriver::LoggedDataDriver(), LoggedDataDriver::plistValueChanged(), plistValueChanged(), plistValueTouched(), and registerSource().
FileSystemDataSource::DataThread poller [protected] |
Referenced by leavingRealtime(), plistValueChanged(), and resetPoller().
bool registered [protected] |
synced to registerSource/deregisterSource so we don't start the thread if not actually in use
Definition at line 181 of file FileSystemDataSource.h.
Referenced by deregisterSource(), isRegistered(), registerSource(), and resetPoller().
const plist::Primitive<double>* timeScale [protected] |
simulator's timeScale parameter, so we can register for updates when it changes
Definition at line 176 of file FileSystemDataSource.h.
Referenced by deregisterSource(), enteringRealtime(), leavingRealtime(), plistValueChanged(), resetPoller(), and ~FileSystemDataSource().
| plist::Primitive<int> verbose |
Controls how much feedback to give on the console regarding progress.
0 - none
1 - report when messages are dropped
2 - also report when a message is sent
3 - also report when loop or end-of-log occurs
4 - also report when each message is preloaded
Definition at line 94 of file FileSystemDataSource.h.
Referenced by advance(), FileSystemDataSource(), nextFrame(), and preprepare().
The documentation for this class was generated from the following files:
