Homepage
Demos
Overview
Downloads
Dev. Resources
Reference
Credits

FilterBankGenerator.h

Go to the documentation of this file.
00001 //-*-c++-*-
00002 #ifndef INCLUDED_FilterBankGenerator_h_
00003 #define INCLUDED_FilterBankGenerator_h_
00004 
00005 #include "Events/EventGeneratorBase.h"
00006 #include "Shared/LoadSave.h"
00007 
00008 //! Abstract base class for generators of FilterBankEvent's
00009 /*! This is needed to provide an interface for the FilterBankEvent to
00010  *  call back when the actual image data is requested from it.  This
00011  *  facilitates lazy calculation of image data...  no sense in
00012  *  processing layers or channels which aren't actually going to be
00013  *  used...
00014  *
00015  *  Also this way we save on allocating/deallocating large memory
00016  *  blocks on each event... the buffers allocated here can be reused
00017  *  frame to frame.
00018  *
00019  *  Larger layer indicies generally indicate higher resolution images
00020  *  in a scaling pyramid, but you are free to store your own data
00021  *  however you wish.
00022  *
00023  *  <h3>Serialization Format</h3>
00024  *
00025  *  First, be sure to get a good overview of the LoadSave style.  Most
00026  *  serialization is handled using this interface.
00027  *
00028  *  When, for instance, RawCameraGenerator::SaveBuffer() is called, it
00029  *  first calls it's super class, FilterBankGenerator::SaveBuffer(),
00030  *  which will write out the general image information, common to all
00031  *  subclasses of FilterBankGenerator. (i'll cover the specifics in a
00032  *  second) Once that's done, the RawCameraGenerator adds it's own bit
00033  *  of header and then saves the image data itself.
00034  *
00035  *  Note that only a single channel is being saved at this point.  So
00036  *  for instance, all the Y information.  No interleaving is going
00037  *  on. (unless you're saving from InterleavedYUVGenerator of course,
00038  *  which treats the 3 interleaved channels as a single image)
00039  *  Otherwise,only one image (selected with selectSaveImage()) of the
00040  *  bank will loaded or saved at a time.
00041  *  
00042  *  So, anyway.  The first header will be the same for all
00043  *  FilterBankGenerator subclasses.  In the specification below, I'm
00044  *  going to use one field per line (the new lines are not literal,
00045  *  it's a binary stream).  Each field is of the form '<@c type:name>
00046  *  <i>(notes)</i>'
00047  *  
00048  *  FilterBankGenerator Header: (from FilterBankGenerator::SaveBuffer())
00049  *  - <@c string: "FbkImage">  <i>(remember a 'string' is len+str+0; so this is the literal "\010\0\0\0FbkImage\0"; also remember "\010" is octal for 8)</i>
00050  *  - <@c unsigned @c int: width> 
00051  *  - <@c unsigned @c int: height> 
00052  *  - <@c unsigned @c int: image layer> 
00053  *  - <@c unsigned @c int: image channel> <i>(so notice you can tell which channel it is after it's been saved)</i>
00054  * 
00055  *  Generator Specific Header (selected examples follow, or similarly, any of the other generators)
00056  *  
00057  *  - RawCameraGenerator: (from RawCameraGenerator::SaveBuffer())
00058  *    - <@c string: "RawImage">
00059  *    - <<tt>char[</tt>width<tt>*</tt>height<tt>]</tt>: image data> <i>(note, just once channel being stored)</i>
00060  *  - InterleavedYUVGenerator: (from InterleavedYUVGenerator::SaveBuffer())
00061  *    - <@c string: "InterleavedYUVImage">
00062  *    - <<tt>char[</tt>width<tt>*</tt>height<tt>*3]</tt>: image data> <i>(in YVU order, technically YCbCr)</i>
00063  *  - SegmentedColorGenerator: (from SegmentedColorGenerator::SaveBuffer())
00064  *    - <@c string: "SegColorImage">
00065  *    - <<tt>char[</tt>width<tt>*</tt>height<tt>]</tt>: image data> <i>(one byte per sample)</i>
00066  *    - <@c unsigned @c int: num_cols> <i>(number of different colors available)</i>
00067  *    - for each of num_col:
00068  *      - <@c char: red> <i>red color to use for display of this index</i>
00069  *      - <@c char: green> <i>green color to use for display of this index</i>
00070  *      - <@c char: blue> <i>blue color to use for display of this index</i>
00071  *  - RLEGenerator: (from RLEGenerator::SaveBuffer())
00072  *    - <@c string: "RLEImage">  <i>(remember a 'string' is len+str+0; so this is the literal "\010\0\0\0RLEImage\0"; also remember "\010" is octal for 8)</i>
00073  *    - <@c unsigned @c int: num_runs> <i>(how many runs will follow)</i>
00074  *    - for each of num_runs:
00075  *      - <@c char: color> <i>(index value of color of run)</i>
00076  *      - <@c short: x> <i>(x position of start of run ("unknown" runs are skipped - assume index 0 for pixels which are jumped))</i>
00077  *      - <@c short: width> <i>(length of run, will not exceed remaining width of image)</i>
00078  *    - <i>notice there's no color information from RLE - it's not (shouldn't be) assuming anything about the data being compressed)</i>
00079  *
00080  *  However, while we're on the topic, I'll mention that although this
00081  *  is the same image format used for streaming to VisionGUI, there's
00082  *  a few more fields added by RawCamBehavior or SegCamBehavior at the
00083  *  beginning of each packet.  See those classes for more information
00084  *  on the wireless protocol. That should tell you everything you need
00085  *  to know to interpret the vision stream as well.
00086  *
00087  *  <h3>Adding New FilterBankGenerator Subclasses</h3>
00088  *
00089  *  If you're doing fancy memory stuff, you probably want to override
00090  *  the freeCaches() and destruct() functions so that the default
00091  *  implementation won't try to free something it shouldn't.  Don't
00092  *  forget to call them from your own destructor though, otherwise
00093  *  your versions won't get called before the default implementation's
00094  *  does.
00095  *
00096  *  If you want to be able to transmit or save your images, you will
00097  *  need to override the LoadSave functions (listed below) to provide
00098  *  your own code for interpreting the image data itself, and then
00099  *  create or modify a behavior to open a socket and transmit the
00100  *  information.  (you could do that from within the generator itself
00101  *  if you like)
00102  *
00103  *  You will probably also want to add a few extra functions to allow
00104  *  users to set compression/data format parameters.
00105  *
00106  *  @see RawCameraGenerator, SegmentedColorGenerator for the basic
00107  *  image access
00108  * 
00109  *  @see RLEGenerator, RegionGenerator for some relatively simple
00110  *  examples of vision stages if you want to make some of your own.
00111  */
00112 class FilterBankGenerator : public EventGeneratorBase, public LoadSave {
00113 public:
00114   // Constructors are all protected - doesn't make sense to
00115   // instantiate this class directly, you want to use a subclass
00116 
00117   //! destructor
00118   /*! Your own subclasses should also have destructors which call
00119    *  freeCaches() and destruct().  Otherwise, if you override these
00120    *  functions to delete any custom memory you allocate, those
00121    *  implementations won't be called by this destructor... a
00122    *  destructor ignores virtual functions, only calls at its own
00123    *  class level.\n
00124    *  So it really doesn't matter if you aren't allocating any extra
00125    *  memory other than what's in the image cache, but it's still good
00126    *  form just in case you add stuff later so you won't forget and
00127    *  leak memory everywhere */
00128   virtual ~FilterBankGenerator() {
00129     freeCaches();
00130     destruct();
00131   }
00132 
00133   //! returns the generator this is receiving its events from (or the last one anyway)
00134   virtual const FilterBankGenerator * getSourceGenerator() const { return src; }
00135 
00136   //! returns the number of image layers (e.g. different resolutions available)
00137   virtual unsigned int getNumLayers() const { return numLayers; }
00138 
00139   //! returns the number of channels per image (e.g. Y, U, or V components)
00140   virtual unsigned int getNumChannels() const { return numChannels; }
00141   
00142   //! returns pointer to the beginning of the image data for the specified layer and channel
00143   /*! this will cause the data to be calculated and cached if it's not already available */
00144   virtual unsigned char * getImage(unsigned int layer, unsigned int channel);
00145 
00146   //! returns width (in samples) of the image in a given layer
00147   unsigned int getWidth(unsigned int layer) const { return widths[layer]; }
00148 
00149   //! returns height (in samples) of the image in a given layer
00150   unsigned int getHeight(unsigned int layer) const { return heights[layer]; }
00151   
00152   //! returns the bytes to skip from the one-past-end of a row to get the beginning of the next
00153   unsigned int getSkip(unsigned int layer) const { return skips[layer]; }
00154   
00155   //! returns the bytes to skip from the beginning of one row to get the beginning of the next
00156   /*! This is just for convenience; the stride is just the skip plus the width, but it's precomputed for you for speed and clarity */
00157   unsigned int getStride(unsigned int layer) const { return strides[layer]; }
00158 
00159   //! returns the increment (in bytes) to use to go from one sample to the next
00160   unsigned int getIncrement(unsigned int layer) const { return increments[layer]; }
00161   
00162   //! returns the frame number of the current frame, see #frameNumber
00163   unsigned int getFrameNumber() const { return frameNumber; }
00164   
00165   //! returns the number of frames processed, see #framesProcessed
00166   unsigned int getFramesProcessed() const { return framesProcessed; }
00167   
00168   //! returns a pointer to a particular sample; if you are using this in an inner loop, consider using the getSkip() and getIncrement() values to iterate with better performance
00169   /*! @param px      the horizontal pizel position, relative to left edge; no boundary checking is done, ranges 0 through width-1
00170    *  @param py      the vertical pixel position, relative to top edge; no boundary checking is done, ranges 0 through height-1
00171    *  @param layer   the resolution layer to extract from
00172    *  @param channel the image channel to extract from */
00173   unsigned char * getPixel(unsigned int px, unsigned int py, unsigned int layer, unsigned int channel) { return getImage(layer,channel)+py*getStride(layer)+px*getIncrement(layer); }
00174   
00175   //! returns a pointer to a particular sample; if you are using this in an inner loop, consider using the getSkip() and getIncrement() values to iterate with better performance
00176   /*! @param x       the horizontal position, relative to center of the image, left edge is -1 and right edge is 1; no boundary checking is done
00177    *  @param y       the vertical pixel position, relative to center of the image, top edge is the negative aspect ratio, bottom edge is positive aspect ratio; no boundary checking is done
00178    *  @param layer   the resolution layer to extract from
00179    *  @param channel the image channel to extract from
00180    *
00181    *  To keep the coordinate system square, the x is defined to range -1,1, but y's range depends on the
00182    *  aspect ratio of the image, height/width.  Thus typically y will approx. -.75,.75 */
00183   unsigned char * getPixel(float x, float y, unsigned int layer, unsigned int channel) {
00184     unsigned int px,py;
00185     getPixelCoordinates(px,py,x,y,layer);
00186     return getPixel(px,py,layer,channel);
00187   }
00188 
00189   //! sets the pixel-coordinate px and py parameters to the corresponding value of x and y
00190   /*! @param[out] px      the pixel position, relative to left edge, positive right, ranges 0 through width-1
00191    *  @param[out] py      the pixel position, relative to top edge, positive down, ranges 0 through height-1
00192    *  @param[in]  x       the horizontal position, relative to center of the image, left edge is -1 and right edge is 1; no boundary checking is done
00193    *  @param[in]  y       the vertical pixel position, relative to center of the image, top edge is the negative aspect ratio, bottom edge is positive aspect ratio; no boundary checking is done
00194    *  @param[in]  layer   the resolution layer the pixel coordinates are relative to
00195    *
00196    *  To keep the coordinate system square, the x is defined to range -1,1, but y's range depends on the
00197    *  aspect ratio of the image, height/width.  Thus typically y will approx. -.75,.75 */
00198   void getPixelCoordinates(unsigned int& px, unsigned int& py, float x, float y, unsigned int layer) const {
00199     //note width sets the scale for both, so coordinate system is square... is good? I'm up for debate.
00200     px=(unsigned int)((getWidth(layer)-1)*(x+1)/2+.5f); //+.5 to round to nearest
00201     float aspect=getHeight(layer)/(float)getWidth(layer);
00202     py=(unsigned int)((getHeight(layer)-1)*(y+aspect)/(aspect*2)+.5f);
00203   }
00204   
00205   //! sets the x and y parameters from the pixel-coordinates px and py
00206   /*! @param[out] x       the horizontal position, relative to center of the image, left edge is -1 and right edge is 1; no boundary checking is done
00207    *  @param[out] y       the vertical pixel position, relative to center of the image, top edge is the negative aspect ratio, bottom edge is positive aspect ratio; no boundary checking is done
00208    *  @param[in]  px      the pixel position, relative to left edge, positive right, ranges 0 through width-1
00209    *  @param[in]  py      the pixel position, relative to top edge, positive down, ranges 0 through height-1
00210    *  @param[in]  layer   the resolution layer the pixel coordinates are relative to
00211    *
00212    *  To keep the coordinate system square, the x is defined to range -1,1, but y's range depends on the
00213    *  aspect ratio of the image, height/width.  Thus typically y will approx. -.75,.75 */
00214   void getRealCoordinates(float& x, float& y, unsigned int px, unsigned int py, unsigned int layer) const {
00215     //note width sets the scale for both, so coordinate system is square... is good? I'm up for debate.
00216     x=px/(float)(getWidth(layer)-1)*2-1;
00217     float aspect=getHeight(layer)/(float)getWidth(layer);
00218     y=py/(float)(getHeight(layer)-1)*aspect*2-aspect;
00219   }
00220   
00221   //! deletes storage of cached images and marks it invalid
00222   /*! you should override this if the images cache pointer isn't actually an array of bytes... 
00223    *  Don't forget to call it in your subclass's destructor or your version won't get called... */
00224   virtual void freeCaches();
00225 
00226   //! marks all of the cached images as invalid (but doesn't free their memory)
00227   /*! You probably want to call this right before you send the FilterBankEvent */
00228   virtual void invalidateCaches();
00229 
00230   //! default implementation does a few common housekeeping chores for you - probably should just take a look at its code
00231   /*! It doesn't throw any events for you - that's probably the main
00232    *  reason you'd still want to override it\n
00233    *  Also, if your class has a set number of layers or channels - for
00234    *  instance, always 1 channel like InterleavedYUVGenerator, you
00235    *  should override setNumImages() to enforce that constraint by
00236    *  throwing away the appropriate argument and passing the your own
00237    *  value to the superclass implementation.*/
00238   virtual void processEvent(const EventBase & event);
00239   
00240   //!@name LoadSave interface
00241 
00242   virtual unsigned int getBinSize() const;
00243 
00244   virtual unsigned int LoadBuffer(const char buf[], unsigned int len);
00245 
00246   virtual unsigned int SaveBuffer(char buf[], unsigned int len) const;
00247 
00248   //! Not actually part of the LoadSave interface, but allows you to select which image of the bank will be saved
00249   /*! Calling this will also cause the image data for that image to be calculated,
00250    *  otherwise SaveBuffer won't have up-to-date data to save.
00251    *  
00252    *  When loading, the saved image's layer and channel will reset this */
00253   virtual void selectSaveImage(unsigned int layer, unsigned int channel) { selectedSaveLayer=layer; selectedSaveChannel=channel; getImage(layer,channel);}
00254 
00255   virtual unsigned int getSelectedSaveLayer() const { return selectedSaveLayer; } //!< returns layer to be saved, or layer of last image loaded
00256   virtual unsigned int getSelectedSaveChannel() const { return selectedSaveChannel; } //!< returns channel to be saved, or channel of last image loaded
00257 
00258   //@}
00259 
00260 
00261 protected:
00262   //! constructor, separate class and instance names, with a raw event specification, excluding type typically for stages which reference the previous stage's data
00263   FilterBankGenerator(const std::string& classname,const std::string& instancename, EventBase::EventGeneratorID_t mgid, unsigned int msid, EventBase::EventGeneratorID_t srcegid, unsigned int srcsrc)
00264     : EventGeneratorBase(classname, instancename, mgid, msid, srcegid, srcsrc),
00265       src(NULL), numLayers(0), numChannels(0), widths(NULL), heights(NULL), skips(NULL),
00266       strides(NULL), increments(NULL), images(NULL), imageValids(NULL), selectedSaveLayer(0),
00267       selectedSaveChannel(0), frameNumber(0), framesProcessed(0)
00268   { }
00269 
00270   //! constructor, separate class and instance names, with a raw event specification, including type typically for stages which will store their own copy of the data
00271   FilterBankGenerator(const std::string& classname,const std::string& instancename, EventBase::EventGeneratorID_t mgid, unsigned int msid, EventBase::EventGeneratorID_t srcegid, unsigned int srcsrc, EventBase::EventTypeID_t srcetid)
00272     : EventGeneratorBase(classname, instancename, mgid, msid, srcegid, srcsrc, srcetid),
00273       src(NULL), numLayers(0), numChannels(0), widths(NULL), heights(NULL), skips(NULL),
00274       strides(NULL), increments(NULL), images(NULL), imageValids(NULL), selectedSaveLayer(0),
00275       selectedSaveChannel(0), frameNumber(0), framesProcessed(0)
00276   { }
00277 
00278   //! constructor, separate class and instance names, with a filter bank source, passes on all types typically for stages which reference the previous stage's data
00279   FilterBankGenerator(const std::string& classname,const std::string& instancename, EventBase::EventGeneratorID_t mgid, unsigned int msid, FilterBankGenerator * fbgsrc)
00280     : EventGeneratorBase(classname, instancename, mgid, msid, fbgsrc!=NULL?fbgsrc->getGeneratorID():EventBase::numEGIDs, fbgsrc!=NULL?fbgsrc->getSourceID():0),
00281       src(fbgsrc), numLayers(0), numChannels(0), widths(NULL), heights(NULL), skips(NULL),
00282       strides(NULL), increments(NULL), images(NULL), imageValids(NULL), selectedSaveLayer(0),
00283       selectedSaveChannel(0), frameNumber(0), framesProcessed(0)
00284   {
00285     if(src!=NULL)
00286       setNumImages(src->getNumLayers(),src->getNumChannels());
00287   }
00288 
00289   //! constructor, separate class and instance names, with a filter bank source, accepts a particular type typically for stages which will store their own data
00290   FilterBankGenerator(const std::string& classname,const std::string& instancename, EventBase::EventGeneratorID_t mgid, unsigned int msid, FilterBankGenerator * fbgsrc, EventBase::EventTypeID_t etid)
00291     : EventGeneratorBase(classname, instancename, mgid, msid, fbgsrc!=NULL?fbgsrc->getGeneratorID():EventBase::numEGIDs, fbgsrc!=NULL?fbgsrc->getSourceID():0,etid),
00292       src(fbgsrc), numLayers(0), numChannels(0), widths(NULL), heights(NULL), skips(NULL),
00293       strides(NULL), increments(NULL), images(NULL), imageValids(NULL), selectedSaveLayer(0),
00294       selectedSaveChannel(0), frameNumber(0), framesProcessed(0)
00295   {
00296     if(src!=NULL)
00297       setNumImages(src->getNumLayers(),src->getNumChannels());
00298   }
00299 
00300   //! resizes the filter bank information storage area, you should override this to do your setup and call it from your constructor
00301   /*! In general, it isn't expected that FilterBankGenerator's should
00302    *  necessarily be dynamically resizeable (although it would be
00303    *  nice), which is why this isn't public.  If yours is, just add
00304    *  some pubic accessor functions which call this.  In general, the
00305    *  included subclasses should be able to handle being resized, but
00306    *  there's no reason to do so since the system won't be changing
00307    *  its available resolutions at run time. 
00308    *
00309    *  The default implementation is a no-op if(numLayers==nLayers && numChannels==nChannels)
00310    */
00311   virtual void setNumImages(unsigned int nLayers, unsigned int nChannels);
00312 
00313   //! resets width and height parameters to that of the #src
00314   /*! You'll probably want to override this to also set #skips and #strides */
00315   virtual void setDimensions();
00316   
00317   //! create new image data storage area for the cache - this called by getImage() only when the corresponding entry in images is NULL
00318   /*! You should return the pointer you want stored in images to be
00319    *  returned by any calls to getFirstRow.  Interpretation of the
00320    *  data it points to is dependant on the the generator which
00321    *  creates it */
00322   virtual unsigned char * createImageCache(unsigned int layer, unsigned int channel) const=0;
00323 
00324   //! should calculate new image data, called by getImage() only when #imageValids indicates the image being requested is dirty (and only after getImage() has already called createImageCache())
00325   /*! This is where you'll want to put your user-specific code for calculating the image data */
00326   virtual void calcImage(unsigned int layer, unsigned int channel) =0;
00327 
00328   //! deletes the arrays
00329   virtual void destruct();
00330 
00331   //! updates the image data to make sure its up to date with what's available from the source
00332   /*! If someone calls getImage on a stage which hadn't been listening for
00333    *  events (an optimization to save time when it doesn't have any listeners
00334    *  of its own -- see EventGeneratorBase), then this will retroactively
00335    *  pull image data from the source even though the event for it was missed
00336    *
00337    *  @return false if no image data is available yet, true otherwise*/
00338   virtual bool refresh();
00339 
00340 
00341   FilterBankGenerator * src; //!< the generator of the last FilterBankEvent received
00342 
00343   unsigned int numLayers;   //!< current number of layers available
00344   unsigned int numChannels; //!< current number of channels available
00345 
00346   unsigned int * widths;    //!< an array of size numLayers, width (in samples) in pixels of each layer
00347   unsigned int * heights;   //!< an array of size numLayers, height (in samples) in pixels of each layer
00348   unsigned int * skips;     //!< an array of size numLayers, skip (in bytes) from row end to next row begin
00349   unsigned int * strides;   //!< an array of size numLayers, stride (in bytes) from a given column in one row to the same column in the next row
00350   unsigned int * increments;//!< an array of size numLayers, increment (in bytes) to use to get from one sample to the next
00351   
00352   mutable unsigned char *** images; //!< an array [numLayers][numChannels], stores pointer to cached image data
00353   mutable bool ** imageValids;      //!< an array [numLayers][numChannels], entry is true if cached data is still valid
00354 
00355   unsigned int selectedSaveLayer;   //!< layer to be manipulated with the LoadSave functions
00356   unsigned int selectedSaveChannel; //!< channel to be manipulated with the LoadSave functions
00357 
00358   //! the frame number of last frame received by processEvent - subclasses will need to set to the source's frameNumber if they don't call FilterBankGenerator::processEvent()
00359   /*! The idea is to use this as a unique serial number for each
00360    *  frame.  That way you can know if the current image in different
00361    *  generators is actually the same camera image before you try to
00362    *  compare or combine them.
00363    *
00364    *  You could also figure out the number of dropped frames by
00365    *  subtracting framesProcessed from this value.  Give some leeway
00366    *  however, because it takes the first 40-70 frames just to boot up
00367    *  (when running on the aibo), so there's no way they can be
00368    *  processed.
00369    */
00370   unsigned int frameNumber; 
00371 
00372   //! the current frame number available from the system - subclasses which receive data directly from the system should set this (and should not use EventGeneratorBase's auto-listen to ensure this is accurate)
00373   static unsigned int sysFrameNumber;
00374 
00375   //! subclasses should increment this any time they make a new filter bank available
00376   /*! this is automatically incremented if you use the FilterBankGenerator::processEvent() */
00377   unsigned int framesProcessed; 
00378 
00379 private:
00380   FilterBankGenerator(const FilterBankGenerator& fbk); //!< don't call
00381   const FilterBankGenerator& operator=(const FilterBankGenerator& fbk); //!< don't call
00382 };
00383 
00384 /*! @file
00385  * @brief Describes abstract base class for generators of FilterBankEvent's
00386  * @author ejt (Creator)
00387  *
00388  * $Author: ejt $
00389  * $Name: tekkotsu-2_4_1 $
00390  * $Revision: 1.18 $
00391  * $State: Exp $
00392  * $Date: 2005/08/07 04:11:04 $
00393  */
00394 
00395 #endif

Tekkotsu v2.4.1
Generated Tue Aug 16 16:32:47 2005 by Doxygen 1.4.4