Homepage | Demos | Overview | Downloads | Dev. Resources | Reference | Credits |
FilterBankGenerator.hGo to the documentation of this file.00001 //-*-c++-*- 00002 #ifndef INCLUDED_FilterBankGenerator_h_ 00003 #define INCLUDED_FilterBankGenerator_h_ 00004 00005 #include "Events/EventGeneratorBase.h" 00006 #include "Shared/LoadSave.h" 00007 00008 //! Abstract base class for generators of FilterBankEvent's 00009 /*! This is needed to provide an interface for the FilterBankEvent to 00010 * call back when the actual image data is requested from it. This 00011 * facilitates lazy calculation of image data... no sense in 00012 * processing layers or channels which aren't actually going to be 00013 * used... 00014 * 00015 * Also this way we save on allocating/deallocating large memory 00016 * blocks on each event... the buffers allocated here can be reused 00017 * frame to frame. 00018 * 00019 * Larger layer indicies generally indicate higher resolution images 00020 * in a scaling pyramid, but you are free to store your own data 00021 * however you wish. 00022 * 00023 * <h3>Serialization Format</h3> 00024 * 00025 * First, be sure to get a good overview of the LoadSave style. Most 00026 * serialization is handled using this interface. 00027 * 00028 * When, for instance, RawCameraGenerator::SaveBuffer() is called, it 00029 * first calls it's super class, FilterBankGenerator::SaveBuffer(), 00030 * which will write out the general image information, common to all 00031 * subclasses of FilterBankGenerator. (i'll cover the specifics in a 00032 * second) Once that's done, the RawCameraGenerator adds it's own bit 00033 * of header and then saves the image data itself. 00034 * 00035 * Note that only a single channel is being saved at this point. So 00036 * for instance, all the Y information. No interleaving is going 00037 * on. (unless you're saving from InterleavedYUVGenerator of course, 00038 * which treats the 3 interleaved channels as a single image) 00039 * Otherwise,only one image (selected with selectSaveImage()) of the 00040 * bank will loaded or saved at a time. 00041 * 00042 * So, anyway. The first header will be the same for all 00043 * FilterBankGenerator subclasses. In the specification below, I'm 00044 * going to use one field per line (the new lines are not literal, 00045 * it's a binary stream). Each field is of the form '<@c type:name> 00046 * <i>(notes)</i>' 00047 * 00048 * FilterBankGenerator Header: (from FilterBankGenerator::SaveBuffer()) 00049 * - <@c string: "FbkImage"> <i>(remember a 'string' is len+str+0; so this is the literal "\010\0\0\0FbkImage\0"; also remember "\010" is octal for 8)</i> 00050 * - <@c unsigned @c int: width> 00051 * - <@c unsigned @c int: height> 00052 * - <@c unsigned @c int: image layer> 00053 * - <@c unsigned @c int: image channel> <i>(so notice you can tell which channel it is after it's been saved)</i> 00054 * 00055 * Generator Specific Header (selected examples follow, or similarly, any of the other generators) 00056 * 00057 * - RawCameraGenerator: (from RawCameraGenerator::SaveBuffer()) 00058 * - <@c string: "RawImage"> 00059 * - <<tt>char[</tt>width<tt>*</tt>height<tt>]</tt>: image data> <i>(note, just once channel being stored)</i> 00060 * - InterleavedYUVGenerator: (from InterleavedYUVGenerator::SaveBuffer()) 00061 * - <@c string: "InterleavedYUVImage"> 00062 * - <<tt>char[</tt>width<tt>*</tt>height<tt>*3]</tt>: image data> <i>(in YVU order, technically YCbCr)</i> 00063 * - SegmentedColorGenerator: (from SegmentedColorGenerator::SaveBuffer()) 00064 * - <@c string: "SegColorImage"> 00065 * - <<tt>char[</tt>width<tt>*</tt>height<tt>]</tt>: image data> <i>(one byte per sample)</i> 00066 * - <@c unsigned @c int: num_cols> <i>(number of different colors available)</i> 00067 * - for each of num_col: 00068 * - <@c char: red> <i>red color to use for display of this index</i> 00069 * - <@c char: green> <i>green color to use for display of this index</i> 00070 * - <@c char: blue> <i>blue color to use for display of this index</i> 00071 * - RLEGenerator: (from RLEGenerator::SaveBuffer()) 00072 * - <@c string: "RLEImage"> <i>(remember a 'string' is len+str+0; so this is the literal "\010\0\0\0RLEImage\0"; also remember "\010" is octal for 8)</i> 00073 * - <@c unsigned @c int: num_runs> <i>(how many runs will follow)</i> 00074 * - for each of num_runs: 00075 * - <@c char: color> <i>(index value of color of run)</i> 00076 * - <@c short: x> <i>(x position of start of run ("unknown" runs are skipped - assume index 0 for pixels which are jumped))</i> 00077 * - <@c short: width> <i>(length of run, will not exceed remaining width of image)</i> 00078 * - <i>notice there's no color information from RLE - it's not (shouldn't be) assuming anything about the data being compressed)</i> 00079 * 00080 * However, while we're on the topic, I'll mention that although this 00081 * is the same image format used for streaming to VisionGUI, there's 00082 * a few more fields added by RawCamBehavior or SegCamBehavior at the 00083 * beginning of each packet. See those classes for more information 00084 * on the wireless protocol. That should tell you everything you need 00085 * to know to interpret the vision stream as well. 00086 * 00087 * <h3>Adding New FilterBankGenerator Subclasses</h3> 00088 * 00089 * If you're doing fancy memory stuff, you probably want to override 00090 * the freeCaches() and destruct() functions so that the default 00091 * implementation won't try to free something it shouldn't. Don't 00092 * forget to call them from your own destructor though, otherwise 00093 * your versions won't get called before the default implementation's 00094 * does. 00095 * 00096 * If you want to be able to transmit or save your images, you will 00097 * need to override the LoadSave functions (listed below) to provide 00098 * your own code for interpreting the image data itself, and then 00099 * create or modify a behavior to open a socket and transmit the 00100 * information. (you could do that from within the generator itself 00101 * if you like) 00102 * 00103 * You will probably also want to add a few extra functions to allow 00104 * users to set compression/data format parameters. 00105 * 00106 * @see RawCameraGenerator, SegmentedColorGenerator for the basic 00107 * image access 00108 * 00109 * @see RLEGenerator, RegionGenerator for some relatively simple 00110 * examples of vision stages if you want to make some of your own. 00111 */ 00112 class FilterBankGenerator : public EventGeneratorBase, public LoadSave { 00113 public: 00114 // Constructors are all protected - doesn't make sense to 00115 // instantiate this class directly, you want to use a subclass 00116 00117 //! destructor 00118 /*! Your own subclasses should also have destructors which call 00119 * freeCaches() and destruct(). Otherwise, if you override these 00120 * functions to delete any custom memory you allocate, those 00121 * implementations won't be called by this destructor... a 00122 * destructor ignores virtual functions, only calls at its own 00123 * class level.\n 00124 * So it really doesn't matter if you aren't allocating any extra 00125 * memory other than what's in the image cache, but it's still good 00126 * form just in case you add stuff later so you won't forget and 00127 * leak memory everywhere */ 00128 virtual ~FilterBankGenerator() { 00129 freeCaches(); 00130 destruct(); 00131 } 00132 00133 //! returns the generator this is receiving its events from (or the last one anyway) 00134 virtual const FilterBankGenerator * getSourceGenerator() const { return src; } 00135 00136 //! returns the number of image layers (e.g. different resolutions available) 00137 virtual unsigned int getNumLayers() const { return numLayers; } 00138 00139 //! returns the number of channels per image (e.g. Y, U, or V components) 00140 virtual unsigned int getNumChannels() const { return numChannels; } 00141 00142 //! returns pointer to the beginning of the image data for the specified layer and channel 00143 /*! this will cause the data to be calculated and cached if it's not already available */ 00144 virtual unsigned char * getImage(unsigned int layer, unsigned int channel); 00145 00146 //! returns width (in samples) of the image in a given layer 00147 unsigned int getWidth(unsigned int layer) const { return widths[layer]; } 00148 00149 //! returns height (in samples) of the image in a given layer 00150 unsigned int getHeight(unsigned int layer) const { return heights[layer]; } 00151 00152 //! returns the bytes to skip from the one-past-end of a row to get the beginning of the next 00153 unsigned int getSkip(unsigned int layer) const { return skips[layer]; } 00154 00155 //! returns the bytes to skip from the beginning of one row to get the beginning of the next 00156 /*! This is just for convenience; the stride is just the skip plus the width, but it's precomputed for you for speed and clarity */ 00157 unsigned int getStride(unsigned int layer) const { return strides[layer]; } 00158 00159 //! returns the increment (in bytes) to use to go from one sample to the next 00160 unsigned int getIncrement(unsigned int layer) const { return increments[layer]; } 00161 00162 //! returns the frame number of the current frame, see #frameNumber 00163 unsigned int getFrameNumber() const { return frameNumber; } 00164 00165 //! returns the number of frames processed, see #framesProcessed 00166 unsigned int getFramesProcessed() const { return framesProcessed; } 00167 00168 //! returns a pointer to a particular sample; if you are using this in an inner loop, consider using the getSkip() and getIncrement() values to iterate with better performance 00169 /*! @param px the horizontal pizel position, relative to left edge; no boundary checking is done, ranges 0 through width-1 00170 * @param py the vertical pixel position, relative to top edge; no boundary checking is done, ranges 0 through height-1 00171 * @param layer the resolution layer to extract from 00172 * @param channel the image channel to extract from */ 00173 unsigned char * getPixel(unsigned int px, unsigned int py, unsigned int layer, unsigned int channel) { return getImage(layer,channel)+py*getStride(layer)+px*getIncrement(layer); } 00174 00175 //! returns a pointer to a particular sample; if you are using this in an inner loop, consider using the getSkip() and getIncrement() values to iterate with better performance 00176 /*! @param x the horizontal position, relative to center of the image, left edge is -1 and right edge is 1; no boundary checking is done 00177 * @param y the vertical pixel position, relative to center of the image, top edge is the negative aspect ratio, bottom edge is positive aspect ratio; no boundary checking is done 00178 * @param layer the resolution layer to extract from 00179 * @param channel the image channel to extract from 00180 * 00181 * To keep the coordinate system square, the x is defined to range -1,1, but y's range depends on the 00182 * aspect ratio of the image, height/width. Thus typically y will approx. -.75,.75 */ 00183 unsigned char * getPixel(float x, float y, unsigned int layer, unsigned int channel) { 00184 unsigned int px,py; 00185 getPixelCoordinates(px,py,x,y,layer); 00186 return getPixel(px,py,layer,channel); 00187 } 00188 00189 //! sets the pixel-coordinate px and py parameters to the corresponding value of x and y 00190 /*! @param[out] px the pixel position, relative to left edge, positive right, ranges 0 through width-1 00191 * @param[out] py the pixel position, relative to top edge, positive down, ranges 0 through height-1 00192 * @param[in] x the horizontal position, relative to center of the image, left edge is -1 and right edge is 1; no boundary checking is done 00193 * @param[in] y the vertical pixel position, relative to center of the image, top edge is the negative aspect ratio, bottom edge is positive aspect ratio; no boundary checking is done 00194 * @param[in] layer the resolution layer the pixel coordinates are relative to 00195 * 00196 * To keep the coordinate system square, the x is defined to range -1,1, but y's range depends on the 00197 * aspect ratio of the image, height/width. Thus typically y will approx. -.75,.75 */ 00198 void getPixelCoordinates(unsigned int& px, unsigned int& py, float x, float y, unsigned int layer) const { 00199 //note width sets the scale for both, so coordinate system is square... is good? I'm up for debate. 00200 px=(unsigned int)((getWidth(layer)-1)*(x+1)/2+.5f); //+.5 to round to nearest 00201 float aspect=getHeight(layer)/(float)getWidth(layer); 00202 py=(unsigned int)((getHeight(layer)-1)*(y+aspect)/(aspect*2)+.5f); 00203 } 00204 00205 //! sets the x and y parameters from the pixel-coordinates px and py 00206 /*! @param[out] x the horizontal position, relative to center of the image, left edge is -1 and right edge is 1; no boundary checking is done 00207 * @param[out] y the vertical pixel position, relative to center of the image, top edge is the negative aspect ratio, bottom edge is positive aspect ratio; no boundary checking is done 00208 * @param[in] px the pixel position, relative to left edge, positive right, ranges 0 through width-1 00209 * @param[in] py the pixel position, relative to top edge, positive down, ranges 0 through height-1 00210 * @param[in] layer the resolution layer the pixel coordinates are relative to 00211 * 00212 * To keep the coordinate system square, the x is defined to range -1,1, but y's range depends on the 00213 * aspect ratio of the image, height/width. Thus typically y will approx. -.75,.75 */ 00214 void getRealCoordinates(float& x, float& y, unsigned int px, unsigned int py, unsigned int layer) const { 00215 //note width sets the scale for both, so coordinate system is square... is good? I'm up for debate. 00216 x=px/(float)(getWidth(layer)-1)*2-1; 00217 float aspect=getHeight(layer)/(float)getWidth(layer); 00218 y=py/(float)(getHeight(layer)-1)*aspect*2-aspect; 00219 } 00220 00221 //! deletes storage of cached images and marks it invalid 00222 /*! you should override this if the images cache pointer isn't actually an array of bytes... 00223 * Don't forget to call it in your subclass's destructor or your version won't get called... */ 00224 virtual void freeCaches(); 00225 00226 //! marks all of the cached images as invalid (but doesn't free their memory) 00227 /*! You probably want to call this right before you send the FilterBankEvent */ 00228 virtual void invalidateCaches(); 00229 00230 //! default implementation does a few common housekeeping chores for you - probably should just take a look at its code 00231 /*! It doesn't throw any events for you - that's probably the main 00232 * reason you'd still want to override it\n 00233 * Also, if your class has a set number of layers or channels - for 00234 * instance, always 1 channel like InterleavedYUVGenerator, you 00235 * should override setNumImages() to enforce that constraint by 00236 * throwing away the appropriate argument and passing the your own 00237 * value to the superclass implementation.*/ 00238 virtual void processEvent(const EventBase & event); 00239 00240 //!@name LoadSave interface 00241 00242 virtual unsigned int getBinSize() const; 00243 00244 virtual unsigned int LoadBuffer(const char buf[], unsigned int len); 00245 00246 virtual unsigned int SaveBuffer(char buf[], unsigned int len) const; 00247 00248 //! Not actually part of the LoadSave interface, but allows you to select which image of the bank will be saved 00249 /*! Calling this will also cause the image data for that image to be calculated, 00250 * otherwise SaveBuffer won't have up-to-date data to save. 00251 * 00252 * When loading, the saved image's layer and channel will reset this */ 00253 virtual void selectSaveImage(unsigned int layer, unsigned int channel) { selectedSaveLayer=layer; selectedSaveChannel=channel; getImage(layer,channel);} 00254 00255 virtual unsigned int getSelectedSaveLayer() const { return selectedSaveLayer; } //!< returns layer to be saved, or layer of last image loaded 00256 virtual unsigned int getSelectedSaveChannel() const { return selectedSaveChannel; } //!< returns channel to be saved, or channel of last image loaded 00257 00258 //@} 00259 00260 00261 protected: 00262 //! constructor, separate class and instance names, with a raw event specification, excluding type typically for stages which reference the previous stage's data 00263 FilterBankGenerator(const std::string& classname,const std::string& instancename, EventBase::EventGeneratorID_t mgid, unsigned int msid, EventBase::EventGeneratorID_t srcegid, unsigned int srcsrc) 00264 : EventGeneratorBase(classname, instancename, mgid, msid, srcegid, srcsrc), 00265 src(NULL), numLayers(0), numChannels(0), widths(NULL), heights(NULL), skips(NULL), 00266 strides(NULL), increments(NULL), images(NULL), imageValids(NULL), selectedSaveLayer(0), 00267 selectedSaveChannel(0), frameNumber(0), framesProcessed(0) 00268 { } 00269 00270 //! constructor, separate class and instance names, with a raw event specification, including type typically for stages which will store their own copy of the data 00271 FilterBankGenerator(const std::string& classname,const std::string& instancename, EventBase::EventGeneratorID_t mgid, unsigned int msid, EventBase::EventGeneratorID_t srcegid, unsigned int srcsrc, EventBase::EventTypeID_t srcetid) 00272 : EventGeneratorBase(classname, instancename, mgid, msid, srcegid, srcsrc, srcetid), 00273 src(NULL), numLayers(0), numChannels(0), widths(NULL), heights(NULL), skips(NULL), 00274 strides(NULL), increments(NULL), images(NULL), imageValids(NULL), selectedSaveLayer(0), 00275 selectedSaveChannel(0), frameNumber(0), framesProcessed(0) 00276 { } 00277 00278 //! constructor, separate class and instance names, with a filter bank source, passes on all types typically for stages which reference the previous stage's data 00279 FilterBankGenerator(const std::string& classname,const std::string& instancename, EventBase::EventGeneratorID_t mgid, unsigned int msid, FilterBankGenerator * fbgsrc) 00280 : EventGeneratorBase(classname, instancename, mgid, msid, fbgsrc!=NULL?fbgsrc->getGeneratorID():EventBase::numEGIDs, fbgsrc!=NULL?fbgsrc->getSourceID():0), 00281 src(fbgsrc), numLayers(0), numChannels(0), widths(NULL), heights(NULL), skips(NULL), 00282 strides(NULL), increments(NULL), images(NULL), imageValids(NULL), selectedSaveLayer(0), 00283 selectedSaveChannel(0), frameNumber(0), framesProcessed(0) 00284 { 00285 if(src!=NULL) 00286 setNumImages(src->getNumLayers(),src->getNumChannels()); 00287 } 00288 00289 //! constructor, separate class and instance names, with a filter bank source, accepts a particular type typically for stages which will store their own data 00290 FilterBankGenerator(const std::string& classname,const std::string& instancename, EventBase::EventGeneratorID_t mgid, unsigned int msid, FilterBankGenerator * fbgsrc, EventBase::EventTypeID_t etid) 00291 : EventGeneratorBase(classname, instancename, mgid, msid, fbgsrc!=NULL?fbgsrc->getGeneratorID():EventBase::numEGIDs, fbgsrc!=NULL?fbgsrc->getSourceID():0,etid), 00292 src(fbgsrc), numLayers(0), numChannels(0), widths(NULL), heights(NULL), skips(NULL), 00293 strides(NULL), increments(NULL), images(NULL), imageValids(NULL), selectedSaveLayer(0), 00294 selectedSaveChannel(0), frameNumber(0), framesProcessed(0) 00295 { 00296 if(src!=NULL) 00297 setNumImages(src->getNumLayers(),src->getNumChannels()); 00298 } 00299 00300 //! resizes the filter bank information storage area, you should override this to do your setup and call it from your constructor 00301 /*! In general, it isn't expected that FilterBankGenerator's should 00302 * necessarily be dynamically resizeable (although it would be 00303 * nice), which is why this isn't public. If yours is, just add 00304 * some pubic accessor functions which call this. In general, the 00305 * included subclasses should be able to handle being resized, but 00306 * there's no reason to do so since the system won't be changing 00307 * its available resolutions at run time. 00308 * 00309 * The default implementation is a no-op if(numLayers==nLayers && numChannels==nChannels) 00310 */ 00311 virtual void setNumImages(unsigned int nLayers, unsigned int nChannels); 00312 00313 //! resets width and height parameters to that of the #src 00314 /*! You'll probably want to override this to also set #skips and #strides */ 00315 virtual void setDimensions(); 00316 00317 //! create new image data storage area for the cache - this called by getImage() only when the corresponding entry in images is NULL 00318 /*! You should return the pointer you want stored in images to be 00319 * returned by any calls to getFirstRow. Interpretation of the 00320 * data it points to is dependant on the the generator which 00321 * creates it */ 00322 virtual unsigned char * createImageCache(unsigned int layer, unsigned int channel) const=0; 00323 00324 //! should calculate new image data, called by getImage() only when #imageValids indicates the image being requested is dirty (and only after getImage() has already called createImageCache()) 00325 /*! This is where you'll want to put your user-specific code for calculating the image data */ 00326 virtual void calcImage(unsigned int layer, unsigned int channel) =0; 00327 00328 //! deletes the arrays 00329 virtual void destruct(); 00330 00331 //! updates the image data to make sure its up to date with what's available from the source 00332 /*! If someone calls getImage on a stage which hadn't been listening for 00333 * events (an optimization to save time when it doesn't have any listeners 00334 * of its own -- see EventGeneratorBase), then this will retroactively 00335 * pull image data from the source even though the event for it was missed 00336 * 00337 * @return false if no image data is available yet, true otherwise*/ 00338 virtual bool refresh(); 00339 00340 00341 FilterBankGenerator * src; //!< the generator of the last FilterBankEvent received 00342 00343 unsigned int numLayers; //!< current number of layers available 00344 unsigned int numChannels; //!< current number of channels available 00345 00346 unsigned int * widths; //!< an array of size numLayers, width (in samples) in pixels of each layer 00347 unsigned int * heights; //!< an array of size numLayers, height (in samples) in pixels of each layer 00348 unsigned int * skips; //!< an array of size numLayers, skip (in bytes) from row end to next row begin 00349 unsigned int * strides; //!< an array of size numLayers, stride (in bytes) from a given column in one row to the same column in the next row 00350 unsigned int * increments;//!< an array of size numLayers, increment (in bytes) to use to get from one sample to the next 00351 00352 mutable unsigned char *** images; //!< an array [numLayers][numChannels], stores pointer to cached image data 00353 mutable bool ** imageValids; //!< an array [numLayers][numChannels], entry is true if cached data is still valid 00354 00355 unsigned int selectedSaveLayer; //!< layer to be manipulated with the LoadSave functions 00356 unsigned int selectedSaveChannel; //!< channel to be manipulated with the LoadSave functions 00357 00358 //! the frame number of last frame received by processEvent - subclasses will need to set to the source's frameNumber if they don't call FilterBankGenerator::processEvent() 00359 /*! The idea is to use this as a unique serial number for each 00360 * frame. That way you can know if the current image in different 00361 * generators is actually the same camera image before you try to 00362 * compare or combine them. 00363 * 00364 * You could also figure out the number of dropped frames by 00365 * subtracting framesProcessed from this value. Give some leeway 00366 * however, because it takes the first 40-70 frames just to boot up 00367 * (when running on the aibo), so there's no way they can be 00368 * processed. 00369 */ 00370 unsigned int frameNumber; 00371 00372 //! the current frame number available from the system - subclasses which receive data directly from the system should set this (and should not use EventGeneratorBase's auto-listen to ensure this is accurate) 00373 static unsigned int sysFrameNumber; 00374 00375 //! subclasses should increment this any time they make a new filter bank available 00376 /*! this is automatically incremented if you use the FilterBankGenerator::processEvent() */ 00377 unsigned int framesProcessed; 00378 00379 private: 00380 FilterBankGenerator(const FilterBankGenerator& fbk); //!< don't call 00381 const FilterBankGenerator& operator=(const FilterBankGenerator& fbk); //!< don't call 00382 }; 00383 00384 /*! @file 00385 * @brief Describes abstract base class for generators of FilterBankEvent's 00386 * @author ejt (Creator) 00387 * 00388 * $Author: ejt $ 00389 * $Name: tekkotsu-2_4_1 $ 00390 * $Revision: 1.18 $ 00391 * $State: Exp $ 00392 * $Date: 2005/08/07 04:11:04 $ 00393 */ 00394 00395 #endif |
Tekkotsu v2.4.1 |
Generated Tue Aug 16 16:32:47 2005 by Doxygen 1.4.4 |