|
CFEL - ASG Software Suite
2.5.0
CASS
|
XFEL HDF5 File Input for cass. More...
#include <xfel_hdf5_file_input.h>
Inheritance diagram for cass::XFELHDF5FileInput: Collaboration diagram for cass::XFELHDF5FileInput:Classes | |
| struct | AcqirisParams |
| parameters needed to retrieve the Acqiris data More... | |
| struct | PixeldetectorParams |
| parameters needed to retrieve the pixeldetector data More... | |
Public Member Functions | |
| void | runthis () |
| function with the main loop More... | |
| void | load () |
| load the parameters used for the multifile input More... | |
| double | progress () |
| retrieve the averaged progress state of all file processors More... | |
| uint64_t | eventcounter () |
| retrieve the number of processed events More... | |
| uint64_t | skippedeventcounter () |
| retrieve the number of skipped processed events More... | |
| Public Member Functions inherited from cass::InputBase | |
| virtual | ~InputBase () |
| destructor More... | |
| void | newEventAdded (const size_t eventsize) |
| increment the numer of events received in the ratemeter More... | |
| RingBuffer< CASSEvent > & | ringbuffer () |
| retrieve a reference to the the ringbuffer More... | |
| rbItem_t | getNextFillable (unsigned timeout=500) |
| retrieve an iterator to the next fillable event More... | |
| Public Member Functions inherited from lmf::PausableThread | |
| PausableThread (control_t control=_run, QObject *parent=0) | |
| constructor More... | |
| virtual | ~PausableThread () |
| destructor More... | |
| void | run () |
| run the thread More... | |
| void | pause (bool block=false) |
| pause the thread More... | |
| void | waitUntilPaused () |
| waits until thread is paused More... | |
| void | resume () |
| resume the thread More... | |
| status_t | status () const |
| return the current status of the thread More... | |
| bool | shouldQuit () const |
| query whether this thread is told to quit More... | |
| void | rethrowException () const |
| rethrow the thrown exception More... | |
| exception_t | exceptionThrown () const |
| Public Member Functions inherited from QThread | |
| QThread (QObject *parent=0) | |
| currentThread () | |
| HANDLE QThread::currentThreadId() | |
| exec () | |
| exit (int returnCode=0) | |
| finished () | |
| idealThreadCount () | |
| isFinished () | |
| isRunning () | |
| msleep (unsigned long msecs) | |
| priority () | |
| quit () | |
| run () | |
| setPriority (Priority priority) | |
| setStackSize (uint stackSize) | |
| setTerminationEnabled (bool enabled=true) | |
| sleep (unsigned long secs) | |
| stackSize () | |
| start (Priority priority=InheritPriority) | |
| started () | |
| terminate () | |
| terminated () | |
| usleep (unsigned long usecs) | |
| wait (unsigned long time=ULONG_MAX) | |
| yieldCurrentThread () | |
Static Public Member Functions | |
| static void | instance (std::string filelistname, RingBuffer< CASSEvent > &, Ratemeter &ratemeter, Ratemeter &loadmeter, bool quitwhendone, QObject *parent=0) |
| create instance of this More... | |
| Static Public Member Functions inherited from cass::InputBase | |
| static shared_pointer | instance () |
| get the signelton instance More... | |
| static shared_pointer::element_type & | reference () |
| get reference to the singelton instance More... | |
Private Member Functions | |
| XFELHDF5FileInput (std::string filelistname, RingBuffer< CASSEvent > &, Ratemeter &ratemeter, Ratemeter &loadmeter, bool quitwhendone, QObject *parent=0) | |
| constructor More... | |
Private Attributes | |
| bool | _quitWhenDone |
| flag to tell the thread to quit when its done with all files More... | |
| std::string | _filelistname |
| name of the file containing all files that we need to process More... | |
| uint64_t | _counter |
| the counter for all events More... | |
| uint64_t | _scounter |
| the counter for all events More... | |
| uint64_t | _totalevents |
| the total # of events to be handled More... | |
Additional Inherited Members | |
| Public Types inherited from cass::InputBase | |
| typedef std::tr1::shared_ptr< InputBase > | shared_pointer |
| shared pointer of this type More... | |
| typedef RingBuffer< CASSEvent >::iter_type | rbItem_t |
| define an item in the ringbuffer More... | |
| Public Types inherited from lmf::PausableThread | |
| enum | status_t { running, paused, notstarted } |
| enum describing the internal status of the thread More... | |
| enum | control_t { _run, _quit, _pause } |
| enum describing the control status of the thread More... | |
| enum | exception_t { NO_EXCEPTION, INVALID_ARGUMENT_EXCEPTION, RUNTIME_ERROR_EXCEPTION, OUT_OF_RANGE_EXCEPTION, LOGIC_ERROR_EXCEPTION, STANDART_EXCEPTION, UNKNOWN_EXCEPTION } |
| enum describing which exception was thrown More... | |
| Public Slots inherited from lmf::PausableThread | |
| virtual void | end () |
| tell the thread to quit More... | |
| Public Attributes inherited from cass::InputBase | |
| QMutex | lock |
| a mutex so that external program can lock access to this More... | |
| Protected Member Functions inherited from cass::InputBase | |
| InputBase (RingBuffer< CASSEvent > &ringbuffer, Ratemeter &ratemeter, Ratemeter &loadmeter, QObject *parent=0) | |
| protected constructor since it should be a singelton More... | |
| Protected Member Functions inherited from lmf::PausableThread | |
| void | pausePoint () |
| point where the thread will be paused More... | |
| Protected Attributes inherited from cass::InputBase | |
| RingBuffer< CASSEvent > & | _ringbuffer |
| reference to the ringbuffer More... | |
| Ratemeter & | _ratemeter |
| ratemeter to measure the rate More... | |
| Ratemeter & | _loadmeter |
| meter to measure the data load More... | |
| Protected Attributes inherited from lmf::PausableThread | |
| QMutex | _pauseMutex |
| mutex to wait on until thread is paused More... | |
| QWaitCondition | _pauseCondition |
| wait condition to wait on until thread is resumed More... | |
| QWaitCondition | _waitUntilPausedCondition |
| wait condition to wait unitl thread is paused More... | |
| status_t | _status |
| the internal status of the thread More... | |
| control_t | _control |
| the internal control status of the thread More... | |
| size_t | _pausecount |
| a counter how many threads have pause this thread More... | |
| exception_t | _exception_thrown |
| flag to show that general exception was thrown More... | |
| std::invalid_argument | _invarg_excep |
| the invalid arguemnt exception thrown More... | |
| std::runtime_error | _runt_excep |
| the invalid arguemnt exception thrown More... | |
| std::out_of_range | _outrange_excep |
| the invalid arguemnt exception thrown More... | |
| std::logic_error | _logic_excep |
| the invalid arguemnt exception thrown More... | |
| Static Protected Attributes inherited from cass::InputBase | |
| static shared_pointer | _instance |
| singelton instance More... | |
XFEL HDF5 File Input for cass.
XFELHDF5FileInput/{MaxPulseId}
The maximum pulse id that will be used to iterate through. Default is 65.
XFELHDF5FileInput/{PreCacheData}
Flag that allows precaching the whole files by reading the contents to memory. Need to ensure that you've got enough memory available first otherwise the program will crash. Default is true.
XFELHDF5FileInput/AGIPD/{DataCASSID}
ID that the agipd data will have in the cassevent. Default is 30
XFELHDF5FileInput/AGIPD/{MaskCASSID}
ID that the agipd mask will have in the cassevent. Default is 31
XFELHDF5FileInput/AGIPD/{GainCASSID}
ID that the agipd gain information will have in the cassevent. Default is 32
XFELHDF5FileInput/AGIPD/Tiles/%index%/{Id}
ID of the tile. Default is -1 which will skip this entry
XFELHDF5FileInput/AGIPD/Tiles/%index%/{HDF5FileName}
HDF5 file that contains the data of the tile. Default is Invalid which will skip this entry
XFELHDF5FileInput/AGIPD/Tiles/%index%/{DataHDF5Key}
Key to the data of the tile in the HDF5 file. Default is Invalid which will skip this entry
XFELHDF5FileInput/AGIPD/Tiles/%index%/{MaskHDF5Key}
Key to the mask of the tile in the HDF5 file. Default is Invalid which will skip this entry
XFELHDF5FileInput/AGIPD/Tiles/%index%/{GainHDF5Key}
Key to the gain data of the tile in the HDF5 file. Default is Invalid which will skip this entry
XFELHDF5FileInput/AGIPD/Tiles/%index%/{TrainIdHDF5Key}
Key to the train Id of the tile in the HDF5 file. Default is Invalid which will skip this entry
XFELHDF5FileInput/AGIPD/Tiles/%index%/{PulseIdHDF5Key}
Key to the pulse Id of the tile in the HDF5 file. Default is Invalid which will skip this entry
XFELHDF5FileInput/AGIPD/Tiles/%index%/{CellIdHDF5Key}
Key to the cell Id of the tile in the HDF5 file. Default is Invalid which will skip this entry
Definition at line 62 of file xfel_hdf5_file_input.h.
|
private |
constructor
| filelistname | name of the file containing all files that should be processed |
| ringbuffer | reference to the ringbuffer containing the CASSEvents |
| ratemeter | reference to the ratemeter to measure the rate of the input |
| loadmeter | reference to the ratemeter to measure the load of the input |
| quitwhendone | flag that tells this class that it should quit the Program when its done reading all events |
| parent | The parent QT Object of this class |
Definition at line 235 of file xfel_hdf5_file_input.cpp.
References cass::Log::add(), and cass::Log::VERBOSEINFO.
|
inlinevirtual |
retrieve the number of processed events
Reimplemented from cass::InputBase.
Definition at line 100 of file xfel_hdf5_file_input.h.
References _counter.
Referenced by runthis().
|
static |
create instance of this
| filelistname | name of the file containing all files that should be processed |
| ringbuffer | reference to the ringbuffer containing the CASSEvents |
| ratemeter | reference to the ratemeter to measure the rate of the input |
| loadmeter | reference to the ratemeter to measure the load of the input |
| quitwhendone | flag that tells this class that it should quit the Program when its done reading all events |
| parent | The parent QT Object of this class |
Definition at line 224 of file xfel_hdf5_file_input.cpp.
|
virtual |
load the parameters used for the multifile input
Implements cass::InputBase.
Definition at line 250 of file xfel_hdf5_file_input.cpp.
|
inlinevirtual |
retrieve the averaged progress state of all file processors
Reimplemented from cass::InputBase.
Definition at line 93 of file xfel_hdf5_file_input.h.
References _counter, and _totalevents.
|
virtual |
function with the main loop
load the settings from the ini file
get the cassids for the individual agipd images
get info for the individual tiles
create a new tile
get the user defined machine values
create a new machineInfo
the filename of the file containing the machine value
the name of the machine data within the cassevent
the name of the machine data within the cassevent
for all the tiles make a map of ids generated from the trainId and the pulseId with the corresponding index to the data Using openmp to parallelize the retrieval of the detector data. The 'pragma omp parallel for' statement says that the for loop should be parallelized. Within the loop all the declared variables are local to the thread and not shared. The 'num_threads' parameter allows to define how many threads should be used. Since the agip detector has 16 tiles, we only need 16 threads and not all that are available (which will be used if nothing is declared).
preload the data to check if it improves the speed of the analysis
create a map that tells the idx for each trainids of the machine values
pre cache the data to see whether its faster to analyze
generate container list of all the unique trainIds available
now go through all the trains
add break point here
go through all the nominal available pulses within a train
add break point here
retrieve a new element from the ringbuffer. If one didn't get a an element (when the end iterator of the buffer is returned). Continue to the next iteration, where it is checked if the thread should quit.
generate an unique eventid from the pulse id and the trainId
get reference to all devices of the CASSEvent and an iterator
retrieve the pixel detector part of the cassevent
retrieve the right detectors from the cassevent and reset it
the data frame
the mask frame
the gain frame
find the tiles that correspond to the the made up eventid and copy them to the right pos within the detector image
ensure that all values in the frame are numbers
retrieve the machine detector part of the cassevent
find the additional data for the machine data
skip if the tile data for the eventid id is not available
tell the ringbuffer that we're done with the event
now go through all set of files
get the shape from the first datafile (should be the same for all data files). The first dimension is the number of images, so we iterate over this number
pre cache the data
get all the trainids and the pulseids from the first datafile, they later used to compile a unique eventid
calculate the which Event in the file should be retrieved
flag that tell whether the data is good
retrieve a new element from the ringbuffer. If one didn't get a an element (when the end iterator of the buffer is returned). Continue to the next iteration, where it is checked if the thread should quit.
create the event id from the trainid and the pulseid
get reference to all devices of the CASSEvent and an iterator
retrieve the pixel detector part of the cassevent
retrieve the right detector from the cassevent and reset it
copy the det data to the frame
set the detector parameters and add the event id
retrieve the right detector from the cassevent and reset it
copy the det data to the frame
set the detector parameters and add the event id
retrieve the right detector to hold the mask from the cassevent and reset it
copy the det data to the frame
set the detector parameters and add the event id
retrieve the machine detector part of the cassevent
tell the ringbuffer that we're done with the event
quit the program if requested otherwise wait until the program is signalled to quit
Implements cass::InputBase.
Definition at line 254 of file xfel_hdf5_file_input.cpp.
References _counter, _quitWhenDone, cass::InputBase::_ringbuffer, lmf::PausableThread::_status, _totalevents, cass::Log::add(), cass::MachineData::Device::BeamlineData(), QSettings::beginGroup(), QSettings::beginReadArray(), hdf5::partiality_t::block, cass::MachineInfo::cassname, cass::AGIPDTile::cellIdDsetName, hdf5::partiality_t::count, data, cass::AGIPDTile::dataDsetName, cass::pixeldetector::Device::dets(), cass::CASSEvent::devices(), hdf5::partiality_t::dims, QSettings::endArray(), QSettings::endGroup(), cass::Log::ERROR, eventcounter(), cass::AGIPDTile::fh, cass::MachineInfo::fh, cass::AGIPDTile::gainDsetName, cass::InputBase::getNextFillable(), cass::MachineInfo::hdf5DsetName, cass::AGIPDTile::id, cass::CASSEvent::id(), cass::pixeldetector::Detector::id(), cass::Log::INFO, cass::CASSEvent::MachineData, mask, cass::AGIPDTile::maskDsetName, cass::AGIPDTile::nCols, cass::InputBase::newEventAdded(), cass::AGIPDTile::nRows, hdf5::partiality_t::offset, cass::CASSEvent::PixelDetectors, cass::AGIPDTile::pulseIdDsetName, lmf::PausableThread::running, QSettings::setArrayIndex(), lmf::PausableThread::shouldQuit(), cass::AGIPDTile::size, size, QThread::sleep(), hdf5::partiality_t::stride, cass::tileToFrames(), cass::toString(), cass::AGIPDTile::trainIdDsetName, cass::MachineInfo::trainIdDsetName, QSettings::value(), and cass::Log::VERBOSEINFO.
|
inlinevirtual |
retrieve the number of skipped processed events
Reimplemented from cass::InputBase.
Definition at line 106 of file xfel_hdf5_file_input.h.
References _scounter.
|
private |
the counter for all events
Definition at line 166 of file xfel_hdf5_file_input.h.
Referenced by eventcounter(), progress(), and runthis().
|
private |
name of the file containing all files that we need to process
Definition at line 163 of file xfel_hdf5_file_input.h.
|
private |
flag to tell the thread to quit when its done with all files
Definition at line 160 of file xfel_hdf5_file_input.h.
Referenced by runthis().
|
private |
the counter for all events
Definition at line 169 of file xfel_hdf5_file_input.h.
Referenced by skippedeventcounter().
|
private |
the total # of events to be handled
Definition at line 172 of file xfel_hdf5_file_input.h.
Referenced by progress(), and runthis().
1.8.10
