saves a selected 2d histogram to hdf5 More...

#include <hdf5_converter.h>

Inheritance diagram for cass::pp1002:

Collaboration diagram for cass::pp1002:

Classes
struct	entry_t
	struct bundleing info for writing an entry to file More...

Public Member Functions
	pp1002 (const name_t &)
	constructor More...

virtual void	processEvent (const CASSEvent &)
	process the event More...

virtual void	aboutToQuit ()
	dump all pp histograms to summary group just before quitting More...

virtual void	loadSettings (size_t)
	load the settings of this pp More...

virtual const result_t &	result (const CASSEvent::id_t eventid=0)
	overwrite the retrieval of an histogram More...

virtual void	releaseEvent (const CASSEvent &)
	overwrite the release More...

Public Member Functions inherited from cass::Processor
	Processor (const name_t &name)
	constructor More...

virtual	~Processor ()
	virtual destructor More...

result_t::shared_pointer	resultCopy (const uint64_t eventid)
	retrieve histogram for id More...

virtual void	load ()
	load the general settings More...

const names_t &	dependencies ()
	Define all processors keys a processor depends on. More...

void	clearDependencies ()
	clear the dependenies More...

void	clearHistograms ()
	clear the histograms More...

virtual void	processCommand (std::string command)
	process command in pp More...

const name_t	name () const
	retrieve the name of this processor More...

bool	hide () const
	retrieve the hide flag of this processor More...

const std::string &	comment () const
	retrieve the comment of this processor More...

Protected Types
typedef std::tr1::shared_ptr< hdf5::WriteEntry >	entryWriter_t
	define pointer to the entry writer More...

Protected Member Functions
void	writeSummaryToMultipleEventsFile ()
	write the summary to a file that contains multiple events More...

void	writeEventToMultipleEventsFile (const CASSEvent &evt)
	function to write the events to a file that contains multiple events More...

void	appendEventToMultipleEventsFile (const CASSEvent &evt)
	function to write the events to a file that contains multiple events More...

void	writeSummaryToSingleFile ()
	function to write the summary to a single file More...

void	writeEventToSingleFile (const CASSEvent &evt)
	function to write the events to a single file More...

Protected Member Functions inherited from cass::Processor
virtual void	process (const CASSEvent &event, result_t &result)
	process the event More...

virtual void	createHistList (result_t::shared_pointer result)
	create result list. More...

void	setupGeneral ()
	general setup of the processor More...

bool	setupCondition (bool defaultConditionType=true)
	setup the condition. More...

shared_pointer	setupDependency (const std::string &depVarName, const name_t &name="")
	setup the dependecy. More...

Protected Attributes
std::string	_basefilename
	the filename that the data will be written to More...

std::list< entry_t >	_procList
	container with all pps that contain the histograms to dump to hdf5 More...

std::list< entry_t >	_procSummaryList
	container for all pps that should be written when program quits More...

int	_maxFilePerSubDir
	the number of files in each subdir More...

int	_filecounter
	counter to count how many files have been written More...

entryWriter_t	_entryWriter
	the entry writer More...

size_t	_maxFileSize
	the maximum file size of the single file More...

std::tr1::function< void(void)>	_writeSummary
	write summary to file More...

std::tr1::function< void(const CASSEvent &)>	_writeEvent
	write event to file More...

Protected Attributes inherited from cass::Processor
const name_t	_name
	the processors name More...

bool	_hide
	flag to tell whether this pp should be hidden in the dropdown list More...

std::string	_comment
	optional comment that one can add to a processor. More...

CachedList	_resultList
	the list of results More...

names_t	_dependencies
	the list of dependencies More...

shared_pointer	_condition
	pointer to the processor that will contain the condition More...

Private Attributes
QMutex	_lock
	a lock to make the process reentrant More...

Additional Inherited Members
Public Types inherited from cass::Processor
typedef std::tr1::shared_ptr< Processor >	shared_pointer
	a shared pointer of this More...

typedef std::string	name_t
	define the name type More...

typedef std::list< name_t >	names_t
	define the list of names More...

typedef CachedList::item_type	result_t
	define the results More...

typedef CachedList::item_sp	resultsp_t
	define the shared pointer to the result More...

Detailed Description

saves a selected 2d histogram to hdf5

Available PostProcessors:: "1002": saves a selected 2d histogram to hdf5

it will just save one histogram in a file and then write the next into the next hdf5 file. Inside the hdf5 it uses the same layout that the Chapman crew is using to be able to read and process the hdf5 with crystfel

User definable Parameters:

Processor/%name%/{CompressLevel}
The compression level. Default is 2

Processor/%name%/{FileBaseName}
Base Name of the Files to be written. In case of writing mutiple Events to the same file an alpha counter will be added to the file name. In case of writing an event to a single file the event id Will be appended to the file name.

Processor/%name%/{WriteMultipleEventsInOneFile}
Flag to tell whether to write multiple events to the same file (true) or each event into a single file (false). Default is false.

Processor/%name%/{WriteToSingleDatasets}
In case one writes multiple events into the same file, this flag allows to tell, that the data will be written into a single dataset. The dataset will have an additional dimension in the slowest axis that holds the data of all the events. In addition to that a dataset is written that lists all the eventids of the events that are written. Default is false.

Processor/%name%/{MaximumNbrFilesPerDir}
In case of single files per event, distribute the files over subdirectories where each subdir contains this amount of files. If -1 it will not distribute the files. Default is -1.

Processor/%name%/{MaximumFileSize_GB}
In case of multiple events per file, this is the maximum file size before the alpha counter of the filename will be increased and a the events will be written to the new file. Default is 200

Processor/%name%/Processor/{size}
How many Processors should be written to the h5 file.

Processor/%name%/Processor/%id%/{Name}
Name of the Processor that should be written into the h5 file. Default is "unknown"

Processor/%name%/Processor/%id%/{GroupName}
Name of the group in the h5 file into which the Processor should be written into. Default is "/"

Processor/%name%/Processor/%id%/{ValName}
Name that the data should have in the h5 file. Default is the name of the Processor.

Processor/%name%/ProcessorSummary/{size}
How many Processors should be written to the h5 file.

Processor/%name%/ProcessorSummary/%id%/{Name}
Name of the Processor that should be written into the h5 file. Default is "unknown"

Processor/%name%/ProcessorSummary/%id%/{GroupName}
Name of the group in the h5 file into which the Processor should be written into. Default is "/"

Processor/%name%/ProcessorSummary/%id%/{ValName}
Name that the data should have in the h5 file. Default is the name of the Processor.

Todo:: enable that one can write into just one h5 file multiple events

Author: Lutz Foucar

Definition at line 87 of file hdf5_converter.h.

Member Typedef Documentation

typedef std::tr1::shared_ptr<hdf5::WriteEntry> cass::pp1002::entryWriter_t

protected

define pointer to the entry writer

Definition at line 185 of file hdf5_converter.h.

Constructor & Destructor Documentation

pp1002::pp1002 ( const name_t & name )

constructor

Definition at line 316 of file hdf5_converter.cpp.

References loadSettings().

Member Function Documentation

void pp1002::aboutToQuit ( )

virtual

dump all pp histograms to summary group just before quitting

check if something to be written

Reimplemented from cass::Processor.

Definition at line 441 of file hdf5_converter.cpp.

References _lock, _procSummaryList, and _writeSummary.

void pp1002::appendEventToMultipleEventsFile ( const CASSEvent & evt )

protected

function to write the events to a file that contains multiple events

append the static size results to a single dataset with one more dimension

Parameters

evt	The event containg the data to write

check the current file size, create a new file with increase ending, if too big

tell the writer which id to use and the corresponding base group

write all entries to file using the writer

Note: we can't use for_each here, since we need to ensure that the entries are written sequentially and for_each can potentially use omp to parallelize the execution.

Definition at line 555 of file hdf5_converter.cpp.

References _basefilename, _entryWriter, _maxFileSize, _procList, cass::CASSEvent::id(), and cass::AlphaCounter::increaseFileCounter().

Referenced by loadSettings().

void pp1002::loadSettings ( size_t )

virtual

load the settings of this pp

when requested add the first subdir to the filename and make sure that the directory exists.

set up the maximum file size and convert to Bytes

set up which kind of file should be written

set up the dir or the filename, depending on the case

Reimplemented from cass::Processor.

Definition at line 322 of file hdf5_converter.cpp.

References _basefilename, cass::Processor::_condition, _entryWriter, _filecounter, cass::Processor::_hide, _maxFilePerSubDir, _maxFileSize, _procList, _procSummaryList, _writeEvent, _writeSummary, cass::Log::add(), appendEventToMultipleEventsFile(), QSettings::beginGroup(), QSettings::beginReadArray(), QSettings::endArray(), QString::fromStdString(), cass::Log::INFO, cass::AlphaCounter::intializeDir(), cass::AlphaCounter::intializeFile(), cass::Processor::name(), QSettings::setArrayIndex(), cass::Processor::setupCondition(), cass::Processor::setupDependency(), cass::Processor::setupGeneral(), size, cass::toString(), QSettings::value(), writeEventToMultipleEventsFile(), writeEventToSingleFile(), writeSummaryToMultipleEventsFile(), and writeSummaryToSingleFile().

Referenced by pp1002().

void pp1002::processEvent ( const CASSEvent & evt )

virtual

process the event

check if there is something to be written or if it should be written

Reimplemented from cass::Processor.

Definition at line 490 of file hdf5_converter.cpp.

References cass::Processor::_condition, _lock, _procList, _writeEvent, and cass::CASSEvent::id().

virtual void cass::pp1002::releaseEvent ( const CASSEvent & )

inlinevirtual

overwrite the release

Reimplemented from cass::Processor.

Definition at line 139 of file hdf5_converter.h.

const Processor::result_t & pp1002::result ( const CASSEvent::id_t eventid = 0 )

virtual

overwrite the retrieval of an histogram

Reimplemented from cass::Processor.

Definition at line 436 of file hdf5_converter.cpp.

References cass::Processor::name().

void pp1002::writeEventToMultipleEventsFile ( const CASSEvent & evt )

protected

function to write the events to a file that contains multiple events

Parameters

evt	The event containg the data to write

check the current file size, create a new file with increase ending, if too big

tell the writer which id to use and the corresponding base group

write all entries to file using the writer

Note: we can't use for_each here, since we need to ensure that the entries are written sequentially and for_each can potentially use omp to parallelize the execution.

Definition at line 527 of file hdf5_converter.cpp.

References _basefilename, _entryWriter, _maxFileSize, _procList, cass::CASSEvent::id(), cass::AlphaCounter::increaseFileCounter(), and cass::toString().

Referenced by loadSettings().

void pp1002::writeEventToSingleFile ( const CASSEvent & evt )

protected

function to write the events to a single file

Parameters

evt	The event containg the data to write

increment subdir in filename when they should be distributed and the counter exeeded the maximum amount of files per subdir

create entry writer with filename using basefilename and event id

write all entries to file using the writer

Note: we can't use for_each here, since we need to ensure that the entries are written sequentially and for_each can potentially use omp to parallelize the execution.

Definition at line 500 of file hdf5_converter.cpp.

References _basefilename, _filecounter, _maxFilePerSubDir, _procList, cass::CASSEvent::id(), cass::AlphaCounter::increaseDirCounter(), and cass::toString().

Referenced by loadSettings().

void pp1002::writeSummaryToMultipleEventsFile ( )

protected

write the summary to a file that contains multiple events

write all entries to file using the writer

Note: we can't use for_each here, since we need to ensure that the entries are written sequentially and for_each can potentially use omp to parallelize the execution.

Definition at line 472 of file hdf5_converter.cpp.

References _entryWriter, and _procSummaryList.

Referenced by loadSettings().

void pp1002::writeSummaryToSingleFile ( )

protected

function to write the summary to a single file

remove subdir from filename when they should be distributed

create filename from base filename and write entries to file

write all entries to file using the writer

Note: we can't use for_each here, since we need to ensure that the entries are written sequentially and for_each can potentially use omp to parallelize the execution.

Definition at line 451 of file hdf5_converter.cpp.

References _basefilename, _maxFilePerSubDir, _procSummaryList, and cass::AlphaCounter::removeAlphaSubdir().

Referenced by loadSettings().