![]() |
threaddb
1.0
A file mapped memory container extension
|
#include "threaddbTypes.h"
Go to the source code of this file.
Functions | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Create (void **pThreadDB_p, size_t PackageSize_p, size_t PackageCacheLimit_p) |
Interface for creating a new database object. More... | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Import (void **pThreadDB_p, const char *pIndexFileUTF8_p, size_t PackageCacheLimit_p) |
Interface for creating a new database object based on a database index file. More... | |
DLLEXPORT_ void | ThreadDB_Destroy (void *pThreadDB_p) |
Interface to destroy an existing database object. More... | |
const DLLEXPORT_ char * | ThreadDB_GetVersionInfo (void *pThreadDB_p) |
Interface for retrieving the package version info. More... | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Save (const char *pIndexFileUTF8_p, void *pThreadDB_p) |
Interface to save the stored data. More... | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_NewThread (const char **pFileName_p, const char *pFolder_p, size_t MaxFileSize_p, void *pThreadDB_p) |
Interface to create and register a new worker thread. More... | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_NewPackage (uint64_t *pPackageID_p, void *pThreadDB_p) |
Registers a new data item package. More... | |
DLLEXPORT_ size_t | ThreadDB_GetPackageCount (void *pThreadDB_p) |
Returns the number of currently registered packages. More... | |
DLLEXPORT_ size_t | ThreadDB_GetFileCount (void *pThreadDB_p) |
Returns the number of currently registered temporary database files. More... | |
const DLLEXPORT_ char * | ThreadDB_GetDatabaseFileName (size_t FileIndex_p, void *pThreadDB_p) |
Returns the filename of a currently registered temporary database files. More... | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_RelocateFileTo (size_t FileID_p, const char *pFilePathUTF8_p, threadDB_RelocationType RelocationType_p, void *pThreadDB_p) |
Renames and copies/moves a temporary database file to a different location or disc. More... | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Store (uint64_t Package_p, size_t Size_p, const char pData_p[], threadDB_ItemInfo *pItemHandle_p, void *pThreadDB_p) |
Stores database package items in the specified package. More... | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Synchronize (void *pThreadDB_p) |
Synchronizes the internal package buffers with the temporary database files. More... | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Open (threadDB_ReadInfo **pReadInfo_p, size_t Package_p, void *pThreadDB_p) |
This operation opens a package for stream reading. More... | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Close (threadDB_ReadInfo **pReadInfo_p, void *pThreadDB_p) |
This operation closes a package after stream reading has finished. More... | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Recover (size_t *pReadBytes_p, size_t Size_p, char pData_p[], threadDB_ReadInfo *pReadInfo_p, void *pThreadDB_p) |
Stream reading operation through the data items stored in a package. More... | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_RecoverItem (size_t Size_p, char pData_p[], threadDB_ItemInfo *pItemHandle_p, void *pThreadDB_p) |
Random access operation to the data items stored in a package. More... | |
DLLEXPORT_ threadDB_ReturnCode | ThreadDB_Replace (size_t Size_p, const char pData_p[], const threadDB_ItemInfo *pItemHandle_p, void *pThreadDB_p) |
Replace operation to modify the contents of a stored data item. More... | |
const DLLEXPORT_ char * | ThreadDB_GetErrorMessage (void *pThreadDB_p) |
Information on error state. More... | |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Close | ( | threadDB_ReadInfo ** | pReadInfo_p, |
void * | pThreadDB_p | ||
) |
This operation closes a package after stream reading has finished.
This operation is required to close a package and remove the pReadInfo_p
after stream reading has finished.
pReadInfo_p | Read handle created during the ThreadDB_Open operation. |
pThreadDB_p | Handle to the database object. |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Create | ( | void ** | pThreadDB_p, |
size_t | PackageSize_p, | ||
size_t | PackageCacheLimit_p | ||
) |
Interface for creating a new database object.
Creates a new database object and returns the related handle. The operation initializes the internal data structures and starts the thread for package buffer flushing. The PackageSize_p
is the size of the memory block allocated for interim buffering of the stored data items. The PackageCacheLimit_p
defines an upper limit of parallel held item buffers of size PackageSize_p
. If the number of packages exceeds this limit the flush threads starts to withdraw package buffers from memory. This treatment requires additional thread synchronization taking place and could reduce the throughput significantly. The major part of consumed memory at a time is given by the number of packages times the package buffer size. Prior storing items in the database, it is important to create at least one worker thread using ThreadDB_NewThread.
pThreadDB_p | Handle to the created database object. |
PackageSize_p | Record size for buffering the data items. |
PackageCacheLimit_p | Maximum number of package buffers held in memory at one point of time. |
DLLEXPORT_ void ThreadDB_Destroy | ( | void * | pThreadDB_p | ) |
Interface to destroy an existing database object.
Destroys an existing database object, removes the related temporary database files and stops all running threads.
pThreadDB_p | Handle to the database object. |
const DLLEXPORT_ char* ThreadDB_GetDatabaseFileName | ( | size_t | FileIndex_p, |
void * | pThreadDB_p | ||
) |
Returns the filename of a currently registered temporary database files.
Returns the filename of the registered temporary database files with index FileIndex_p
. During execution of this command registering of new packages using ThreadDB_GetDatabaseFileName is synchronized.
FileIndex_p | Index of the temporary database file. |
pThreadDB_p | Handle to the database object. |
const DLLEXPORT_ char* ThreadDB_GetErrorMessage | ( | void * | pThreadDB_p | ) |
Information on error state.
This function returns information on current error state of the database.
pThreadDB_p | Handle to the database object. return Information on the occured error. |
DLLEXPORT_ size_t ThreadDB_GetFileCount | ( | void * | pThreadDB_p | ) |
Returns the number of currently registered temporary database files.
Returns the number of currently registered temporary database files. During execution of this command registering of new packages using ThreadDB_NewPackage is synchronized.
pThreadDB_p | Handle to the database object. |
DLLEXPORT_ size_t ThreadDB_GetPackageCount | ( | void * | pThreadDB_p | ) |
Returns the number of currently registered packages.
Returns the number of currently registered packages. During execution of this command registering of new packages using ThreadDB_NewPackage is synchronized.
pThreadDB_p | Handle to the database object. |
const DLLEXPORT_ char* ThreadDB_GetVersionInfo | ( | void * | pThreadDB_p | ) |
Interface for retrieving the package version info.
Provides the current version of the ThreadDB project.
pThreadDB_p | Handle to the database object. |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Import | ( | void ** | pThreadDB_p, |
const char * | pIndexFileUTF8_p, | ||
size_t | PackageCacheLimit_p | ||
) |
Interface for creating a new database object based on a database index file.
Creates a new database object based on a previously saved database. The operation initializes the internal data structures, starts the thread for package buffer flushing and builds up the internal package table. The PackageSize_p
is the size of the memory block allocated for interim buffering of the stored package items. The pIndexFileUTF8_p
needs to specify the full path to the location of the database index file. The pIndexFileUTF8_p
path characters have to be provided in UTF8 representation. Threads registered on an imported database will create new temporary database files and the imported database files are kept mainly untouched.
pThreadDB_p | Handle to the created database object. |
pIndexFileUTF8_p | Record size for buffering the data items. |
PackageCacheLimit_p | Maximum number of package buffers held in memory at one point of time. |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_NewPackage | ( | uint64_t * | pPackageID_p, |
void * | pThreadDB_p | ||
) |
Registers a new data item package.
Creates a new package to receive the stored data items. Each package definition is held in memory and consumes an amount of about 180 Bytes. Storing data items within a package synchronization of threads takes place.
pPackageID_p | Generated package ID. |
pThreadDB_p | Handle to the database object. |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_NewThread | ( | const char ** | pFileName_p, |
const char * | pFolder_p, | ||
size_t | MaxFileSize_p, | ||
void * | pThreadDB_p | ||
) |
Interface to create and register a new worker thread.
Creates a new worker thread receiving and managing the stored item data. Each worker thread gets a temporary database file assigned to. It is possible to define multiple worker threads and define temporary folders on different discs. The created files have the format threaddb.XXXXXX with XXXXX reflecting an unique identifying number. A basic prinicple of storing data itmes is, that the data is directed to the first available thread registered. This leads to a priority to which thread/temporary file the data is written. It is therefore sensible to specify folders on discs with faster write speed (e.g. SSD discs) first.
pFileName_p | Full path to the created temporary database file in ASCII character representation. |
pFolder_p | Path to the temporary database file folder in ASCII character representation. |
MaxFileSize_p | Maximum size in bytes a database file is allowed to grow. Exceeding this limit a new temporary file will be created. |
pThreadDB_p | Handle to the database object. |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Open | ( | threadDB_ReadInfo ** | pReadInfo_p, |
size_t | Package_p, | ||
void * | pThreadDB_p | ||
) |
This operation opens a package for stream reading.
This operation opens a package for stream reading the package contents. Stream reading is sensible for continious reading through the stored data items of the given package.
pReadInfo_p | Generated ReadInfo handle. |
Package_p | index of the package whose contents should be recovered. |
pThreadDB_p | Handle to the database object. |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Recover | ( | size_t * | pReadBytes_p, |
size_t | Size_p, | ||
char | pData_p[], | ||
threadDB_ReadInfo * | pReadInfo_p, | ||
void * | pThreadDB_p | ||
) |
Stream reading operation through the data items stored in a package.
This function allows to continiously read through the stored data items. The calling process has to preallocate the pData_p
transfer buffer and provide the size of data to be read through the Size_p
parameter. It is possible to read amongst several data items at once.
pReadBytes_p | Number of read data bytes. |
Size_p | number of bytes to be read from the selected package. |
pData_p | preallocated transfer buffer, requires to be preallocated and embrace at least Size_p bytes. |
pReadInfo_p | read handle created during the ThreadDB_Open operation. |
pThreadDB_p | Handle to the database object. |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_RecoverItem | ( | size_t | Size_p, |
char | pData_p[], | ||
threadDB_ItemInfo * | pItemHandle_p, | ||
void * | pThreadDB_p | ||
) |
Random access operation to the data items stored in a package.
This function allows to read stored data items individually. The calling process has to preallocate the pData_p
transfer buffer and provide the size of data to be read through the Size_p
parameter. It is possible to read amongst several data items at once. The parameter pItemHandle_p
points to the selected data item. During read the handle is moved on to the following item entry. This allows to read through a series of elements continously. Since this operation requires to individually seek for the entry on disc, throughput es in general less in comparison to continous reading.
Size_p | number of bytes to be read from the selected package. |
pData_p | preallocated transfer buffer, requires to embrace at least Size_p bytes. |
pItemHandle_p | handle to the stored item. |
pThreadDB_p | Handle to the database object. |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_RelocateFileTo | ( | size_t | FileID_p, |
const char * | pFilePathUTF8_p, | ||
threadDB_RelocationType | RelocationType_p, | ||
void * | pThreadDB_p | ||
) |
Renames and copies/moves a temporary database file to a different location or disc.
The function allows to rename or copy/move a database file to a different location or disc. The command is synchronized and available also while store or read operations are in progress.
FileID_p | the file index of the temporary database file to be renamed/moved. |
pFilePathUTF8_p | full path to the destination to which the database file should be moved. |
RelocationType_p | type of action copy/move to take place. |
pThreadDB_p | Handle to the database object. |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Replace | ( | size_t | Size_p, |
const char | pData_p[], | ||
const threadDB_ItemInfo * | pItemHandle_p, | ||
void * | pThreadDB_p | ||
) |
Replace operation to modify the contents of a stored data item.
This function allows to modify existing data item entries in the database. It replaces the given Size_p
of bytes by the data provided in the pData_p
transfer buffer. The Size_p parameter has to match the size of the previously stored data item.
Size_p | number of bytes to be replaced. |
pData_p | preallocated transfer buffer, requires to be embrace at least Size_p bytes. |
pItemHandle_p | handle to the stored item to be replaced. |
pThreadDB_p | Handle to the database object. |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Save | ( | const char * | pIndexFileUTF8_p, |
void * | pThreadDB_p | ||
) |
Interface to save the stored data.
Saves the database control structures to the given database index file. The related temporary database files are removed from the purging process executed during the database destruction phase.
pIndexFileUTF8_p | Full path to the database index file location in UTF8 representation. |
pThreadDB_p | Handle to the database object. |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Store | ( | uint64_t | Package_p, |
size_t | Size_p, | ||
const char | pData_p[], | ||
threadDB_ItemInfo * | pItemHandle_p, | ||
void * | pThreadDB_p | ||
) |
Stores database package items in the specified package.
The function allows to append a new data item to the package @ Package_p. The item is asynchronously copied and appended to the given package. The parameter pItemHandle_p
handle can be omitted by specifying a NULL pointer. If available, the stored index entry of the item is provided. A pItemHandle_p
allows random access of individual package items. The pItemHandle_p
has to be allocated and managed separately and provided by the caller. The item handles are registered in the database and updated during the ThreadDB_Synchronize operation. Please note, that the item handle is usually invalid until the ThreadDB_Synchronize operation is executed. The Size_p
parameter specifies the size of data to be transferred. The size of the data item is NOT stored in the database and has to be managed separately. It needs to be provided during the ThreadDB_Recover and ThreadDB_RecoverItem operations. The maximum size specified by the Size_p
parameter. Data items exceeding the specified PackageSize_p
need are split into multiple records and stored performing several @ ref ThreadDB_Store calls. Please note that each data item requires to be temporary duplicated to allow asynchronous storage. A special form of the ThreadDB_Store call is to flush specific packages to disc. This can be achieved by calling the function with a Size_p
of zero. In this case only the buffer of the selected Package_p
is synchronized while other packages stay open for writing and are not prematurely flushed. This allows to start reading of package information while writing to other packages is still in progress.
Package_p | package index to which the data item should be appended. |
Size_p | size of the package item to be stored. |
pData_p | pointer to the data for being stored. |
pItemHandle_p | optional parameter for random data item access. |
pThreadDB_p | Handle to the database object. |
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Synchronize | ( | void * | pThreadDB_p | ) |
Synchronizes the internal package buffers with the temporary database files.
Synchronizes the package buffers by writing their contents to the database files. This operation also updates and validates the registered item handles specified during the ThreadDB_Store operation. Synchronization is required prior calling ThreadDB_Recover or ThreadDB_RecoverItem operations. This operation also releases the internal package buffers.
pThreadDB_p | Handle to the database object. |