Database File Cache Classes

Contents

Overview
Usage
Conditional Directives
Constants
Public Data Members
Database Segment Functions
Database Cache Functions
Database Cache Node Functions

Overview

The database file cache classes are experimental. They are a special adaptation of the Device Cache classes that use raw file data as the segment type. The cache links blocks of file data to memory-based cache segments, so an application can read and write cached segments in place of the disk file. The goal is to reduce disk reads and writes.

The cache works directly on a gxdFPTR file pointer, not on a gxDatabase object. It reads and writes raw bytes at the file addresses you give it. It does not allocate database blocks, write block headers, or calculate checksums. Use the gxDatabase engine to allocate blocks and use the cache to move the object data.

The database file cache uses three cooperating classes:

gxDatabaseSegment - A variable length memory segment that holds a copy of file data and the file address it came from. Each segment has a dirty flag that tells the cache the in-memory copy has changed and must be written to disk. Segments also have previous and next pointers used to link them into a list.

gxDatabaseCache - Allocates and frees a fixed number of segments and handles requests for file data. When data is requested the cache first looks for a segment already holding that file address. If none is found it reuses a segment that is not dirty, or flushes the least recently reserved segment if every segment is dirty. The reused segment is cleared, moved to the head of the list, and loaded with data from disk. Segments are kept in a circular doubly linked list. The head of the list is the most recently reserved segment and the tail is the least recently reserved segment.

gxDBCacheNode - A lightweight pointer used to read and write cached segments. Each cache node stores a file address, a pointer to the segment holding that address, and a pointer to its cache.

The cache owns all segment memory. The segment array is allocated when the cache is constructed and freed when the cache is destroyed. Each segment owns a MemoryBuffer that grows to fit the largest block of data loaded into it. Cache nodes never own memory.

The cache does not write changes to disk until a segment is reused or one of the flush functions is called. The destructor flushes all dirty segments to the connected file. If the file will be closed before the cache is destroyed, call Flush() and then ReleaseFile() first.

A file address of -1 marks a segment or cache node that is not linked to any file data.

Usage

The following code is taken from the examples_database/database_cache test program. It writes objects through the cache and reads them back:

const unsigned NUM_SEGMENTS = 10;
gxDatabaseCache cache(NUM_SEGMENTS, f->GetFP());
gxDBCacheNode p(&cache);

// Allocate room for the object in the data file
FAU_t object_address = f->Alloc(sizeof(DatabaseObject));

// Write the object to the database file cache
ds = p.CachedWrite(&ob, sizeof(DatabaseObject), object_address);

// Read the object from the database file cache
ds = p.CachedRead(offset, sizeof(DatabaseObject));
if(ds) {
  memmove(&buf, p->ds.m_buf(), sizeof(DatabaseObject));
}

// Flush the cache and release file the cache is bound to
// before closing the file.
cache.Flush();
cache.ReleaseFile();
f->Close();

Conditional Directives

__USE_DATABASE_CACHE__ - The database file cache classes are only compiled when this directive is defined. If it is not defined the dbfcache.h header declares nothing. The Linux build environment (env/linux.env) defines it by default.

Constants

Minimum number of cache segments. If fewer segments are requested the cache allocates this many instead.

const int MIN_DATABASE_SEGMENTS = 3;

Public Data Members

class gxDatabaseSegment
{
public:
  gxDatabaseSegment *prev; // Previous database segment in the list
  gxDatabaseSegment *next; // Next database segment in the list
  FAU_t ds_address;        // File address of this database segment

public: // Persistent data members
  MemoryBuffer ds; // In memory copy of raw file data
};

class gxDatabaseCache
{
public: // Cache statistics
  unsigned uncached_reads; // Counts the number of disk reads
  unsigned cached_reads;   // Counts the number of cached reads
};

Read the cached data through the segment's "ds" MemoryBuffer. If you change the data in "ds" directly, call SetDirty() so the change is written to disk. The "prev" and "next" pointers are managed by the cache and should not be changed by the application.

The statistics counters are updated by gxDatabaseCache::AllocSegment(). A request for an address already in the cache increments "cached_reads". A request for an address not in the cache increments "uncached_reads", even if no data is read from disk. Use ResetStats() to set both counters to zero.

Database Segment Functions

gxDatabaseSegment::Flush()
gxDatabaseSegment::IsDirty()
gxDatabaseSegment::IsNull()
gxDatabaseSegment::Read()
gxDatabaseSegment::ResetSegment()
gxDatabaseSegment::SetDirty()
gxDatabaseSegment::Write()
gxDatabaseSegment::gxDatabaseSegment()
gxDatabaseSegment::~gxDatabaseSegment()

gxDatabaseError gxDatabaseSegment::Flush(gxdFPTR *fp)

Public member function used to write this segment to the file "fp" if it is dirty and has a valid file address. After a successful write the file address is set to -1 and the dirty flag is cleared. The data stays in the memory buffer. A segment that is not dirty is left unchanged. Returns gxDBASE_NULL_PTR if "fp" is null, zero if successful, or a non-zero value if the write fails.

int gxDatabaseSegment::IsDirty()

Public member function that returns true if this segment has been marked dirty.

int gxDatabaseSegment::IsDirty() const

Public member function that returns true if this segment has been marked dirty. Const version.

int gxDatabaseSegment::IsNull()

Public member function that returns true if the memory buffer has no memory allocated or a logical length of zero.

int gxDatabaseSegment::IsNull() const

Public member function that returns true if the memory buffer is empty. Const version.

gxDatabaseError gxDatabaseSegment::Read(gxdFPTR *fp, __ULWORD__ bytes)

Public member function used to read "bytes" bytes from the file "fp", starting at this segment's file address, into the memory buffer. The buffer is resized as needed. The dirty flag is cleared after a successful read. Returns gxDBASE_NULL_PTR, gxDBASE_MEM_ALLOC_ERROR, gxDBASE_FILE_SEEK_ERROR, or gxDBASE_FILE_READ_ERROR on failure, or zero if successful.

void gxDatabaseSegment::ResetSegment()

Public member function used to clear the dirty flag, set the file address to -1, and set the logical length of the memory buffer to zero. The buffer memory is kept for reuse.

void gxDatabaseSegment::SetDirty()

Public member function used to mark this segment as dirty. A dirty segment is written to disk when it is flushed or reused.

gxDatabaseError gxDatabaseSegment::Write(gxdFPTR *fp)

Public member function used to write the whole memory buffer to the file "fp" at this segment's file address. The dirty flag is cleared after a successful write and the file address is kept. If the buffer is empty nothing is written and zero is returned. Returns gxDBASE_NULL_PTR, gxDBASE_FILE_SEEK_ERROR, or gxDBASE_FILE_WRITE_ERROR on failure, or zero if successful.

gxDatabaseSegment::gxDatabaseSegment()

Default class constructor. Sets the list pointers to null and resets the segment. No buffer memory is allocated.

gxDatabaseSegment::gxDatabaseSegment(unsigned segment_size)

Class constructor used to reset the segment and allocate "segment_size" bytes for the memory buffer.

gxDatabaseSegment::~gxDatabaseSegment()

Class destructor. The memory buffer frees its own memory. Dirty data is not written to disk.

Database Cache Functions

gxDatabaseSegment *gxDatabaseCache::AllocSegment(FAU_t ds_address, __ULWORD__ bytes = (__ULWORD__)0)

Public member function used to reserve a segment for the file address "ds_address". If the address is already in the cache its segment is moved to the head of the list and returned, and "bytes" is ignored. Otherwise a free segment is found, cleared, moved to the head of the list, assigned to the address, and, if "bytes" is greater than zero, loaded with "bytes" bytes from the file. Returns a pointer to the segment or a null value if "ds_address" is -1, no segment can be freed, the cache is not connected when a read is needed, or the read fails. If the read fails the segment is cleared and moved to the tail of the list. NOTE: 10/14/2026: Fixed three problems. A reused segment that was not dirty was not moved to the head of the list, so it was the first to be reused again. A reused segment kept the data from its previous use. A failed read left the segment assigned to the address, so the next request returned unread data as a cache hit.

unsigned gxDatabaseCache::CacheSize()

Public member function that returns the number of bytes used by the cache. This is the size of the segment array plus the memory allocated by every segment buffer. The value is calculated each time this function is called. Returns zero if no segments are allocated.

gxDatabaseError gxDatabaseCache::Connect(gxdFPTR *fptr)

Public member function used to connect the cache to an open file. If the cache is already connected, all dirty segments are flushed to the old file first. Returns gxDBASE_NULL_PTR if "fptr" is null, zero if successful, or a non-zero value if the flush fails. If the flush fails the cache stays connected to the old file.

gxDatabaseSegment *gxDatabaseCache::ConstructCache(unsigned num_segs)

Public member function used to allocate the segment array as one contiguous block and link the segments into a circular list. If "num_segs" is less than MIN_DATABASE_SEGMENTS, the minimum is used. If a segment array already exists it is destroyed first, which also flushes and disconnects the cache. The statistics counters are reset. Returns a pointer to the segment array.

gxDatabaseSegment *gxDatabaseCache::DemoteSegment(gxDatabaseSegment *segment)

Public member function used to move the specified segment to the tail of the list, making it the least recently reserved segment. Returns the segment, or a null value if the cache is empty.

void gxDatabaseCache::DestroyCache()

Public member function used to flush all dirty segments, disconnect the cache from its file, and free the segment array.

gxDatabaseSegment *gxDatabaseCache::FindSegment(FAU_t ds_address)

Public member function used to search the cache for the segment holding "ds_address". Does not change the order of the list or the statistics counters. Returns a pointer to the segment or a null value if the address is not in the cache.

gxDatabaseError gxDatabaseCache::Flush()

Public member function used to write every dirty segment to the connected file, starting at the head of the list. Flushed segments keep their data but their file address is set to -1, so the data will be read from disk again on the next request. Returns gxDBASE_NULL_PTR if the cache is not connected or has no segments, zero if successful, or the error code of the first failed write.

gxDatabaseSegment *gxDatabaseCache::GetHead()

Public member function that returns a pointer to the head of the segment list, which is the most recently reserved segment.

gxDatabaseSegment *gxDatabaseCache::GetHead() const

Public member function that returns a pointer to the most recently reserved segment. Const version.

gxDatabaseSegment *gxDatabaseCache::GetTail()

Public member function that returns a pointer to the tail of the segment list, which is the least recently reserved segment.

gxDatabaseSegment *gxDatabaseCache::GetTail() const

Public member function that returns a pointer to the least recently reserved segment. Const version.

gxdFPTR *gxDatabaseCache::gxFilePtr()

Public member function that returns the file pointer the cache is connected to, or a null value if the cache is not connected.

int gxDatabaseCache::IsEmpty()

Public member function that returns true if no segment array is allocated.

int gxDatabaseCache::IsEmpty() const

Public member function that returns true if no segment array is allocated. Const version.

gxDatabaseError gxDatabaseCache::LRUFlush(unsigned num_to_flush)

Public member function used to flush up to "num_to_flush" segments, starting at the tail of the list (the least recently reserved segment) and moving toward the head. Clean segments count toward "num_to_flush" but are not written. Returns the same values as Flush().

int gxDatabaseCache::NumSegments()

Public member function that returns the total number of segments allocated.

int gxDatabaseCache::NumSegments() const

Public member function that returns the total number of segments allocated. Const version.

gxDatabaseCache::operator int()

Conversion operator that returns true if a segment array is allocated.

gxDatabaseCache::operator const int() const

Const conversion operator that returns true if a segment array is allocated.

int gxDatabaseCache::operator!()

Overloaded not operator that returns true if no segment array is allocated.

int gxDatabaseCache::operator!() const

Overloaded not operator that returns true if no segment array is allocated. Const version.

gxDatabaseSegment *gxDatabaseCache::PromoteSegment(gxDatabaseSegment *segment)

Public member function used to move the specified segment to the head of the list, making it the most recently reserved segment. Returns the segment, or a null value if the cache is empty or "segment" is null.

void gxDatabaseCache::ReleaseFile()

Public member function used to disconnect the cache from its file without flushing. Any dirty segments will not be written. Call this after Flush() and before closing the file so the destructor does not write to a closed file.

gxDatabaseSegment *gxDatabaseCache::ReleaseSegments()

Public member function that releases the segment array without freeing it and returns it. The cache is left empty: the head, tail, segment count and cache size are reset, so the cache will not use or free the array. The caller must free the returned array with delete[]. Dirty segments are not flushed, so call Flush() first to keep their data. NOTE: 10/24/2026: Changed in version 6.0. The function returned nothing and set only the array pointer to null, so the head and tail still pointed into the released array and the array was lost unless the caller had kept a pointer to it.

void gxDatabaseCache::ResetStats()

Public member function used to set the uncached_reads and cached_reads counters to zero.

unsigned gxDatabaseCache::SegmentsInUse()

Public member function that returns the number of dirty segments in the cache.

gxDatabaseCache::gxDatabaseCache(unsigned num_segs)

Class constructor used to build a cache of "num_segs" segments that is not connected to a file. Call Connect() before reading or flushing.

gxDatabaseCache::gxDatabaseCache(unsigned num_segs, gxdFPTR *fptr)

Class constructor used to build a cache of "num_segs" segments connected to the open file "fptr".

gxDatabaseCache::~gxDatabaseCache()

Class destructor. Calls DestroyCache(), which flushes all dirty segments to the connected file and frees the segment array.

Database Cache Node Functions

gxDBCacheNode::Bind()
gxDBCacheNode::CachedRead()
gxDBCacheNode::CachedWrite()
gxDBCacheNode::Delete()
gxDBCacheNode::GetCache()
gxDBCacheNode::GetNodeAddress()
gxDBCacheNode::GetNodeLength()
gxDBCacheNode::SetNodeLength()
gxDBCacheNode::GetSegment()
gxDBCacheNode::HasSegment()
gxDBCacheNode::operator FAU_t()
gxDBCacheNode::operator*()
gxDBCacheNode::operator->()
gxDBCacheNode::operator=()
gxDBCacheNode::Release()
gxDBCacheNode::ReleaseCache()
gxDBCacheNode::gxDBCacheNode()
gxDBCacheNode::~gxDBCacheNode()

gxDatabaseSegment *gxDBCacheNode::Bind()

Public member function used to bind this cache node to the segment holding its file address. If the current segment still holds that address it is returned. Otherwise the segment is loaded with the node length, the number of bytes given to the last CachedRead(), CachedWrite() or Bind(bytes) call. If no node length is set, only a segment already in the cache can be bound. A null value is then returned for an address that is not cached, and the file address is kept so Bind(bytes) can load it. If no segment can be reserved the file address is set to -1 and a null value is returned. NOTE: 10/24/2026: Fixed in version 6.0. A segment not already in the cache was loaded with a read length of one byte, so it held only the first byte of the node.

gxDatabaseSegment *gxDBCacheNode::Bind(__ULWORD__ bytes)

Public member function that sets the node length to "bytes" and calls Bind(). A segment not already in the cache is loaded with "bytes" bytes of file data. NOTE: 10/24/2026: Added in version 6.0.

gxDatabaseSegment *gxDBCacheNode::CachedRead(FAU_t address, __ULWORD__ bytes = ( __ULWORD__)0)

Public member function used to read "bytes" bytes of file data at "address" through the cache. If the address is already cached the cached segment is returned as it is. Otherwise a segment is reserved and loaded from disk. If "bytes" is zero no data is read and the segment is empty. On success the file address of this cache node is set. Returns a pointer to the segment, or a null value if an error occurs, in which case the file address is set to -1. NOTE: 10/14/2026: Fixed a reused segment returning data from its previous use when "bytes" was zero.

gxDatabaseSegment *gxDBCacheNode::CachedWrite(const void *buf, __ULWORD__ bytes, FAU_t address)

Public member function used to copy "bytes" bytes from "buf" into a cache segment for the file address "address" and mark the segment dirty. The data is written to disk when the segment is flushed or reused. The address must be the file offset where the data belongs, such as the object address returned by gxDatabase::Alloc(). Returns a pointer to the segment, or a null value if no segment can be reserved or the data cannot be copied into it, in which case the file address is set to -1. NOTE: 10/14/2026: Fixed a failed copy of the data still marking the segment dirty. The segment is now cleared and a null value is returned.

int gxDBCacheNode::Delete()

Public member function used to release the segment this cache node points to. The segment is reset, which clears its dirty flag so the data is not written to disk, and the segment is moved to the tail of the list for reuse. The file address of this cache node is set to -1. No file data is deleted. Returns true if successful or false if this cache node has no segment, the cache is not connected, or the segment now holds a different address.

gxDatabaseCache *gxDBCacheNode::GetCache()

Public member function that returns a pointer to the cache this cache node is connected to.

gxDatabaseCache *gxDBCacheNode::GetCache() const

Public member function that returns a pointer to the cache this cache node is connected to. Const version.

FAU_t gxDBCacheNode::GetNodeAddress()

Public member function that returns the file address this cache node refers to.

FAU_t gxDBCacheNode::GetNodeAddress() const

Public member function that returns the file address this cache node refers to. Const version.

__ULWORD__ gxDBCacheNode::GetNodeLength()

Public member function that returns the number of bytes Bind() loads for a segment that is not in the cache. It is set by CachedRead() (when "bytes" is not zero), CachedWrite(), Bind(bytes) and SetNodeLength(). A value of 0 means not set. NOTE: 10/24/2026: Added in version 6.0.

void gxDBCacheNode::SetNodeLength(__ULWORD__ bytes)

Public member function used to set the number of bytes Bind() loads for a segment that is not in the cache. NOTE: 10/24/2026: Added in version 6.0.

gxDatabaseSegment *gxDBCacheNode::GetSegment()

Public member function that returns the current segment pointer without binding. The segment may have been reused for another address.

gxDatabaseSegment *gxDBCacheNode::GetSegment() const

Public member function that returns the current segment pointer without binding. Const version.

int gxDBCacheNode::HasSegment()

Public member function that returns true if this cache node has a segment pointer.

int gxDBCacheNode::HasSegment() const

Public member function that returns true if this cache node has a segment pointer. Const version.

gxDBCacheNode::operator FAU_t()

Conversion operator that returns the file address of this cache node.

gxDBCacheNode::operator FAU_t() const

Const conversion operator that returns the file address of this cache node.

gxDatabaseSegment &gxDBCacheNode::operator*()

Overloaded indirection operator. Calls Bind() and returns a reference to the segment. The result is not checked, so the segment must be valid.

gxDatabaseSegment *gxDBCacheNode::operator->()

Overloaded arrow operator. Calls Bind() and returns a pointer to the segment, which may be null. After a successful CachedRead() this returns the segment just read, for example p->ds.m_buf().

gxDBCacheNode &gxDBCacheNode::operator=(const gxDBCacheNode &ob)

Assignment operator. Copies the segment and cache pointers from "ob" and sets the file address of this cache node to -1 (not linked). NOTE: 10/14/2026: The file address was set to 0, which is a valid file offset for this cache.

void gxDBCacheNode::operator=(FAU_t address)

Assignment operator used to set the file address and call Bind(). The node length is not changed, so the segment is loaded with the length of the last read or write.

void gxDBCacheNode::Release()

Public member function used to set the segment pointer to null. The segment and the file address are not changed.

void gxDBCacheNode::ReleaseCache()

Public member function used to set the cache pointer to null. The cache node must not be used again until it is connected to a cache, because the other member functions do not test for a null cache.

gxDBCacheNode::gxDBCacheNode(gxDatabaseCache &ds_cache, FAU_t address = (FAU_t)-1)

Class constructor used to connect this cache node to a cache and set its file address. No segment is bound until the node is used.

gxDBCacheNode::gxDBCacheNode(gxDatabaseCache *ds_cache, FAU_t address = (FAU_t)-1)

Class constructor used to connect this cache node to a cache, given a pointer, and set its file address.

gxDBCacheNode::gxDBCacheNode(const gxDBCacheNode &ob)

Copy constructor. Copies the segment and cache pointers from "ob" and sets the file address to -1 (not linked). NOTE: 10/14/2026: The file address was set to 0, which is a valid file offset for this cache.

gxDBCacheNode::~gxDBCacheNode()

Class destructor. Does nothing. The segment remains in the cache.