Database Overview

Chapter Updated 2/1/99


The objective of this chapter is to provide information regarding how the database files are utilized and document the various record structures. With the exception of the brief section on the record buffer, the information presented in this chapter is not required to use the Xbase library. It is mainly information describing internal file structures utilized by the Xbase routines.

Xbase files are comprised of a variable length header record which stores information about the file and describes the fixed length record format, followed by a series of fixed length data records.

Each fixed length data record is preceded by a one byte indicator which identifiies if the record has been deleted. If the record is not deleted, the indicator is a space (0x20). If deleted, the indicator contains an asterisk (0x2A). Data fields are stored in records without field separators or record terminators.

In earlier releases of dBASE, there is an ASCII NULL character between the $0D end of header indicator and the start of the data. This NULL was removed starting with dBASE III Plus, making a Plus header one byte shorter than an identically structured III file. The methods documented in the Xbase software and documentation follow the more recent version where the NULL character is not included.

Each database file is comprised of zero, one or many records. A record is comprised of fields. Only one record is accessed at a time.

Zero, one or many database files can be open simultaneously.


The Record Buffer

When using the Xbase routines, each open data file has a record buffer which is manipulated by calling the database, index and field routines. Updates are committed from the record buffer to the database when a write, or append is performed. The library will automatically write updates to the database if the buffer has been updated and the record is repositioned or the database is closed.

The record buffer is not used for handling the actual data portion of memo fields. When working with memo fields, the application program must allocate enough buffer space for reading and writing memo fields.

Internal to the library, there is an additional record buffer which stores the original value of the data record before any changes are made. This is used by the index routines for finding and deleting original key values from any open indices before adding the new keys. If the key values are not changed, no index updates occur.



Xbase Database File Header

The Xbase file header, located at the beginning of the database, describes the .DBF database. Knowledge of this structure is not necessary to effectively utilize the Xbase libraries.


PositionLengthDescription
01 bytefile version number
(03H without a .DBT file)
(83H with a .DBT file)
1-33 bytesdate of last update
(YY MM DD) in binary format
4-732 bit numbernumber of records in data file
8-916 bit numberlength of header structure
10-1116 bit numberlength of the record
12-3120 bytesreserved
32-n32 bytes eachfield descriptor record (see below)
n+11 byte0DH as the field terminator




Xbase Field Descriptor Record

The Xbase field descriptor record stores information about each field in the database. Each database has from 1 to 1024 fields. Knowledge of this structure is not necessary to effectively utilize the Xbase libraries.


PositionLengthDescription
0-1011 bytesfield name in ASCII zero-filled
111 bytefield type in ASCII (C N L D or M)
12-1532 bit numberfield data address
161 bytefield length in binary
171 bytefield decimal count in binary
18-3114 bytesreserved bytes (version 1.00)




Field Data Format

Data are stored in ASCII format in the database as follows:

DATA TYPEDATA RECORD STORAGE
CharacterASCII characters, left justified, right blank filled
Date(8 digits in YYYYMMDD format, such as
19601007 for October 7, 1960)
Logical? Y y N n T t F f (? when not initialized)
Memo10 digits representing a .DBT block number
Numeric. 0 1 2 3 4 5 6 7 8 9 + -, right justified, left blank filled
Float. 0 1 2 3 4 5 6 7 8 9 + -, right justified, left blank filled



Memo Fields

Memo fields store variable length data elements in a seperate .DBT file. The main .DBF file maintains a ten byte field which is used by the Xbase routines for determining the location of the data in the .DBT file.

Xbase DBMS supports both dBASE III+ and dBASE IV version memo files. The version IV files are somewhat more efficient in that they reuse unused memo space when data are deleted or freed from use. With version III files, all new updates are appended to the end of the file and the unused space is not reclaimed until the datafiles are packed.

Memo fields can be used for storing a variety of date type. However, type 3 files are limited to storing textual data because most internal memo field processing in a type 3 file relies on two contiguous 0x1a charaters.

Type 4 memo fields can be used for storing BLOB (binary large object) data reliably, as the internal file structure does not rely on any special characters embedded in the data.

A special not on storing string data in a memo field. For those users that are new to C/C++ programming, string fields typically end with a null (0x00) terminator character. As a general rule of thumb when using the library, add one to the length of any string when specifying the length of the data. This stores the null terminating byte with the data. For example, when storing string "This is a string" specified size if 17, not 16.

Technical memo file information

The following info on memo fields is for the curious. It is not required reading if you don't need to know the internals.

  • Memo files are made up of one or more blocks
  • For version III files, the block size is 512
  • For version IV files, the block size is a multiple of 512
  • The minimum amout of space necessary to store one memo field is one block or 512 bytes.
  • The default block size can be adjusted by manipulating the XB_DBT_BLOCK_SIZE macro in the options.h file.
  • The main .DBF file maintains a ten byte numeric field which is blank if no memo data exists for a given field. Otherwise it contains a number, which when multiplied by the block size, points to the offset in the file of the head block in the file/

    For version 3 memo field files, there are two fields in the head block of the file, NextBlockNo and Version. Depending on the Xbase software, some vendors products update these two fields, some do not. The Xbase library keeps the fields updated, but does not rely on them to be valued with correct data. This helps to support maximum compatibility amoungst all Xbase tools available.

    For version 4 memo field files, the first block in the .DBT file is a header block which is comprised of 8 bytes of data which maintain the file's block size and the next free block available in the file. Blocks two through n contain the actual memo data. A chain of empty blocks is maintained within the file for potential future use. When an add or update routine executes, it first attempts to find a spot in a set of blocks which were earlier allocated, but not currently in use for the data. If no free spot is found, data are appended to the end of the file. The free block chain is sorted in block number order. When blocks of data are freed and added to the free block chain, the routines will attempt to concatonate free block chains togethor where possible. When a delete occurs, or an update which requires less space occurs, the new free space is added to the free block chain.

    Various Memo File Block Types

    Valid Block Types
    Head Block
    Only data block for memo field
    First of several contiguous data block set
    2-n of contiguous data block set
    Only data block in free chain (version IV only)
    First of several contiguous free block set (version IV only)
    2-n of contiguous free block set (type 4 only)


    Head Block Structure

    1-4LONGNext Block ID
    5-8LONGNot used all 0x00's
    9-16CHAR(8)Filename (Version IV Only)
    17CHARVersion (0x03 = Version III, 0x00 = Version IV)
    18-20CHAR(3)Not used all 0x00's
    21-22SHORTBlock Size (Version IV only )
    23-Remainder of blockCHARNot used


    Version IV Head Data Block Structure

    xbShort0-1-1
    xbShort2-3Starting position of data (always 8 ?)
    xbLong4-7Length of data includes first 8 bytes
    char (9) - Blocksize8-15Data


    Version IV Head Free Block Structure

    xbLong0-3Next free block in the free block chain
    xbLong4-7Number of free blocks in this contiguous free block set


    Version 3 and 4 memo fields are terminated with two contiguous 0x1A bytes of data.


    Send Me Mail - xbase@startech.keller.tx.us

    (c)1997 StarTech