Servertec
Persistent Object Store
1.4.1 09/04/2005

stec.xpos
Class ObjectStore

java.lang.Object
  extended by stec.xpos.ObjectStore

public final class ObjectStore
extends java.lang.Object

Contains static methods used to create, delete, open, rehash, expand and pack extended persistent object stores.

Extended persistent object stores are simular to java.util.Hashtables, except they do not reside in memory, but on storage media, are not automaically rehashed and support null values.

Each extended persistent object store physically exists in at least five (5) physical random access volume files, Store and Table, Mapper, Keys and Values. The Store file contains a header and volume descriptors for Table, Mapper, Keys and Values volume files. The Table volume file(s) contains a header and the lookup table. The Mapper volume file(s) contains a header and and zero (0) or more mapper pages pointing to corresponding keys, value pairs stored in Keys and Values and zero (0) or more freed mapper pages. The Keys volume file(s) contains a header and and zero (0) or more serialized keys and zero (0) or more freed key pages. The Values volume file(s) contains a header and and zero (0) or more serialized values and zero (0) or more freed value pages.

Volume Header

1 byte - type - the file type. Currently 0x00 for Volume.
2 bytes - version - the version number. Currently 0x0100 (1.0.0).
4 byte - lookup_table_volume_count - the number of lookup table volumes. one (1) or more.
n bytes - lookup_table_volume_descriptors - one (1) or more lookup table volume descriptors.
4 byte - mapper_volume_count - the number of mapper volumes. one (1) or more.
n bytes - mapper_volume_descriptors - one (1) or more mapper volume descriptors.
4 byte - keys_store_volume_count - the number of keys store volumes. one (1) or more.
n bytes - keys_store_volume_descriptors - one (1) or more keys store volume descriptors.
4 byte - values_store_volume_count - the number of values store volumes. one (1) or more.
n bytes - values_store_volume_descriptors - one (1) or more values store volume descriptors.

Volume Descriptor

n bytes - volume_filename - the utf8 encoded file name of the volume file.
8 bytes - maximum_page_count - the maximum number of pages that can be allocated. 1 or more. -1 for unlimited.

Store Header

1 byte - type - the file type. Currently 0x03 for Store.
2 bytes - version - the version number. Currently 0x0100 (1.0.0).
4 bytes - table_descriptor_count - the number of Table volume descriptors, one (1) or more.
n bytes - table_descriptors - one (1) or more volume descriptors for Table.
4 bytes - mapper_descriptor_count - the number of Mapper volume descriptors, one (1) or more.
n bytes - mapper_descriptors - one (1) or more volume descriptors for Mapper.
4 bytes - keys_descriptor_count - the number of Keys volume descriptors, one (1) or more.
n bytes - keys_descriptors - one (1) or more volume descriptors for Keys.
4 bytes - values_descriptor_count - the number of Values volume descriptors, one (1) or more.
n bytes - values_descriptors - one (1) or more volume descriptors for Values.

Table Header

1 byte - type - the file type. Currently 0x04 for Table.
2 bytes - version - the version number. Currently 0x0100 (1.0.0).
8 bytes - lookup_table_page_count - the number of pages in the lookup table as specified when the extended persistent object store was created. 1 or more.

Lookup Table

The lookup table is an array of eight (8) byte pointers to the first mapper pages of size lookup_table_page_count as defined in the above header.

Mapper Header

1 byte - type - the file type. Currently 0x05 for Mapper.
2 bytes - version - the version number. Currently 0x0100 (1.0.0).
8 bytes - page_count - the number of mapper pages currently in use or free. 0 or more.
8 bytes - maximum_page_count - the maximum number of pages that can be allocated. 1 or more. -1 for unlimited.
8 bytes - free_list - pointer to the first free mapper page. -1 if none.
8 bytes - free_count - the number of free pages. 0 or more.
8 bytes - record_count - the number of existing keys and associated values. 0 or more.

Mapper Page

1 byte - flags - flags used to specify whether the mapper page is in use or has been freed. 1 for in use and 0 for free.
8 bytes - previous_page - pointer to any previous mapper page. -1 if none.
8 bytes - next_page - pointer to any next mapper page. -1 if none.
8 bytes - hash_code - the hash code value of the key object.
8 bytes - key_base_pointer - the base pointer to thedata page containing the encoded key.
4 bytes - key_length - the number of bytes used to store the encoded key.
8 bytes - value_base_pointer - the base pointer to the data page containing the encoded value.
4 bytes - value_length - the number of bytes used to store the encoded value.

Once a key and associated value are removed, mapper pages no longer used are linked to the beginning of the free_list as defined in the above header.

Keys Header

1 byte - type - the file type. Currently 0x06 for Keys.
2 bytes - version - the version number. Currently 0x0100 (1.0.0).
4 bytes - page_length - the size in bytes of the average page as specified when the extended persistent object store was created.
8 bytes - page_count - the number of pages currently in use or free. 0 or more.
8 bytes - maximum_page_count - the maximum number of pages that can be allocated. 1 or more. -1 for unlimited.
8 bytes - free_list - pointer to first free page. -1 if none.
8 bytes - free_count - the number of free pages. 0 or more.

Each page is composed of a header and data area of size page_length as specified in the above header.

Keys Page

1 byte - flags - flags used to specify whether the page is in use or has been freed. 1 for in use and 0 for free.
8 bytes - next_data_page - pointer to any page that follows. -1 if none.
n bytes - entry - the encoded key mapped to fixed data page of page_length as specified in the header.

The persistant object store stores variable length objects by storing them into one (1) or more fixed length pages.

Once a key is changed or removed, pages no longer used are linked to the beginning of the free_list as defined in the above header.

Values Header

1 byte - type - the file type. Currently 0x07 for Values.
2 bytes - version - the version number. Currently 0x0100 (1.0.0).
4 bytes - page_length - the size in bytes of the average page as specified when the extended persistent object store was created.
8 bytes - page_count - the number of pages currently in use or free. 0 or more.
8 bytes - maximum_page_count - the maximum number of pages that can be allocated. 1 or more. -1 for unlimited.
8 bytes - free_list - pointer to first free page. -1 if none.
8 bytes - free_count - the number of free pages. 0 or more.

Each page is composed of a header and data area of size page_length as specified in the above header.

.htstore Page

1 byte - flags - flags used to specify whether the page is in use or has been freed. 1 for in use and 0 for free.
8 bytes - next_data_page - pointer to any page that follows. -1 if none.
n bytes - entry - the encoded value mapped to fixed page of page_length as specified in the header.

The persistant object store stores variable length objects by storing them into one (1) or more fixed length pages.

Once a value is changed or removed, pages no longer used are linked to the beginning of the free_list as defined in the above header.

Keys and associated values are encoded as following:

null - 1 byte - 0x00
Byte - 2 bytes - 0x01 byte
Character - 3 bytes - 0x02 char
Short - 3 bytes - 0x03 short
Integer - 5 bytes - 0x04 int
Long - 9 bytes - 0x05 long
Float - 5 bytes - 0x06 float
Double - 9 bytes - 0x07 double
String - 1 + bytes - 0x08 utf8_encoded_string
Object - 1 + bytes - 0x09 serialized_object
Boolean - 2 bytes - 0x0A boolean_value
Blob - 1 + bytes - 0x0B byte_array
Date - 9 bytes - 0x0C date
BigInteger - 1 + bytes - 0x0D byte_array
BigDecimal - 1 + bytes - 0x0E byte_array
byte[] - 1 + bytes - 0x0F byte_array

Limits

In general Servertec Persistent Object Stores (POS) are limited by size of signed long (64-1 bits).

Limit Size
Maximum Number of Pages in Lookup Table 264-1
Maximum Number of Pages in Mapper 264-1
Maximum Number of Pages in Keys 264-1
Maximum Number of Pages in Values 264-1
Maximum Number of Volumes Containing Lookup Table 232-1
Maximum Number of Volumes Containing Mapper 232-1
Maximum Number of Volumes Containing Keys 232-1
Maximum Number of Volumes Containing Values 232-1
Maximum Number of Keys 264-1
Maximum Number of Values 232-1
Maximum Size of Key 232-1
Maximum Size of Value 232-1
Maximum File Size of Lookup Table Volume Files Platform/Java VM Specific
Maximum File Size of Mapper Volume Files Platform/Java VM Specific
Maximum File Size of Keys Volume Files Platform/Java VM Specific
Maximum File Size of Values Volume Files Platform/Java VM Specific
Maximum Number of Open Stores Platform/Java VM Specific
Number of File Handles Per Opened Lookup Table Volume File 1
Number of File Handles Per Opened Mapper Volume File 1
Number of File Handles Per Opened Keys Volume File 1
Number of File Handles Per Opened Values Volume File 1
Number of File Handles Per Opened Store At Least 4

Locking

Object locking is at a per stec.xpos.ObjectStoreReference basis with each store maintaining its own in-memory lock table. When an ObjectStoreReference is closed all its locks are released.

Multi-Threading

Each thread should have its own stec.xpos.ObjectStoreReference that is not shared with other threads.

Internally stec.xpos.ObjectStoreReference operations block other threads from accessing the single instance to the object store.

Since:
1.0.0 08/12/2001
See Also:
Hashtable, ByteArray

Field Summary
static boolean displayLogo
           
 
Constructor Summary
ObjectStore()
           
 
Method Summary
static void copy(java.lang.String from_filename, java.lang.String to_filename, boolean overwrite)
          Copies the specified persistent object store.
static void create(java.lang.String filename, long lookup_table_page_count, long mapper_maximum_page_count, long keys_maximum_page_count, long values_maximum_page_count, int keys_page_length, int values_page_length, boolean overwrite)
          Used to create a new extended persistent object store or to overwrite an existing extended persistent object store.
static void create(java.lang.String filename, long lookup_table_page_count, long mapper_maximum_page_count, long keys_maximum_page_count, long values_maximum_page_count, int volume_count, int keys_page_length, int values_page_length, boolean overwrite)
          Used to create a new extended persistent object store or to overwrite an existing extended persistent object store.
static void create(java.lang.String filename, VolumeDescriptor[] lookup_table_descriptors, VolumeDescriptor[] mapper_descriptors, VolumeDescriptor[] keys_descriptors, int keys_page_length, VolumeDescriptor[] values_descriptors, int values_page_length, boolean overwrite)
          Used to create a new extended persistent object store or to overwrite an existing extended persistent object store.
static void delete(java.lang.String filename)
          Deletes the specified extended persistent object store.
static boolean exists(java.lang.String filename)
          Returns whether the specified extended persistent object store currently exists.
static void expand(java.lang.String filename, long mapper_maximum_page_count, long keys_maximum_page_count, long values_maximum_page_count)
          Changes the maximum number of pages that the last mapper, keys and values volume can grow to.
static void expand(java.lang.String filename, VolumeDescriptor[] new_mapper_descriptors, VolumeDescriptor[] new_keys_descriptors, VolumeDescriptor[] new_values_descriptors)
          Appends the specified mapper, keys and values volumes to the specified extended persistent object store.
static boolean inUse(java.lang.String filename)
          Returns whether the specified extended persistent object store is being used.
static void move(java.lang.String from_filename, java.lang.String to_filename, boolean overwrite)
          Moves the specified persistent object store.
static ObjectStoreReference open(java.lang.String filename, boolean exclusive)
          Opens the specified extended persistent object store and returns a stec.xpos.ObjectStoreReference to it.
static void pack(java.lang.String filename)
          Releases unused space at the end of the specified extended persistent object store back to the operating system.
static void rehash(java.lang.String filename, VolumeDescriptor[] new_lookup_table_descriptors, VolumeDescriptor[] new_mapper_descriptors)
          Used to change the number of lookup table pages for the specified extended persistent object store.
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

displayLogo

public static boolean displayLogo
Constructor Detail

ObjectStore

public ObjectStore()
Method Detail

create

public static final void create(java.lang.String filename,
                                long lookup_table_page_count,
                                long mapper_maximum_page_count,
                                long keys_maximum_page_count,
                                long values_maximum_page_count,
                                int keys_page_length,
                                int values_page_length,
                                boolean overwrite)
                         throws java.lang.IllegalArgumentException,
                                java.io.IOException
Used to create a new extended persistent object store or to overwrite an existing extended persistent object store.

Parameters:
filename - the path and base file name of the extended persistent object store to create.
lookup_table_page_count - the number of pages in the lookup table. 1 or more.
mapper_maximum_page_count - the maximum number of mapper pages. -1 for unlimited.
keys_maximum_page_count - the maximum number of keys store pages. -1 for unlimited.
values_maximum_page_count - the maximum number of values store pages. -1 for unlimited.
keys_page_length - the size in bytes of the average page in the keys store.
values_page_length - the size in bytes of the average page in the values store.
overwrite - whether to overwrite any existing extended persistent object store.

Creates a single volume for the lookup table store of size lookup_table_page_count whose name is generated by appending .table to the specified object store file name.

Creates a single volume for the mapper store of size mapper_maximum_page_count whose name is generated by appending .mapper to the specified object store file name.

Creates a single volume for the keys store of size keys_maximum_page_count whose name is generated by appending .keys to the specified object store file name.

Creates a single volume for the values store of size values_maximum_page_count whose name is generated by appending .values to the specified object store file name.

Throws:
java.lang.IllegalArgumentException - if any of the parameters specified were not valid.
java.io.IOException - if the store is in use, already exists or cannot be created or if an I/O exception occurred.
See Also:
create, create

create

public static final void create(java.lang.String filename,
                                long lookup_table_page_count,
                                long mapper_maximum_page_count,
                                long keys_maximum_page_count,
                                long values_maximum_page_count,
                                int volume_count,
                                int keys_page_length,
                                int values_page_length,
                                boolean overwrite)
                         throws java.lang.IllegalArgumentException,
                                java.io.IOException
Used to create a new extended persistent object store or to overwrite an existing extended persistent object store.

Parameters:
filename - the path and base file name of the extended persistent object store to create.
lookup_table_page_count - the number of pages in the lookup table. 1 or more.
mapper_maximum_page_count - the maximum number of mapper pages. 1 or more.
keys_maximum_page_count - the maximum number of keys store pages. 1 or more.
values_maximum_page_count - the maximum number of values store pages. 1 or more.
volume_count - the number of volume files to create. 1 or more.
keys_page_length - the size in bytes of the average page in the keys store.
values_page_length - the size in bytes of the average page in the values store.
overwrite - whether to overwrite any existing extended persistent object store.

Creates the specified number of volumes for the lookup table store of size lookup_table_page_count whose name is generated by appending .table.volume_number to the specified object store file name.

Creates the specified number of volumes for the mapper store of size mapper_maximum_page_count whose name is generated by appending .mapper.volume_number to the specified object store file name.

Creates the specified number of volumes for the keys store of size keys_maximum_page_count whose name is generated by appending .keys.volume_number to the specified object store file name.

Creates the specified number of volumes for the values store of size values_maximum_page_count whose name is generated by appending .values.volume_number to the specified object store file name.

Throws:
java.lang.IllegalArgumentException - if any of the parameters specified were not valid.
java.io.IOException - if the store is in use, already exists or cannot be created or if an I/O exception occurred.
See Also:
create, create

create

public static final void create(java.lang.String filename,
                                VolumeDescriptor[] lookup_table_descriptors,
                                VolumeDescriptor[] mapper_descriptors,
                                VolumeDescriptor[] keys_descriptors,
                                int keys_page_length,
                                VolumeDescriptor[] values_descriptors,
                                int values_page_length,
                                boolean overwrite)
                         throws java.lang.IllegalArgumentException,
                                java.io.IOException
Used to create a new extended persistent object store or to overwrite an existing extended persistent object store.

Parameters:
filename - the path and base file name of the extended persistent object store to create.
lookup_table_descriptors - an array of one (1) or more VolumeDescriptors defining the lookup table store.
mapper_descriptors - an array of one (1) or more VolumeDescriptors defining the mapper store.
keys_descriptors - an array of one (1) or more VolumeDescriptors defining the keys store.
keys_page_length - the size in bytes of the average page in the keys store.
values_descriptors - an array of one (1) or more VolumeDescriptors defining the values store.
values_page_length - the size in bytes of the average page in the values store.
overwrite - whether to overwrite any existing extended persistent object store.
Throws:
java.lang.IllegalArgumentException - if any of the parameters specified were not valid.
java.io.IOException - if the store is in use, already exists or cannot be created or if an I/O exception occurred.
See Also:
create, create

exists

public static final boolean exists(java.lang.String filename)
                            throws java.io.IOException
Returns whether the specified extended persistent object store currently exists.

Parameters:
filename - the path and base file name of the extended persistent object store to test.
Returns:
whether the specified extended persistent object store currently exists.
Throws:
java.io.IOException - if an I/O exception occurred.

delete

public static final void delete(java.lang.String filename)
                         throws java.io.FileNotFoundException,
                                java.io.IOException
Deletes the specified extended persistent object store.

Parameters:
filename - the path and base file name of the extended persistent object store to delete.
Throws:
java.io.FileNotFoundException - if the store does not exist.
java.io.IOException - if the store is in use or cannot be deleted or if an I/O exception occurred.
See Also:
create, create, create

open

public static final ObjectStoreReference open(java.lang.String filename,
                                              boolean exclusive)
                                       throws java.io.FileNotFoundException,
                                              java.io.IOException
Opens the specified extended persistent object store and returns a stec.xpos.ObjectStoreReference to it.

Parameters:
filename - the path and base file name of the extended persistent object store to open.
exclusive - whether to open the extended persistent object store reference for exlusive use.
Returns:
the stec.xpos.ObjectStoreReference to the extended persistent object store.
Throws:
java.io.FileNotFoundException - if the store does not exist.
java.io.IOException - if the store is exclusively opened or cannot be opened or if an I/O exception occurred.
See Also:
ObjectStoreReference, close

rehash

public static final void rehash(java.lang.String filename,
                                VolumeDescriptor[] new_lookup_table_descriptors,
                                VolumeDescriptor[] new_mapper_descriptors)
                         throws java.io.FileNotFoundException,
                                java.io.IOException
Used to change the number of lookup table pages for the specified extended persistent object store.

Rehashing a large extended persistent object store may take a long time.

Internally creates a new store, lookup table and mapper, then performs a rehash using the newly created store and after successfully completing renames the original store to a temporary name and then renames the new store to the original store's name and then finally deletes the renamed original store.

Parameters:
filename - the path and base file name of the extended persistent object store to rehash.
new_lookup_table_descriptors - an array of one (1) or more VolumeDescriptors defining the new lookup table store.
new_mapper_descriptors - an array of one (1) or more VolumeDescriptors defining the new mapper store.
Throws:
java.lang.IllegalArgumentException - if any of the parameters specified were not valid.
java.io.FileNotFoundException - if the store does not exist.
java.io.IOException - if the store is in use or cannot be rehashed or if an I/O exception occurred.

pack

public static final void pack(java.lang.String filename)
                       throws java.io.FileNotFoundException,
                              java.io.IOException
Releases unused space at the end of the specified extended persistent object store back to the operating system.

Packing a large extended persistent object store may take a long time.

Java 1.2.0 or later is required to change the physical size of store files.

Parameters:
filename - the path and base file name of the extended persistent object store to pack.
Throws:
java.io.FileNotFoundException - if the store does not exist.
java.io.IOException - if the store is in use or cannot be packed or if an I/O exception occurred.

expand

public static final void expand(java.lang.String filename,
                                long mapper_maximum_page_count,
                                long keys_maximum_page_count,
                                long values_maximum_page_count)
                         throws java.lang.IllegalArgumentException,
                                java.io.FileNotFoundException,
                                java.io.IOException
Changes the maximum number of pages that the last mapper, keys and values volume can grow to.

Parameters:
filename - the path and base file name of the persistent object store to expand.
mapper_maximum_page_count - the maximum number of mapper pages. -1 for unlimited.
keys_maximum_page_count - the maximum number keys of store pages. -1 for unlimited.
values_maximum_page_count - the maximum number values of store pages. -1 for unlimited.
Throws:
java.lang.IllegalArgumentException - if any of the parameters specified were not valid.
java.io.FileNotFoundException - if the store does not exist.
java.io.IOException - if the store is in use or cannot be expanded or if an I/O exception occurred.

expand

public static final void expand(java.lang.String filename,
                                VolumeDescriptor[] new_mapper_descriptors,
                                VolumeDescriptor[] new_keys_descriptors,
                                VolumeDescriptor[] new_values_descriptors)
                         throws java.lang.IllegalArgumentException,
                                java.io.FileNotFoundException,
                                java.io.IOException
Appends the specified mapper, keys and values volumes to the specified extended persistent object store.

Parameters:
filename - the path and base file name of the persistent object store to expand.
new_mapper_descriptors - an array of one (1) or more VolumeDescriptors defining the new mapper store. null if not expanding the mapper store.
new_keys_descriptors - an array of one (1) or more VolumeDescriptors defining the new keys store. null if not expanding the keys store.
new_values_descriptors - an array of one (1) or more VolumeDescriptors defining the new values store. null if not expanding the values store.
Throws:
java.lang.IllegalArgumentException - if any of the parameters specified were not valid.
java.io.FileNotFoundException - if the store does not exist.
java.io.IOException - if the store is in use or cannot be expanded or if an I/O exception occurred.

inUse

public static final boolean inUse(java.lang.String filename)
                           throws java.io.IOException
Returns whether the specified extended persistent object store is being used.

Parameters:
filename - the path and base file name of the extended persistent object store to test.
Returns:
whether the specified extended persistent object store is being used.
Throws:
java.io.IOException - if an I/O exception occurred.
Since:
1.1.0 09/01/2002

copy

public static final void copy(java.lang.String from_filename,
                              java.lang.String to_filename,
                              boolean overwrite)
                       throws java.lang.IllegalArgumentException,
                              java.io.FileNotFoundException,
                              java.io.IOException
Copies the specified persistent object store.

Parameters:
from_filename - the path and base file name of the persistent object store to copy.
to_filename - the path and base file name of the persistent object store to copy to.
overwrite - whether to overwrite any existing persistent object store.
Throws:
java.lang.IllegalArgumentException - if any of the parameters specified were not valid.
java.io.FileNotFoundException - if the store does not exist.
java.io.IOException - if the store is in use or cannot be copied or if an I/O exception occurred.
Since:
1.2.2 03/03/2003

move

public static final void move(java.lang.String from_filename,
                              java.lang.String to_filename,
                              boolean overwrite)
                       throws java.lang.IllegalArgumentException,
                              java.io.FileNotFoundException,
                              java.io.IOException
Moves the specified persistent object store.

Parameters:
from_filename - the path and base file name of the persistent object store to move.
to_filename - the path and base file name of the persistent object store to move to.
overwrite - whether to overwrite any existing persistent object store.
Throws:
java.lang.IllegalArgumentException - if any of the parameters specified were not valid.
java.io.FileNotFoundException - if the store does not exist.
java.io.IOException - if the store is in use or cannot be moved or if an I/O exception occurred.
Since:
1.2.2 03/03/2003

Servertec
Persistent Object Store
1.4.1 09/04/2005

Copyright © 2001-2005 Servertec. All rights reserved.