Class StoredPage

  • All Implemented Interfaces:
    Cacheable, TypedFormat, DerbyObserver, Page
    Direct Known Subclasses:
    AllocPage

    public class StoredPage
    extends CachedPage
    StoredPage is a sub class of CachedPage that stores page data in a fixed size byte array and is designed to be written out to a file through a DataInput/DataOutput interface. A StoredPage can exist in its clean or dirty state without the FileContainer it was created from being in memory.

    Page Format
    The page is broken into five sections

      +----------+-------------+-------------------+-------------------+----------+
      | formatId | page header | records           | slot offset table | checksum |
      +----------+-------------+-------------------+-------------------+----------+
      

    FormatId
    The formatId is a 4 bytes array, it contains the format Id of this page.
    Page Header
    The page header is a fixed size, 56 bytes
      1 byte  boolean           is page an overflow page
      1 byte  byte              page status (a field maintained in base page)
      8 bytes long              pageVersion (a field maintained in base page)
      2 bytes unsigned short    number of slots in slot offset table
      4 bytes integer           next record identifier
      4 bytes integer           generation number of this page (Future Use)
      4 bytes integer           previous generation of this page (Future Use)
      8 bytes bipLocation       the location of the beforeimage page (Future Use)
      2 bytes unsigned short    number of deleted rows on page. (new release 2.0)
      2 bytes unsigned short    % of the page to keep free for updates
      2 bytes short             spare for future use
      4 bytes long              spare for future use (encryption uses to write 
                                                      random bytes here).
      8 bytes long              spare for future use
      8 bytes long              spare for future use
    
      
    Note that spare space has been guaranteed to be writen with "0", so that future use of field should not either not use "0" as a valid data item or pick 0 as a valid default value so that on the fly upgrade can assume that 0 means field was never assigned.
    Records The records section contains zero or more records, the format of each record follows. minimumRecordSize is the minimum user record size, excluding the space we use for the record header and field headers. When a record is inserted, it is stored in a space at least as large as the sum of the minimumRecordSize and total header size. For example, If minimumRecordSize is 10 bytes, the user record is 7 bytes, we used 5 bytes for record and field headers, this record will take (10 + 5) bytes of space, extra 3 bytes is put into reserve. If minimumRecordSize is 10 bytes, user record is 17 bytes, we used 5 bytes for record and field headers, this record will take (17 + 5) bytes of space, no reserve space here. minimumRecordSize is defined by user on per container basis. The default for minimumRecordSize is set to 1. This implementation always keeps occupied bytes at the low end of the record section. Thus removing (purging) a record moves all other records down, and their slots are also moved down. A page has no empty slot (an empty page has no slot)
    Record and Field Format Record Header format is defined in the StoredRecordHeader class.
       
      
    Fields 1 byte Boolean - is null, if true no more data follows. 4 bytes Integer - length of field that follows (excludes these four bytes). StoredPage will use the static method provided by StoredFieldHeader to read/write field status and field data length. Field Header format is defined in the StoredFieldHeader class.

    Slot Offset Table
    The slot offset table is a table of 6 or 12 bytes per record, depending on the pageSize being less or greater than 64K: 2 bytes (unsigned short) or 4 bytes (int) page offset for the record that is assigned to the slot, and 2 bytes (unsigned short) or 4 bytes (int) for the length of the record on this page. 2 bytes (unsigned short) or 4 bytes (int) for the length of the reserved number of bytes for this record on this page. First slot is slot 0. The slot table grows backwards. Slots are never left empty.
    Checksum
    8 bytes of a java.util.zip.CRC32 checksum of the entire's page contents without the 8 bytes representing the checksum.

    Page Access The page data is accessed in this class by one of three methods.

    1. As a byte array using pageData (field in cachedPage). This is the fastest.
    2. As an ArrayInputStream (rawDataIn) and ArrayOutputStream (rawDataOut), this is used to set limits on any one reading the page logically.
    3. Logically through rawDataIn (ArrayInputStream) and logicalDataOut (FormatIdOutputStream), this provides the methods to write logical data (e.g. booleans and integers etc.) and the ObjectInput and ObjectOutput interfaces for DataValueDescriptor's. These logical streams are constructed using the array streams.
    See Also:
    CRC32, ArrayInputStream, ArrayOutputStream
    • Field Detail

      • FORMAT_NUMBER

        public static final int FORMAT_NUMBER
        static final Fields of the class
        See Also:
        Constant Field Values
      • PAGE_HEADER_OFFSET

        protected static final int PAGE_HEADER_OFFSET
        Start of page, formatId must fit in 4 bytes.

        where the page header starts - page format is mandated by cached page

        See Also:
        Constant Field Values
      • PAGE_HEADER_SIZE

        protected static final int PAGE_HEADER_SIZE
        Fixed size of the page header
        See Also:
        Constant Field Values
      • RECORD_SPACE_OFFSET

        protected static final int RECORD_SPACE_OFFSET
        Start of the record storage area.

        Note: a subclass may change the start of the record storage area. Don't always count on this number.

        See Also:
        Constant Field Values
      • PAGE_VERSION_OFFSET

        protected static final int PAGE_VERSION_OFFSET
        offset of the page version number
        See Also:
        Constant Field Values
      • SMALL_SLOT_SIZE

        protected static final int SMALL_SLOT_SIZE
        SMALL_SLOT_SIZE are for pages smaller than 64K, LARGE_SLOT_SIZE is for pages bigger than 64K.
        See Also:
        Constant Field Values
      • CHECKSUM_SIZE

        protected static final int CHECKSUM_SIZE
        Size of the checksum stored on the page. The checksum is stored in the last 8 bytes of the page, the slot table grows backward up the page starting at the end of the page just before the checksum.
        See Also:
        Constant Field Values
      • OVERFLOW_POINTER_SIZE

        protected static final int OVERFLOW_POINTER_SIZE
        OVERFLOW_POINTER_SIZE - Number of bytes to reserve for overflow pointer The overflow pointer is the pointer that the takes the place of the last column of a row if the row can't fit on the page. The pointer then points to another page where the next column of the row can be found. The overflow pointer can be bigger than a row, so when overflowing a row the code must overflow enough columns so that there is enough free space to write the row. Note this means that the minimum space a row can take on a page must allow for at least the size of the overflow pointers so that if the row is updated it can write the over flow pointer.
        See Also:
        Constant Field Values
      • OVERFLOW_PTR_FIELD_SIZE

        protected static final int OVERFLOW_PTR_FIELD_SIZE
        OVERFLOW_PTR_FIELD_SIZE - Number of bytes of an overflow field This is the length to reserve for either a column or row overflow pointer field. It includes the size of the field header plus the maxium length of the overflow pointer (it could be shorter due to compressed storage). The calcualtion is: OVERFLOW_PTR_FIELD_SIZE = OVERFLOW_POINTER_SIZE + sizeof(status byte) + sizeof(field length field for a field which is just an overflow ptr)
        See Also:
        Constant Field Values
      • bh

        ByteHolder bh
        In memory buffer used as scratch space for streaming columns.
      • COLUMN_NONE

        protected static final int COLUMN_NONE
        Constants used in call to logColumn.

        Action taken in this routine is determined by the kind of column as specified in the columnFlag: COLUMN_NONE - the column is insignificant COLUMN_FIRST - this is the first column in a logRow() call COLUMN_LONG - this is a known long column, therefore we will store part of the column on the current page and overflow the rest if necessary. COLUMN_CREATE_NULL - the column was recently added. it doesn't actually exist in the on-disk row yet. we will need to put a null in it as soon as possible. see DERBY-5679.

        See Also:
        Constant Field Values
      • maxFieldSize

        private int maxFieldSize
        maxFieldSize is a worst case calculation for the size of a record on an empty page, with a single field, but still allow room for an overflow pointer if another field is to be added. See initSpace(). maxFieldSize is a worst case calculation for the size of a record This is used as the threshold for a long column. maxFieldSize = totalSpace - slotEntrySize - 16 - OVERFLOW_POINTER_SIZE;
      • isOverflowPage

        private boolean isOverflowPage
        The page header is a fixed size, 56 bytes, following are variables used to access the fields in the header:

        1 byte boolean isOverflowPage is page an overflow page 1 byte byte pageStatus page status (field in base page) 8 bytes long pageVersion page version (field in base page) 2 bytes ushort slotsInUse number of slots in slot offset table 4 bytes integer nextId next record identifier 4 bytes integer generation generation number of this page(FUTURE USE) 4 bytes integer prevGeneration previous generation of page (FUTURE USE) 8 bytes long bipLocation the location of the BI page (FUTURE USE) 2 bytes ushort deletedRowCount number of deleted rows on page.(rel 2.0) 2 bytes long spare for future use 4 bytes long spare (encryption writes random bytes) 8 bytes long spare for future use 8 bytes long spare for future use Note that spare space has been guaranteed to be writen with "0", so that future use of field should not either not use "0" as a valid data item or pick 0 as a valid default value so that on the fly upgrade can assume that 0 means field was never assigned.

      • slotsInUse

        private int slotsInUse
      • nextId

        private int nextId
      • generation

        private int generation
      • prevGeneration

        private int prevGeneration
      • bipLocation

        private long bipLocation
      • deletedRowCount

        private int deletedRowCount
      • headerOutOfDate

        private boolean headerOutOfDate
        Is the header in the byte array out of date wrt the fields.

        this field must be set to true whenever one of the above header fields is modified. Ie any of (isOverflowPage, slotsInUse, nextId, generation, prevGeneration, bipLocation, deletedRowCount)

      • checksum

        private java.util.zip.CRC32 checksum
        holder for the checksum.
      • minimumRecordSize

        protected int minimumRecordSize
        Minimum space to reserve for record portion length of row.

        minimumRecordSize is stored in the container handle. It is used to reserve minimum space for recordPortionLength. Default is 1. To get the value from the container handle: myContainer.getMinimumRecordSize(); minimumRecordSize is the minimum user record size, excluding the space we use for the record header and field headers. When a record is inserted, it is stored in a space at least as large as the sum of the minimumRecordSize and total header size. For example, If minimumRecordSize is 10 bytes, the user record is 7 bytes, we used 5 bytes for record and field headers, this record will take (10 + 5) bytes of space, extra 3 bytes is put into reserve. If minimumRecordSize is 10 bytes, user record is 17 bytes, we used 5 bytes for record and field headers, this record will take (17 + 5) bytes of space, no reserve space here. minimumRecordSize is defined by user on per container basis. The default for minimumRecordSize is set to 1.

      • userRowSize

        private int userRowSize
        scratch variable used to keep track of the total user size for the row. the information is used by logRow to maintain minimumRecordSize on Page. minimumRecordSize is only considered for main data pages, therefore, the page must be latched during an insert operation.
      • slotFieldSize

        private int slotFieldSize
        slot field and slot entry size.

        The size of these fields is dependant on the page size. These 2 variables should be set when pageSize is determined, and should not be changed for that page. Each slot entry contains 3 fields (slotOffet, recordPortionLength and reservedSpace) for the record the slot is pointing to. slotFieldSize is the size for each of the slot field. slotEntrySize is the total space used for a single slot entry.

      • slotEntrySize

        private int slotEntrySize
      • slotTableOffsetToFirstEntry

        private int slotTableOffsetToFirstEntry
        Offset of the first entry in the slot table.

        Offset table is located at end of page, just before checksum. It grows backward as an array from this point toward the middle of the page.

        slotTableOffsetToFirstEntry is the offset to the beginning of the first entry (slot[0]) in the slot table. This allows the following math to get to the offset of N'th entry in the slot table: offset of slot[N] = slotTableOffsetToFirstEntry + (N * slotEntrySize)

      • slotTableOffsetToFirstRecordLengthField

        private int slotTableOffsetToFirstRecordLengthField
        Offset of the record length entry in the 1st slot table entry.

        Offset table is located at end of page, just before checksum. It grows backward as an array from this point toward the middle of the page. The record length is stored as the second "field" of the slot table entry.

        slotTableOffsetToFirstRecordLengthField is the offset to the beginning of the record length field in the first entry (slot[0]) in the slot table. This allows the following math to get to the record length field of N'th entry in the slot table: offset of record length of slot[N] slot entry = slotTableOffsetToFirstRecordLengthField + (N * slotEntrySize)

      • slotTableOffsetToFirstReservedSpaceField

        private int slotTableOffsetToFirstReservedSpaceField
        Offset of the reserved space length entry in the 1st slot table entry.

        Offset table is located at end of page, just before checksum. It grows backward as an array from this point toward the middle of the page. The reserved space length is stored as the third "field" of the slot table entry.

        slotTableOffsetToFirstReservedSpaceField is the offset to the beginning of the reserved space field in the first entry (slot[0]) in the slot table. This allows the following math to get to the reserved space field of N'th entry in the slot table: offset of reserved space of slot[N] slot entry = slotTableOffsetToFirstReservedSpaceField + (N * slotEntrySize)

      • totalSpace

        protected int totalSpace
        total usable space on a page.

        This is the space not taken by page hdr, page table, and existing slot entries/rows.

      • freeSpace

        protected int freeSpace
      • firstFreeByte

        private int firstFreeByte
      • spareSpace

        protected int spareSpace
        % of page to keep free for updates.

        How much of a head page should be reserved as "free" so that the space can be used by update which expands the row without needing to overflow it. 1 means save 1% of the free space for expansion.

      • overflowRecordHeader

        private StoredRecordHeader overflowRecordHeader
        Scratch variable used when you need a overflowRecordHeader. Declared globally so that object is only allocated once per page.
      • rawDataIn

        protected ArrayInputStream rawDataIn
        Input streams used to read/write bytes to/from the page byte array.
    • Constructor Detail

      • StoredPage

        public StoredPage()
        Simple no-arg constructor for StoredPage.
    • Method Detail

      • getTypeFormatId

        public int getTypeFormatId()
        Return my format identifier.
        Returns:
        The identifier. (A UUID stuffed in an array of 16 bytes).
      • initialize

        protected void initialize()
        Initialize the StoredPage.

        Initialize the object, ie. perform work normally perfomed in constructor. Called by setIdentity() and createIdentity() - the Cacheable interfaces which are used to move a page in/out of cache.

        Overrides:
        initialize in class CachedPage
      • createOutStreams

        private void createOutStreams()
        Create the output streams.

        Create the output streams, these are created on demand to avoid creating unrequired objects for pages that are never modified during their lifetime in the cache.

        Throws:
        StandardException - Standard exception policy.
      • setOutputStream

        private void setOutputStream​(java.io.OutputStream out)
        Tie the logical output stream to a passed in OutputStream.

        Tie the logical output stream to a passed in OutputStream with no limit as to the number of bytes that can be written.

      • resetOutputStream

        private void resetOutputStream()
        Reset the logical output stream.

        Reset the logical output stream (logicalDataOut) to be attached to the page array stream as is the norm, no limits are placed on any writes.

      • usePageBuffer

        protected void usePageBuffer​(byte[] pageBuffer)
        use this passed in page buffer as this object's page data.

        The page content may not have been read in from disk yet. For pagesize smaller than 64K: Size of the record offset stored in a slot (unsigned short) Size of the record portion length stored in a slot (unsigned short) Size of the record portion length stored in a slot (unsigned short) For pagesize greater than 64K, but less than 2gig: Size of the record offset stored in a slot (int) Size of the record portion length stored in a slot (int) Size of the record portion length stored in a slot (int)

        Specified by:
        usePageBuffer in class CachedPage
        Parameters:
        pageBuffer - The array of bytes to use as the page buffer.
      • calculateSlotFieldSize

        private int calculateSlotFieldSize​(int pageSize)
        Calculate the slot field size from the page size.
        Parameters:
        pageSize - page size in bytes
        Returns:
        slot field size in bytes
      • createPage

        protected void createPage​(PageKey newIdentity,
                                  PageCreationArgs args)
                           throws StandardException
        Create a new StoredPage.

        Make this object represent a new page (ie. a page that never existed before, as opposed to reading in an existing page from disk).

        Specified by:
        createPage in class CachedPage
        Parameters:
        newIdentity - The key describing page (segment,container,page).
        args - information stored about the page, once in the container header and passed in through the object.
        Throws:
        StandardException - Standard exception policy.
      • initFromData

        protected void initFromData​(FileContainer myContainer,
                                    PageKey newIdentity)
                             throws StandardException
        Initialize the page from values in the page buffer.

        Initialize in memory structure using the buffer in pageData. This is how a StoredPage object is intialized to represent page read in from disk.

        Specified by:
        initFromData in class CachedPage
        Parameters:
        myContainer - The container to read the page in from.
        newIdentity - The key representing page being read in (segment, container, page number)
        Throws:
        StandardException - If the page cannot be read correctly, or is inconsistent.
      • validateChecksum

        protected void validateChecksum​(PageKey id)
                                 throws StandardException
        Validate the check sum on the page.

        Compare the check sum stored in the page on disk with the checksum calculated from the bytes on the page.

        Parameters:
        id - The key that describes the page.
        Throws:
        StandardException - Standard exception policy.
      • updateChecksum

        protected void updateChecksum()
                               throws java.io.IOException
        Recalculate checksum and write it to the page array.

        Recalculate the checksum of the page, and write the result back into the last bytes of the page.

        Throws:
        java.io.IOException - if writing to end of array fails.
      • writePage

        protected void writePage​(PageKey identity)
                          throws StandardException
        Write information about page from variables into page byte array.

        This routine insures that all information about the page is reflected in the page byte buffer. This involves moving information from local variables into encoded version on the page in page header and checksum.

        Specified by:
        writePage in class CachedPage
        Parameters:
        identity - The key of this page.
        Throws:
        StandardException - Standard exception policy.
      • getTotalSpace

        public int getTotalSpace​(int slot)
                          throws StandardException
        Return the total number of bytes used, reserved, or wasted by the record at this slot.

        The amount of space the record on this slot is currently taking on the page. If there is any reserve space or wasted space, count that in also Do NOT count the slot entry size

        Specified by:
        getTotalSpace in class BasePage
        Parameters:
        slot - look at row at this slot.
        Returns:
        The number of bytes used by the row at slot "slot".
        Throws:
        StandardException - Standard exception policy.
      • spaceForInsert

        public boolean spaceForInsert()
                               throws StandardException
        Is there minimal space for insert?

        Does quick calculation to see if average size row on this page could be inserted on the page. This is done because the actual row size being inserted isn't known until we actually copy the columns from their object form into their on disk form which is expensive. So we use this calculation so that in the normal case we only do one copy of the row directly onto the page.

        Returns:
        true if we think the page will allow an insert, false otherwise.
        Throws:
        StandardException - Standard exception policy.
      • spaceForInsert

        public boolean spaceForInsert​(java.lang.Object[] row,
                                      FormatableBitSet validColumns,
                                      int overflowThreshold)
                               throws StandardException
        Is row guaranteed to be inserted successfully on this page?

        Return true if this record is guaranteed to be inserted successfully using insert() or insertAtSlot(). This guarantee is only valid while the row remains unchanged and the page latch is held.

        Parameters:
        row - The row to check for insert.
        validColumns - bit map to interpret valid columns in row.
        overflowThreshold - The percentage of the page to use for the insert. 100 means use 100% of the page, 50 means use 50% of page (ie. make sure 2 rows fit per page).
        Returns:
        bolean indicating if row can be inserted on this page.
        Throws:
        StandardException - Standard exception policy.
      • spaceForInsert

        private boolean spaceForInsert​(java.lang.Object[] row,
                                       FormatableBitSet validColumns,
                                       int spaceNeeded,
                                       int startColumn,
                                       int overflowThreshold)
                                throws StandardException
        Is row guaranteed to be inserted successfully on this page?

        Return true if this record is guaranteed to be inserted successfully using insert() or insertAtSlot(). This guarantee is only valid while the row remains unchanged and the page latch is held.

        This is a private call only used when calculating whether an overflow page can be used to insert part of an overflow row/column.

        Parameters:
        row - The row to check for insert.
        validColumns - bit map to interpret valid columns in row.
        overflowThreshold - The percentage of the page to use for the insert. 100 means use 100% of the page, 50 means use 50% of page (ie. make sure 2 rows fit per page).
        Returns:
        bolean indicating if row can be inserted on this page.
        Throws:
        StandardException - Standard exception policy.
      • unfilled

        public boolean unfilled()
        Is this page unfilled?

        Returns true if page is relatively unfilled, which means the page is < 1/2 full and has enough space to insert an "average" sized row onto the page.

        Specified by:
        unfilled in class BasePage
        Returns:
        true if page is relatively unfilled.
      • allowInsert

        public boolean allowInsert()
        Is there enough space on the page to insert a minimum size row?

        Calculate whether there is enough space on the page to insert a minimum size row. The calculation includes maintaining the required reserved space on the page for existing rows to grow on the page.

        Specified by:
        allowInsert in class BasePage
        Returns:
        boolean indicating if a minimum sized row can be inserted.
      • spaceForCopy

        public boolean spaceForCopy​(int num_rows,
                                    int[] spaceNeeded)
        Does this page have enough space to insert the input rows?

        Can the rows with lengths spaceNeeded[0..num_rows-1] be copied onto this page?

        Specified by:
        spaceForCopy in class BasePage
        Parameters:
        num_rows - number of rows to check for.
        spaceNeeded - array of lengths of the rows to insert.
        Returns:
        true if the sum of the lengths will fit on the page.
      • spaceForCopy

        protected boolean spaceForCopy​(int spaceNeeded,
                                       int source_id)
        Does this page have enough space to move the row to it.

        Calculate if a row of length "spaceNeeded" with current record id "source_id" will fit on this page.

        Parameters:
        spaceNeeded - length of the row encoded with source_id record id.
        source_id - record id of the row being moved.
        Returns:
        true if the record will fit on this page, after being given a new record id as the next id on this page.
      • restoreRecordFromSlot

        protected boolean restoreRecordFromSlot​(int slot,
                                                java.lang.Object[] row,
                                                FetchDescriptor fetchDesc,
                                                RecordHandle recordToLock,
                                                StoredRecordHeader recordHeader,
                                                boolean isHeadRow)
                                         throws StandardException
        Read the record at the given slot into the given row.

        This reads and initializes the columns in the row array from the raw bytes stored in the page associated with the given slot. If validColumns is non-null then it will only read those columns indicated by the bit set, otherwise it will try to read into every column in row[].

        If there are more columns than entries in row[] then it just stops after every entry in row[] is full.

        If there are more entries in row[] than exist on disk, the requested excess columns will be set to null by calling the column's object's restoreToNull() routine (ie. ((Object) column).restoreToNull() ).

        If a qualifier list is provided then the row will only be read from disk if all of the qualifiers evaluate true. Some of the columns may have been read into row[] in the process of evaluating the qualifier.

        This routine should only be called on the head portion of a row, it will call a utility routine to read the rest of the row if it is a long row.

        Specified by:
        restoreRecordFromSlot in class BasePage
        Parameters:
        slot - the slot number
        row - (out) filled in sparse row
        fetchDesc - Information describing fetch, including what columns to fetch and qualifiers.
        recordToLock - the record handle for the row at top level, and is used in OverflowInputStream to lock the row for Blobs/Clobs.
        isHeadRow - The row on this page includes the head record handle. Will be false for the overflow portions of a "long" row, where columns of a row span multiple pages.
        Returns:
        false if a qualifier_list is provided and the row does not qualifier (no row read in that case), else true.
        Throws:
        StandardException - Standard Derby error policy
      • newRecordId

        public int newRecordId()
        Create a new record handle.

        Return the next record id for allocation. Callers of this interface expect the next id to get bumped some where else - probably by storeRecordForInsert().

        Specified by:
        newRecordId in class BasePage
        Returns:
        The next id to assing to a row.
      • newRecordIdAndBump

        public int newRecordIdAndBump()
        Create a new record handle, and bump the id.

        Create a new record handle, and bump the id while holding the latch so that no other user can ever see this record id. This will lead to unused record id's in the case where an insert fails because there is not enough space on the page.

        Specified by:
        newRecordIdAndBump in class BasePage
        Returns:
        The next id to assing to a row.
      • newRecordId

        protected int newRecordId​(int recordId)
        Create a new record id based on current one passed in.

        This interface is used for the "copy" insert interface of raw store where multiple rows are inserted into a page in a single logged operation. We don't want to bump the id until the operation is logged so we just allocated each id in order and then bump the next id at the end of the operation.

        Specified by:
        newRecordId in class BasePage
        Parameters:
        recordId - The id caller just used, return the next one.
        Returns:
        the next id based on the input id.
      • isOverflowPage

        public boolean isOverflowPage()
        Description copied from class: BasePage
        Return true if the page is an overflow page, false if not. For implementation that don't have overflow pages, return false.
        Specified by:
        isOverflowPage in class BasePage
      • getPageSize

        public final int getPageSize()
        Get the full size of the page.
      • clearSection

        protected final void clearSection​(int offset,
                                          int length)
        Zero out a portion of the page.
        Parameters:
        offset - position of first byte to clear
        length - how many bytes to clear
      • getMaxFreeSpace

        protected int getMaxFreeSpace()
        The maximum free space on this page possible.

        The the maximum amount of space that can be used on the page for the records and the slot offset table. NOTE: subclass may have overwitten it to report less freeSpace

        Returns:
        the maximum free space on this page possible.
      • getCurrentFreeSpace

        protected int getCurrentFreeSpace()
        The current free space on the page.
      • readPageHeader

        private void readPageHeader()
                             throws java.io.IOException
        Read the page header from the page array.

        Read the page header from byte form in the page array into in memory variables.

        Throws:
        java.io.IOException
      • updatePageHeader

        private void updatePageHeader()
                               throws java.io.IOException
        Update the page header in the page array.

        Write the bytes of the page header, taking the values from those in the in memory variables.

        Throws:
        java.io.IOException
      • updatePageVersion

        private void updatePageVersion()
                                throws java.io.IOException
        Update the page version number in the byte array
        Throws:
        java.io.IOException
      • getSlotOffset

        private int getSlotOffset​(int slot)
        Get the page offset of a given slot entry.

        Get the page offset of a slot entry, this is not the offset of the record stored in the slot, but the offset of the actual slot.

        Parameters:
        slot - The array entry of the slot to find.
        Returns:
        The page offset of a given slot entry.
      • getRecordOffset

        private int getRecordOffset​(int slot)
        Get the page offset of the record associated with the input slot.

        This is the actual offset on the page of the beginning of the record.

        Parameters:
        slot - The array entry of the slot to find.
        Returns:
        The page offset of the record associated with the input slot.
      • setRecordOffset

        private void setRecordOffset​(int slot,
                                     int recordOffset)
                              throws java.io.IOException
        Set the page offset of the record associated with the input slot.

        This is the actual offset on the page of the beginning of the record.

        Parameters:
        slot - The array entry of the slot to set.
        recordOffset - the new offset to set.
        Throws:
        java.io.IOException
      • getRecordPortionLength

        protected int getRecordPortionLength​(int slot)
                                      throws java.io.IOException
        Return length of row on this page.

        Return the total length of data and header stored on this page for this record. This length is stored as the second "field" of the slot table entry.

        Parameters:
        slot - the slot of the row to look up the length of.
        Returns:
        The length of the row on this page.
        Throws:
        java.io.IOException
      • getReservedCount

        public int getReservedCount​(int slot)
                             throws java.io.IOException
        Return reserved length of row on this page.

        Return the reserved length of this record. This length is stored as the third "field" of the slot table entry.

        Specified by:
        getReservedCount in class BasePage
        Parameters:
        slot - the slot of the row to look up the length of.
        Returns:
        The reserved length of the row on this page.
        Throws:
        java.io.IOException - Thrown by InputStream methods potential I/O errors
      • updateRecordPortionLength

        private void updateRecordPortionLength​(int slot,
                                               int delta,
                                               int reservedDelta)
                                        throws java.io.IOException
        Update the length of data stored on this page for this record

        Update both the record length "field" and the reserved space "field" of the slot table entry associated with "slot". This length is stored as the second "field" of the slot table entry. The changes to these 2 fields are represented as the delta to apply to each field as input in "delta" and "reservedDelta."

        Parameters:
        slot - the slot of the record to set.
        delta - The amount the record length changed.
        reservedDelta - The amount the reserved length changed.
        Throws:
        StandardException - Standard exception policy.
        java.io.IOException
      • initSlotTable

        private void initSlotTable​(PageKey newIdentity)
                            throws StandardException
        Initialize the in-memory slot table.

        Initialize the in-memory slot table, ie. that of our super-class BasePage. Go through all the records on the page and set the freeSpace and firstFreeByte on page.

        Parameters:
        newIdentity - The identity of the page we are trying to initialize, since we are in the middle of trying to build the page existing info in the class is not set up yet (like getIdentity()).
        Throws:
        StandardException - Standard exception policy.
      • setSlotEntry

        private void setSlotEntry​(int slot,
                                  int recordOffset,
                                  int recordPortionLength,
                                  int reservedSpace)
                           throws java.io.IOException
        Set up a new slot entry.

        Parameters:
        slot - the slot to initialize.
        recordOffset - the offset on the page to find the record.
        recordPortionLength - the actual length of record+hdr on page.
        reservedSpace - the reserved length associated with record.
        Throws:
        StandardException - Standard exception policy.
        java.io.IOException
      • addSlotEntry

        private void addSlotEntry​(int slot,
                                  int recordOffset,
                                  int recordPortionLength,
                                  int reservedSpace)
                           throws java.io.IOException
        Insert a new slot entry into the current slot array.

        Shift the existing slots from slot to (slotsInUse - 1) up by one. Up here means from low slot to high slot (e.g from slot 2 to slot 3). Our slot table grows backward so we have to be careful here.

        Parameters:
        slot - Position the new slot will take
        recordOffset - Offset of the record for the new slot
        recordPortionLength - Length of the record stored in the new slot
        reservedSpace - Length of reserved space of record in slot
        Throws:
        java.io.IOException
      • removeSlotEntry

        private void removeSlotEntry​(int slot)
                              throws java.io.IOException
        Remove slot entry from slot array.

        Remove a storage slot at slot. Shift the existing slots from slot+1 to (slotsInUse - 1) down by one.. Down here means from high slot to low slot (e.g from slot 3 to slot 2)

        Parameters:
        slot - The slot to delete.
        Throws:
        java.io.IOException
      • recordHeaderOnDemand

        public StoredRecordHeader recordHeaderOnDemand​(int slot)
        create the record header for the specific slot.

        Create a new record header object, initialize it, and add it to the array of cache'd record headers on this page. Finally return reference to the initialized record header.

        Specified by:
        recordHeaderOnDemand in class BasePage
        Parameters:
        slot - return record header of this slot.
        Returns:
        The record header for the specific slot.
      • entireRecordOnPage

        public boolean entireRecordOnPage​(int slot)
                                   throws StandardException
        Is entire record on the page?

        Specified by:
        entireRecordOnPage in class BasePage
        Parameters:
        slot - Check record at this slot.
        Returns:
        true if the entire record at slot is on this page, i.e, no overflow row or long columns.
        Throws:
        StandardException - Standard exception policy.
      • purgeOverflowAtSlot

        protected void purgeOverflowAtSlot​(int slot,
                                           RecordHandle headRowHandle,
                                           boolean needDataLogged)
                                    throws StandardException
        Purge one row on an overflow page.

        HeadRowHandle is the recordHandle pointing to the head row piece.

        Parameters:
        slot - slot number of row to purge.
        headRowHandle - recordHandle of the head row piece.
        needDataLogged - when true data is logged for purges otherwise just headers.
        Throws:
        StandardException - Standard exception policy.
      • purgeOneColumnChain

        private void purgeOneColumnChain​(long overflowPageId,
                                         int overflowRecordId)
                                  throws StandardException
        Purge the column chain that starts at overflowPageId, overflowRecordId

        Purge just the column chain that starts at the input address. The long column chain is pointed at by a field in a row. The long column is then chained as a sequence of "rows", the last column then points to the next segment of the chain on each page. Long columns chains currently are only one row per page so the next slot of a row in a long row chain should always be the first slot.

        Parameters:
        overflowPageId - The page where the long column chain starts.
        overflowRecordId - The record id where long column chain starts.
        Throws:
        StandardException - Standard exception policy.
      • purgeColumnChains

        private void purgeColumnChains​(RawTransaction t,
                                       int slot,
                                       RecordHandle headRowHandle)
                                throws StandardException
        purge long columns chains which eminate from this page.

        Purge all the long column chains emanating from the record on this slot of this page. The headRowHandle is the record handle of the head row piece of this row - if this page is the head row, then headRowHandle is the record handle at the slot. Otherwise, headRowHandle points to a row on a different page, i.e., the head page.

        Parameters:
        t - The raw transaction doing the purging.
        slot - The slot of the row to purge.
        headRowHandle - The RecordHandle of the head row.
        Throws:
        StandardException - Standard exception policy.
      • purgeRowPieces

        protected void purgeRowPieces​(RawTransaction t,
                                      int slot,
                                      RecordHandle headRowHandle,
                                      boolean needDataLogged)
                               throws StandardException
        Purge all the overflow columns and overflow rows of the record at slot.

        Purge all the overflow columns and overflow rows of the record at slot. This is called by BasePage.purgeAtSlot, the head row piece is purged there.

        Specified by:
        purgeRowPieces in class BasePage
        Parameters:
        t - The raw transaction doing the purging.
        slot - The slot of the row to purge.
        headRowHandle - The RecordHandle of the head row.
        needDataLogged - when true data is logged for purges otherwise just headers.
        Throws:
        StandardException - Standard exception policy.
      • removeOrphanedColumnChain

        void removeOrphanedColumnChain​(ReclaimSpace work,
                                       ContainerHandle containerHdl)
                                throws StandardException
        Remove a column chain that may have been orphaned by an update.

        Remove a column chain that may have been orphaned by an update. This is executed as a post commit operation. This page is the head page of the row which used to point to the column chain in question. The location of the orphaned column chain is in the ReclaimSpace record.
        MT - latched. No lock will be gotten, the head record must already be locked exclusive with no outstanding changes that can be rolled back.

        Parameters:
        work - object describing the chain to remove.
        containerHdl - open container handle to use to remove chain.
        Throws:
        StandardException - Standard exception policy.
      • isColumnOrphaned

        private boolean isColumnOrphaned​(StoredRecordHeader recordHeader,
                                         int columnId,
                                         long oldPageId,
                                         long oldRecordId)
                                  throws StandardException,
                                         java.io.IOException
        See if there is a orphaned long colum chain or not.

        See if there is a orphaned long colum chain or not. This is a helper function for removeOrphanedChain. This page, which may be a head page or overflow page, contains the column specified in columnId. It used to point to a long column chain at oldPageId and oldRecordId. Returns true if it no longer points to that long column chain.

        Parameters:
        recordHeader - record header which used to point at the long column
        columnId - column id of the long column in head.
        oldPageId - the page id where the long column used to be.
        oldRecordId - the record id where the long column used to be.
        Returns:
        true if page no longer points to the long column chain.
        Throws:
        StandardException - Standard exception policy.
        java.io.IOException
      • getNextColumnPiece

        private RecordHandle getNextColumnPiece​(int slot)
                                         throws StandardException
        Return the next recordHandle in a long column chain.

        Return a recordHandle pointing to the next piece of the column chain. This page must be an overflow page that is in a column chain. If this is the last piece of the overflow colum, return null.

        Parameters:
        slot - The slot of the current long column piece.
        Returns:
        The next record handle in a long column chain.
        Throws:
        StandardException - Standard exception policy.
      • initSpace

        private void initSpace()
        initialize the in memory variables associated with space maintenance.

        Get the total available space on an empty page. initSlotTable() must be called after the page has been read in.

      • clearAllSpace

        private void clearAllSpace()
        Initialize the freeSpace count and set the firstFreeByte on page
      • compressPage

        private void compressPage​(int startByte,
                                  int endByte)
                           throws java.io.IOException
        Compress out the space specified by startByte and endByte.

        As part of moving rows, updating rows, purging rows compact the space left between rows.

        Parameters:
        startByte - compress out space starting at startByte offset
        endByte - compress out space ending at endByte offset
        Throws:
        java.io.IOException
      • expandPage

        protected void expandPage​(int startOffset,
                                  int requiredBytes)
                           throws java.io.IOException
        Free up required bytes by shifting rows "down" the page.

        Expand page, move all the data from start Offset down the page by the amount required to free up the required bytes.

        Parameters:
        startOffset - offset on page to begin the shift
        requiredBytes - the number of bytes that must be freed.
        Throws:
        java.io.IOException - If IOException is raised during the page mod.
      • shrinkPage

        private void shrinkPage​(int startOffset,
                                int shrinkBytes)
                         throws java.io.IOException
        Shrink page.

        move all the data from start Offset up the page by the amount shrunk.

        Parameters:
        startOffset - offset on page to begin the shift
        shrinkBytes - the number of bytes that must be moved.
        Throws:
        java.io.IOException - some IOException is raised during the page mod, (unlikely as this is just writing to array).
      • getRecordLength

        public int getRecordLength​(int slot)
                            throws java.io.IOException
        Description copied from class: BasePage
        Get the stored length of a record. This must match the amount of data written by logColumn and logField.
        MT - latched - page latch must be held
        Specified by:
        getRecordLength in class BasePage
        Throws:
        java.io.IOException
      • getIsOverflow

        protected boolean getIsOverflow​(int slot)
                                 throws java.io.IOException
        Throws:
        java.io.IOException
      • logRow

        public int logRow​(int slot,
                          boolean forInsert,
                          int recordId,
                          java.lang.Object[] row,
                          FormatableBitSet validColumns,
                          DynamicByteArrayOutputStream out,
                          int startColumn,
                          byte insertFlag,
                          int realStartColumn,
                          int realSpaceOnPage,
                          int overflowThreshold)
                   throws StandardException,
                          java.io.IOException
        Log a row into the StoreOuput stream.

        Write the row in its record format to the stream. Record format is a record header followed by each field with its field header. See this class's description for the specifics of these headers. startColumn is used to specified which column for this logRow to start logging. When realStartColumn is specified, that means part of the row has already been logged. startColumn here indicates that the first column was logged in the logBuffer, need to continue log the rest of the row starting at realStartColumn. This is used when a longColumn is encountered during a long row. After done logging the long column, we need to continue logging the rest of the row. A -1 value for realStartColumn, means that it is not significant. logRow will not throw an noSpaceOnPage exception, if it is an overflow page, and the record we are inserting is the only record on the page. We are supporting rows expanding multiple pages through this mechanism. logRow expects row to be a sparse row.

        Specified by:
        logRow in class BasePage
        Parameters:
        slot - the slot of the row being logged.
        forInsert - this is logging an insert (not update/delete).
        recordId - record id of the row being logged.
        row - actual data of row in object form. If row is null then we are logging an overflow pointer.
        validColumns - bit map describing valid columns in row.
        out - stream to log to.
        startColumn - what column to start with (see above for detail)
        insertFlag - flag indicating mode we are in, INSERT_DEFAULT - default insert INSERT_SPLIT - splitting a row/column across pages.
        realStartColumn - If -1 ignore variable, else part of row has already been logged, and should continue with this column.
        realSpaceOnPage - Use this as space on page if realStartColumn is not -1.
        overflowThreshold - How much of the page to use before deciding to overflow a row.
        Returns:
        the "realStartColumn" value, -1 if not a long row.
        Throws:
        java.io.IOException - RESOLVE
        StandardException - Standard exception policy.
        See Also:
        BasePage.logRow(int, boolean, int, java.lang.Object[], org.apache.derby.iapi.services.io.FormatableBitSet, org.apache.derby.iapi.services.io.DynamicByteArrayOutputStream, int, byte, int, int, int)
      • handleIncompleteLogRow

        private void handleIncompleteLogRow​(int slot,
                                            int startColumn,
                                            FormatableBitSet columnList,
                                            DynamicByteArrayOutputStream out)
                                     throws StandardException
        Handle an update of a record portion that is incomplete.

        Handle an update of a record portion that is incomplete. Ie. Columns have expanded that require other columns to move off the page into a new portion.

        This method works out of the columns that need to be moved which are not being updated and makes a copy of their data. It then throws an exception with this data, much like the long column exception which will then allow the original insert to complete.

        If no columns need to be saved (ie all the ones that would move are being updated) then no exception is thrown, logRow() will return and the update completes normally.

        Parameters:
        slot - slot of the current update.
        startColumn - column to start at, handles start in middle of row
        columnList - bit map indicating which columns are being updated.
        out - place to lot to.
        Throws:
        StandardException - Standard exception policy.
      • restoreRecordFromStream

        public void restoreRecordFromStream​(LimitObjectInput in,
                                            java.lang.Object[] row)
                                     throws StandardException,
                                            java.io.IOException
        Restore a storable row from a LimitInputStream.

        Restore a storable row from an LimitInputStream - user must supply two streams on top of the same data, one implements ObjectInput interface that knows how to restore the object, the other one implements LimitInputStream.

        Specified by:
        restoreRecordFromStream in class BasePage
        Parameters:
        in - the limit input stream
        row - (IN/OUT) row that is to be restored (sparse representation)
        Throws:
        StandardException - Standard exception policy.
        java.io.IOException - object exceeds the available data in the stream.
      • qualifyRecordFromRow

        private boolean qualifyRecordFromRow​(java.lang.Object[] row,
                                             Qualifier[][] qual_list)
                                      throws StandardException
        Process the qualifier list on the row, return true if it qualifies.

        A two dimensional array is to be used to pass around a AND's and OR's in conjunctive normal form. The top slot of the 2 dimensional array is optimized for the more frequent where no OR's are present. The first array slot is always a list of AND's to be treated as described above for single dimensional AND qualifier arrays. The subsequent slots are to be treated as AND'd arrays or OR's. Thus the 2 dimensional array qual[][] argument is to be treated as the following, note if qual.length = 1 then only the first array is valid and it is and an array of and clauses: (qual[0][0] and qual[0][0] ... and qual[0][qual[0].length - 1]) and (qual[1][0] or qual[1][1] ... or qual[1][qual[1].length - 1]) and (qual[2][0] or qual[2][1] ... or qual[2][qual[2].length - 1]) ... and (qual[qual.length - 1][0] or qual[1][1] ... or qual[1][2])

        Parameters:
        row - The row being qualified.
        qual_list - 2 dimensional array representing conjunctive normal form of simple qualifiers.
        Returns:
        true if the row qualifies.
        Throws:
        StandardException - Standard exception policy.
      • readOneColumnFromPage

        private final void readOneColumnFromPage​(java.lang.Object[] row,
                                                 int colid,
                                                 int offset_to_field_data,
                                                 StoredRecordHeader recordHeader,
                                                 RecordHandle recordToLock)
                                          throws StandardException,
                                                 java.io.IOException
        Read just one column from stream into row.

        The routine reads just one column from the row, it is mostly code taken from readRecordFromStream, but highly optimized to just get one column from a non-overflow row. It can only be called to read a row from the pageData array as it directly accesses the page array to avoid the Stream overhead while processing non-user data which does not need the limit functionality.

        It is expected that this code will be called to read in a column associated with a qualifiers which are applied one column at a time, and has been specialized to proved the greatest peformance for processing qualifiers. This kind of access is done when scanning large datasets while applying qualifiers and thus any performance gain at this low level is multiplied by the large number of rows that may be iterated over.

        The column is read into the object located in row[qual_colid].

        Parameters:
        row - col is read into object in row[qual_colid].
        offset_to_field_data - offset in bytes from top of page to field
        colid - the column id to read, colid N is row[N]
        recordHeader - record header of row to read column from.
        recordToLock - record handle to lock, used by overflow column code.
        Throws:
        StandardException - Standard exception policy.
        java.io.IOException
      • qualifyRecordFromSlot

        private final boolean qualifyRecordFromSlot​(java.lang.Object[] row,
                                                    int offset_to_row_data,
                                                    FetchDescriptor fetchDesc,
                                                    StoredRecordHeader recordHeader,
                                                    RecordHandle recordToLock)
                                             throws StandardException,
                                                    java.io.IOException
        Process the list of qualifiers on the row in the stream.

        The rawDataIn stream is expected to be positioned after the record header.

        Check all qualifiers in the qualifier array against row. Return true if all compares specified by the qualifier array return true, else return false.

        This routine assumes client caller has already checked if the row is deleted or not. The row that it get's is expected to match the partial column list of the scan.

        On entering this routine the stream should be positioned to the beginning of the row data, just after the row header. On exit the stream will also be positioned there. A two dimensional array is to be used to pass around a AND's and OR's in conjunctive normal form. The top slot of the 2 dimensional array is optimized for the more frequent where no OR's are present. The first array slot is always a list of AND's to be treated as described above for single dimensional AND qualifier arrays. The subsequent slots are to be treated as AND'd arrays or OR's. Thus the 2 dimensional array qual[][] argument is to be treated as the following, note if qual.length = 1 then only the first array is valid and it is and an array of and clauses: (qual[0][0] and qual[0][0] ... and qual[0][qual[0].length - 1]) and (qual[1][0] or qual[1][1] ... or qual[1][qual[1].length - 1]) and (qual[2][0] or qual[2][1] ... or qual[2][qual[2].length - 1]) ... and (qual[qual.length - 1][0] or qual[1][1] ... or qual[1][2])

        Parameters:
        row - restore row into this object array.
        offset_to_row_data - offset in bytes from top of page to row
        fetchDesc - Description of fetch including which cols and qualifiers.
        recordHeader - The record header of the row, it was read in from stream and dataIn is positioned after it.
        recordToLock - The head row to use for locking, used to lock head row of overflow columns/rows.
        Returns:
        Whether or not the row input qualifies.
        Throws:
        StandardException - Standard exception policy.
        java.io.IOException
      • readRecordFromStream

        private final boolean readRecordFromStream​(java.lang.Object[] row,
                                                   int max_colid,
                                                   int[] vCols,
                                                   int[] mCols,
                                                   LimitObjectInput dataIn,
                                                   StoredRecordHeader recordHeader,
                                                   RecordHandle recordToLock)
                                            throws StandardException,
                                                   java.io.IOException
        restore a record from a stream.

        The rawDataIn stream is expected to be positioned after the record header.

        Parameters:
        row - restore row into this object array.
        max_colid - The maximum numbered column id that will be requested by caller. It should be: min(row.length - 1, maximum bit set in vCols) It is used to stop the inner most loop from looking at more columns in the row.
        vCols - If not null, bit map indicates valid cols.
        mCols - If not null, int array indicates columns already read in from the stream. A non-zero entry means the column has already been read in.
        dataIn - restore row from this stream.
        recordHeader - The record header of the row, it was read in from stream and dataIn is positioned after it.
        recordToLock - The head row to use for locking, used to lock head row of overflow columns/rows.
        Returns:
        The identifier to be used to open the conglomerate later.
        Throws:
        StandardException - Standard exception policy.
        java.io.IOException
      • restorePortionLongColumn

        public void restorePortionLongColumn​(OverflowInputStream fetchStream)
                                      throws StandardException,
                                             java.io.IOException
        Restore a portion of a long column.

        Restore a portion of a long column - user must supply two streams on top of the same data, one implements ObjectInput interface that knows how to restore the object, the other one implements LimitInputStream.

        Specified by:
        restorePortionLongColumn in class BasePage
        Parameters:
        fetchStream - the stream to read the next portion of long col from
        Throws:
        StandardException - Standard exception policy.
        java.io.IOException
      • logColumn

        public void logColumn​(int slot,
                              int fieldId,
                              java.lang.Object column,
                              DynamicByteArrayOutputStream out,
                              int overflowThreshold)
                       throws StandardException,
                              java.io.IOException
        Log a Storable to a stream.

        Log a Storable into a stream. This is used by update field operations

        Write the column in its field format to the stream. Field format is a field header followed the data of the column as defined by the data itself. See this class's description for the specifics of the header.

        Specified by:
        logColumn in class BasePage
        Parameters:
        slot - slot of the current record
        fieldId - field number of the column being updated
        column - column version of the field.
        out - Where to write the logged form.
        Throws:
        StandardException - Standard Derby error policy
        java.io.IOException - RESOLVE
      • logColumn

        private int logColumn​(java.lang.Object[] row,
                              int arrayPosition,
                              DynamicByteArrayOutputStream out,
                              int spaceAvailable,
                              int columnFlag,
                              int overflowThreshold)
                       throws StandardException,
                              java.io.IOException
        Log column from input row to the given output stream.

        Read data from row[arrayPosition], and write the column data in raw store page format to the given column. Along the way determine if the column will fit on the current page.

        Action taken in this routine is determined by the kind of column as specified in the columnFlag: COLUMN_NONE - the column is insignificant COLUMN_FIRST - this is the first column in a logRow() call COLUMN_LONG - this is a known long column, therefore we will store part of the column on the current page and overflow the rest if necessary.

        Upon entry to this routine logicalDataOut is tied to the DynamicByteArrayOutputStream out.
        If a column is a long column and it does not totally fit on the current page, then a LongColumnException is thrown. We package up info about the current long column in the partially filled in exception so that callers can take correct action. The column will now be set a as a stream.

        Parameters:
        row - array of column from which to read the column from.
        arrayPosition - The array position of column to be reading from row.
        out - The stream to write the raw store page format of the the column to.
        spaceAvailable - The number of bytes available on the page for this column, this may differ from current page as it may include bytes used by previous columns.
        columnFlag - one of: COLUMN_NONE, COLUMN_FIRST, or COLUMN_LONG.
        Returns:
        The spaceAvailable after accounting for space for this column.
        Throws:
        StandardException - Standard exception policy.
        LongColumnException - Thrown if column will not fit on a single page. See notes above
        java.io.IOException
      • logOverflowRecord

        private int logOverflowRecord​(int slot,
                                      int spaceAvailable,
                                      DynamicByteArrayOutputStream out)
                               throws StandardException,
                                      java.io.IOException
        Create and write a long row header to the log stream.

        Called to log a new overflow record, will check for space available and throw an exception if the record header will not fit on the page.

        Parameters:
        slot - slot of record to log.
        spaceAvailable - spaceAvaliable on page.
        out - stream to log the record to.
        Returns:
        -1
        Throws:
        StandardException - Standard exception policy.
        java.io.IOException
      • logField

        public void logField​(int slot,
                             int fieldNumber,
                             java.io.OutputStream out)
                      throws StandardException,
                             java.io.IOException
        Log a field to the ObjectOutput stream.

        Find the field in the record and then write out the complete field, i.e. header and data.

        Specified by:
        logField in class BasePage
        Parameters:
        slot - Slot number the record is stored in.
        fieldNumber - Number of the field (starts at 0).
        out - Where to write the logged form.
        Throws:
        StandardException - Standard Derby error policy
        java.io.IOException - RESOLVE
        See Also:
        BasePage.logField(int, int, java.io.OutputStream)
      • moveRecordForCompressAtSlot

        public int moveRecordForCompressAtSlot​(int slot,
                                               java.lang.Object[] row,
                                               RecordHandle[] old_handle,
                                               RecordHandle[] new_handle)
                                        throws StandardException
        Move record to a page toward the beginning of the file.

        As part of compressing the table records need to be moved from the end of the file toward the beginning of the file. Only the contiguous set of free pages at the very end of the file can be given back to the OS. This call is used to purge the row from the current page, insert it into a previous page, and return the new row location Mark the record identified by position as deleted. The record may be undeleted sometime later using undelete() by any transaction that sees the record.

        The interface is optimized to work on a number of rows at a time, optimally processing all rows on the page at once. The call will process either all rows on the page, or the number of slots in the input arrays - whichever is smaller. Locking Policy

        MUST be called with table locked, no locks are requested. Because it is called with table locks the call will go ahead and purge any row which is marked deleted. It will also use purge rather than delete to remove the old row after it moves it to a new page. This is ok since the table lock insures that no other transaction will use space on the table before this transaction commits.
        A page latch on the new page will be requested and released.

        Parameters:
        slot - slot of original row to move.
        row - a row template to hold all columns of row.
        old_handle - An array to be filled in by the call with the old handles of all rows moved.
        new_handle - An array to be filled in by the call with the new handles of all rows moved.
        Returns:
        the number of rows processed.
        Throws:
        StandardException - Standard Derby error policy
        See Also:
        LockingPolicy
      • cleanPage

        private void cleanPage()
      • initPage

        public void initPage​(LogInstant instant,
                             byte status,
                             int recordId,
                             boolean overflow,
                             boolean reuse)
                      throws StandardException
        Initialize the page. If reuse, then Clean up any in memory or on disk structure to ready the page for reuse. This is not only reusing the page buffer, but reusing a free page which may or may not be cleaned up the the client of raw store when it was deallocated.
        Specified by:
        initPage in class BasePage
        Throws:
        StandardException - Derby Standard Error Policy
      • storeRecord

        public void storeRecord​(LogInstant instant,
                                int slot,
                                boolean insert,
                                java.io.ObjectInput in)
                         throws StandardException,
                                java.io.IOException
        Store a record at the given slot.
        Specified by:
        storeRecord in class BasePage
        Throws:
        StandardException - Standard Derby error policy
        java.io.IOException - RESOLVE
      • storeRecordForInsert

        private void storeRecordForInsert​(int slot,
                                          java.io.ObjectInput in)
                                   throws StandardException,
                                          java.io.IOException
        Throws:
        StandardException
        java.io.IOException
      • storeRecordForUpdate

        private void storeRecordForUpdate​(int slot,
                                          java.io.ObjectInput in)
                                   throws StandardException,
                                          java.io.IOException
        Throws:
        StandardException
        java.io.IOException
      • createSpaceForUpdate

        private void createSpaceForUpdate​(int slot,
                                          int offset,
                                          int oldLength,
                                          int newLength)
                                   throws StandardException,
                                          java.io.IOException
        Create the space to update a portion of a record. This method ensures there is enough room to replace the old data of length oldLength at the given offset, with the new data of length newLength. This method does put any new data on the page, it moves old data around and zeros out any old data when newLength < oldLength. This method does update the information in the slot table. The passed in offset is the correct place to put the data when this method returns, ie. it only moves data that has an offset greater then this.
        Throws:
        StandardException - Standard Derby error policy
        java.io.IOException - RESOLVE
      • reserveSpaceForSlot

        public void reserveSpaceForSlot​(LogInstant instant,
                                        int slot,
                                        int spaceToReserve)
                                 throws StandardException,
                                        java.io.IOException
        reserveSpaceForSlot This method will reserve at least specified "spaceToReserve" bytes for the record in the slot.
        Specified by:
        reserveSpaceForSlot in class BasePage
        Throws:
        StandardException - Standard Derby error policy
        java.io.IOException - RESOLVE
      • skipField

        public void skipField​(java.io.ObjectInput in)
                       throws java.io.IOException
        Skip a field header and its data on the given stream.
        Specified by:
        skipField in class BasePage
        Throws:
        java.io.IOException - corrupt stream
      • skipRecord

        public void skipRecord​(java.io.ObjectInput in)
                        throws java.io.IOException
        Specified by:
        skipRecord in class BasePage
        Throws:
        java.io.IOException
      • shiftRemainingData

        private int shiftRemainingData​(int slot,
                                       int offset,
                                       int oldLength,
                                       int newLength)
                                throws java.io.IOException
        Shift data within a record to account for an update.
        Parameters:
        offset - Offset where the update starts, need not be on a field boundry.
        oldLength - length of the data being replaced
        newLength - length of the data replacing the old data
        Returns:
        the length of the data in the record after the replaced data.
        Throws:
        java.io.IOException
      • internalDeletedRecordCount

        protected int internalDeletedRecordCount()
        get record count without checking for latch
        Specified by:
        internalDeletedRecordCount in class BasePage
      • purgeRecord

        public void purgeRecord​(LogInstant instant,
                                int slot,
                                int recordId)
                         throws StandardException,
                                java.io.IOException
        purgeRecord from page. Move following slots up by one.
        Specified by:
        purgeRecord in class BasePage
        slot - the slot to purge
        recordId - the id of the record that is to be purged
        Throws:
        StandardException - Standard Derby error policy
        java.io.IOException - RESOLVE
      • getFieldOffset

        private int getFieldOffset​(int slot,
                                   int fieldNumber)
                            throws java.io.IOException
        Get the offset of the field header of the given field for the record in the given slot. Field number is the absolute number for the complete record, not just this portion. E.g. if this is a record portion that starts at field 3 and has 6 fields then the second field on this *page* has field number 4.
        Throws:
        java.io.IOException
      • currentTimeStamp

        public PageTimeStamp currentTimeStamp()
        Get a time stamp for this page
        Returns:
        page time stamp
      • setTimeStamp

        public void setTimeStamp​(PageTimeStamp ts)
                          throws StandardException
        Set given pageVersion to be the as what is on this page
        Throws:
        StandardException - given time stamp is null or is not a time stamp implementation this page knows how to deal with
      • equalTimeStamp

        public boolean equalTimeStamp​(PageTimeStamp ts)
                               throws StandardException
        compare given PageVersion with pageVersion on page
        Parameters:
        ts - the page version gotton from this page via a currentTimeStamp or setTimeStamp call earlier
        Returns:
        true if the same
        Throws:
        StandardException - given time stamp not gotton from this page
        See Also:
        PageTimeStamp
      • toString

        public java.lang.String toString()
        debugging, print this page
        Overrides:
        toString in class java.lang.Object
      • toUncheckedString

        public java.lang.String toUncheckedString()
      • pagedataToHexDump

        private static java.lang.String pagedataToHexDump​(byte[] data)
        Provide a hex dump of the data in the in memory version of the page.

        The output looks like: 00000000: 4d5a 9000 0300 0000 0400 0000 ffff 0000 MZ.............. 00000010: b800 0000 0000 0000 4000 0000 0000 0000 ........@....... 00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000030: 0000 0000 0000 0000 0000 0000 8000 0000 ................ 00000040: 0e1f ba0e 00b4 09cd 21b8 014c cd21 5468 ........!..L.!Th 00000050: 6973 2070 726f 6772 616d 2063 616e 6e6f is program canno 00000060: 7420 6265 2072 756e 2069 6e20 444f 5320 t be run in DOS 00000070: 6d6f 6465 2e0d 0a24 0000 0000 0000 0050 mode...$.......P 00000080: 4500 004c 0109 008b abfd 3000 0000 0000 E..L......0..... 00000090: 0000 00e0 000e 210b 0102 3700 3405 0000 ......!...7.4... 000000a0: 8401 0000 6400 0000 6004 0000 1000 0000 ....d...`....... 000000b0: 5005 0000 0008 6000 1000 0000 0200 0001 P.....`......... 000000c0: 0000 0000 0000 0004 0000 0000 0000 0000 ................ 000000d0: 9007 0000 0400 0009 a207 0002 0000 0000 ................ 000000e0: 0010 0000 1000 0000 0010 0000 1000 0000 ................ 000000f0: 0000 0010 0000 0000 6006 00ef 8100 0000 ........`....... 00000100: 5006 00e6 0c00 0000 0007 00d0 0400 0000 P............... 00000110: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000120: 1007 00c8 7100 0000 0000 0000 0000 0000 ....q........... 00000130: 0000 0000 0000 0000 0000 0000 0000 0000 ................

        Parameters:
        data - array of bytes to dump.
        Returns:
        The string with the hex dump in it.
      • pageHeaderToString

        private java.lang.String pageHeaderToString()
      • getPageDumpString

        java.lang.String getPageDumpString()
      • recordToString

        private java.lang.String recordToString​(int slot)
      • getOverflowPage

        protected StoredPage getOverflowPage​(long pageNumber)
                                      throws StandardException
        Get the overflow page for a record that has already overflowed.
        Throws:
        StandardException - Standard Derby error policy
      • getOverflowPageForInsert

        public BasePage getOverflowPageForInsert​(int currentSlot,
                                                 java.lang.Object[] row,
                                                 FormatableBitSet validColumns)
                                          throws StandardException
        Get a overflow page that potentially can handle a new overflowed record.
        Throws:
        StandardException - Standard Derby error policy
      • updateOverflowDetails

        public void updateOverflowDetails​(RecordHandle handle,
                                          RecordHandle overflowHandle)
                                   throws StandardException
        Update a record handle to point to an overflowed record portion. Note that the record handle need not be the current page.
        Specified by:
        updateOverflowDetails in class BasePage
        Parameters:
        handle - handle of the record for long row
        overflowHandle - the overflow (continuation) pointer for the long row
        Throws:
        StandardException - Standard Derby error policy
      • updateFieldOverflowDetails

        public void updateFieldOverflowDetails​(RecordHandle handle,
                                               RecordHandle overflowHandle)
                                        throws StandardException
        Description copied from class: BasePage
        Update the overflow pointer for a long column
        MT - latched - page latch must be held
        Specified by:
        updateFieldOverflowDetails in class BasePage
        Parameters:
        handle - handle of the record for long row
        overflowHandle - the overflow (continuation) pointer for the long row
        Throws:
        StandardException - Standard Derby error policy
      • appendOverflowFieldHeader

        public int appendOverflowFieldHeader​(DynamicByteArrayOutputStream logBuffer,
                                             RecordHandle overflowHandle)
                                      throws StandardException,
                                             java.io.IOException
        Description copied from class: BasePage
        Append an overflow pointer to a partly logged row, to point to a long column that just been logged.
        MT - latched - page latch must be held
        Specified by:
        appendOverflowFieldHeader in class BasePage
        Parameters:
        logBuffer - The buffer that contains the partially logged row.
        overflowHandle - the overflow (continuation) pointer to the beginning of the long column
        Throws:
        StandardException - Standard Derby error policy
        java.io.IOException
      • getSlotsInUse

        protected int getSlotsInUse()
      • getMaxDataLength

        private int getMaxDataLength​(int spaceAvailable,
                                     int overflowThreshold)
        return the max datalength allowed with the space available
      • isLong

        private boolean isLong​(int fieldSize,
                               int overflowThreshold)
        return whether the field has exceeded the max threshold for this page it compares the fieldSize with the largest possible field for this page
      • checkRowReservedSpace

        private boolean checkRowReservedSpace​(int slot)
                                       throws StandardException
        See if reserved space should be reclaimed for the input row.

        See if the row on this page has reserved space that should be shrunk once the update commits. Will only indicate space should be reclaimed if at least RawTransaction.MINIMUM_RECORD_SIZE_DEFAULT bytes can be reclaimed.

        Returns:
        true if space should be reclaimed from this row post commit.
        Throws:
        StandardException