Class FileContainer
- java.lang.Object
-
- org.apache.derby.impl.store.raw.data.BaseContainer
-
- org.apache.derby.impl.store.raw.data.FileContainer
-
- All Implemented Interfaces:
Cacheable
,TypedFormat
,Lockable
- Direct Known Subclasses:
InputStreamContainer
,RAFContainer
abstract class FileContainer extends BaseContainer implements Cacheable, TypedFormat
FileContainer is an abstract base class for containers which are based on files. This class extends BaseContainer and implements Cacheable and TypedFormat
-
-
Field Summary
Fields Modifier and Type Field Description protected AllocationCache
allocCache
private boolean
bulkIncreaseContainerSize
protected boolean
canUpdate
private java.util.zip.CRC32
checksum
protected static int
CHECKSUM_SIZE
private static int
CONTAINER_FORMAT_ID_SIZE
the container format must fit in this many bytesprotected static int
CONTAINER_INFO_SIZE
The size of the persistently stored container info ContainerHeader contains the following information: 4 bytes int FormatId 4 bytes int status 4 bytes int pageSize 4 bytes int spareSpace 4 bytes int minimumRecordSize 2 bytes short initialPages 2 bytes short spare1 8 bytes long first Allocation page number 8 bytes long first Allocation page offset 8 bytes long container version 8 bytes long estimated number of rows 8 bytes long reusable recordId sequence number 8 bytes long spare3 8 bytes long checksum container info size is 80 bytes, with 10 bytes of spare spaceprotected CacheManager
containerCache
(package private) byte[]
containerInfo
protected long
containerVersion
protected BaseDataFileFactory
dataFactory
private static int
DEFAULT_PRE_ALLOC_SIZE
private byte[]
encryptionBuffer
private long
estimatedPageCount
An estimated page count.protected long
estimatedRowCount
private static int
FILE_COMMITTED_DROP
private static int
FILE_DROPPED
private static int
FILE_REUSABLE_RECORDID
static long
FIRST_ALLOC_PAGE_NUMBER
where the first alloc page is located - the logical page number and the physical page offset NOTE if it is not 0 this is not going to work for Stream file which doesn't support seekstatic long
FIRST_ALLOC_PAGE_OFFSET
protected long
firstAllocPageNumber
protected long
firstAllocPageOffset
protected static int
formatIdInteger
protected short
initialPages
protected boolean
isDirty
private long
lastAllocatedPage
The last allocated page.private long[]
lastInsertedPage
The page that was last inserted into.private int
lastInsertedPage_index
protected LogInstant
lastLogInstant
private long
lastUnfilledPage
The last unfilled page found.private static int
MAX_PRE_ALLOC_SIZE
private static int
MIN_PRE_ALLOC_SIZE
protected int
minimumRecordSize
protected CacheManager
pageCache
protected int
pageSize
private static int
PRE_ALLOC_THRESHOLD
private int
PreAllocSize
private int
PreAllocThreshold
protected boolean
preDirty
private long
reusableRecordIdSequenceNumber
The sequence number for reusable recordIds .protected static java.lang.String
SPACE_TRACE
protected int
spareSpace
-
Fields inherited from class org.apache.derby.impl.store.raw.data.BaseContainer
identity, isCommittedDrop, isDropped, isReusableRecordId
-
-
Constructor Summary
Constructors Constructor Description FileContainer(BaseDataFileFactory factory)
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description protected abstract void
backupContainer(BaseContainerHandle handle, java.lang.String backupLocation)
backup the container.protected void
bumpContainerVersion(LogInstant instant)
increment the version by one and return the new version.protected boolean
canUpdate()
Can the container be updated.void
clearIdentity()
Put the object into the No Identity state.protected void
clearPreallocThreshold()
The container will have no pre-allocate threshold, i.e., if the implementation supports it, page preallocation will happen the next time a new page is allocated.(package private) abstract void
closeContainer()
protected void
compressContainer(RawTransaction ntt, BaseContainerHandle allocHandle)
Compress free space from container.(package private) abstract void
createContainer(ContainerKey newIdentity)
Create a new container.protected Cacheable
createIdent(ContainerKey newIdentity, java.lang.Object createParameter)
Cacheable
createIdentity(java.lang.Object key, java.lang.Object createParameter)
Create a new item.private void
createInfoFromLog(ByteArray byteArray)
Set container properties from the passed in ByteArray, which is created by logCreateContainerInfo.private void
createInfoFromProp(java.util.Properties createArgs)
Set container properties from the passed in createArgs.protected void
deallocatePage(BaseContainerHandle handle, BasePage page)
Deallocate a page from the container.private void
deallocatePagenum(BaseContainerHandle handle, long pnum)
deallocate the page from the alloc pageprotected void
decryptPage(byte[] pageData, int pageSize)
Decrypts a page
MT - MT safe.protected int
doPreAllocatePages(long lastPreallocPagenum, int preAllocSize)
Preallocate the pages - actually doing it, called by subclass onlyprotected void
dropContainer(LogInstant instant, boolean isDropped)
Drop Container.protected byte[]
encryptPage(byte[] pageData, int pageSize, byte[] encryptionBuffer, boolean newEngine)
Encrypts a page.private AllocPage
findAllocPageForAdd(BaseContainerHandle allocHandle, RawTransaction ntt, long lastAllocatedPage)
private AllocPage
findLastAllocPage(BaseContainerHandle handle, RawTransaction tran)
Find the last alloc page, returns null if no alloc page is foundprotected BasePage
getAllocPage(long pageNumber)
Get an alloc page - only accessible to the raw store (container and recovery)protected BasePage
getAnyPage(BaseContainerHandle handle, long pageNumber)
Get any old page - turn off all validationvoid
getContainerProperties(java.util.Properties prop)
Request the system properties associated with a container.protected long
getContainerVersion()
Get the logged container version.(package private) static ContextService
getContextService()
Privileged lookup of the ContextService.protected byte[]
getEmbryonicPage(java.io.DataInput fileData)
Get an embryonic page from the dataInput stream.(package private) byte[]
getEmbryonicPage(StorageRandomAccessFile file, long offset)
Read an embryonic page (that is, a section of the first alloc page that is so large that we know all the borrowed space is included in it) from the specified offset in aStorageRandomAccessFile
.protected byte[]
getEncryptionBuffer()
Get encryption buffer.long
getEstimatedPageCount(BaseContainerHandle handle, int flag)
long
getEstimatedRowCount(int flag)
MT - this routine is NOT MT-safe and clients don't need to provide synchronization.protected BasePage
getFirstHeadPage(BaseContainerHandle handle, boolean wait)
Get the first valid page in the containerprotected BasePage
getHeadPage(BaseContainerHandle handle, long pageNumber, boolean wait)
Get only a valid, non-overflow page.private BasePage
getInsertablePage(BaseContainerHandle handle, long pageNumber, boolean wait, boolean overflowOK)
private long
getLastInsertedPage()
protected long
getLastPageNumber(BaseContainerHandle handle)
private long
getLastUnfilledPage()
protected BasePage
getLatchedPage(BaseContainerHandle handle, long pageNumber)
Get a latched page.protected int
getMinimumRecordSize()
protected BasePage
getNextHeadPage(BaseContainerHandle handle, long pageNumber, boolean wait)
Get the next page in the container.protected BasePage
getPage(BaseContainerHandle handle, long pageNumber, boolean wait)
Get a valid (non-deallocated or free) page in the container.protected BasePage
getPageForCompress(BaseContainerHandle handle, int flag, long pageno)
Get candidate page to move a row for compressing the table.protected BasePage
getPageForInsert(BaseContainerHandle handle, int flag)
Get a potentially suitable page for insert and latch it.protected int
getPageSize()
long
getReusableRecordIdSequenceNumber()
Get the reusable RecordId sequence number for the container.private static java.lang.Object
getServiceModule(java.lang.Object serviceModule, java.lang.String factoryInterface)
Privileged module lookup.SpaceInfo
getSpaceInfo(BaseContainerHandle handle)
Get information about space used by the container.protected int
getSpareSpace()
int
getTypeFormatId()
Return my format identifier.private long
getUnfilledPageNumber(BaseContainerHandle handle, long pagenum)
private BasePage
getUserPage(BaseContainerHandle handle, long pageNumber, boolean overflowOK, boolean wait)
Get a page in the container.protected void
incrementReusableRecordIdSequenceNumber()
Increment the reusable RecordId version sequence number.private void
initContainerHeader(boolean changeContainer)
private void
initializeLastInsertedPage(int size)
protected BasePage
initPage(BaseContainerHandle allochandle, PageKey pkey, PageCreationArgs createArgs, long pageOffset, boolean reuse, boolean overflow)
Initialize a pageboolean
isDirty()
We treat this container as dirty if it has the container file open.protected BasePage
latchPage(BaseContainerHandle handle, BasePage foundPage, boolean wait)
protected void
letGo(BaseContainerHandle handle)
The container is kept by the find() in File.openContainer.protected ByteArray
logCreateContainerInfo()
Log all information on the container creation necessary to recreate the container during a load tran.private AllocPage
makeAllocPage(RawTransaction ntt, BaseContainerHandle handle, long pageNumber, long pageOffset, int containerInfoSize)
protected BasePage
newPage(BaseContainerHandle userHandle, RawTransaction ntt, BaseContainerHandle allocHandle, boolean isOverflow)
Create a new page in the container.(package private) abstract boolean
openContainer(ContainerKey newIdentity)
Open a container.private boolean
pageValid(BaseContainerHandle handle, long pagenum)
protected abstract int
preAllocate(long lastPreallocPagenum, int preAllocSize)
preAllocate writes out the preallocated pages to disk if necessary.void
preDirty(boolean preDirtyOn)
The container is about to be modified.protected void
prepareForBulkLoad(BaseContainerHandle handle, int numPage)
The container will be grown vastly, prepare for it.protected void
readHeader(byte[] epage)
Read the container's header.private void
readHeaderFromArray(byte[] a)
Read containerInfo from a byte array The container Header array must be written by or of the same format as put together by writeHeaderFromArray.protected abstract void
readPage(long pageNumber, byte[] pageData)
Read a page into the supplied array.protected BasePage
reCreatePageForRedoRecovery(BaseContainerHandle handle, int pageFormat, long pageNumber, long pageOffset)
ReCreate a page for rollforward recovery.protected void
setDirty(boolean dirty)
void
setEstimatedRowCount(long count, int flag)
protected Cacheable
setIdent(ContainerKey newIdentity)
Open the container.Cacheable
setIdentity(java.lang.Object key)
Open the container.private void
setLastInsertedPage(long val)
private void
setLastUnfilledPage(long val)
private void
switchToMultiInsertPageMode(BaseContainerHandle handle)
protected void
trackUnfilledPage(long pagenumber, boolean unfilled)
protected void
updateEstimatedRowCount(int delta)
Update estimated row count by page as it leaves the cache.(package private) void
writeAtOffset(StorageRandomAccessFile file, byte[] bytes, long offset)
Write a sequence of bytes at the given offset in a file.protected void
writeHeader(java.lang.Object identity, byte[] pageData)
Write the container header to a page array (the first allocation page)protected void
writeHeader(java.lang.Object identity, StorageRandomAccessFile file, boolean create, byte[] epage)
Write the container header directly to file.private void
writeHeaderToArray(byte[] a)
Write containerInfo into a byte array The container Header thus put together can be read by readHeaderFromArray.protected abstract void
writePage(long pageNumber, byte[] pageData, boolean syncPage)
Write a page from the supplied array.-
Methods inherited from class org.apache.derby.impl.store.raw.data.BaseContainer
addPage, compressContainer, encryptOrDecryptContainer, fillInIdentity, flushAll, getAllocPage, getAnyPage, getCommittedDropState, getContainerId, getContainerStatus, getDeallocLock, getDroppedState, getFirstPage, getIdentity, getNextPage, getSegmentId, isReusableRecordId, lockAttributes, lockerAlwaysCompatible, lockEvent, removeContainer, removePage, requestCompatible, setCommittedDropState, setDroppedState, setReusableRecordIdState, truncate, truncatePages, unlockEvent, use
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.derby.iapi.services.cache.Cacheable
clean, getIdentity
-
-
-
-
Field Detail
-
formatIdInteger
protected static final int formatIdInteger
- See Also:
- Constant Field Values
-
pageCache
protected final CacheManager pageCache
-
containerCache
protected final CacheManager containerCache
-
dataFactory
protected final BaseDataFileFactory dataFactory
-
pageSize
protected int pageSize
-
spareSpace
protected int spareSpace
-
minimumRecordSize
protected int minimumRecordSize
-
initialPages
protected short initialPages
-
canUpdate
protected boolean canUpdate
-
PreAllocThreshold
private int PreAllocThreshold
-
PreAllocSize
private int PreAllocSize
-
bulkIncreaseContainerSize
private boolean bulkIncreaseContainerSize
-
PRE_ALLOC_THRESHOLD
private static final int PRE_ALLOC_THRESHOLD
- See Also:
- Constant Field Values
-
MIN_PRE_ALLOC_SIZE
private static final int MIN_PRE_ALLOC_SIZE
- See Also:
- Constant Field Values
-
DEFAULT_PRE_ALLOC_SIZE
private static final int DEFAULT_PRE_ALLOC_SIZE
- See Also:
- Constant Field Values
-
MAX_PRE_ALLOC_SIZE
private static final int MAX_PRE_ALLOC_SIZE
- See Also:
- Constant Field Values
-
firstAllocPageNumber
protected long firstAllocPageNumber
-
firstAllocPageOffset
protected long firstAllocPageOffset
-
containerVersion
protected long containerVersion
-
estimatedRowCount
protected long estimatedRowCount
-
lastLogInstant
protected LogInstant lastLogInstant
-
reusableRecordIdSequenceNumber
private long reusableRecordIdSequenceNumber
The sequence number for reusable recordIds . As long as this number does not change, recordIds will be stable within the container.
-
lastInsertedPage
private long[] lastInsertedPage
The page that was last inserted into. Use this for getPageForInsert. Remember the last allocated non-overflow page, and remember it in memory only. Use Get/Set method to access this field except when we know it is being single thread access.
-
lastInsertedPage_index
private int lastInsertedPage_index
-
lastUnfilledPage
private long lastUnfilledPage
The last unfilled page found. Use this for getPageForInsert. Remember the last unfilled page found, and remember it in memory only. Use Get/Set method to access this field except when we know it is being single thread access.
-
lastAllocatedPage
private long lastAllocatedPage
The last allocated page. This global var is access *without* synchronization. It is used as a hint for page allocation to find the next reusable page.
-
estimatedPageCount
private long estimatedPageCount
An estimated page count. Use this for getEstimatedPagecount. Remember it in memory only.
-
preDirty
protected boolean preDirty
-
isDirty
protected boolean isDirty
-
allocCache
protected AllocationCache allocCache
-
containerInfo
byte[] containerInfo
-
checksum
private java.util.zip.CRC32 checksum
-
encryptionBuffer
private byte[] encryptionBuffer
-
CONTAINER_FORMAT_ID_SIZE
private static final int CONTAINER_FORMAT_ID_SIZE
the container format must fit in this many bytes- See Also:
- Constant Field Values
-
CHECKSUM_SIZE
protected static final int CHECKSUM_SIZE
- See Also:
- Constant Field Values
-
CONTAINER_INFO_SIZE
protected static final int CONTAINER_INFO_SIZE
The size of the persistently stored container info ContainerHeader contains the following information: 4 bytes int FormatId 4 bytes int status 4 bytes int pageSize 4 bytes int spareSpace 4 bytes int minimumRecordSize 2 bytes short initialPages 2 bytes short spare1 8 bytes long first Allocation page number 8 bytes long first Allocation page offset 8 bytes long container version 8 bytes long estimated number of rows 8 bytes long reusable recordId sequence number 8 bytes long spare3 8 bytes long checksum container info size is 80 bytes, with 10 bytes of spare space- See Also:
- Constant Field Values
-
FIRST_ALLOC_PAGE_NUMBER
public static final long FIRST_ALLOC_PAGE_NUMBER
where the first alloc page is located - the logical page number and the physical page offset NOTE if it is not 0 this is not going to work for Stream file which doesn't support seek- See Also:
- Constant Field Values
-
FIRST_ALLOC_PAGE_OFFSET
public static final long FIRST_ALLOC_PAGE_OFFSET
- See Also:
- Constant Field Values
-
FILE_DROPPED
private static final int FILE_DROPPED
- See Also:
- Constant Field Values
-
FILE_COMMITTED_DROP
private static final int FILE_COMMITTED_DROP
- See Also:
- Constant Field Values
-
FILE_REUSABLE_RECORDID
private static final int FILE_REUSABLE_RECORDID
- See Also:
- Constant Field Values
-
SPACE_TRACE
protected static final java.lang.String SPACE_TRACE
-
-
Constructor Detail
-
FileContainer
FileContainer(BaseDataFileFactory factory)
-
-
Method Detail
-
getTypeFormatId
public int getTypeFormatId()
Return my format identifier.- Specified by:
getTypeFormatId
in interfaceTypedFormat
- Returns:
- The identifier. (A UUID stuffed in an array of 16 bytes).
-
getSpaceInfo
public SpaceInfo getSpaceInfo(BaseContainerHandle handle) throws StandardException
Get information about space used by the container.- Specified by:
getSpaceInfo
in classBaseContainer
- Throws:
StandardException
-
setIdentity
public Cacheable setIdentity(java.lang.Object key) throws StandardException
Open the container.- Specified by:
setIdentity
in interfaceCacheable
- Returns:
- a valid object if the container was successfully opened, null if it does not exist.
- Throws:
StandardException
- Some problem in opening a container.- See Also:
Cacheable.setIdentity(java.lang.Object)
-
setIdent
protected Cacheable setIdent(ContainerKey newIdentity) throws StandardException
Open the container.Open the container with key "newIdentity".
should be same name as setIdentity but seems to cause method resolution ambiguities
- Throws:
StandardException
- Some problem in opening a container.- See Also:
Cacheable.setIdentity(java.lang.Object)
-
createIdentity
public Cacheable createIdentity(java.lang.Object key, java.lang.Object createParameter) throws StandardException
Description copied from interface:Cacheable
Create a new item.Create a new item and set the identity of the object to represent it. The object will be in the No Identity state, ie. it will have just been created or clearIdentity() was just called.
The object must copy the information out of key, not just store a reference to key if the key is not immutable. After this call the expression getIdentity().equals(key) must return true.
If the class of the object needs to change (e.g. to support a different format) then the object should create a new object, call its initParameter() with the parameters the original object was called with, set its identity and return a reference to it. The cache manager will discard the reference to the old object.
If an exception is thrown the object must be left in the no-identity state.
MT - single thread required - Method must only be called be cache manager and the cache manager will guarantee only one thread can be calling it.- Specified by:
createIdentity
in interfaceCacheable
- Returns:
- an object reference if the object can take on the identity, null otherwise.
- Throws:
StandardException
- If forCreate is true and the object cannot be created.- See Also:
CacheManager.create(java.lang.Object, java.lang.Object)
-
createIdent
protected Cacheable createIdent(ContainerKey newIdentity, java.lang.Object createParameter) throws StandardException
- Throws:
StandardException
-
clearIdentity
public void clearIdentity()
Description copied from interface:Cacheable
Put the object into the No Identity state.
MT - single thread required - Method must only be called be cache manager and the cache manager will guarantee only one thread can be calling it.- Specified by:
clearIdentity
in interfaceCacheable
- Overrides:
clearIdentity
in classBaseContainer
-
isDirty
public boolean isDirty()
We treat this container as dirty if it has the container file open.- Specified by:
isDirty
in interfaceCacheable
- See Also:
Cacheable.isDirty()
-
preDirty
public void preDirty(boolean preDirtyOn)
Description copied from class:BaseContainer
The container is about to be modified. Loggable actions use this to make sure the container gets cleaned if a checkpoint is taken after any log record is sent to the log stream but before the container is actually dirtied.- Specified by:
preDirty
in classBaseContainer
-
setDirty
protected void setDirty(boolean dirty)
-
createContainer
abstract void createContainer(ContainerKey newIdentity) throws StandardException
Create a new container.Create a new container, all references to identity must be through the passed in identity, this object will no identity until after this method returns.
- Throws:
StandardException
- Derby Standard error policy
-
openContainer
abstract boolean openContainer(ContainerKey newIdentity) throws StandardException
Open a container.Longer descrption of routine.
Open a container. Open the file that maps to this container, if the file does not exist then we assume the container was never created. If the file exists but we have trouble opening it then we throw some exception.
MT - single thread required - Enforced by cache manager.- Throws:
StandardException
- Standard exception policy.
-
closeContainer
abstract void closeContainer()
-
dropContainer
protected void dropContainer(LogInstant instant, boolean isDropped)
Drop Container.- Specified by:
dropContainer
in classBaseContainer
- See Also:
Transaction.dropContainer(org.apache.derby.iapi.store.raw.ContainerKey)
-
bumpContainerVersion
protected final void bumpContainerVersion(LogInstant instant)
increment the version by one and return the new version.
MT - caller must synchronized this in the same sync block that modifies the container header.
-
getContainerVersion
protected long getContainerVersion()
Description copied from class:BaseContainer
Get the logged container version.- Specified by:
getContainerVersion
in classBaseContainer
-
getContainerProperties
public void getContainerProperties(java.util.Properties prop) throws StandardException
Request the system properties associated with a container.Request the value of properties that are associated with a container. The following properties can be requested: derby.storage.pageSize derby.storage.pageReservedSpace derby.storage.minimumRecordSize derby.storage.reusableRecordId derby.storage.initialPages
To get the value of a particular property add it to the property list, and on return the value of the property will be set to it's current value. For example: get_prop(ConglomerateController cc) { Properties prop = new Properties(); prop.put("derby.storage.pageSize", ""); cc.getContainerProperties(prop); System.out.println( "table's page size = " + prop.getProperty("derby.storage.pageSize"); }
- Specified by:
getContainerProperties
in classBaseContainer
- Parameters:
prop
- Property list to fill in.- Throws:
StandardException
- Standard exception policy.
-
readHeader
protected void readHeader(byte[] epage) throws java.io.IOException, StandardException
Read the container's header. When this method is called, the embryonic page that is passed in must have been read directly from the file or the input stream, even if the alloc page may still be in cache. This is because a stubbify operation only writes the stub to disk, it does not get rid of any stale page from the page cache. So if it so happens that the stubbified container object is aged out of the container cache but the first alloc page hasn't, then when any stale page of this container wants to be written out, the container needs to be reopened, which is when this routine is called. We must not get the alloc page in cache because it may be stale page and it may still say the container has not been dropped.
MT - single thread required - Enforced by caller.- Parameters:
epage
- the embryonic page to read the header from- Throws:
StandardException
- Derby Standard error policyjava.io.IOException
- error in reading the header from file
-
initContainerHeader
private void initContainerHeader(boolean changeContainer)
-
readHeaderFromArray
private void readHeaderFromArray(byte[] a) throws StandardException, java.io.IOException
Read containerInfo from a byte array The container Header array must be written by or of the same format as put together by writeHeaderFromArray.- Throws:
StandardException
- Derby Standard error policyjava.io.IOException
- error in reading the header from file
-
writeHeader
protected void writeHeader(java.lang.Object identity, byte[] pageData) throws StandardException, java.io.IOException
Write the container header to a page array (the first allocation page)- Throws:
StandardException
- Derby Standard error policyjava.io.IOException
- error in writing the header to file
-
writeHeader
protected void writeHeader(java.lang.Object identity, StorageRandomAccessFile file, boolean create, byte[] epage) throws java.io.IOException, StandardException
Write the container header directly to file. Subclasses that can writes the container header is expected to manufacture a DataOutput stream which is used here.
MT - single thread required - Enforced by caller- Throws:
StandardException
- Derby Standard error policyjava.io.IOException
- error in writing the header to file
-
writeAtOffset
void writeAtOffset(StorageRandomAccessFile file, byte[] bytes, long offset) throws java.io.IOException, StandardException
Write a sequence of bytes at the given offset in a file. This method is not thread safe, so the caller must make sure that no other thread is performing operations that may change current position in the file.- Parameters:
file
- the file to write tobytes
- the bytes to writeoffset
- the offset to start writing at- Throws:
java.io.IOException
- if an I/O error occurs while writingStandardException
- Derby Standard error policy
-
getEmbryonicPage
protected byte[] getEmbryonicPage(java.io.DataInput fileData) throws java.io.IOException, StandardException
Get an embryonic page from the dataInput stream. The embryonic page will be read in from the input stream (fileData), which is assumed to be positioned at the beginning of the first allocation page.- Throws:
java.io.IOException
- error in read the embryonic page from fileStandardException
- Derby Standard error policy
-
getEmbryonicPage
byte[] getEmbryonicPage(StorageRandomAccessFile file, long offset) throws java.io.IOException, StandardException
Read an embryonic page (that is, a section of the first alloc page that is so large that we know all the borrowed space is included in it) from the specified offset in aStorageRandomAccessFile
. This method is not thread safe, so the caller must make sure that no other thread is performing operations that may change current position in the file.- Parameters:
file
- the file to read fromoffset
- where to start reading (normallyFileContainer.FIRST_ALLOC_PAGE_OFFSET
)- Returns:
- a byte array containing the embryonic page
- Throws:
java.io.IOException
- if an I/O error occurs while readingStandardException
- Derby Standard error policy
-
writeHeaderToArray
private void writeHeaderToArray(byte[] a) throws java.io.IOException
Write containerInfo into a byte array The container Header thus put together can be read by readHeaderFromArray.- Throws:
java.io.IOException
- error in writing the header
-
logCreateContainerInfo
protected ByteArray logCreateContainerInfo() throws StandardException
Log all information on the container creation necessary to recreate the container during a load tran.- Specified by:
logCreateContainerInfo
in classBaseContainer
- Throws:
StandardException
- Derby Standard error policy
-
createInfoFromLog
private void createInfoFromLog(ByteArray byteArray) throws StandardException
Set container properties from the passed in ByteArray, which is created by logCreateContainerInfo. This information is used to recreate the container during recovery load tran. The following container properties are set: pageSize spareSpace minimumRecordSize isReusableRecordId initialPages- Throws:
StandardException
-
createInfoFromProp
private void createInfoFromProp(java.util.Properties createArgs) throws StandardException
Set container properties from the passed in createArgs. The following container properties are set: pageSize spareSpace minimumRecordSize isReusableRecordId initialPages RESOLVE - in the future setting parameters should be overridable by sub-class, e.g. one implementation of Container may require a minimum page size of 4k.- Throws:
StandardException
-
canUpdate
protected boolean canUpdate()
Description copied from class:BaseContainer
Can the container be updated.- Specified by:
canUpdate
in classBaseContainer
- Returns:
- true if the container can be updated, false otherwise.
-
deallocatePage
protected void deallocatePage(BaseContainerHandle handle, BasePage page) throws StandardException
Deallocate a page from the container.- Specified by:
deallocatePage
in classBaseContainer
- Parameters:
handle
- the container handle doing the deallocationpage
- the page to be deallocated. It is latched upon entry and will be unlatched by the caller of this function- Throws:
StandardException
- Derby Standard error policy
-
deallocatePagenum
private void deallocatePagenum(BaseContainerHandle handle, long pnum) throws StandardException
deallocate the page from the alloc page- Throws:
StandardException
-
compressContainer
protected void compressContainer(RawTransaction ntt, BaseContainerHandle allocHandle) throws StandardException
Compress free space from container.
MT - thread aware - It is assumed that our caller (our super class) has already arranged a logical lock on page allocation to only allow a single thread through here. Compressing free space is done in allocation page units, working it's way from the end of the container to the beginning. Each loop operates on the last allocation page in the container. Freeing space in the container page involves 2 transactions, an update to an allocation page, N data pages, and possibly the delete of the allocation page. The User Transaction (UT) initiated the compress call. The Nested Top Transaction (NTT) is the transaction started by RawStore inside the compress call. This NTT is committed before compress returns. The NTT is used to access high traffic data structures such as the AllocPage. This is outline of the algorithm used in compressing the container. Until a non free page is found loop, in each loop return to the OS all space at the end of the container occupied by free pages, including the allocation page itself if all of it's pages are free. 1) Find last 2 allocation pages in container (last if there is only one). 2) invalidate the allocation information cached by the container. Without the cache no page can be gotten from the container. Pages already in the page cache are not affected. Thus by latching the allocPage and invalidating the allocation cache, this NTT blocks out all page gets from this container until it commits. 3) the allocPage determines which pages can be released to the OS, mark that in its data structure (the alloc extent). Mark the contiguous block of nallocated/free pages at the end of the file as unallocated. This change is associated with the NTT. 4) The NTT calls the OS to deallocate the space from the file. Note that the system can handle being booted and asked to get an allocated page which is past end of file, it just extends the file automatically. 5) If freeing all space on the alloc page, and there is more than one alloc page, then free the alloc page - this requires an update to the previous alloc page which the loop has kept latched also. 6) if the last alloc page was deleted, restart loop at #1 All NTT latches are released before this routine returns. If we use an NTT, the caller has to commit the NTT to release the allocPage latch. If we don't use an NTT, the allocPage latch is released as this routine returns.- Specified by:
compressContainer
in classBaseContainer
- Parameters:
ntt
- - the nested top transaction for the purpose of freeing space. If ntt is null, use the user transaction for allocation. #param allocHandle - the container handle opened by the ntt, use this to latch the alloc page- Throws:
StandardException
- Standard Derby error policy
-
getReusableRecordIdSequenceNumber
public final long getReusableRecordIdSequenceNumber()
Get the reusable RecordId sequence number for the container.- Specified by:
getReusableRecordIdSequenceNumber
in classBaseContainer
- Returns:
- reusable RecordId sequence number for the container.
- See Also:
BaseContainer.getReusableRecordIdSequenceNumber()
-
incrementReusableRecordIdSequenceNumber
protected final void incrementReusableRecordIdSequenceNumber()
Increment the reusable RecordId version sequence number.- Specified by:
incrementReusableRecordIdSequenceNumber
in classBaseContainer
-
newPage
protected BasePage newPage(BaseContainerHandle userHandle, RawTransaction ntt, BaseContainerHandle allocHandle, boolean isOverflow) throws StandardException
Create a new page in the container.
MT - thread aware - It is assumed that our caller (our super class) has already arranged a logical lock on page allocation to only allow a single thread through here. Adding a new page involves 2 transactions and 2 pages. The User Transaction (UT) initiated the addPage call and expects a latched page (owns by the UT) to be returned. The Nested Top Transaction (NTT) is the transaction started by RawStore inside an addPage call. This NTT is committed before the page is returned. The NTT is used to accessed high traffic data structure such as the AllocPage. This is outline of the algorithm used in adding a page: 1) find or make an allocPage which can handle the addding of a new page. Latch the allocPage with the NTT. 2) invalidate the allocation information cached by the container. Without the cache no page can be gotten from the container. Pages already in the page cache is not affected. Thus by latching the allocPage and invalidating the allocation cache, this NTT blocks out all page gets from this container until it commits. 3) the allocPage determines which page can be allocated, mark that in its data structure (the alloc extent) and returns the page number of the new page. This change is associated with the NTT. 4) the NTT gets or creates the new page in the page cache (bypassing the lookup of the allocPage since that is already latched by the NTT and will deadlock). 5) the NTT initializes the page (mark it is being a VALID page). 6) the page latch is transfered to the UT from the NTT. 7) the new page is returned, latched by UT If we use an NTT, the caller has to commit the NTT to release the allocPage latch. If we don't use an NTT, the allocPage latch is released as this routine returns.- Specified by:
newPage
in classBaseContainer
- Parameters:
userHandle
- - the container handle opened by the user transaction, use this to latch the new user pagentt
- - the nested top transaction for the purpose of allocating the new page If ntt is null, use the user transaction for allocation. #param allocHandle - the container handle opened by the ntt, use this to latch the alloc page- Throws:
StandardException
- Standard Derby error policy
-
clearPreallocThreshold
protected void clearPreallocThreshold()
Description copied from class:BaseContainer
The container will have no pre-allocate threshold, i.e., if the implementation supports it, page preallocation will happen the next time a new page is allocated.- Specified by:
clearPreallocThreshold
in classBaseContainer
-
prepareForBulkLoad
protected void prepareForBulkLoad(BaseContainerHandle handle, int numPage)
Description copied from class:BaseContainer
The container will be grown vastly, prepare for it.- Specified by:
prepareForBulkLoad
in classBaseContainer
-
pageValid
private boolean pageValid(BaseContainerHandle handle, long pagenum) throws StandardException
- Throws:
StandardException
-
getLastPageNumber
protected long getLastPageNumber(BaseContainerHandle handle) throws StandardException
- Throws:
StandardException
-
findAllocPageForAdd
private AllocPage findAllocPageForAdd(BaseContainerHandle allocHandle, RawTransaction ntt, long lastAllocatedPage) throws StandardException
- Throws:
StandardException
-
findLastAllocPage
private AllocPage findLastAllocPage(BaseContainerHandle handle, RawTransaction tran)
Find the last alloc page, returns null if no alloc page is found
-
makeAllocPage
private AllocPage makeAllocPage(RawTransaction ntt, BaseContainerHandle handle, long pageNumber, long pageOffset, int containerInfoSize) throws StandardException
- Throws:
StandardException
-
initPage
protected BasePage initPage(BaseContainerHandle allochandle, PageKey pkey, PageCreationArgs createArgs, long pageOffset, boolean reuse, boolean overflow) throws StandardException
Initialize a page- Parameters:
allochandle
- the contianer handle to initialize the page with - the nttpkey
- the page number of the page to be initializedcreateArgs
- the arguments for page creationreuse
- is true if we are reusing a page that has already been initialized once- Returns:
- a latched page that has been initialized.
- Throws:
StandardException
- Derby Standard error policy
-
getUserPage
private BasePage getUserPage(BaseContainerHandle handle, long pageNumber, boolean overflowOK, boolean wait) throws StandardException
Get a page in the container. Get User page is the generic base routine for all user (client to raw store) getPage. This routine coordinate with allocation/deallocation to ensure that no page can be gotten from the container while page is in the middle of being allocated or deallocated. This routine latches the page.- Parameters:
handle
- the container handlepageNumber
- the page number of the page to getoverflowOK
- if true then an overflow page is OK, if false, then only non-overflow page is OKwait
- if true then wait for a latch- Returns:
- the latched page
MT - thread safe - Throws:
StandardException
- Standard Derby error policy
-
trackUnfilledPage
protected void trackUnfilledPage(long pagenumber, boolean unfilled)
-
getPage
protected BasePage getPage(BaseContainerHandle handle, long pageNumber, boolean wait) throws StandardException
Get a valid (non-deallocated or free) page in the container. Overflow page is OK. Resulting page is latched.
MT - thread safe- Specified by:
getPage
in classBaseContainer
- Throws:
StandardException
- Standard Derby error policy
-
getAnyPage
protected BasePage getAnyPage(BaseContainerHandle handle, long pageNumber) throws StandardException
Get any old page - turn off all validation- Specified by:
getAnyPage
in classBaseContainer
- Throws:
StandardException
- Derby Standard error policy
-
reCreatePageForRedoRecovery
protected BasePage reCreatePageForRedoRecovery(BaseContainerHandle handle, int pageFormat, long pageNumber, long pageOffset) throws StandardException
ReCreate a page for rollforward recovery.During redo recovery it is possible for the system to try to redo the creation of a page (ie. going from non-existence to version 0). It first trys to read the page from disk, but a few different types of errors can occur: o the page does not exist at all on disk, this can happen during rollforward recovery applied to a backup where the file was copied and the page was added to the file during the time frame of the backup but after the physical file was copied. o space in the file exists, but it was never initalized. This can happen if you happen to crash at just the right moment during the allocation process. Also on some OS's it is possible to read from a part of the file that was not ever written - resulting in garbage from the store's point of view (often the result is all 0's). All these errors are easy to recover from as the system can easily create a version 0 from scratch and write it to disk. Because the system does not sync allocation of data pages, it is also possible at this point that whlie writing the version 0 to disk to create it we may encounter an out of disk space error (caught in this routine as a StandardException from the create() call. We can't recovery from this without help from outside, so the caught exception is nested and a new exception thrown which the recovery system will output to the user asking them to check their disk for space/errors.
- Specified by:
reCreatePageForRedoRecovery
in classBaseContainer
- Throws:
StandardException
- Standard exception policy.
-
getAllocPage
protected BasePage getAllocPage(long pageNumber) throws StandardException
Get an alloc page - only accessible to the raw store (container and recovery)- Specified by:
getAllocPage
in classBaseContainer
- Throws:
StandardException
- Derby Standard error policy
-
getHeadPage
protected BasePage getHeadPage(BaseContainerHandle handle, long pageNumber, boolean wait) throws StandardException
Get only a valid, non-overflow page. If page number is either invalid or overflow, returns null- Specified by:
getHeadPage
in classBaseContainer
- Throws:
StandardException
- Derby Standard error policy
-
getFirstHeadPage
protected BasePage getFirstHeadPage(BaseContainerHandle handle, boolean wait) throws StandardException
Get the first valid page in the container- Specified by:
getFirstHeadPage
in classBaseContainer
- Throws:
StandardException
- Derby Standard error policy
-
getNextHeadPage
protected BasePage getNextHeadPage(BaseContainerHandle handle, long pageNumber, boolean wait) throws StandardException
Get the next page in the container.- Specified by:
getNextHeadPage
in classBaseContainer
- Throws:
StandardException
- Standard Derby error policy
-
getInsertablePage
private BasePage getInsertablePage(BaseContainerHandle handle, long pageNumber, boolean wait, boolean overflowOK) throws StandardException
- Throws:
StandardException
-
getPageForCompress
protected BasePage getPageForCompress(BaseContainerHandle handle, int flag, long pageno) throws StandardException
Get candidate page to move a row for compressing the table.The caller is moving rows from the end of the table toward the beginning, with the goal of freeing up a block of empty pages at the end of the container which can be returned to the OS.
On entry pageno will be latched by the caller. Only return pages with numbers below pageno. Attempting to return pageno will result in a latch/latch deadlock on the same thread.
- Specified by:
getPageForCompress
in classBaseContainer
- Throws:
StandardException
- Standard exception policy.
-
getPageForInsert
protected BasePage getPageForInsert(BaseContainerHandle handle, int flag) throws StandardException
Get a potentially suitable page for insert and latch it.- Specified by:
getPageForInsert
in classBaseContainer
- Throws:
StandardException
- Standard Derby error policy
-
getLatchedPage
protected BasePage getLatchedPage(BaseContainerHandle handle, long pageNumber) throws StandardException
Get a latched page. Incase of backup page Latch is necessary to prevent modification to the page when it is being written to the backup. Backup process relies on latches to get consistent snap shot of the page , user level table/page/row locks are NOT acquired by the online backup mechanism.- Parameters:
handle
- the container handle used to latch the pagepageNumber
- the page number of the page to get- Returns:
- the latched page
- Throws:
StandardException
- Standard Derby error policy
-
getUnfilledPageNumber
private long getUnfilledPageNumber(BaseContainerHandle handle, long pagenum) throws StandardException
- Throws:
StandardException
-
getEstimatedRowCount
public long getEstimatedRowCount(int flag)
MT - this routine is NOT MT-safe and clients don't need to provide synchronization.- Specified by:
getEstimatedRowCount
in classBaseContainer
- See Also:
ContainerHandle.getEstimatedRowCount(int)
-
setEstimatedRowCount
public void setEstimatedRowCount(long count, int flag)
- Specified by:
setEstimatedRowCount
in classBaseContainer
- See Also:
ContainerHandle.setEstimatedRowCount(long, int)
-
updateEstimatedRowCount
protected void updateEstimatedRowCount(int delta)
Update estimated row count by page as it leaves the cache. The estimated row count is updated without logging!
-
getEstimatedPageCount
public long getEstimatedPageCount(BaseContainerHandle handle, int flag) throws StandardException
- Specified by:
getEstimatedPageCount
in classBaseContainer
- Throws:
StandardException
- Standard Derby error policy- See Also:
ContainerHandle.getEstimatedPageCount(int)
-
readPage
protected abstract void readPage(long pageNumber, byte[] pageData) throws java.io.IOException, StandardException
Read a page into the supplied array.
MT - thread safe- Throws:
java.io.IOException
- error reading pageStandardException
- standard Derby error message
-
writePage
protected abstract void writePage(long pageNumber, byte[] pageData, boolean syncPage) throws java.io.IOException, StandardException
Write a page from the supplied array.
MT - thread safe- Throws:
java.io.IOException
- error writing pageStandardException
- Standard Derby error policy
-
decryptPage
protected void decryptPage(byte[] pageData, int pageSize) throws StandardException
Decrypts a page
MT - MT safe.- Throws:
StandardException
- Standard Derby error policy
-
encryptPage
protected byte[] encryptPage(byte[] pageData, int pageSize, byte[] encryptionBuffer, boolean newEngine) throws StandardException
Encrypts a page.
MT - not safe, call within synchronized block and only use the returned byte array withing synchronized block.- Throws:
StandardException
- Standard Derby error policy
-
getEncryptionBuffer
protected byte[] getEncryptionBuffer()
Get encryption buffer. MT - not safe, call within synchronized block and only use the returned byte array withing synchronized block.- Returns:
- byte array to be used for encryping a page.
-
preAllocate
protected abstract int preAllocate(long lastPreallocPagenum, int preAllocSize)
preAllocate writes out the preallocated pages to disk if necessary.
Make sure the container is large enough and the pages are well formatted. The only reason to do this is to save some I/O during page initialization. Once the initPage log record is written, it is expected that the page really do exist and is well formed or recovery will fail. However, we can gain some performance by writing a bunch of pages at a time rather than one at a time.
If it doesn't make sense for the the implementation to have pre-allocation, just return 0.
If the container is not being logged, don't actually do anything, just return 0.- Parameters:
lastPreallocPagenum
- the last preallocated page number as known by the allocation pagepreAllocSize
- try to preallocate this page number of pages. Since only the container knows how many pages are actually on disk, it may determine that certain number of pages that the allocation page thinks need to be preallocated is already allocated, in those case, act as if the preallocation is successful.- Returns:
- number of successfully preallocated page, or 0 if no page has been preallocated
-
doPreAllocatePages
protected int doPreAllocatePages(long lastPreallocPagenum, int preAllocSize)
Preallocate the pages - actually doing it, called by subclass only
-
getPageSize
protected int getPageSize()
-
getSpareSpace
protected int getSpareSpace()
-
getMinimumRecordSize
protected int getMinimumRecordSize()
-
switchToMultiInsertPageMode
private void switchToMultiInsertPageMode(BaseContainerHandle handle) throws StandardException
- Throws:
StandardException
-
getLastInsertedPage
private long getLastInsertedPage()
-
getLastUnfilledPage
private long getLastUnfilledPage()
-
initializeLastInsertedPage
private void initializeLastInsertedPage(int size)
-
setLastInsertedPage
private void setLastInsertedPage(long val)
-
setLastUnfilledPage
private void setLastUnfilledPage(long val)
-
letGo
protected void letGo(BaseContainerHandle handle)
The container is kept by the find() in File.openContainer.- Overrides:
letGo
in classBaseContainer
-
latchPage
protected BasePage latchPage(BaseContainerHandle handle, BasePage foundPage, boolean wait) throws StandardException
- Overrides:
latchPage
in classBaseContainer
- Throws:
StandardException
-
backupContainer
protected abstract void backupContainer(BaseContainerHandle handle, java.lang.String backupLocation) throws StandardException
backup the container.- Specified by:
backupContainer
in classBaseContainer
- Parameters:
handle
- the container handle.backupLocation
- location of the backup container.- Throws:
StandardException
- Standard Derby error policy
-
getContextService
static ContextService getContextService()
Privileged lookup of the ContextService. Must be limited to package visibility so that user code can't call this entry point.
-
getServiceModule
private static java.lang.Object getServiceModule(java.lang.Object serviceModule, java.lang.String factoryInterface)
Privileged module lookup. Must be private so that user code can't call this entry point.
-
-