Class RAFContainer4
- java.lang.Object
-
- org.apache.derby.impl.store.raw.data.BaseContainer
-
- org.apache.derby.impl.store.raw.data.FileContainer
-
- org.apache.derby.impl.store.raw.data.RAFContainer
-
- org.apache.derby.impl.store.raw.data.RAFContainer4
-
- All Implemented Interfaces:
java.security.PrivilegedExceptionAction<java.lang.Object>
,Cacheable
,TypedFormat
,Lockable
class RAFContainer4 extends RAFContainer
RAFContainer4 overrides a few methods in FileContainer/RAFContainer in order to use FileChannel from Java 1.4's New IO framework to issue multiple IO operations to the same file concurrently instead of strictly serializing IO operations using a mutex on the container object. Since we compile with Java 1.4, the override "annotations" are inside the method javadoc headers.Note that our requests for multiple concurrent IOs may be serialized further down in the IO stack - this is entirely up to the JVM and OS. However, at least in Linux on Sun's 1.4.2_09 JVM we see the desired behavior: The FileChannel.read/write(ByteBuffer buf, long position) calls map to pread/pwrite system calls, which enable efficient IO to the same file descriptor by multiple threads.
This whole class should be merged back into RAFContainer when Derby officially stops supporting Java 1.3.
Significant behavior changes from RAFContainer:
- Multiple concurrent IOs permitted.
- State changes to the container (create, open, close) can now happen while IO is in progress due to the lack of locking. Closing a container while IO is in progress will cause IOExceptions in the thread calling readPage or writePage. If this happens something is probably amiss anyway. The iosInProgress variable is used in an attempt to detect this should it happen while running a debug build.
- See Also:
FileChannel
-
-
Field Summary
Fields Modifier and Type Field Description private java.lang.Object
channelCleanupMonitor
private ContainerKey
currentIdentity
private boolean
giveUpIO
private java.lang.Object
giveUpIOm
private int
iosInProgress
For debugging - will be incremented when an IO is started, decremented when it is done.private java.nio.channels.FileChannel
ourChannel
This channel will be retrieved from RAFContainer's fileData member when fileData is set.private boolean
restoreChannelInProgress
private int
threadsInPageIO
-
Fields inherited from class org.apache.derby.impl.store.raw.data.RAFContainer
fileData, needsSync
-
Fields inherited from class org.apache.derby.impl.store.raw.data.FileContainer
allocCache, canUpdate, CHECKSUM_SIZE, CONTAINER_INFO_SIZE, containerCache, containerInfo, containerVersion, dataFactory, estimatedRowCount, FIRST_ALLOC_PAGE_NUMBER, FIRST_ALLOC_PAGE_OFFSET, firstAllocPageNumber, firstAllocPageOffset, formatIdInteger, initialPages, isDirty, lastLogInstant, minimumRecordSize, pageCache, pageSize, preDirty, SPACE_TRACE, spareSpace
-
Fields inherited from class org.apache.derby.impl.store.raw.data.BaseContainer
identity, isCommittedDrop, isDropped, isReusableRecordId
-
-
Constructor Summary
Constructors Constructor Description RAFContainer4(BaseDataFileFactory factory)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description private void
awaitRestoreChannel(java.lang.Exception e, boolean stealthMode)
Use when seeing an exception during IO and when another thread is presumably doing the recovery.(package private) void
closeContainer()
override of RAFContainer#closeContainer(package private) void
createContainer(ContainerKey newIdentity)
override of RAFContainer#createContainerprivate static void
debugTrace(java.lang.String msg)
private java.nio.channels.FileChannel
getChannel()
Return the file channel for the current value of thefileData
field.private java.nio.channels.FileChannel
getChannel(StorageRandomAccessFile file)
Return theFileChannel
for the specifiedStorageRandomAccessFile
if it is aRandomAccessFile
.(package private) byte[]
getEmbryonicPage(StorageRandomAccessFile file, long offset)
Read an embryonic page (that is, a section of the first alloc page that is so large that we know all the borrowed space is included in it) from the specified offset in aStorageRandomAccessFile
.private void
handleClosedChannel(java.nio.channels.ClosedChannelException e, boolean stealthMode, int retries)
This method handles what to do when, during a NIO operation we receive aClosedChannelException
.(package private) boolean
openContainer(ContainerKey newIdentity)
Open a container.private void
readFull(java.nio.ByteBuffer dstBuffer, java.nio.channels.FileChannel srcChannel, long position)
Attempts to fill buf completely from start until it's full.protected void
readPage(long pageNumber, byte[] pageData)
Read a page into the supplied array.private void
readPage(long pageNumber, byte[] pageData, long offset)
Read a page into the supplied array.private void
readPage0(long pageNumber, byte[] pageData, long offset)
private boolean
recoverContainerAfterInterrupt(java.lang.String whence, boolean stealthMode)
Use this when the thread has received a ClosedByInterruptException (or, prior to JDK 1.7 it may also be AsynchronousCloseException - a bug) exception during IO and its interruped flag is also set.private void
reopen()
When the existing channel (ourChannel
) has been closed due to interrupt, we need to reopen the underlying RAF to get a fresh channel so we can resume IO.(package private) void
writeAtOffset(StorageRandomAccessFile file, byte[] bytes, long offset)
Write a sequence of bytes at the given offset in a file.private void
writeFull(java.nio.ByteBuffer srcBuffer, java.nio.channels.FileChannel dstChannel, long position)
Attempts to write buf completely from start until end, at the given position in the destination fileChannel.protected void
writePage(long pageNumber, byte[] pageData, boolean syncPage)
Write a page from the supplied array.private void
writePage0(long pageNumber, byte[] pageData, boolean syncPage)
-
Methods inherited from class org.apache.derby.impl.store.raw.data.RAFContainer
backupContainer, clean, encryptOrDecryptContainer, flushAll, getFileName, getRandomAccessFile, isDirty, preAllocate, privGetFileName, removeContainer, removeFile, reopenContainer, run, truncatePages, updatePageArray
-
Methods inherited from class org.apache.derby.impl.store.raw.data.FileContainer
bumpContainerVersion, canUpdate, clearIdentity, clearPreallocThreshold, compressContainer, createIdent, createIdentity, deallocatePage, decryptPage, doPreAllocatePages, dropContainer, encryptPage, getAllocPage, getAnyPage, getContainerProperties, getContainerVersion, getContextService, getEmbryonicPage, getEncryptionBuffer, getEstimatedPageCount, getEstimatedRowCount, getFirstHeadPage, getHeadPage, getLastPageNumber, getLatchedPage, getMinimumRecordSize, getNextHeadPage, getPage, getPageForCompress, getPageForInsert, getPageSize, getReusableRecordIdSequenceNumber, getSpaceInfo, getSpareSpace, getTypeFormatId, incrementReusableRecordIdSequenceNumber, initPage, latchPage, letGo, logCreateContainerInfo, newPage, preDirty, prepareForBulkLoad, readHeader, reCreatePageForRedoRecovery, setDirty, setEstimatedRowCount, setIdent, setIdentity, trackUnfilledPage, updateEstimatedRowCount, writeHeader, writeHeader
-
Methods inherited from class org.apache.derby.impl.store.raw.data.BaseContainer
addPage, compressContainer, fillInIdentity, getAllocPage, getAnyPage, getCommittedDropState, getContainerId, getContainerStatus, getDeallocLock, getDroppedState, getFirstPage, getIdentity, getNextPage, getSegmentId, isReusableRecordId, lockAttributes, lockerAlwaysCompatible, lockEvent, removePage, requestCompatible, setCommittedDropState, setDroppedState, setReusableRecordIdState, truncate, unlockEvent, use
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.derby.iapi.services.cache.Cacheable
getIdentity
-
-
-
-
Field Detail
-
ourChannel
private java.nio.channels.FileChannel ourChannel
This channel will be retrieved from RAFContainer's fileData member when fileData is set. We wrap a couple of RAFContainer's methods to accomplish this.
-
channelCleanupMonitor
private final java.lang.Object channelCleanupMonitor
-
threadsInPageIO
private volatile int threadsInPageIO
-
restoreChannelInProgress
private volatile boolean restoreChannelInProgress
-
giveUpIO
private boolean giveUpIO
-
giveUpIOm
private final java.lang.Object giveUpIOm
-
iosInProgress
private int iosInProgress
For debugging - will be incremented when an IO is started, decremented when it is done. Should be == 0 when container state is changed.
-
currentIdentity
private ContainerKey currentIdentity
-
-
Constructor Detail
-
RAFContainer4
public RAFContainer4(BaseDataFileFactory factory)
-
-
Method Detail
-
getChannel
private java.nio.channels.FileChannel getChannel(StorageRandomAccessFile file)
Return theFileChannel
for the specifiedStorageRandomAccessFile
if it is aRandomAccessFile
. Otherwise, returnnull
.- Parameters:
file
- the file to get the channel for- Returns:
- a
FileChannel
iffile
is an instance ofRandomAccessFile
,null
otherwise
-
getChannel
private java.nio.channels.FileChannel getChannel()
Return the file channel for the current value of the
fileData
field. IffileData
doesn't support file channels, returnnull
.Callers of this method must synchronize on the container object since two shared fields (
fileData
andourChannel
) are accessed.- Returns:
- a
FileChannel
object, if supported, ornull
-
openContainer
boolean openContainer(ContainerKey newIdentity) throws StandardException
Description copied from class:FileContainer
Open a container.Longer descrption of routine.
Open a container. Open the file that maps to this container, if the file does not exist then we assume the container was never created. If the file exists but we have trouble opening it then we throw some exception.
MT - single thread required - Enforced by cache manager.- Overrides:
openContainer
in classRAFContainer
- Throws:
StandardException
- Standard exception policy.
-
createContainer
void createContainer(ContainerKey newIdentity) throws StandardException
override of RAFContainer#createContainer- Overrides:
createContainer
in classRAFContainer
- Throws:
StandardException
- Derby Standard error policy
-
reopen
private void reopen() throws StandardException
When the existing channel (ourChannel
) has been closed due to interrupt, we need to reopen the underlying RAF to get a fresh channel so we can resume IO.- Throws:
StandardException
-
closeContainer
void closeContainer()
override of RAFContainer#closeContainer- Overrides:
closeContainer
in classRAFContainer
-
readPage
protected void readPage(long pageNumber, byte[] pageData) throws java.io.IOException, StandardException
Read a page into the supplied array. override of RAFContainer#readPage
MT - thread safe- Overrides:
readPage
in classRAFContainer
- Throws:
java.io.IOException
- exception reading pageStandardException
- Standard Derby error policy
-
readPage
private void readPage(long pageNumber, byte[] pageData, long offset) throws java.io.IOException, StandardException
Read a page into the supplied array. override of RAFContainer#readPage
MT - thread safe- Parameters:
pageNumber
- the page number to read data from, or -1 (called from getEmbryonicPage)pageData
- the buffer to read data intooffset
- -1 normally (not used since offset is computed from pageNumber), but used if pageNumber == -1 (getEmbryonicPage)- Throws:
java.io.IOException
- exception reading pageStandardException
- Standard Derby error policy
-
readPage0
private void readPage0(long pageNumber, byte[] pageData, long offset) throws java.io.IOException, StandardException
- Throws:
java.io.IOException
StandardException
-
writePage
protected void writePage(long pageNumber, byte[] pageData, boolean syncPage) throws java.io.IOException, StandardException
Write a page from the supplied array. override of RAFContainer#writePage
MT - thread safe- Overrides:
writePage
in classRAFContainer
- Throws:
StandardException
- Standard Derby error policyjava.io.IOException
- IO error accessing page
-
handleClosedChannel
private void handleClosedChannel(java.nio.channels.ClosedChannelException e, boolean stealthMode, int retries) throws StandardException
This method handles what to do when, during a NIO operation we receive a
ClosedChannelException
. Note the specialization hierarchy:ClosedChannelException
->AsynchronousCloseException
->ClosedByInterruptException
If
e
is a ClosedByInterruptException, we normally start container recovery, i.e. we need to reopen the random access file so we get get a new interruptible channel and continue IO.If
e
is aAsynchronousCloseException
or a plainClosedChannelException
, the behavior depends ofstealthMode
:If
stealthMode == false
, the method will wait for another thread tp finish recovering the IO channel before returning.If
stealthMode == true
, the method throwsInterruptDetectedException
, allowing retry at a higher level in the code. The reason for this is that we sometimes need to release monitors on objects needed by the recovery thread.- Parameters:
e
- Should be an instance ofClosedChannelException
.stealthMode
- Iftrue
, do retry at a higher levelretries
- Give up waiting for another thread to reopen the channel whenretries
reaches 0. Only applicable ifstealthMode == false
.- Throws:
InterruptDetectedException
- if retry at higher level is requiredstealthMode == true
.StandardException
- standard error policy, incl. when we give up waiting for another thread to reopen channel
-
awaitRestoreChannel
private void awaitRestoreChannel(java.lang.Exception e, boolean stealthMode) throws StandardException
Use when seeing an exception during IO and when another thread is presumably doing the recovery. IfstealthMode == false
, wait for another thread to recover the container after an interrupt. IfstealthMode == true
, throw internal exceptionInterruptDetectedException
to do retry from higher in the stack. IfstealthMode == false
, maximum wait time for the container to become available again is determined by the productInterruptStatus.MAX_INTERRUPT_RETRIES * InterruptStatus.INTERRUPT_RETRY_SLEEP
. There is a chance this thread will not see any recovery occuring (yet), in which case it waits for a bit and just returns, so the caller must retry IO until success. If for some reason the recovering thread has given up on resurrecting the container, cf#giveUpIO
, the method throwsFILE_IO_INTERRUPTED
.- Parameters:
e
- the exception we saw during IOstealthMode
- true if the thread doing IO in stealth mode- Throws:
StandardException
-InterruptDetectedException
and normal error policy
-
recoverContainerAfterInterrupt
private boolean recoverContainerAfterInterrupt(java.lang.String whence, boolean stealthMode) throws StandardException
Use this when the thread has received a ClosedByInterruptException (or, prior to JDK 1.7 it may also be AsynchronousCloseException - a bug) exception during IO and its interruped flag is also set. This makes this thread a likely candicate to do container recovery, unless another thread started it already, cf. return value.- Parameters:
whence
- caller site (debug info)stealthMode
- don't update threadsInPageIO if true- Returns:
- true if we did recovery, false if we saw someone else do it and abstained
- Throws:
StandardException
-
writePage0
private void writePage0(long pageNumber, byte[] pageData, boolean syncPage) throws java.io.IOException, StandardException
- Throws:
java.io.IOException
StandardException
-
writeAtOffset
void writeAtOffset(StorageRandomAccessFile file, byte[] bytes, long offset) throws java.io.IOException, StandardException
Write a sequence of bytes at the given offset in a file. This method operates in stealth mode, see doc forhandleClosedChannel
. This presumes that IO retry happens at a higher level, i.e. the caller(s) must be prepared to handleInterruptDetectedException
. This method overrides FileContainer#writeAtOffset.- Overrides:
writeAtOffset
in classFileContainer
- Parameters:
file
- the file to write tobytes
- the bytes to writeoffset
- the offset to start writing at- Throws:
java.io.IOException
- if an I/O error occurs while writingStandardException
- Derby Standard error policy
-
getEmbryonicPage
byte[] getEmbryonicPage(StorageRandomAccessFile file, long offset) throws java.io.IOException, StandardException
Read an embryonic page (that is, a section of the first alloc page that is so large that we know all the borrowed space is included in it) from the specified offset in aStorageRandomAccessFile
. override of FileContainer#getEmbryonicPage- Overrides:
getEmbryonicPage
in classFileContainer
- Parameters:
file
- the file to read fromoffset
- where to start reading (normallyFileContainer.FIRST_ALLOC_PAGE_OFFSET
)- Returns:
- a byte array containing the embryonic page
- Throws:
java.io.IOException
- if an I/O error occurs while readingStandardException
- if thread is interrupted.
-
readFull
private void readFull(java.nio.ByteBuffer dstBuffer, java.nio.channels.FileChannel srcChannel, long position) throws java.io.IOException, StandardException
Attempts to fill buf completely from start until it's full. FileChannel has no readFull() method, so we roll our own.- Parameters:
dstBuffer
- buffer to read intosrcChannel
- channel to read fromposition
- file position from where to read- Throws:
java.io.IOException
- if an I/O error occurs while readingStandardException
- If thread is interrupted.
-
writeFull
private void writeFull(java.nio.ByteBuffer srcBuffer, java.nio.channels.FileChannel dstChannel, long position) throws java.io.IOException
Attempts to write buf completely from start until end, at the given position in the destination fileChannel. FileChannel has no writeFull() method, so we roll our own.- Parameters:
srcBuffer
- buffer to writedstChannel
- channel to write toposition
- file position to start writing at- Throws:
java.io.IOException
- if an I/O error occurs while writingStandardException
- If thread is interrupted.
-
debugTrace
private static void debugTrace(java.lang.String msg)
-
-