Class LogToFile
- java.lang.Object
-
- org.apache.derby.impl.store.raw.log.LogToFile
-
- All Implemented Interfaces:
java.security.PrivilegedExceptionAction<java.lang.Object>
,Serviceable
,ModuleControl
,ModuleSupportable
,Corruptable
,LogFactory
public final class LogToFile extends java.lang.Object implements LogFactory, ModuleControl, ModuleSupportable, Serviceable, java.security.PrivilegedExceptionAction<java.lang.Object>
This is an implementation of the log using a non-circular file system file. No support for incremental log backup or media recovery. Only crash recovery is supported.The 'log' is a stream of log records. The 'log' is implemented as a series of numbered log files. These numbered log files are logically continuous so a transaction can have log records that span multiple log files. A single log record cannot span more then one log file. The log file number is monotonically increasing.
The log belongs to a log factory of a RawStore. In the current implementation, each RawStore only has one log factory, so each RawStore only has one log (which composed of multiple log files). At any given time, a log factory only writes new log records to one log file, this log file is called the 'current log file'.
A log file is named loglogNumber.dat
Everytime a checkpoint is taken, a new log file is created and all subsequent log records will go to the new log file. After a checkpoint is taken, old and useless log files will be deleted.
RawStore exposes a checkpoint method which clients can call, or a checkpoint is taken automatically by the RawStore when
- the log file grows beyond a certain size (configurable, default 100K bytes)
- RawStore is shutdown and a checkpoint hasn't been done "for a while"
- RawStore is recovered and a checkpoint hasn't been done "for a while"
This LogFactory is responsible for the formats of 2 kinds of file: the log file and the log control file. And it is responsible for the format of the log record wrapper.
Format of log control file
FILE_STREAM_LOG_FILEnone. The format is implied by the FILE_STREAM_LOG_FILEFormat ID FILE_STREAM_LOG_FILE Purpose The log control file contains information about which log files are present and where the last checkpoint log record is located. The log file contains log record which record all the changes to the database. The complete transaction log is composed of a series of log files. The log record wrapper provides information for the log scan. Upgrade int format id - the format Id of this log file int obsolete log file version - not used long log file number - this number orders the log files in a series to form the complete transaction log long prevLogRecord - log instant of the previous log record, in the previous log file. [log record wrapper]* one or more log records with wrapper int endMarker - value of zero. The beginning of a log record wrapper is the length of the log record, therefore it is never zero [int fuzzy end]* zero or more int's of value 0, in case this log file has been recovered and any incomplete log record set to zero.length(int) length of the log record (for forward scan) instant(long) LogInstant of the log record logRecord(byte[length]) byte array that is written by the FileLogger length(int) length of the log record (for backward scan)Disk Layout int format id int obsolete log file version long the log instant (LogCounter) of the last completed checkpoint (logfile counter, position) int Derby major version int Derby minor version int subversion revision/build number byte Flags (beta flag (0 or 1), test durability flag (0 or 1)) byte spare (0) byte spare (0) byte spare (0) long spare (value set to 0) long checksum for control data written
-
-
Field Summary
Fields Modifier and Type Field Description private int
action
private StorageFile
activeFile
private java.lang.String
activePerms
private long
allowedToReadFileNumber
private boolean
backupInProgress
(package private) long
bootTimeLogFileNumber
private static int
CHECKPOINT_INTERVAL_MAX
private static int
CHECKPOINT_INTERVAL_MIN
private DaemonService
checkpointDaemon
private boolean
checkpointDaemonCalled
(package private) long
checkpointInstant
private int
checkpointInterval
private java.util.zip.CRC32
checksum
protected StandardException
corrupt
If not null then something is corrupt in the raw store and this represents the original error.private CheckpointOperation
currentCheckpoint
private boolean
databaseEncrypted
(package private) java.lang.String
dataDirectory
protected DataFactory
dataFactory
static java.lang.String
DBG_FLAG
private static int
DEFAULT_CHECKPOINT_INTERVAL
private static java.lang.String
DEFAULT_LOG_ARCHIVE_DIRECTORY
private static int
DEFAULT_LOG_BUFFER_SIZE
private static int
DEFAULT_LOG_SWITCH_INTERVAL
static java.lang.String
DUMP_LOG_FROM_LOG_FILE
static java.lang.String
DUMP_LOG_ONLY
protected long
endPosition
private static int
fid
private StorageRandomAccessFile
firstLog
(package private) long
firstLogFileNumber
private boolean
inCheckpoint
private boolean
inLogSwitch
private boolean
inRedo
private boolean
inReplicationMasterMode
private boolean
inReplicationSlaveMode
private boolean
inReplicationSlavePreMode
True if the database has been booted in replication slave pre mode, effectively turning off writes to the log file.private static long
INT_LENGTH
private static byte
IS_BETA_FLAG
private static byte
IS_DURABILITY_TESTMODE_NO_SYNC_FLAG
When the derby.system.durability property is set to 'test', the store system will not force sync calls in the following cases - for the log file at each commit - for the log file before data page is forced to disk - for page allocation when file is grown - for data writes during checkpoint This means it is possible that the recovery system may not work properly, committed transactions may be lost, and/or database may not be in a consistent state.private boolean
isFrozen
If frozen, don't allow anything on disk to change.private boolean
isWriteSynced
Note: Why logging system support file sync and write sync ?(package private) ProductVersionHolder
jbmsVersion
Product Version information.private boolean
jvmSyncErrorChecked
Status for whether the check on the sync error on some JVMs has been done or not.private boolean
keepAllLogs
(package private) long
lastFlush
private static int
LOG_BUFFER_SIZE_MAX
private static int
LOG_BUFFER_SIZE_MIN
protected static int
LOG_FILE_HEADER_PREVIOUS_LOG_INSTANT_OFFSET
static int
LOG_FILE_HEADER_SIZE
static int
LOG_RECORD_OVERHEAD
private static int
LOG_SWITCH_INTERVAL_MAX
private static int
LOG_SWITCH_INTERVAL_MIN
protected static java.lang.String
LOG_SYNC_STATISTICS
private boolean
logArchived
private boolean
logBeingFlushed
private int
logBufferSize
(package private) java.lang.String
logDevice
(package private) long
logFileNumber
private long
logFileToBackup
private boolean
logNotSynced
protected LogAccessFile
logOut
private WritableStorageFactory
logStorageFactory
private int
logSwitchInterval
private boolean
logSwitchRequired
private long
logWrittenFromLastCheckPoint
private MasterFactory
masterFactory
private long
maxLogFileNumber
private int
mon_flushCalls
private boolean
mon_LogSyncStatistics
private int
mon_numBytesToLog
private int
mon_numLogFlushWaits
private int
mon_syncCalls
private int
myClientNumber
private static int
OBSOLETE_LOG_VERSION_NUMBER
private boolean
onDiskBeta
private int
onDiskMajorVersion
On disk database version information.private int
onDiskMinorVersion
private RawStoreFactory
rawStoreFactory
protected boolean
ReadOnlyDB
private boolean
recoveryNeeded
private StandardException
replicationSlaveException
If this exception is set while in replication slave mode, the exception will be thrown by the thread doing recovery will.private java.lang.Object
slaveRecoveryMonitor
private boolean
stopped
static java.lang.String
TEST_LOG_FULL
Set to true if we want to simulate a log full conditionstatic java.lang.String
TEST_LOG_INCOMPLETE_LOG_WRITE
Set to true if we want the upcoming log record to be only partially written.static java.lang.String
TEST_LOG_PARTIAL_LOG_WRITE_NUM_BYTES
Set to the number of bytes we want the next log record to actually write out, only used when TEST_LOG_INCOMPLETE_LOG_WRITE is on.static java.lang.String
TEST_LOG_SWITCH_LOG
Set to true if we want the checkpoint to only switch the log but not actually do the checkpoint(package private) int
test_logWritten
DEBUG test onlystatic java.lang.String
TEST_MAX_LOGFILE_NUMBER
Set to true if we want to simulate max possible log file number is being used.(package private) int
test_numRecordToFillLog
static java.lang.String
TEST_RECORD_TO_FILL_LOG
Set to the number of log record we want to write before the log is simulated to be full.static java.lang.String
TEST_SWITCH_LOG_FAIL1
Set to true if we want to simulate a log full condition while switching logstatic java.lang.String
TEST_SWITCH_LOG_FAIL2
private java.io.File
toFile
private static boolean
wasDBInDurabilityTestModeNoSync
keeps track of if the database was booted previously at any time with derby.system.durability=test-
Fields inherited from interface org.apache.derby.iapi.store.raw.log.LogFactory
LOG_DIRECTORY_NAME, MODULE, RT_READONLY, RUNTIME_ATTRIBUTES
-
Fields inherited from interface org.apache.derby.iapi.services.daemon.Serviceable
DONE, REQUEUE
-
-
Constructor Summary
Constructors Constructor Description LogToFile()
MT- not needed for constructor
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
abortLogBackup()
long
appendLogRecord(byte[] data, int offset, int length, byte[] optionalData, int optionalDataOffset, int optionalDataLength)
Append length bytes of data to the log prepended by a long log instant and followed by 4 bytes of length information.private void
backupLogFiles(java.io.File toDir, long lastLogFileToBackup)
void
boot(boolean create, java.util.Properties startParams)
Boot up the log factory.boolean
canSupport(java.util.Properties startParams)
See if this implementation can support any attributes that are listed in properties.private void
checkCorrupt()
protected void
checkForReplication(LogAccessFile log)
Used by LogAccessFile to check if it should take the replication master role, and thereby send log records to the MasterFactory.private boolean
checkJvmSyncError(StorageFile logFile)
In Java 1.4.2 and newer rws and rwd modes for RandomAccessFile are supported.boolean
checkpoint(RawStoreFactory rsf, DataFactory df, TransactionFactory tf, boolean wait)
Checkpoint the rawStore.void
checkpointInRFR(LogInstant cinstant, long redoLWM, long undoLWM, DataFactory df)
redo a checkpoint during rollforward recoveryprivate boolean
checkpointWithTran(RawTransaction cptran, RawStoreFactory rsf, DataFactory df, TransactionFactory tf, boolean wait)
checkpoint with pre-start transaction(package private) boolean
checkVersion(int requiredMajorVersion, int requiredMinorVersion)
Check to see if a database has been upgraded to the required level in order to use a store feature.boolean
checkVersion(int requiredMajorVersion, int requiredMinorVersion, java.lang.String feature)
Check to see if a database has been upgraded to the required level in order to use a store feature.void
createDataWarningFile()
Create readme file in log directory warning users against touching any files in the directoryprivate void
createLogDirectory()
Create the directory where transaction log should go.protected long
currentInstant()
Get the current log instant - this is the log instant of the Next log record to be written outboolean
databaseEncrypted()
int
decrypt(byte[] ciphertext, int offset, int length, byte[] cleartext, int outputOffset)
void
deleteLogFileAfterCheckpointLogFile()
private void
deleteObsoleteLogfiles()
void
deleteOnlineArchivedLogFiles()
void
disableLogArchiveMode()
void
enableLogArchiveMode()
int
encrypt(byte[] cleartext, int offset, int length, byte[] ciphertext, int outputOffset)
void
endLogBackup(java.io.File toDir)
protected long
endPosition()
void
failoverSlave()
Used to make the slave stop appending log records, complete recovery and boot the database.private CheckpointOperation
findCheckpoint(long checkpointInstant, FileLogger logger)
private static java.lang.Object
findServiceModule(java.lang.Object serviceModule, java.lang.String factoryInterface)
Privileged startup.private long
firstLogInstant()
Get the first valid log instant - this is the beginning of the first log fileprotected void
flush(long fileNumber, long wherePosition)
Flush the log such that the log record written with the instant wherePosition is guaranteed to be on disk.void
flush(LogInstant where)
Flush all unwritten log record up to the log instance indicated to disk and sync.void
flushAll()
Flush all unwritten log record to disk and sync.private void
flushBuffer(long fileNumber, long wherePosition)
Flush all unwritten log record up to the log instance indicated to disk without syncing.void
freezePersistentStore()
Backup restore - stop sending log record to the log streamjava.lang.String
getCanonicalLogPath()
Return the canonical directory of the PARENT of the log directory.private static ContextService
getContextService()
Privileged lookup of the ContextService.private StorageFile
getControlFileName()
Return the control file nameint
getEncryptedDataLength(int length)
returns the length that will make the data to be multiple of encryption block size based on the given length.int
getEncryptionBlockSize()
return the encryption block size used during encrypted db creationprivate long
getFirstLogNeeded(CheckpointOperation checkpoint)
Return the "oldest" log file still needed by recovery.LogInstant
getFirstUnflushedInstant()
Get the instant of the first record which was not flushed.long
getFirstUnflushedInstantAsLong()
Get the log instant long value of the first log record that has not been flushed.StorageFile
getLogDirectory()
Return the location of the log directory.private java.lang.String
getLogDirPath(StorageFile logDir)
void
getLogFactoryProperties(PersistentSet set)
Get JBMS properties relevant to the log factoryprotected StorageRandomAccessFile
getLogFileAtBeginning(long filenumber)
Open a log file and position the file at the beginning.protected StorageRandomAccessFile
getLogFileAtPosition(long logInstant)
Get a read-only handle to the log file positioned at the stated positionprivate StorageFile
getLogFileName(long filenumber)
Given a log file number, return its file nameprivate long
getLogFileNumber()
Return the current log file number.StorageRandomAccessFile
getLogFileToSimulateCorruption(long filenum)
Get the log file to Simulate a log corruption FOR UNIT TESTING USAGE ONLYLogger
getLogger()
MT- not neededprivate void
getLogStorageFactory()
private StorageFile
getMirrorControlFileName()
Return the mirror control file nameprivate static ModuleFactory
getMonitor()
Privileged Monitor lookup.private static java.lang.Object
getServiceModule(java.lang.Object serviceModule, java.lang.String factoryInterface)
Privileged module lookup.int
getTypeFormatId()
Return my format identifier.void
initializeReplicationSlaveRole()
Initializes logOut so that log received from the replication master can be appended to the log file.private boolean
initLogFile(StorageRandomAccessFile newlog, long number, long prevLogRecordEndInstant)
Initialize the log to the correct format with the given version and log file number.boolean
inReplicationMasterMode()
Used to determine if the replication master mode has been started, and the logging for unlogged operations needs to be enabled.boolean
inRFR()
boolean
isCheckpointInLastLogFile()
private static boolean
isFullUpgrade(java.util.Properties startParams, java.lang.String oldVersionInfo)
Privileged startup.boolean
logArchived()
Backup restore - is the log being archived to some directory?protected void
logErrMsg(java.lang.String msg)
Print error message to user about the log MT - not needed, informational onlyprotected void
logErrMsg(java.lang.Throwable t)
Print error message to user about the log MT - not needed, informational onlyprivate void
logErrMsgForDurabilityTestModeNoSync()
In case of boot errors, and if database is either booted with derby.system.durability=test or was previously at any time booted in this mode, mention in the error message that the error is probably because the derby.system.durability was set.private long
logtest_appendPartialLogRecord(byte[] data, int offset, int length, byte[] optionalData, int optionalDataOffset, int optionalDataLength)
Writes out a partial log record - takes the appendLogRecord.StandardException
markCorrupt(StandardException originalError)
Once the log factory is marked as corrupt then the raw store will shut down.protected LogScan
openBackwardsScan(long startAt, LogInstant stopAt)
Scan backward from start position.protected LogScan
openBackwardsScan(LogInstant stopAt)
Scan backward from end of log.ScanHandle
openFlushedScan(DatabaseInstant start, int groupsIWant)
Get a ScanHandle to scan flushed records from the log.LogScan
openForwardsFlushedScan(LogInstant startAt)
Open a forward scan of the transaction log.protected LogScan
openForwardsScan(long startAt, LogInstant stopAt)
Scan Forward from start position.LogScan
openForwardsScan(LogInstant startAt, LogInstant stopAt)
Get a forwards scanprivate StorageRandomAccessFile
openLogFileInWriteMode(StorageFile logFile)
open the given log file name for writes; if file can not be be opened in write sync mode then disable the write sync mode and open the file in "rw" mode.int
performWork(ContextManager context)
Do whatever it is that you want the daemon to do for you.private void
preAllocateNewLogFile(StorageRandomAccessFile log)
private void
printErrorStack(java.lang.Throwable t)
print stack trace from the Throwable including its nested exceptionsprotected boolean
privCanWrite(StorageFile file)
private boolean
privCopyFile(java.io.File from, StorageFile to)
private boolean
privCopyFile(StorageFile from, java.io.File to)
protected boolean
privDelete(StorageFile file)
protected boolean
privExists(StorageFile file)
private java.io.OutputStreamWriter
privGetOutputStreamWriter(StorageFile file)
private java.lang.String[]
privList(java.io.File file)
private java.lang.String[]
privList(StorageFile file)
protected boolean
privMkdirs(StorageFile file)
private StorageRandomAccessFile
privRandomAccessFile(StorageFile file, java.lang.String perms)
private boolean
privRemoveDirectory(StorageFile file)
private long
readControlFile(StorageFile logControlFileName, java.util.Properties startParams)
void
recover(DataFactory df, TransactionFactory tf)
Recover the rawStore to a consistent state using the log.private boolean
restoreLogs(java.util.Properties properties)
This function restores logs based on the following attributes are specified on connection URL: Attribute.CREATE_FROM (Create database from backup if it does not exist) Attribute.RESTORE_FROM (Delete the whole database if it exists and then restore it from backup) Attribute.ROLL_FORWARD_RECOVERY_FROM:(Perform Rollforward Recovery; except for the log directory everything else is replaced by the copy from backup. log files in the backup are copied to the existing online log directory.java.lang.Object
run()
private boolean
runBooleanAction(int action, StorageFile file)
boolean
serviceASAP()
If this work should be done as soon as possible, then return true.boolean
serviceImmediately()
If this work should be done immediately on the user thread then return true.void
setDatabaseEncrypted(boolean isEncrypted, boolean flushLog)
Sets whether the database is encrypted, all the transaction log has to be encrypted, and flush the log if requested.private void
setEndPosition(long newPosition)
set the endPosition of the log and make sure the new position won't spill off the end of the logvoid
setRawStoreFactory(RawStoreFactory rsf)
Make log factory aware of which raw store factory it belongs tovoid
startLogBackup(java.io.File toDir)
void
startNewLogFile()
void
startReplicationMasterRole(MasterFactory masterFactory)
Make this LogFactory pass log records to the MasterFactory every time a log record is appended to the log on disk, and notify the MasterFactory when a log disk flush has taken place.void
stop()
Stop the log factoryvoid
stopReplicationMasterRole()
Stop this LogFactory from passing log records to the MasterFactory and from notifying the MasterFactory when a log disk flush has taken place.void
stopReplicationSlaveRole()
Stop the slave functionality for this LogFactory.void
switchLogFile()
Switch to the next log file if possible.private void
syncFile(StorageRandomAccessFile raf)
Utility routine to call sync() on the input file descriptor.protected void
testLogFull()
Simulate a log full condition if TEST_LOG_FULL is set to true, then the property TEST_RECORD_TO_FILL_LOG indicates the number of times this function is call before an IOException simulating a log full condition is raised.java.lang.String
toString()
private void
truncateLog(long firstLogNeeded)
Get rid of old and unnecessary log filesprivate void
truncateLog(CheckpointOperation checkpoint)
Get rid of old and unnecessary log filesvoid
unfreezePersistentStore()
Backup restore - start sending log record to the log streamprivate boolean
verifyLogFormat(StorageFile logFileName, long number)
Verify that we the log file is of the right format and of the right version and log file number.private boolean
verifyLogFormat(StorageRandomAccessFile log, long number)
Verify that we the log file is of the right format and of the right version and log file number.(package private) boolean
writeControlFile(StorageFile logControlFileName, long value)
Carefully write out this value to the control file.
-
-
-
Field Detail
-
INT_LENGTH
private static final long INT_LENGTH
- See Also:
- Constant Field Values
-
fid
private static int fid
-
LOG_FILE_HEADER_SIZE
public static final int LOG_FILE_HEADER_SIZE
- See Also:
- Constant Field Values
-
LOG_FILE_HEADER_PREVIOUS_LOG_INSTANT_OFFSET
protected static final int LOG_FILE_HEADER_PREVIOUS_LOG_INSTANT_OFFSET
- See Also:
- Constant Field Values
-
LOG_RECORD_OVERHEAD
public static final int LOG_RECORD_OVERHEAD
- See Also:
- Constant Field Values
-
DBG_FLAG
public static final java.lang.String DBG_FLAG
-
DUMP_LOG_ONLY
public static final java.lang.String DUMP_LOG_ONLY
-
DUMP_LOG_FROM_LOG_FILE
public static final java.lang.String DUMP_LOG_FROM_LOG_FILE
-
LOG_SYNC_STATISTICS
protected static final java.lang.String LOG_SYNC_STATISTICS
- See Also:
- Constant Field Values
-
OBSOLETE_LOG_VERSION_NUMBER
private static final int OBSOLETE_LOG_VERSION_NUMBER
- See Also:
- Constant Field Values
-
DEFAULT_LOG_SWITCH_INTERVAL
private static final int DEFAULT_LOG_SWITCH_INTERVAL
- See Also:
- Constant Field Values
-
LOG_SWITCH_INTERVAL_MIN
private static final int LOG_SWITCH_INTERVAL_MIN
- See Also:
- Constant Field Values
-
LOG_SWITCH_INTERVAL_MAX
private static final int LOG_SWITCH_INTERVAL_MAX
- See Also:
- Constant Field Values
-
CHECKPOINT_INTERVAL_MIN
private static final int CHECKPOINT_INTERVAL_MIN
- See Also:
- Constant Field Values
-
CHECKPOINT_INTERVAL_MAX
private static final int CHECKPOINT_INTERVAL_MAX
- See Also:
- Constant Field Values
-
DEFAULT_CHECKPOINT_INTERVAL
private static final int DEFAULT_CHECKPOINT_INTERVAL
- See Also:
- Constant Field Values
-
DEFAULT_LOG_BUFFER_SIZE
private static final int DEFAULT_LOG_BUFFER_SIZE
- See Also:
- Constant Field Values
-
LOG_BUFFER_SIZE_MIN
private static final int LOG_BUFFER_SIZE_MIN
- See Also:
- Constant Field Values
-
LOG_BUFFER_SIZE_MAX
private static final int LOG_BUFFER_SIZE_MAX
- See Also:
- Constant Field Values
-
logBufferSize
private int logBufferSize
-
IS_BETA_FLAG
private static final byte IS_BETA_FLAG
- See Also:
- Constant Field Values
-
IS_DURABILITY_TESTMODE_NO_SYNC_FLAG
private static final byte IS_DURABILITY_TESTMODE_NO_SYNC_FLAG
When the derby.system.durability property is set to 'test', the store system will not force sync calls in the following cases - for the log file at each commit - for the log file before data page is forced to disk - for page allocation when file is grown - for data writes during checkpoint This means it is possible that the recovery system may not work properly, committed transactions may be lost, and/or database may not be in a consistent state. In order that we recognize this case that the database was previously at any time booted in this mode, this value is written out into the log control file. This will help prevent us from wasting time to resolve issues in such cases.
-
wasDBInDurabilityTestModeNoSync
private static boolean wasDBInDurabilityTestModeNoSync
keeps track of if the database was booted previously at any time with derby.system.durability=test
-
DEFAULT_LOG_ARCHIVE_DIRECTORY
private static final java.lang.String DEFAULT_LOG_ARCHIVE_DIRECTORY
- See Also:
- Constant Field Values
-
logSwitchInterval
private int logSwitchInterval
-
checkpointInterval
private int checkpointInterval
-
dataDirectory
java.lang.String dataDirectory
-
logStorageFactory
private WritableStorageFactory logStorageFactory
-
logBeingFlushed
private boolean logBeingFlushed
-
logOut
protected LogAccessFile logOut
-
firstLog
private StorageRandomAccessFile firstLog
-
endPosition
protected long endPosition
-
lastFlush
long lastFlush
-
logFileNumber
long logFileNumber
-
bootTimeLogFileNumber
long bootTimeLogFileNumber
-
firstLogFileNumber
long firstLogFileNumber
-
maxLogFileNumber
private long maxLogFileNumber
-
currentCheckpoint
private CheckpointOperation currentCheckpoint
-
checkpointInstant
long checkpointInstant
-
checkpointDaemon
private DaemonService checkpointDaemon
-
myClientNumber
private int myClientNumber
-
checkpointDaemonCalled
private volatile boolean checkpointDaemonCalled
-
logWrittenFromLastCheckPoint
private long logWrittenFromLastCheckPoint
-
rawStoreFactory
private RawStoreFactory rawStoreFactory
-
dataFactory
protected DataFactory dataFactory
-
ReadOnlyDB
protected boolean ReadOnlyDB
-
masterFactory
private MasterFactory masterFactory
-
inReplicationMasterMode
private boolean inReplicationMasterMode
-
inReplicationSlaveMode
private boolean inReplicationSlaveMode
-
replicationSlaveException
private volatile StandardException replicationSlaveException
If this exception is set while in replication slave mode, the exception will be thrown by the thread doing recovery will. Effectively, this will shut down the database.
-
inReplicationSlavePreMode
private boolean inReplicationSlavePreMode
True if the database has been booted in replication slave pre mode, effectively turning off writes to the log file.- See Also:
SlaveFactory
-
slaveRecoveryMonitor
private java.lang.Object slaveRecoveryMonitor
-
allowedToReadFileNumber
private long allowedToReadFileNumber
-
keepAllLogs
private boolean keepAllLogs
-
databaseEncrypted
private boolean databaseEncrypted
-
recoveryNeeded
private boolean recoveryNeeded
-
inCheckpoint
private boolean inCheckpoint
-
inRedo
private boolean inRedo
-
inLogSwitch
private boolean inLogSwitch
-
stopped
private boolean stopped
-
logDevice
java.lang.String logDevice
-
logNotSynced
private boolean logNotSynced
-
logArchived
private volatile boolean logArchived
-
logSwitchRequired
private boolean logSwitchRequired
-
test_logWritten
int test_logWritten
DEBUG test only
-
test_numRecordToFillLog
int test_numRecordToFillLog
-
mon_flushCalls
private int mon_flushCalls
-
mon_syncCalls
private int mon_syncCalls
-
mon_numLogFlushWaits
private int mon_numLogFlushWaits
-
mon_LogSyncStatistics
private boolean mon_LogSyncStatistics
-
mon_numBytesToLog
private int mon_numBytesToLog
-
corrupt
protected volatile StandardException corrupt
If not null then something is corrupt in the raw store and this represents the original error.
-
isFrozen
private boolean isFrozen
If frozen, don't allow anything on disk to change.
-
jbmsVersion
ProductVersionHolder jbmsVersion
Product Version information. Invarient after boot.
-
onDiskMajorVersion
private int onDiskMajorVersion
On disk database version information. When running in soft upgrade this version may be different to jbmsVersion.
-
onDiskMinorVersion
private int onDiskMinorVersion
-
onDiskBeta
private boolean onDiskBeta
-
checksum
private java.util.zip.CRC32 checksum
-
isWriteSynced
private boolean isWriteSynced
Note: Why logging system support file sync and write sync ? Note : The reason to support file and write sync of logs is there was no support to do write sync until jdk1.4 and then there was write sync jvm bug in jdk1.4.1, only in jdk1.4.2 write sync(rws and rwd modes) mechanism can be used correctly. Default in JVMS >= jdk1.4.2 is write sync(see the boot method for jvm checks). Write sync mechanism support is added for performance reasons. On commits, logging system has to make sure the log for committed transaction is on disk. With out write sync , log is written to the disk and then fsync() is used on commits to make log is written to the disk for sure. On most of the OS , fsync() calls are expensive. On heavy commit oriented systems, file sync make the system run slow. This problem is solved by using write sync on preallocated log file. write sync is much faster than doing write and file sync to a file. File should be preallocated for write syncs to perform better than the file sync method. Whenever a new log file is created, logSwitchInterval size is preallocated by writing zeros after file after the header.
-
jvmSyncErrorChecked
private boolean jvmSyncErrorChecked
Status for whether the check on the sync error on some JVMs has been done or not. See the checkJvmSyncError method for details.
-
logFileToBackup
private volatile long logFileToBackup
-
backupInProgress
private volatile boolean backupInProgress
-
TEST_LOG_SWITCH_LOG
public static final java.lang.String TEST_LOG_SWITCH_LOG
Set to true if we want the checkpoint to only switch the log but not actually do the checkpoint
-
TEST_LOG_INCOMPLETE_LOG_WRITE
public static final java.lang.String TEST_LOG_INCOMPLETE_LOG_WRITE
Set to true if we want the upcoming log record to be only partially written. The database is corrupted if not immediately shutdown. Set TEST_LOG_PARTIAL_LOG_WRITE_NUM_BYTES to the number of bytes to write out, default is 1 byte.
-
TEST_LOG_PARTIAL_LOG_WRITE_NUM_BYTES
public static final java.lang.String TEST_LOG_PARTIAL_LOG_WRITE_NUM_BYTES
Set to the number of bytes we want the next log record to actually write out, only used when TEST_LOG_INCOMPLETE_LOG_WRITE is on. Default is 1 byte.
-
TEST_LOG_FULL
public static final java.lang.String TEST_LOG_FULL
Set to true if we want to simulate a log full condition
-
TEST_SWITCH_LOG_FAIL1
public static final java.lang.String TEST_SWITCH_LOG_FAIL1
Set to true if we want to simulate a log full condition while switching log
-
TEST_SWITCH_LOG_FAIL2
public static final java.lang.String TEST_SWITCH_LOG_FAIL2
-
TEST_RECORD_TO_FILL_LOG
public static final java.lang.String TEST_RECORD_TO_FILL_LOG
Set to the number of log record we want to write before the log is simulated to be full.
-
TEST_MAX_LOGFILE_NUMBER
public static final java.lang.String TEST_MAX_LOGFILE_NUMBER
Set to true if we want to simulate max possible log file number is being used.
-
action
private int action
-
activeFile
private StorageFile activeFile
-
toFile
private java.io.File toFile
-
activePerms
private java.lang.String activePerms
-
-
Method Detail
-
getTypeFormatId
public int getTypeFormatId()
Return my format identifier.
-
markCorrupt
public StandardException markCorrupt(StandardException originalError)
Once the log factory is marked as corrupt then the raw store will shut down.- Specified by:
markCorrupt
in interfaceCorruptable
- Returns:
- Must always return its parameter.
-
checkCorrupt
private void checkCorrupt() throws StandardException
- Throws:
StandardException
-
getLogger
public Logger getLogger()
MT- not needed- Specified by:
getLogger
in interfaceLogFactory
-
setRawStoreFactory
public void setRawStoreFactory(RawStoreFactory rsf)
Make log factory aware of which raw store factory it belongs to- Specified by:
setRawStoreFactory
in interfaceLogFactory
-
recover
public void recover(DataFactory df, TransactionFactory tf) throws StandardException
Recover the rawStore to a consistent state using the log.In this implementation, the log is a stream of log records stored in one or more flat files. Recovery is done in 2 passes: redo and undo.
Redo pass
In the redo pass, reconstruct the state of the rawstore by repeating exactly what happened before as recorded in the log.
Undo pass
In the undo pass, all incomplete transactions are rolled back in the order from the most recently started to the oldest.MT - synchronization provided by caller - RawStore boot. This method is guaranteed to be the only method being called and can assume single thread access on all fields.
- Specified by:
recover
in interfaceLogFactory
- Parameters:
df
- - the data factorytf
- - the transaction factory- Throws:
StandardException
- Standard Derby error policy- See Also:
Loggable.needsRedo(org.apache.derby.iapi.store.raw.Transaction)
,FileLogger.redo(org.apache.derby.iapi.store.raw.xact.RawTransaction, org.apache.derby.iapi.store.raw.xact.TransactionFactory, org.apache.derby.impl.store.raw.log.StreamLogScan, long, long)
-
checkpoint
public boolean checkpoint(RawStoreFactory rsf, DataFactory df, TransactionFactory tf, boolean wait) throws StandardException
Checkpoint the rawStore.MT- Only one checkpoint is to be taking place at any given time.
The steps of a checkpoint are
- switch to a new log file if possible
freeze the log (for the transition to a new log file) flush current log file create and flush the new log file (with file number 1 higher than the previous log file). The new log file becomes the current log file. unfreeze the log
- start checkpoint transaction
- gather interesting information about the rawStore: the current log instant (redoLWM) the earliest active transaction begin tran log record instant (undoLWM), all the truncation LWM set by clients of raw store (replication)
- clean the buffer cache
- log the next checkpoint log record, which contains (repPoint, undoLWM, redoLWM) and commit checkpoint transaction.
- synchronously write the control file containing the next checkpoint log record log instant
- the new checkpoint becomes the current checkpoint. Somewhere near the beginning of each log file should be a checkpoint log record (not guarenteed to be there)
- see if the log can be truncated
The earliest useful log record is determined by the repPoint and the undoLWM, whichever is earlier.
Every log file whose log file number is smaller than the earliest useful log record's log file number can be deleted.
Transactions can be at the following states w/r to a checkpoint - consider the log as a continous stream and not as series of log files for the sake of clarity. |(BT)-------(ET)| marks the begin and end of a transaction. . checkpoint started . |__undoLWM | . V |___redoLWM . |___TruncationLWM . | . V 1 |-----------------| 2 |--------------------------------| 3 |-------| 4 |--------------------------------------(end of log) 5 |-^-| . Checkpoint Log Record ---A--->|<-------B--------->|<-------------C-----------
There are only 3 periods of interest :
A) before undoLWM, B) between undo and redo LWM, C) after redoLWM.Transaction 1 started in A and terminates in B.
During redo, we should only see log records and endXact from this transaction in the first phase (between undoLWM and redoLWM). No beginXact log record for this transaction will be seen.Transaction 2 started in B (right on the undoLWM) and terminated in C.
Any transaction that terminates in C must have a beginXact at or after undoLWM. In other words, no transaction can span A, B and C. During redo, we will see beginXact, other log records and endXact for this transaction.Transaction 3 started in B and ended in B.
During redo, we will see beginXact, other log records and endXact for this transaction.Transaction 4 begins in B and never ends.
During redo, we will see beginXact, other log records. In undo, this loser transaction will be rolled back.Transaction 5 is the transaction taking the checkpoint.
The checkpoint action started way back in time but the checkpoint log record is only written after the buffer cache has been flushed.Note that if any time elapse between taking the undoLWM and the redoLWM, then it will create a 4th period of interest.
- Specified by:
checkpoint
in interfaceLogFactory
- Parameters:
rsf
- The RawStoreFactory to use to do the checkpoint.df
- The DataFactory to use to do the checkpoint.tf
- The TransactionFactory to use to do the checkpoint.wait
- If an existing checkpoint is in progress, then if wait=true then this routine will wait for the checkpoint to complete and the do another checkpoint and wait for it to finish before returning.- Returns:
- true if checkpoint is successful, Will return false if wait is false and the routine finds another thread executing a checkpoint.
- Throws:
StandardException
- - encounter exception while doing checkpoint.
- switch to a new log file if possible
-
checkpointWithTran
private boolean checkpointWithTran(RawTransaction cptran, RawStoreFactory rsf, DataFactory df, TransactionFactory tf, boolean wait) throws StandardException
checkpoint with pre-start transaction- Parameters:
rsf
- The RawStoreFactory to use to do the checkpoint.df
- The DataFactory to use to do the checkpoint.tf
- The TransactionFactory to use to do the checkpoint.wait
- If an existing checkpoint is in progress, then if wait=true then this routine will wait for the checkpoint to complete and the do another checkpoint and wait for it to finish before returning.- Throws:
StandardException
- Derby Standard Error Policy
-
flush
public void flush(LogInstant where) throws StandardException
Flush all unwritten log record up to the log instance indicated to disk and sync. Also check to see if database is frozen or corrupt.MT - not needed, wrapper method
- Specified by:
flush
in interfaceLogFactory
- Parameters:
where
- flush log up to here- Throws:
StandardException
- Standard Derby error policy
-
flushAll
public void flushAll() throws StandardException
Flush all unwritten log record to disk and sync. Also check to see if database is frozen or corrupt.MT - not needed, wrapper method
- Throws:
StandardException
- Standard Derby error policy
-
verifyLogFormat
private boolean verifyLogFormat(StorageFile logFileName, long number) throws StandardException
Verify that we the log file is of the right format and of the right version and log file number.MT - not needed, no global variables used
- Parameters:
logFileName
- the name of the log filenumber
- the log file number- Returns:
- true if the log file is of the current version and of the correct format
- Throws:
StandardException
- Standard Derby error policy
-
verifyLogFormat
private boolean verifyLogFormat(StorageRandomAccessFile log, long number) throws StandardException
Verify that we the log file is of the right format and of the right version and log file number. The log file position is set to the beginning.MT - MT-unsafe, caller must synchronize
- Parameters:
log
- the log filenumber
- the log file number- Returns:
- true if the log file is of the current version and of the correct format
- Throws:
StandardException
- Standard Derby error policy
-
initLogFile
private boolean initLogFile(StorageRandomAccessFile newlog, long number, long prevLogRecordEndInstant) throws java.io.IOException, StandardException
Initialize the log to the correct format with the given version and log file number. The new log file must be empty. After initializing, the file is synchronously written to disk.MT - synchornization provided by caller
- Parameters:
newlog
- the new log file to be initializednumber
- the log file numberprevLogRecordEndInstant
- the end position of the previous log record- Returns:
- true if the log file is empty, else false.
- Throws:
java.io.IOException
- if new log file cannot be accessed or initializedStandardException
-
switchLogFile
public void switchLogFile() throws StandardException
Switch to the next log file if possible.MT - log factory is single threaded thru a log file switch, the log is frozen for the duration of the switch
- Throws:
StandardException
-
flushBuffer
private void flushBuffer(long fileNumber, long wherePosition) throws java.io.IOException, StandardException
Flush all unwritten log record up to the log instance indicated to disk without syncing.MT - not needed, wrapper method
- Parameters:
wherePosition
- flush log up to here- Throws:
java.io.IOException
- Failed to flush to the logStandardException
-
truncateLog
private void truncateLog(CheckpointOperation checkpoint)
Get rid of old and unnecessary log filesMT- only one truncate log is allowed to be taking place at any given time. Synchronized on this.
-
truncateLog
private void truncateLog(long firstLogNeeded)
Get rid of old and unnecessary log files- Parameters:
firstLogNeeded
- The log file number of the oldest log file needed for recovery.
-
getFirstLogNeeded
private long getFirstLogNeeded(CheckpointOperation checkpoint)
Return the "oldest" log file still needed by recovery.Returns the log file that contains the undoLWM, ie. the oldest log record of all uncommitted transactions in the given checkpoint. If no checkpoint is given then returns -1, indicating all log records may be necessary.
-
writeControlFile
boolean writeControlFile(StorageFile logControlFileName, long value) throws java.io.IOException, StandardException
Carefully write out this value to the control file. We do safe write of this data by writing the data into two files every time we write the control data. we write checksum at the end of the file, so if by chance system crashes while writing into the file, using the checksum we find that the control file is hosed then we use the mirror file, which will have the control data written at last check point. see comment at beginning of file for log control file format.MT- synchronized by caller
- Throws:
java.io.IOException
StandardException
-
readControlFile
private long readControlFile(StorageFile logControlFileName, java.util.Properties startParams) throws java.io.IOException, StandardException
- Throws:
java.io.IOException
StandardException
-
createLogDirectory
private void createLogDirectory() throws StandardException
Create the directory where transaction log should go.- Throws:
StandardException
- Standard Error Policy
-
createDataWarningFile
public void createDataWarningFile() throws StandardException
Create readme file in log directory warning users against touching any files in the directory- Specified by:
createDataWarningFile
in interfaceLogFactory
- Throws:
StandardException
-
getLogDirectory
public StorageFile getLogDirectory() throws StandardException
Description copied from interface:LogFactory
Return the location of the log directory.- Specified by:
getLogDirectory
in interfaceLogFactory
- Throws:
StandardException
- Standard Derby Error Policy
-
getCanonicalLogPath
public java.lang.String getCanonicalLogPath()
Description copied from interface:LogFactory
Return the canonical directory of the PARENT of the log directory. The log directory live in the "log" subdirectory of this path. If the log is at the default location (underneath the database directory), this returns null. Should only be called after the log factory is booted.- Specified by:
getCanonicalLogPath
in interfaceLogFactory
-
getControlFileName
private StorageFile getControlFileName() throws StandardException
Return the control file nameMT- read only
- Throws:
StandardException
-
getMirrorControlFileName
private StorageFile getMirrorControlFileName() throws StandardException
Return the mirror control file nameMT- read only
- Throws:
StandardException
-
getLogFileName
private StorageFile getLogFileName(long filenumber) throws StandardException
Given a log file number, return its file nameMT- read only
- Throws:
StandardException
-
findCheckpoint
private CheckpointOperation findCheckpoint(long checkpointInstant, FileLogger logger) throws java.io.IOException, StandardException, java.lang.ClassNotFoundException
- Throws:
java.io.IOException
StandardException
java.lang.ClassNotFoundException
-
openBackwardsScan
protected LogScan openBackwardsScan(long startAt, LogInstant stopAt) throws java.io.IOException, StandardException
Scan backward from start position.MT- read only
- Throws:
java.io.IOException
- cannot access the logStandardException
- Standard Derby error policy
-
openBackwardsScan
protected LogScan openBackwardsScan(LogInstant stopAt) throws java.io.IOException, StandardException
Scan backward from end of log.MT- read only
- Throws:
java.io.IOException
- cannot access the logStandardException
- Standard Derby error policy
-
openFlushedScan
public ScanHandle openFlushedScan(DatabaseInstant start, int groupsIWant) throws StandardException
Description copied from interface:LogFactory
Get a ScanHandle to scan flushed records from the log.MT- read only
- Specified by:
openFlushedScan
in interfaceLogFactory
- Parameters:
start
- - the LogInstant where we start our scan. null means start at the beginning of the log. This function raises an error if startAt is a LogInstant which is not in the log.groupsIWant
- - log record groups the scanner wants.- Returns:
- the LogScan.
- Throws:
StandardException
- Ooops.- See Also:
LogFactory.openFlushedScan(org.apache.derby.iapi.store.access.DatabaseInstant, int)
-
openForwardsScan
protected LogScan openForwardsScan(long startAt, LogInstant stopAt) throws java.io.IOException, StandardException
Scan Forward from start position.MT- read only
- Parameters:
startAt
- - if startAt == INVALID_LOG_INSTANT, start from the beginning of the log. Otherwise, start scan from startAt.stopAt
- - if not null, stop at this log instant (inclusive). Otherwise, stop at the end of the log- Throws:
java.io.IOException
- cannot access the logStandardException
- Standard Derby error policy
-
getLogFileAtBeginning
protected StorageRandomAccessFile getLogFileAtBeginning(long filenumber) throws java.io.IOException, StandardException
Open a log file and position the file at the beginning. Used by scan to switch to the next log fileMT- read only
When the database is in slave replication mode only: Assumes that only recover() will call this method after initializeReplicationSlaveRole() has been called, and until slave replication has ended. If this changes, the current implementation will fail.
- Throws:
StandardException
- Standard Derby error policyjava.io.IOException
- cannot access the log at the new position.
-
getLogFileAtPosition
protected StorageRandomAccessFile getLogFileAtPosition(long logInstant) throws java.io.IOException, StandardException
Get a read-only handle to the log file positioned at the stated positionMT- read only
- Returns:
- null if file does not exist or of the wrong format
- Throws:
java.io.IOException
- cannot access the log at the new position.StandardException
- Standard Derby error policy
-
canSupport
public boolean canSupport(java.util.Properties startParams)
Description copied from interface:ModuleSupportable
See if this implementation can support any attributes that are listed in properties. This call may be made on a newly created instance before the boot() method has been called, or after the boot method has been called for a running module.The module can check for attributes in the properties to see if it can fulfill the required behaviour. E.g. the raw store may define an attribute called RawStore.Recoverable. If a temporary raw store is required the property RawStore.recoverable=false would be added to the properties before calling bootServiceModule. If a raw store cannot support this attribute its canSupport method would return null. Also see the Monitor class's prologue to see how the identifier is used in looking up properties.
Actually a better way maybe to have properties of the form RawStore.Attributes.mandatory=recoverable,smallfootprint and RawStore.Attributes.requested=oltp,fast- Specified by:
canSupport
in interfaceModuleSupportable
- Returns:
- true if this instance can be used, false otherwise.
-
boot
public void boot(boolean create, java.util.Properties startParams) throws StandardException
Boot up the log factory.MT- caller provide synchronization
- Specified by:
boot
in interfaceModuleControl
- Throws:
StandardException
- log factory cannot start up- See Also:
Monitor
,ModuleFactory
-
getLogStorageFactory
private void getLogStorageFactory() throws StandardException
- Throws:
StandardException
-
stop
public void stop()
Stop the log factoryMT- caller provide synchronization (RESOLVE: this should be called AFTER dataFactory and transFactory are stopped)
- Specified by:
stop
in interfaceModuleControl
- See Also:
Monitor
,ModuleFactory
-
deleteObsoleteLogfiles
private void deleteObsoleteLogfiles()
-
serviceASAP
public boolean serviceASAP()
Description copied from interface:Serviceable
If this work should be done as soon as possible, then return true. If it doesn't make any difference if it is done sooner rather than later, then return false. The difference is whether or not the daemon service will be notified to work on this when this work is enqueued or subscribed, in case the serviceable work is put together but not sent to the daemon service directly, like in post commit processingMT - MT safe
- Specified by:
serviceASAP
in interfaceServiceable
-
serviceImmediately
public boolean serviceImmediately()
Description copied from interface:Serviceable
If this work should be done immediately on the user thread then return true. If it doesn't make any difference if this work is done on a the user thread immediately or if it is performed by another thread asynchronously later, then return false.- Specified by:
serviceImmediately
in interfaceServiceable
-
getLogFactoryProperties
public void getLogFactoryProperties(PersistentSet set) throws StandardException
Description copied from interface:LogFactory
Get JBMS properties relevant to the log factory- Specified by:
getLogFactoryProperties
in interfaceLogFactory
- Throws:
StandardException
- Standard Derby Error Policy
-
performWork
public int performWork(ContextManager context)
Description copied from interface:Serviceable
Do whatever it is that you want the daemon to do for you. There may be multiple daemon objects on different thread calling performWork at the same time. The DaemonService will always call performWork with a context manager set up. the DaemonService will clean up the context if an exception is thrown. However, it is up to performWork to manage its own transaction. If you start a transaction in performWork, you must commit or abort it at the end. You may leave the transaction open so that other serviceable may use the transaction and context without starting a new one. On the same token, there may already be an opened transaction on the context. Serviceable performWork should always check the state of the context before use. A Serviceable object should be well behaved while it is performing the daemon work, i.e., it should not take too many resources or hog the CPU for too long or deadlock with anyone else.- Specified by:
performWork
in interfaceServiceable
- Parameters:
context
- the contextManager set up by the DaemonService. There may or may not be the necessary context on it, depending on which other Serviceable object it has done work for.- Returns:
- the return status is only significant if the Serviceable client was enqueued instead of subscribed. For subscribed client, the return status is ignored. For enqueue client, it returns DONE or REQUEUE. If a REQUEUEd is returned, it would be desirable if this should not be serviceASAP, although no harm is done if this still maintains that this should be serviced ASAP ...
-
appendLogRecord
public long appendLogRecord(byte[] data, int offset, int length, byte[] optionalData, int optionalDataOffset, int optionalDataLength) throws StandardException
Append length bytes of data to the log prepended by a long log instant and followed by 4 bytes of length information.This method is synchronized to ensure log records are added sequentially to the end of the log.
MT- single threaded through this log factory. Log records are appended one at a time.
- Throws:
StandardException
- Log Full.
-
currentInstant
protected long currentInstant()
Get the current log instant - this is the log instant of the Next log record to be written outMT - This method is synchronized to ensure that it always points to the end of a log record, not the middle of one.
-
endPosition
protected long endPosition()
-
getLogFileNumber
private long getLogFileNumber()
Return the current log file number.MT - this method is synchronized so that it is not in the middle of being changed by swithLogFile
-
firstLogInstant
private long firstLogInstant()
Get the first valid log instant - this is the beginning of the first log fileMT- synchronized on this
-
flush
protected void flush(long fileNumber, long wherePosition) throws StandardException
Flush the log such that the log record written with the instant wherePosition is guaranteed to be on disk.MT - only one flush is allowed to be taking place at any given time (RESOLVE: right now it single thread thru the log factory while the log is frozen)
- Throws:
StandardException
- cannot sync log file
-
syncFile
private void syncFile(StorageRandomAccessFile raf) throws StandardException
Utility routine to call sync() on the input file descriptor.- Throws:
StandardException
-
openForwardsFlushedScan
public LogScan openForwardsFlushedScan(LogInstant startAt) throws StandardException
Open a forward scan of the transaction log.MT- read only
- Specified by:
openForwardsFlushedScan
in interfaceLogFactory
- Parameters:
startAt
- - the LogInstant where we start our scan. null means start at the beginning of the log. This function raises an error if startAt is a LogInstant which is not in the log.- Returns:
- the LogScan.
- Throws:
StandardException
- Standard Derby exception policy
-
openForwardsScan
public LogScan openForwardsScan(LogInstant startAt, LogInstant stopAt) throws StandardException
Get a forwards scan- Specified by:
openForwardsScan
in interfaceLogFactory
- Parameters:
startAt
- - the LogInstant where we start our scan. null means start at the beginning of the log. This function raises an error if startAt is a LogInstant which is not in the log.stopAt
- - the LogInstant where we stop our scan. null means stop at the end of the log. This function raises an error if stopAt is a LogInstant which is not in the log.- Returns:
- the LogScan.
- Throws:
StandardException
- Standard Derby error policy
-
databaseEncrypted
public final boolean databaseEncrypted()
-
setDatabaseEncrypted
public void setDatabaseEncrypted(boolean isEncrypted, boolean flushLog) throws StandardException
Sets whether the database is encrypted, all the transaction log has to be encrypted, and flush the log if requested.Log needs to be flushed first if the cryptographic state of the database changes (for instance re-encryption with a new key).
- Specified by:
setDatabaseEncrypted
in interfaceLogFactory
- Parameters:
isEncrypted
-true
if the database is encrypted,false
if notflushLog
-true
if log needs to be flushed,false
otherwise- Throws:
StandardException
-
startNewLogFile
public void startNewLogFile() throws StandardException
- Specified by:
startNewLogFile
in interfaceLogFactory
- Throws:
StandardException
-
isCheckpointInLastLogFile
public boolean isCheckpointInLastLogFile() throws StandardException
- Specified by:
isCheckpointInLastLogFile
in interfaceLogFactory
- Throws:
StandardException
-
deleteLogFileAfterCheckpointLogFile
public void deleteLogFileAfterCheckpointLogFile() throws StandardException
- Specified by:
deleteLogFileAfterCheckpointLogFile
in interfaceLogFactory
- Throws:
StandardException
-
encrypt
public int encrypt(byte[] cleartext, int offset, int length, byte[] ciphertext, int outputOffset) throws StandardException
- Throws:
StandardException
- Standard Derby Error Policy- See Also:
RawStoreFactory.encrypt(byte[], int, int, byte[], int, boolean)
-
decrypt
public int decrypt(byte[] ciphertext, int offset, int length, byte[] cleartext, int outputOffset) throws StandardException
- Throws:
StandardException
- Standard Derby Error Policy- See Also:
RawStoreFactory.decrypt(byte[], int, int, byte[], int)
-
getEncryptionBlockSize
public int getEncryptionBlockSize()
return the encryption block size used during encrypted db creation
-
getEncryptedDataLength
public int getEncryptedDataLength(int length)
returns the length that will make the data to be multiple of encryption block size based on the given length. Block cipher algorithms like DES and Blowfish ..etc require their input to be an exact multiple of the block size.
-
getFirstUnflushedInstant
public LogInstant getFirstUnflushedInstant()
Get the instant of the first record which was not flushed.This only works after running recovery the first time.
MT - RESOLVE:
- Specified by:
getFirstUnflushedInstant
in interfaceLogFactory
-
getFirstUnflushedInstantAsLong
public long getFirstUnflushedInstantAsLong()
Description copied from interface:LogFactory
Get the log instant long value of the first log record that has not been flushed. Only works after recover() has finished, or (if in slave replication mode) after calling initializeReplicationSlaveRole.- Specified by:
getFirstUnflushedInstantAsLong
in interfaceLogFactory
- Returns:
- the log instant long value of the first log record that has not been flushed
-
freezePersistentStore
public void freezePersistentStore() throws StandardException
Backup restore - stop sending log record to the log stream- Specified by:
freezePersistentStore
in interfaceLogFactory
- Throws:
StandardException
- Standard Derby error policy
-
unfreezePersistentStore
public void unfreezePersistentStore() throws StandardException
Backup restore - start sending log record to the log stream- Specified by:
unfreezePersistentStore
in interfaceLogFactory
- Throws:
StandardException
- Standard Derby error policy
-
logArchived
public boolean logArchived()
Backup restore - is the log being archived to some directory? if log archive mode is enabled return true else false- Specified by:
logArchived
in interfaceLogFactory
- Returns:
- true if the log is being archived.
-
checkVersion
boolean checkVersion(int requiredMajorVersion, int requiredMinorVersion)
Check to see if a database has been upgraded to the required level in order to use a store feature.- Parameters:
requiredMajorVersion
- required database Engine major versionrequiredMinorVersion
- required database Engine minor version- Returns:
- True if the database has been upgraded to the required level, false otherwise.
-
checkVersion
public boolean checkVersion(int requiredMajorVersion, int requiredMinorVersion, java.lang.String feature) throws StandardException
Check to see if a database has been upgraded to the required level in order to use a store feature.- Specified by:
checkVersion
in interfaceLogFactory
- Parameters:
requiredMajorVersion
- required database Engine major versionrequiredMinorVersion
- required database Engine minor versionfeature
- Non-null to throw an exception, null to return the state of the version match.- Returns:
true
if the database has been upgraded to the required level,false
otherwise.- Throws:
StandardException
- if the database is not at the require version whenfeature
feature is notnull
.
-
logErrMsg
protected void logErrMsg(java.lang.String msg)
Print error message to user about the log MT - not needed, informational only
-
logErrMsg
protected void logErrMsg(java.lang.Throwable t)
Print error message to user about the log MT - not needed, informational only
-
logErrMsgForDurabilityTestModeNoSync
private void logErrMsgForDurabilityTestModeNoSync()
In case of boot errors, and if database is either booted with derby.system.durability=test or was previously at any time booted in this mode, mention in the error message that the error is probably because the derby.system.durability was set. Dont want to waste time to resolve issues in such casesMT - not needed, informational only
-
printErrorStack
private void printErrorStack(java.lang.Throwable t)
print stack trace from the Throwable including its nested exceptions- Parameters:
t
- trace starts from this error
-
logtest_appendPartialLogRecord
private long logtest_appendPartialLogRecord(byte[] data, int offset, int length, byte[] optionalData, int optionalDataOffset, int optionalDataLength) throws StandardException
Writes out a partial log record - takes the appendLogRecord. Need to shutdown the database before another log record gets written, or the database is not recoverable.- Throws:
StandardException
-
testLogFull
protected void testLogFull() throws java.io.IOException
Simulate a log full condition if TEST_LOG_FULL is set to true, then the property TEST_RECORD_TO_FILL_LOG indicates the number of times this function is call before an IOException simulating a log full condition is raised. If TEST_RECORD_TO_FILL_LOG is not set, it defaults to 100 log record- Throws:
java.io.IOException
-
getLogFileToSimulateCorruption
public StorageRandomAccessFile getLogFileToSimulateCorruption(long filenum) throws java.io.IOException, StandardException
Get the log file to Simulate a log corruption FOR UNIT TESTING USAGE ONLY- Throws:
java.io.IOException
StandardException
-
inReplicationMasterMode
public boolean inReplicationMasterMode()
Used to determine if the replication master mode has been started, and the logging for unlogged operations needs to be enabled.- Specified by:
inReplicationMasterMode
in interfaceLogFactory
- Returns:
- true If the master replication mode is turned on and the unlogged operations need to be logged. false If the master replication mode is turned off and the unlogged operations need not be logged.
-
enableLogArchiveMode
public void enableLogArchiveMode() throws StandardException
- Specified by:
enableLogArchiveMode
in interfaceLogFactory
- Throws:
StandardException
-
disableLogArchiveMode
public void disableLogArchiveMode() throws StandardException
- Specified by:
disableLogArchiveMode
in interfaceLogFactory
- Throws:
StandardException
-
deleteOnlineArchivedLogFiles
public void deleteOnlineArchivedLogFiles()
- Specified by:
deleteOnlineArchivedLogFiles
in interfaceLogFactory
-
startLogBackup
public void startLogBackup(java.io.File toDir) throws StandardException
- Specified by:
startLogBackup
in interfaceLogFactory
- Throws:
StandardException
-
backupLogFiles
private void backupLogFiles(java.io.File toDir, long lastLogFileToBackup) throws StandardException
- Throws:
StandardException
-
endLogBackup
public void endLogBackup(java.io.File toDir) throws StandardException
- Specified by:
endLogBackup
in interfaceLogFactory
- Throws:
StandardException
-
abortLogBackup
public void abortLogBackup()
- Specified by:
abortLogBackup
in interfaceLogFactory
-
inRFR
public boolean inRFR()
- Specified by:
inRFR
in interfaceLogFactory
-
checkpointInRFR
public void checkpointInRFR(LogInstant cinstant, long redoLWM, long undoLWM, DataFactory df) throws StandardException
redo a checkpoint during rollforward recovery- Specified by:
checkpointInRFR
in interfaceLogFactory
- Parameters:
cinstant
- The LogInstant of the checkpointredoLWM
- Redo Low Water Mark in the check point recordundoLWM
- Undo Low Water Mark in the checkpointdf
- - the data factory- Throws:
StandardException
-
startReplicationMasterRole
public void startReplicationMasterRole(MasterFactory masterFactory) throws StandardException
Make this LogFactory pass log records to the MasterFactory every time a log record is appended to the log on disk, and notify the MasterFactory when a log disk flush has taken place.- Specified by:
startReplicationMasterRole
in interfaceLogFactory
- Parameters:
masterFactory
- The MasterFactory service responsible for controlling the master side replication behaviour.- Throws:
StandardException
- Standard Derby exception policy, thrown on replication startup error. Will only be thrown if replication is attempted started on a readonly database, i.e, never thrown here.
-
stopReplicationMasterRole
public void stopReplicationMasterRole()
Stop this LogFactory from passing log records to the MasterFactory and from notifying the MasterFactory when a log disk flush has taken place.- Specified by:
stopReplicationMasterRole
in interfaceLogFactory
-
stopReplicationSlaveRole
public void stopReplicationSlaveRole() throws StandardException
Stop the slave functionality for this LogFactory. Calling this method causes the thread currently doing recovery to stop the recovery process and throw a StandardException with SQLState SHUTDOWN_DATABASE. This should only be done when the database will be shutdown.- Throws:
StandardException
- Standard Derby exception policy- See Also:
SlaveDatabase
-
checkForReplication
protected void checkForReplication(LogAccessFile log)
Used by LogAccessFile to check if it should take the replication master role, and thereby send log records to the MasterFactory.- Parameters:
log
- The LogAccessFile that will take the replication master role iff this database is master.
-
initializeReplicationSlaveRole
public void initializeReplicationSlaveRole() throws StandardException
Initializes logOut so that log received from the replication master can be appended to the log file. Normally, logOut (the file log records are appended to) is set up as part of the recovery process. When the database is booted in replication slave mode, however, recovery will not get to the point where logOut is initialized until this database is no longer in slave mode. Since logOut is needed to append log records received from the master, logOut needs to be set up for replication slave mode. This method finds the last log record in the log file with the highest number. logOut is set up so that log records will be appended to the end of that file, and the endPosition and lastFlush variables are set to point to the end of the same file. All this is normally done as part of recovery. After the first log file switch resulting from applying log received from the master, recovery will be allowed to read up to, but not including, the current log file which is the file numbered logFileNumber. Note that this method must not be called until LogToFile#boot() has completed. Currently, this is ensured because RawStore#boot starts the SlaveFactory (in turn calling this method) after LogFactory.boot() has completed. Race conditions for logFileNumber may occur if this is changed.- Throws:
StandardException
- Standard Derby error policy
-
failoverSlave
public void failoverSlave()
Used to make the slave stop appending log records, complete recovery and boot the database.
-
restoreLogs
private boolean restoreLogs(java.util.Properties properties) throws StandardException
This function restores logs based on the following attributes are specified on connection URL: Attribute.CREATE_FROM (Create database from backup if it does not exist) Attribute.RESTORE_FROM (Delete the whole database if it exists and then restore it from backup) Attribute.ROLL_FORWARD_RECOVERY_FROM:(Perform Rollforward Recovery; except for the log directory everything else is replaced by the copy from backup. log files in the backup are copied to the existing online log directory. In case of RESTORE_FROM, the whole database directory is removed in Directory.java while restoring service.properties so even the log directory is removed. In case of CREATE_FROM, log directory will not exist if we came so far because it should fail if a database already exists. In case ROLL_FORWARD_RECOVERY_FROM log directory should not be removed. So only thing that needs to be done here is create a a log directory if it does not exists and copy the log files(including control files) that exists in the backup from which we are are trying to restore the database to the online log directory.- Throws:
StandardException
-
preAllocateNewLogFile
private void preAllocateNewLogFile(StorageRandomAccessFile log) throws java.io.IOException, StandardException
- Throws:
java.io.IOException
StandardException
-
openLogFileInWriteMode
private StorageRandomAccessFile openLogFileInWriteMode(StorageFile logFile) throws java.io.IOException
open the given log file name for writes; if file can not be be opened in write sync mode then disable the write sync mode and open the file in "rw" mode.- Throws:
java.io.IOException
-
getLogDirPath
private java.lang.String getLogDirPath(StorageFile logDir)
-
checkJvmSyncError
private boolean checkJvmSyncError(StorageFile logFile) throws java.io.IOException
In Java 1.4.2 and newer rws and rwd modes for RandomAccessFile are supported. Still, on some JVMs (e.g. early versions of 1.4.2 and 1.5 on Mac OS and FreeBSD) the support for rws and rwd is not working. This method attempts to detect this by opening an existing file in "rws" mode. If this fails, Derby should fall back to use "rw" mode for the log files followed by explicit syncing of the log. Note: it is important to use "rws" for the test. If "rwd" is used, no exception is thrown when opening the file, but the syncing does not take place. For more details see DERBY-1 (and DERBY-2020).- Parameters:
logFile
- information about the log file to be opened- Returns:
- true if a JVM error is detected, false otherwise
- Throws:
StandardException
- Standard Derby exceptionjava.io.IOException
-
privExists
protected boolean privExists(StorageFile file)
-
privDelete
protected boolean privDelete(StorageFile file)
-
privRandomAccessFile
private StorageRandomAccessFile privRandomAccessFile(StorageFile file, java.lang.String perms) throws java.io.IOException
- Throws:
java.io.IOException
-
privGetOutputStreamWriter
private java.io.OutputStreamWriter privGetOutputStreamWriter(StorageFile file) throws java.io.IOException
- Throws:
java.io.IOException
-
privCanWrite
protected boolean privCanWrite(StorageFile file)
-
privMkdirs
protected boolean privMkdirs(StorageFile file) throws java.io.IOException
- Throws:
java.io.IOException
-
privList
private java.lang.String[] privList(java.io.File file)
-
privList
private java.lang.String[] privList(StorageFile file)
-
privCopyFile
private boolean privCopyFile(StorageFile from, java.io.File to) throws StandardException
- Throws:
StandardException
-
privCopyFile
private boolean privCopyFile(java.io.File from, StorageFile to)
-
privRemoveDirectory
private boolean privRemoveDirectory(StorageFile file)
-
runBooleanAction
private boolean runBooleanAction(int action, StorageFile file)
-
setEndPosition
private void setEndPosition(long newPosition)
set the endPosition of the log and make sure the new position won't spill off the end of the log
-
run
public final java.lang.Object run() throws java.io.IOException, StandardException
- Specified by:
run
in interfacejava.security.PrivilegedExceptionAction<java.lang.Object>
- Throws:
java.io.IOException
StandardException
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
getContextService
private static ContextService getContextService()
Privileged lookup of the ContextService. Must be private so that user code can't call this entry point.
-
getMonitor
private static ModuleFactory getMonitor()
Privileged Monitor lookup. Must be private so that user code can't call this entry point.
-
findServiceModule
private static java.lang.Object findServiceModule(java.lang.Object serviceModule, java.lang.String factoryInterface) throws StandardException
Privileged startup. Must be private so that user code can't call this entry point.- Throws:
StandardException
-
getServiceModule
private static java.lang.Object getServiceModule(java.lang.Object serviceModule, java.lang.String factoryInterface)
Privileged module lookup. Must be private so that user code can't call this entry point.
-
isFullUpgrade
private static boolean isFullUpgrade(java.util.Properties startParams, java.lang.String oldVersionInfo) throws StandardException
Privileged startup. Must be private so that user code can't call this entry point.- Throws:
StandardException
-
-