Class AsynchronousLogShipper
- java.lang.Object
-
- java.lang.Thread
-
- org.apache.derby.impl.store.replication.master.AsynchronousLogShipper
-
- All Implemented Interfaces:
java.lang.Runnable
,LogShipper
public class AsynchronousLogShipper extends java.lang.Thread implements LogShipper
Does asynchronous shipping of log records from the master to the slave being replicated to. The implementation does not ship log records as soon as they become available in the log buffer (synchronously), instead it does log shipping in the following two-fold scenarios 1) Periodically (i.e.) at regular intervals of time. 2) when a request is sent from the master controller (force flushing of the log buffer). 3) when a notification is received from the log shipper about a log buffer element becoming full and the load on the log buffer so warrants a ship.
-
-
Field Summary
Fields Modifier and Type Field Description static int
DEFAULT_FORCEFLUSH_TIMEOUT
The number of millis a call to forceFlush will wait before giving up sending a chunk of log to the slaveprivate ReplicationMessage
failedChunk
Store the log chunk that failed during a previous shipping attempt so that it can be re-shipped to the slave.private long
failedChunkHighestInstant
The highest log instant in failedChunkprivate static int
FI_HIGH
Fill information value indicative of a high load in the log buffer.private static int
FI_LOW
Fill information value indicative of a low load in the log buffer.private java.lang.Object
forceFlushSemaphore
Used to synchronize forceFlush callsprivate long
highestShippedInstant
The highest log instant shipped so farprivate long
lastShippingTime
Will store the time at which the last shipping happened.private ReplicationLogBuffer
logBuffer
Replication log buffer that contains the log records that need to be transmitted to the slave.private MasterController
masterController
The master controller that initialized this log shipper.private static long
MAX
If the fill information is less than FI_LOW the log shipper will ship with a MAX ms delay or when a buffer becomes full whichever comes first.private long
maxShippingInterval
Minimum interval (in milliseconds) between log shipping.private static long
MIN
If the fill information (obtained from the log buffer) is less than FI_HIGH but greater than FI_LOW the log shipper will ship with a MIN ms delay.private long
minShippingInterval
Minimum interval (in milliseconds) between log shipping.private java.lang.Object
objLSTSync
Object used to synchronize on while the log shipper thread is moved into the wait state, or while notifying it.private ReplicationLogger
repLogger
private long
shippingInterval
Time interval (in milliseconds) at which the log shipping takes place.private boolean
stopShipping
Indicates whether a stop shipping request has been sent.private ReplicationMessageTransmit
transmitter
Replication message transmitter that is used for the network transmission of the log records retrieved from the log buffer (on the master) to the slave being replicated to.
-
Constructor Summary
Constructors Constructor Description AsynchronousLogShipper(ReplicationLogBuffer logBuffer, ReplicationMessageTransmit transmitter, MasterController masterController, ReplicationLogger repLogger)
Constructor initializes the log buffer, the replication message transmitter, the shipping interval and the master controller.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description private long
calculateSIfromFI()
Will be used to calculate the shipping interval based on the fill information obtained from the log buffer.void
flushBuffer()
Transmits all the log records in the log buffer to the slave.void
flushedInstance(long latestInstanceFlushedToDisk)
updates the information about the latest instance of the log record that has been flushed to the disk.void
forceFlush()
Transmits a chunk of log record from the log buffer to the slave, used by the master controller when the log buffer is full and some space needs to be freed for further log records.long
getHighestShippedInstant()
Get the highest log instant shipped so farprivate void
getLogShipperProperties()
Load relevant system properties: max and min log shipping intervalvoid
run()
Ships log records from the log buffer to the slave being replicated to.private boolean
shipALogChunk()
Retrieves a chunk of log records, if available, from the log buffer and transmits them to the slave.void
stopLogShipment()
Stop shipping log records.void
workToDo()
Used to notify the log shipper that a log buffer element is full.-
Methods inherited from class java.lang.Thread
activeCount, checkAccess, clone, countStackFrames, currentThread, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, onSpinWait, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, suspend, toString, yield
-
-
-
-
Field Detail
-
logBuffer
private final ReplicationLogBuffer logBuffer
Replication log buffer that contains the log records that need to be transmitted to the slave.
-
transmitter
private ReplicationMessageTransmit transmitter
Replication message transmitter that is used for the network transmission of the log records retrieved from the log buffer (on the master) to the slave being replicated to.
-
shippingInterval
private long shippingInterval
Time interval (in milliseconds) at which the log shipping takes place.
-
minShippingInterval
private long minShippingInterval
Minimum interval (in milliseconds) between log shipping. Defaults to MIN, but can be configured using system property derby.replication.minLogShippingInterval- See Also:
MIN
-
maxShippingInterval
private long maxShippingInterval
Minimum interval (in milliseconds) between log shipping. Defaults to MAX, but can be configured using system property derby.replication.maxLogShippingInterval- See Also:
MAX
-
lastShippingTime
private long lastShippingTime
Will store the time at which the last shipping happened. Will be used to calculate the interval between the log ships upon receiving a notification from the log buffer.
-
stopShipping
private volatile boolean stopShipping
Indicates whether a stop shipping request has been sent. true - stop shipping log records false - shipping can continue without interruption.
-
masterController
private MasterController masterController
The master controller that initialized this log shipper.
-
objLSTSync
private java.lang.Object objLSTSync
Object used to synchronize on while the log shipper thread is moved into the wait state, or while notifying it.
-
forceFlushSemaphore
private java.lang.Object forceFlushSemaphore
Used to synchronize forceFlush calls
-
DEFAULT_FORCEFLUSH_TIMEOUT
public static final int DEFAULT_FORCEFLUSH_TIMEOUT
The number of millis a call to forceFlush will wait before giving up sending a chunk of log to the slave- See Also:
- Constant Field Values
-
failedChunk
private ReplicationMessage failedChunk
Store the log chunk that failed during a previous shipping attempt so that it can be re-shipped to the slave.
-
failedChunkHighestInstant
private long failedChunkHighestInstant
The highest log instant in failedChunk
-
highestShippedInstant
private long highestShippedInstant
The highest log instant shipped so far
-
FI_LOW
private static final int FI_LOW
Fill information value indicative of a low load in the log buffer.- See Also:
- Constant Field Values
-
FI_HIGH
private static final int FI_HIGH
Fill information value indicative of a high load in the log buffer.- See Also:
- Constant Field Values
-
MIN
private static final long MIN
If the fill information (obtained from the log buffer) is less than FI_HIGH but greater than FI_LOW the log shipper will ship with a MIN ms delay. MIN is a value that is only as large as not to affect the performance of the master database significantly.- See Also:
- Constant Field Values
-
MAX
private static final long MAX
If the fill information is less than FI_LOW the log shipper will ship with a MAX ms delay or when a buffer becomes full whichever comes first. The delay however will not be smaller than MIN. max(MAX, DEFAULT_NUMBER_LOG_BUFFERS*MIN) is the maximum delay between a log record is committed at the master until it is replicated to the slave. Hence the default latency should be atleast greater than the maximum latency offered by the choice of MIN, hence MAX > DEFAULT_NUMBER_LOG_BUFFERS*MIN.- See Also:
- Constant Field Values
-
repLogger
private final ReplicationLogger repLogger
-
-
Constructor Detail
-
AsynchronousLogShipper
public AsynchronousLogShipper(ReplicationLogBuffer logBuffer, ReplicationMessageTransmit transmitter, MasterController masterController, ReplicationLogger repLogger)
Constructor initializes the log buffer, the replication message transmitter, the shipping interval and the master controller.- Parameters:
logBuffer
- the replication log buffer that contains the log record chunks to be transmitted to the slave.transmitter
- the replication message transmitter that is used for network transmission of retrieved log records.masterController
- The master controller that initialized this log shipper.repLogger
- The replication logger that will write messages to the log file (typically derby.log)
-
-
Method Detail
-
run
public void run()
Ships log records from the log buffer to the slave being replicated to. The log shipping happens between shipping intervals of time, the shipping interval being derived from the fill information (an indicator of load in the log buffer) obtained from the log buffer. The shipping can also be triggered in the following situations, 1) Based on notifications from the log buffer, where the fill information is again used as the basis to decide whether a shipping should happen or not 2) On a forceFlush triggered by the log buffer becoming full and the LogBufferFullException being thrown.- Specified by:
run
in interfacejava.lang.Runnable
- Overrides:
run
in classjava.lang.Thread
-
shipALogChunk
private boolean shipALogChunk() throws java.io.IOException, StandardException
Retrieves a chunk of log records, if available, from the log buffer and transmits them to the slave. Used for both periodic and forced shipping.- Returns:
- true if a chunk of log records was shipped. false if no log records were shipped because log buffer is empty.
- Throws:
java.io.IOException
- If an exception occurs while trying to ship the replication message (containing the log records) across the network.StandardException
- If an exception occurs while trying to read log records from the log buffer.
-
flushBuffer
public void flushBuffer() throws java.io.IOException, StandardException
Transmits all the log records in the log buffer to the slave.- Specified by:
flushBuffer
in interfaceLogShipper
- Throws:
java.io.IOException
- If an exception occurs while trying to ship the replication message (containing the log records) across the network.StandardException
- If an exception occurs while trying to read log records from the log buffer.
-
forceFlush
public void forceFlush() throws java.io.IOException, StandardException
Transmits a chunk of log record from the log buffer to the slave, used by the master controller when the log buffer is full and some space needs to be freed for further log records.- Specified by:
forceFlush
in interfaceLogShipper
- Throws:
java.io.IOException
- If an exception occurs while trying to ship the replication message (containing the log records) across the network.StandardException
- If an exception occurs while trying to read log records from the log buffer.
-
getHighestShippedInstant
public long getHighestShippedInstant()
Get the highest log instant shipped so far- Returns:
- the highest log instant shipped so far
-
flushedInstance
public void flushedInstance(long latestInstanceFlushedToDisk)
updates the information about the latest instance of the log record that has been flushed to the disk. Calling this method has no effect in this asynchronous implementation of the log shipper.- Specified by:
flushedInstance
in interfaceLogShipper
- Parameters:
latestInstanceFlushedToDisk
- a long that contains the latest instance of the log record that has been flushed to the disk.
-
stopLogShipment
public void stopLogShipment()
Stop shipping log records. If a ship is currently in progress it will not be interrupted, shipping will stop only after the current shipment is done.
-
workToDo
public void workToDo()
Used to notify the log shipper that a log buffer element is full. This method would basically use the following steps to decide on the action to be taken when a notification from the log shipper is received, a) Get FI from log buffer b) If FI >= FI_HIGH b.1) notify the log shipper thread. c) Else If the time elapsed since last ship is greater than minShippingInterval c.1) notify the log shipper thread.- Specified by:
workToDo
in interfaceLogShipper
-
calculateSIfromFI
private long calculateSIfromFI()
Will be used to calculate the shipping interval based on the fill information obtained from the log buffer. This method uses the following steps to arrive at the shipping interval, a) FI >= FI_HIGH return -1 (signifies that the waiting time should be 0) b) FI > FI_LOW and FI < FI_HIGH return minShippingInterval c) FI <= FI_LOW return maxShippingInterval.- Returns:
- the shipping interval based on the fill information.
-
getLogShipperProperties
private void getLogShipperProperties()
Load relevant system properties: max and min log shipping interval
-
-