Class ByteMappedBigList
- java.lang.Object
-
- java.util.AbstractCollection<java.lang.Byte>
-
- it.unimi.dsi.fastutil.bytes.AbstractByteCollection
-
- it.unimi.dsi.fastutil.bytes.AbstractByteBigList
-
- it.unimi.dsi.fastutil.bytes.ByteMappedBigList
-
- All Implemented Interfaces:
BigList<java.lang.Byte>
,ByteBigList
,ByteCollection
,ByteIterable
,ByteStack
,Size64
,Stack<java.lang.Byte>
,java.lang.Comparable<BigList<? extends java.lang.Byte>>
,java.lang.Iterable<java.lang.Byte>
,java.util.Collection<java.lang.Byte>
public class ByteMappedBigList extends AbstractByteBigList
A bridge between byte buffers and type-specific big lists.Java's memory-mapping facilities have the severe limitation of mapping at most
Integer.MAX_VALUE
bytes, as they expose the content of a file using aMappedByteBuffer
. This class can expose a file of primitive types of arbitrary length as aBigList
that is actually based on an array ofMappedByteBuffer
s, each mapping a chunk ofCHUNK_SIZE
longs.Mapping can happen with a specified byte order: saving and mapping data in native order using methods from
BinIO
will enhance performance significantly.Instances of this class are not thread safe, but the
copy()
method provides a lightweight duplicate that can be read independently by another thread. Only chunks that are actually used will be duplicated lazily. If you are modifiying the content of list, however, you will need to provide external synchronization.- Author:
- Sebastiano Vigna
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class it.unimi.dsi.fastutil.bytes.AbstractByteBigList
AbstractByteBigList.ByteRandomAccessSubList, AbstractByteBigList.ByteSubList
-
-
Field Summary
Fields Modifier and Type Field Description static long
CHUNK_SIZE
The size in elements of a chunk created bymap(FileChannel, ByteOrder, FileChannel.MapMode)
.static int
LOG2_BITS
Deprecated.UseLOG2_BYTES
.static int
LOG2_BYTES
The logarithm of the number of bytes of the primitive type of this list.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description ByteMappedBigList
copy()
Returns a lightweight duplicate that can be read independently by another thread.byte
getByte(long index)
Returns the element at the specified position.void
getElements(long from, byte[] a, int offset, int length)
Copies (hopefully quickly) elements of this type-specific big list into the given array.static ByteMappedBigList
map(java.nio.channels.FileChannel fileChannel)
Creates a new mapped big list by read-only mapping a given file channel using the standard Java (i.e.,DataOutput
) byte order (ByteOrder.BIG_ENDIAN
).static ByteMappedBigList
map(java.nio.channels.FileChannel fileChannel, java.nio.ByteOrder byteOrder)
Creates a new mapped big list by read-only mapping a given file channel.static ByteMappedBigList
map(java.nio.channels.FileChannel fileChannel, java.nio.ByteOrder byteOrder, java.nio.channels.FileChannel.MapMode mapMode)
Creates a new mapped big list by mapping a given file channel.byte
set(long index, byte value)
Replaces the element at the specified position in this big list with the specified element (optional operation).long
size64()
Returns the size of this data structure as a long.-
Methods inherited from class it.unimi.dsi.fastutil.bytes.AbstractByteBigList
add, add, add, addAll, addAll, addAll, addAll, addElements, addElements, clear, compareTo, contains, equals, forEach, get, getElements, hashCode, indexOf, indexOf, intSpliterator, iterator, lastIndexOf, lastIndexOf, listIterator, listIterator, peek, peekByte, pop, popByte, push, push, rem, remove, removeByte, removeElements, set, setElements, size, size, subList, top, topByte, toString
-
Methods inherited from class it.unimi.dsi.fastutil.bytes.AbstractByteCollection
add, contains, containsAll, containsAll, remove, removeAll, removeAll, retainAll, retainAll, toArray, toByteArray, toByteArray
-
Methods inherited from interface it.unimi.dsi.fastutil.bytes.ByteBigList
addAll, addAll, addAll, addAll, setElements, setElements, spliterator
-
Methods inherited from interface it.unimi.dsi.fastutil.bytes.ByteCollection
add, contains, containsAll, intIterator, intParallelStream, intStream, parallelStream, remove, removeAll, removeIf, removeIf, removeIf, retainAll, stream, toArray, toByteArray, toByteArray
-
Methods inherited from interface it.unimi.dsi.fastutil.bytes.ByteIterable
forEach, forEach
-
-
-
-
Field Detail
-
LOG2_BYTES
public static int LOG2_BYTES
The logarithm of the number of bytes of the primitive type of this list.
-
LOG2_BITS
@Deprecated public static int LOG2_BITS
Deprecated.UseLOG2_BYTES
.
-
CHUNK_SIZE
public static final long CHUNK_SIZE
The size in elements of a chunk created bymap(FileChannel, ByteOrder, FileChannel.MapMode)
.
-
-
Method Detail
-
map
public static ByteMappedBigList map(java.nio.channels.FileChannel fileChannel) throws java.io.IOException
Creates a new mapped big list by read-only mapping a given file channel using the standard Java (i.e.,DataOutput
) byte order (ByteOrder.BIG_ENDIAN
).- Parameters:
fileChannel
- the file channel that will be mapped.- Returns:
- a new read-only mapped big list over the contents of
fileChannel
. - Throws:
java.io.IOException
- See Also:
map(FileChannel, ByteOrder, MapMode)
-
map
public static ByteMappedBigList map(java.nio.channels.FileChannel fileChannel, java.nio.ByteOrder byteOrder) throws java.io.IOException
Creates a new mapped big list by read-only mapping a given file channel.- Parameters:
fileChannel
- the file channel that will be mapped.byteOrder
- a prescribed byte order.- Returns:
- a new read-only mapped big list over the contents of
fileChannel
. - Throws:
java.io.IOException
- See Also:
map(FileChannel, ByteOrder, MapMode)
-
map
public static ByteMappedBigList map(java.nio.channels.FileChannel fileChannel, java.nio.ByteOrder byteOrder, java.nio.channels.FileChannel.MapMode mapMode) throws java.io.IOException
Creates a new mapped big list by mapping a given file channel.- Parameters:
fileChannel
- the file channel that will be mapped.byteOrder
- a prescribed byte order.mapMode
- the mapping mode: usuallyFileChannel.MapMode.READ_ONLY
, but if intend to make the list mutable, you have to passFileChannel.MapMode.READ_WRITE
.- Returns:
- a new mapped big list over the contents of
fileChannel
. - Throws:
java.io.IOException
-
copy
public ByteMappedBigList copy()
Returns a lightweight duplicate that can be read independently by another thread.Only chunks that are actually used will be duplicated lazily.
- Returns:
- a lightweight duplicate that can be read independently by another thread.
-
getByte
public byte getByte(long index)
Description copied from interface:ByteBigList
Returns the element at the specified position.- See Also:
BigList.get(long)
-
getElements
public void getElements(long from, byte[] a, int offset, int length)
Description copied from interface:ByteBigList
Copies (hopefully quickly) elements of this type-specific big list into the given array.- Parameters:
from
- the start index (inclusive).a
- the destination array.offset
- the offset into the destination array where to store the first element copied.length
- the number of elements to be copied.
-
set
public byte set(long index, byte value)
Description copied from class:AbstractByteBigList
Replaces the element at the specified position in this big list with the specified element (optional operation).- Specified by:
set
in interfaceByteBigList
- Overrides:
set
in classAbstractByteBigList
- See Also:
BigList.set(long,Object)
-
size64
public long size64()
Description copied from interface:Size64
Returns the size of this data structure as a long.- Returns:
- the size of this data structure.
-
-