Package org.apache.datasketches.memory

Interface Resource

All Superinterfaces:
AutoCloseable
All Known Subinterfaces:
Buffer, Memory, WritableBuffer, WritableMemory
public interface Resource extends AutoCloseable
Resource relies on preview features of the Java platform:
Programs can only use Resource when preview features are enabled.
Preview features may be removed in a future release, or upgraded to permanent features of the Java platform.
The base class for Memory and Buffer plus some common static variables and check methods.
Author:
Lee Rhodes

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final MemoryRequestServer
    defaultMemReqSvr
    The default MemoryRequestServer used primarily by test.

  • Method Summary

    Modifier and Type
    Method
    Description
    ByteBuffer
    asByteBufferView(ByteOrder order)
    Returns a ByteBuffer view of this Memory object with the given ByteOrder.
    void
    close()
    From Java 21 java.lang.foreign.Arena::close(): Closes this arena.
    int
    compareTo(long thisOffsetBytes, long thisLengthBytes, Resource that, long thatOffsetBytes, long thatLengthBytes)
    Compares the bytes of this Resource to that Resource.
    boolean
    equalTo(long thisOffsetBytes, Resource that, long thatOffsetBytes, long lengthBytes)
    Returns true if the given object is an instance of this class and has equal contents to this object in the given range of bytes.
    default boolean
    equalTo(Resource that)
    Returns true if the given object is an instance of this class and has equal contents to this object.
    void
    force()
    Forces any changes made to the contents of this mapped segment to be written to the storage device described by the mapped segment's file descriptor.
    long
    getCapacity()
    Gets the capacity of this object in bytes
    MemoryRequestServer
    getMemoryRequestServer()
    Gets the MemoryRequestServer to request additional memory for writable resources that are not file-memory-mapped.
    long
    getRelativeOffset(Resource that)
    Gets the relative base offset of this with respect to that, defined as: this - that.
    ByteOrder
    getTypeByteOrder()
    Gets the current Type ByteOrder.
    boolean
    hasByteBuffer()
    Returns true if this Memory is backed by a ByteBuffer.
    boolean
    hasMemoryRequestServer()
    Returns true if the MemoryRequestServer has been configured by the user.
    boolean
    isAlive()
    Is the underlying resource scope alive?
    boolean
    isBuffer()
    Returns true if this instance is a Buffer or WritableBuffer instance.
    boolean
    isByteOrderCompatible(ByteOrder byteOrder)
    Returns true if the Native ByteOrder is the same as the ByteOrder of the current Buffer or Memory and the same ByteOrder as the given byteOrder.
    boolean
    isCloseable()
    Return true if this resource is likely to be closeable, but not guaranteed.
    boolean
    isDirect()
    Returns true if the backing resource is direct (off-heap) memory.
    boolean
    isDuplicate()
    Returns true if this instance is a duplicate of a Buffer instance.
    boolean
    isHeap()
    Returns true if the backing resource is on the Java heap.
    boolean
    isLoaded()
    Returns true if it is likely that the contents of this segment is resident in physical memory.
    boolean
    isMapped()
    Returns true if this instance is of a memory mapped file.
    boolean
    isMemory()
    Returns true if this instance is of a Memory or WritableMemory instance
    boolean
    isReadOnly()
    Returns true if this object or the backing resource is read-only.
    boolean
    isRegion()
    Returns true if this instance is a region view of another Memory or Buffer
    boolean
    isSameResource(Resource that)
    Returns true if the underlying resource is the same underlying resource as that.
    void
    load()
    Loads the contents of this mapped segment into physical memory.
    long
    mismatch(Resource that)
    Finds the first byte mismatch with that.
    long
    mismatch(Resource src, long srcFromOffset, long srcToOffset, Resource dst, long dstFromOffset, long dstToOffset)
    Finds the first byte mismatch based on the given offsets
    long
    nativeOverlap(Resource that)
    Returns a positive number if this overlaps that and this base address is ≤ that base address.
    MemorySegment.ScopePREVIEW
    scope()
    Returns the resource scope associated with this memory segment.
    void
    setMemoryRequestServer(MemoryRequestServer memReqSvr)
    Sets the MemoryRequestServer to be used in case of capacity overflow of on-heap or off-heap allocated Memory.
    ByteBuffer
    toByteBuffer(ByteOrder order)
    Returns a new ByteBuffer with a copy of the data from this Memory object.
    MemorySegmentPREVIEW
    toMemorySegment()
    Returns a copy of the underlying MemorySegment.
    String
    toString()
    Returns a brief description of this object.
    String
    toString(String comment, long offsetBytes, int lengthBytes, boolean withData)
    Returns a description of this object with an optional formatted hex string of the data for the specified a range.
    void
    unload()
    Unloads the contents of this mapped segment from physical memory.
    long
    xxHash64(long in, long seed)
    Returns a 64-bit hash from a single long.
    long
    xxHash64(long offsetBytes, long lengthBytes, long seed)
    Returns the 64-bit hash of the sequence of bytes in this object specified by offsetBytes, lengthBytes and a seed.

  • Field Details

    • defaultMemReqSvr

      static final MemoryRequestServer defaultMemReqSvr
      The default MemoryRequestServer used primarily by test.

  • Method Details

    • getMemoryRequestServer

      MemoryRequestServer getMemoryRequestServer()
      Gets the MemoryRequestServer to request additional memory for writable resources that are not file-memory-mapped. Read-only, file-memory-mapped resources will return null.

      The user can customize the actions of the MemoryRequestServer by implementing the MemoryRequestServer interface and set it using the setMemoryRequestServer(MemoryRequestServer) method or optionally with the WritableMemory.allocateDirect(long, long, ByteOrder, MemoryRequestServer, Arena)PREVIEW method.

      If the MemoryRequestServer is not set by the user and additional memory is needed by the sketch, null will be returned and the sketch will abort. Simple implementation examples include the DefaultMemoryRequestServer in the main tree, as well as the ExampleMemoryRequestServerTest and the use with ByteBuffer documented in the DruidIssue11544Test in the test tree.

      Returns:
      a MemoryRequestServer object or null.

    • hasMemoryRequestServer

      boolean hasMemoryRequestServer()
      Returns true if the MemoryRequestServer has been configured by the user.
      Returns:
      true if the MemoryRequestServer has been configured by the user..

    • setMemoryRequestServer

      void setMemoryRequestServer(MemoryRequestServer memReqSvr)
      Sets the MemoryRequestServer to be used in case of capacity overflow of on-heap or off-heap allocated Memory.
      Parameters:
      memReqSvr - the given MemoryRequestServer

    • asByteBufferView

      ByteBuffer asByteBufferView(ByteOrder order)
      Returns a ByteBuffer view of this Memory object with the given ByteOrder. Some of the properties of the returned buffer are linked to the properties of this Memory object. For instance, if this Memory object is immutable (i.e., read-only, see isReadOnly()), then the resulting buffer is read-only (see Buffer.isReadOnly(). Additionally, if this is a native memory segment, the resulting buffer is direct (see ByteBuffer.isDirect()). The endianness of the returned buffer will be set to the given ByteOrder.
      Parameters:
      order - the given ByteOrder.
      Returns:
      a ByteBuffer view of this Memory object with the given ByteOrder.
      Throws:
      UnsupportedOperationException - - if this segment cannot be mapped onto a ByteBuffer instance, e.g. because it models an heap-based segment that is not based on a byte[]), or if its size is greater than Integer.MAX_VALUE.

    • close

      void close()
      From Java 21 java.lang.foreign.Arena::close(): Closes this arena. If this method completes normally, the arena scope is no longer alivePREVIEW, and all the memory segments associated with it can no longer be accessed. Furthermore, any off-heap region of memory backing the segments obtained from this arena are also released.

      This operation is not idempotent; that is, closing an already closed arena always results in an exception being thrown. This reflects a deliberate design choice: failure to close an arena might reveal a bug in the underlying application logic.

      If this method completes normally, then java.lang.foreign.Arena.scope().isAlive() == false. Implementations are allowed to throw UnsupportedOperationException if an explicit close operation is not supported.

      Specified by:
      close in interface AutoCloseable
      Throws:
      IllegalStateException - if the arena has already been closed.
      IllegalStateException - if a segment associated with this arena is being accessed concurrently, e.g. by a downcall method handlePREVIEW.
      WrongThreadException - if this arena is confined, and this method is called from a thread other than the arena's owner thread.
      UnsupportedOperationException - if this arena cannot be closed explicitly.
      See Also:

    • compareTo

      int compareTo(long thisOffsetBytes, long thisLengthBytes, Resource that, long thatOffsetBytes, long thatLengthBytes)
      Compares the bytes of this Resource to that Resource. Returns (this < that) ? (some negative value) : (this > that) ? (some positive value) : 0;. If all bytes are equal up to the shorter of the two lengths, the shorter length is considered to be less than the other.
      Parameters:
      thisOffsetBytes - the starting offset for this Resource
      thisLengthBytes - the length of the region to compare from this Resource
      that - the other Memory to compare with
      thatOffsetBytes - the starting offset for that Resource
      thatLengthBytes - the length of the region to compare from that Resource
      Returns:
      (this < that) ? (some negative value) : (this > that) ? (some positive value) : 0;

    • equalTo

      default boolean equalTo(Resource that)
      Returns true if the given object is an instance of this class and has equal contents to this object.
      Parameters:
      that - the given Resource object
      Returns:
      true if the given object has equal contents to this object.

    • equalTo

      boolean equalTo(long thisOffsetBytes, Resource that, long thatOffsetBytes, long lengthBytes)
      Returns true if the given object is an instance of this class and has equal contents to this object in the given range of bytes. This will also check two distinct ranges within the same object for equals.
      Parameters:
      thisOffsetBytes - the starting offset in bytes for this object.
      that - the given Resource object
      thatOffsetBytes - the starting offset in bytes for the given object
      lengthBytes - the size of the range in bytes
      Returns:
      true if the given object has equal contents to this object in the given range of bytes.

    • force

      void force()
      Forces any changes made to the contents of this mapped segment to be written to the storage device described by the mapped segment's file descriptor.
      See Also:

    • getCapacity

      long getCapacity()
      Gets the capacity of this object in bytes
      Returns:
      the capacity of this object in bytes

    • getRelativeOffset

      long getRelativeOffset(Resource that)
      Gets the relative base offset of this with respect to that, defined as: this - that. This method is only valid for native (off-heap) allocated resources.
      Parameters:
      that - the given resource.
      Returns:
      this - that offset
      Throws:
      IllegalArgumentException - if one of the resources is on-heap.

    • getTypeByteOrder

      ByteOrder getTypeByteOrder()
      Gets the current Type ByteOrder. This may be different from the ByteOrder of the backing resource and of the Native Byte Order.
      Returns:
      the current Type ByteOrder.

    • hasByteBuffer

      boolean hasByteBuffer()
      Returns true if this Memory is backed by a ByteBuffer.
      Returns:
      true if this Memory is backed by a ByteBuffer.

    • isCloseable

      boolean isCloseable()
      Return true if this resource is likely to be closeable, but not guaranteed. There is no way to determine if the specific type of Arena is explicitly closeable.
      Returns:
      true if this resource is likely to be closeable.

    • isAlive

      boolean isAlive()
      Is the underlying resource scope alive?
      Returns:
      true, if the underlying resource scope is alive.

    • isBuffer

      boolean isBuffer()
      Returns true if this instance is a Buffer or WritableBuffer instance.
      Returns:
      true if this instance is a Buffer or WritableBuffer instance.

    • isByteOrderCompatible

      boolean isByteOrderCompatible(ByteOrder byteOrder)
      Returns true if the Native ByteOrder is the same as the ByteOrder of the current Buffer or Memory and the same ByteOrder as the given byteOrder.
      Parameters:
      byteOrder - the given ByteOrder
      Returns:
      true if the Native ByteOrder is the same as the ByteOrder of the current Buffer or Memory and the same ByteOrder as the given byteOrder.

    • isDirect

      boolean isDirect()
      Returns true if the backing resource is direct (off-heap) memory. This can be true for allocated direct memory, memory mapped files, or from a wrapped ByteBuffer that was allocated direct.
      Returns:
      true if the backing resource is direct (off-heap) memory.

    • isDuplicate

      boolean isDuplicate()
      Returns true if this instance is a duplicate of a Buffer instance.
      Returns:
      true if this instance is a duplicate of a Buffer instance.

    • isHeap

      boolean isHeap()
      Returns true if the backing resource is on the Java heap. This can be true for wrapped heap primitive arrays or from a wrapped ByteBuffer that was allocated on the Java heap.
      Returns:
      true if the backing resource is on the Java heap.

    • isLoaded

      boolean isLoaded()
      Returns true if it is likely that the contents of this segment is resident in physical memory.
      Returns:
      true if it is likely that the contents of this segment is resident in physical memory.
      See Also:

    • isMapped

      boolean isMapped()
      Returns true if this instance is of a memory mapped file.
      Returns:
      true if this instance is of a memory mapped file.

    • isMemory

      boolean isMemory()
      Returns true if this instance is of a Memory or WritableMemory instance
      Returns:
      true if this instance is of a Memory or WritableMemory instance

    • isReadOnly

      boolean isReadOnly()
      Returns true if this object or the backing resource is read-only.
      Returns:
      true if this object or the backing resource is read-only.

    • isRegion

      boolean isRegion()
      Returns true if this instance is a region view of another Memory or Buffer
      Returns:
      true if this instance is a region view of another Memory or Buffer

    • isSameResource

      boolean isSameResource(Resource that)
      Returns true if the underlying resource is the same underlying resource as that.
      Parameters:
      that - the other Resource object
      Returns:
      a long value representing the ordering and size of overlap between this and that

    • load

      void load()
      Loads the contents of this mapped segment into physical memory.
      See Also:

    • nativeOverlap

      long nativeOverlap(Resource that)
      Returns a positive number if this overlaps that and this base address is ≤ that base address. Returns a negative number if this overlaps that and this base address is > that base address. Returns a zero if there is no overlap or if one or both objects are null, not active or on heap.
      Parameters:
      that - the other Resource object
      Returns:
      a long value representing the ordering and size of overlap between this and that.

    • mismatch

      long mismatch(Resource that)
      Finds the first byte mismatch with that.
      Parameters:
      that - the other Resource
      Returns:
      the relative offset, in bytes, of the first mismatch between this and the given other Resource object, otherwise -1 if no mismatch.
      See Also:

    • mismatch

      long mismatch(Resource src, long srcFromOffset, long srcToOffset, Resource dst, long dstFromOffset, long dstToOffset)
      Finds the first byte mismatch based on the given offsets
      Parameters:
      src - the given source Resource
      srcFromOffset - the given start offset of the source region, inclusive.
      srcToOffset - the given end offset of the source region, exclusive.
      dst - the given destination Resource
      dstFromOffset - the given start of the destination destination region, inclusive.
      dstToOffset - the given end offset of the destination destination region, exclusive.
      Returns:
      the byte offset of the first mismatch relative to the start of each of the above two regions.
      See Also:

    • scope

      MemorySegment.ScopePREVIEW scope()
      Returns the resource scope associated with this memory segment.
      Returns:
      the resource scope associated with this memory segment.

    • toByteBuffer

      ByteBuffer toByteBuffer(ByteOrder order)
      Returns a new ByteBuffer with a copy of the data from this Memory object. This new ByteBuffer will be writable, on heap, and with the endianness specified by the given ByteOrder.
      Parameters:
      order - the given ByteOrder. It must be non-null.
      Returns:
      a new ByteBuffer with a copy of the data from this Memory object.

    • toMemorySegment

      MemorySegmentPREVIEW toMemorySegment()
      Returns a copy of the underlying MemorySegment.
      Returns:
      a copy of the underlying MemorySegment.

    • toString

      String toString()
      Returns a brief description of this object.
      Overrides:
      toString in class Object
      Returns:
      a brief description of this object.

    • toString

      String toString(String comment, long offsetBytes, int lengthBytes, boolean withData)
      Returns a description of this object with an optional formatted hex string of the data for the specified a range. Used primarily for testing.
      Parameters:
      comment - a description
      offsetBytes - offset bytes relative to this object start
      lengthBytes - number of bytes to convert to a hex string
      withData - include output listing of byte data in the given range
      Returns:
      a description and hex output in a human readable format.

    • unload

      void unload()
      Unloads the contents of this mapped segment from physical memory.
      See Also:

    • xxHash64

      long xxHash64(long in, long seed)
      Returns a 64-bit hash from a single long. This method has been optimized for speed when only a single hash of a long is required.
      Parameters:
      in - A long.
      seed - A long valued seed.
      Returns:
      the hash.

    • xxHash64

      long xxHash64(long offsetBytes, long lengthBytes, long seed)
      Returns the 64-bit hash of the sequence of bytes in this object specified by offsetBytes, lengthBytes and a seed. Note that the sequence of bytes is always processed in the same order independent of endianness.
      Parameters:
      offsetBytes - the given offset in bytes to the first byte of the byte sequence.
      lengthBytes - the given length in bytes of the byte sequence.
      seed - the given long seed.
      Returns:
      the 64-bit hash of the sequence of bytes in this object specified by offsetBytes and lengthBytes.