Package org.apache.datasketches.memory

Interface Resource

    • Method Summary

      All Methods Instance Methods Abstract Methods Default Methods 
      Modifier and Type Method Description
      void close()
      Closes this resource if this can be closed via AutoCloseable.
      boolean equalTo​(long thisOffsetBytes, Resource that, long thatOffsetBytes, long lengthBytes)
      Returns true if the given Resource has equal contents to this object in the given range of bytes.
      default boolean equalTo​(Resource that)
      Returns true if the given object (that) is an instance of this class and has contents equal to this object.
      void force()
      Forces any changes made to the contents of this memory-mapped Resource to be written to the storage device described by the configured file descriptor.
      long getCapacity()
      Gets the capacity of this object in bytes
      default long getCumulativeOffset()
      Gets the cumulative offset in bytes of this object from the backing resource.
      long getCumulativeOffset​(long offsetBytes)
      Gets the cumulative offset in bytes of this object from the backing resource including the given offsetBytes.
      MemoryRequestServer getMemoryRequestServer()
      Gets the MemoryRequestServer object, if set, for the below resources to request additional memory.
      long getRelativeOffset()
      Returns the offset of address zero of this object relative to the base address of the backing resource.
      ByteOrder getTypeByteOrder()
      Gets the current ByteOrder.
      boolean hasByteBuffer()
      Returns true if this Memory is backed by a ByteBuffer.
      boolean hasMemoryRequestServer()
      Returns true if the MemoryRequestServer has been configured by the user.
      boolean isAlive()
      Returns true if this object is alive and has not been closed.
      boolean isByteOrderCompatible​(ByteOrder byteOrder)
      Returns true if the Native ByteOrder is the same as the ByteOrder of the current Buffer or Memory and the same ByteOrder as the given byteOrder.
      boolean isCloseable()
      Return true if this resource is closeable.
      boolean isDirect()
      If true, the backing resource is off-heap memory.
      boolean isDuplicate()
      Returns true if this instance is a duplicate of a Buffer instance.
      boolean isHeap()
      Returns true if this object is backed by an on-heap primitive array or an on-heap ByteBuffer.
      boolean isLoaded()
      Tells whether or not the contents of this memory-mapped Resource is resident in physical memory.
      boolean isMapped()
      Returns true if the backing resource is a memory-mapped file.
      boolean isMemory()
      If true, this is a Memory or WritableMemory instance, which provides the Memory API.
      boolean isNonNativeOrder()
      If true, all put and get operations will assume the non-native ByteOrder.
      boolean isReadOnly()
      Returns true if this object or the backing resource is read-only.
      boolean isRegionView()
      Returns true if this instance is a region view of another Memory or Buffer
      boolean isSameResource​(Resource that)
      Returns true if the backing resource of this is identical with the backing resource of that.
      void load()
      Loads the contents of this memory-mapped Resource into physical memory.
      void setMemoryRequestServer​(MemoryRequestServer memReqSvr)
      Sets the Default MemoryRequestServer
      String toString()
      Returns a brief description of this object.
      String toString​(String header, long offsetBytes, int lengthBytes, boolean withData)
      Returns a description of this object with an optional formatted hex string of the data for the specified a range.
      long xxHash64​(long in, long seed)
      Returns a 64-bit hash from a single long.
      long xxHash64​(long offsetBytes, long lengthBytes, long seed)
      Returns the 64-bit hash of the sequence of bytes in this object specified by offsetBytes, lengthBytes and a seed.

    • Field Detail

      • defaultMemReqSvr

        static final MemoryRequestServer defaultMemReqSvr
        The default MemoryRequestServer is used if not specified by the user.

    • Method Detail

      • getMemoryRequestServer

        MemoryRequestServer getMemoryRequestServer()
        Gets the MemoryRequestServer object, if set, for the below resources to request additional memory.

        WritableMemory enables this for ByteBuffer, Heap and Off-heap Memory backed resources.

        WritableBuffer enables this for ByteBuffer backed resources. However, the object returned is in the form of a WritableMemory. To convert to WritableBuffer use asWritableBuffer(). To enable for Heap and Off-heap Buffer resources, use the WritableMemory to configure and then call asWritableBuffer().

        Map backed resources will always return null.

        The user must customize the actions of the MemoryRequestServer by implementing the MemoryRequestServer interface.

        For WritableMemory, to enable at runtime set your custom MemoryRequestServer using one of these methods:

        ForWritableBuffer, to enable at runtime set your custom MemoryRequestServer using the following method:

        Simple implementation examples include the DefaultMemoryRequestServer in the main source tree, as well as the ExampleMemoryRequestServerTest and the use with ByteBuffer documented in the DruidIssue11544Test in the test source tree.

        Returns:
        the MemoryRequestServer object or null.

      • hasMemoryRequestServer

        boolean hasMemoryRequestServer()
        Returns true if the MemoryRequestServer has been configured by the user.
        Returns:
        true if the MemoryRequestServer has been configured by the user..

      • setMemoryRequestServer

        void setMemoryRequestServer​(MemoryRequestServer memReqSvr)
        Sets the Default MemoryRequestServer
        Parameters:
        memReqSvr - the given MemoryRequestServer.

      • close

        void close()
        Closes this resource if this can be closed via AutoCloseable. If this operation completes without exceptions, this resource will be marked as not alive, and subsequent operations on this resource will fail with IllegalStateException.

        This operation is not idempotent; that is, closing an already closed resource always results in an exception being thrown. This reflects a deliberate design choice: resource state transitions should be manifest in the client code; a failure in any of these transitions reveals a bug in the underlying application logic.

        Specified by:
        close in interface AutoCloseable
        Throws:
        IllegalStateException - if this Resource is not alive.
        IllegalStateException - if this method is not accessed from the owning thread.
        UnsupportedOperationException - if this resource is not AutoCloseable.

      • isCloseable

        boolean isCloseable()
        Return true if this resource is closeable.
        Returns:
        true if this resource is closeable.

      • equalTo

        default boolean equalTo​(Resource that)
        Returns true if the given object (that) is an instance of this class and has contents equal to this object.
        Parameters:
        that - the given Resource object
        Returns:
        true if the given object has equal contents to this object.
        See Also:
        equalTo(long, Resource, long, long)

      • equalTo

        boolean equalTo​(long thisOffsetBytes,
                Resource that,
                long thatOffsetBytes,
                long lengthBytes)
        Returns true if the given Resource has equal contents to this object in the given range of bytes. This will also check two distinct ranges within the same object for equals.
        Parameters:
        thisOffsetBytes - the starting offset in bytes for this object.
        that - the given Resource
        thatOffsetBytes - the starting offset in bytes for the given Resource object
        lengthBytes - the size of the range in bytes
        Returns:
        true if the given Resource object has equal contents to this object in the given range of bytes.
        Throws:
        IllegalStateException - if either resource is not alive.
        MemoryBoundsException - if there is a bounds violation.

      • force

        void force()
        Forces any changes made to the contents of this memory-mapped Resource to be written to the storage device described by the configured file descriptor.

        If the file descriptor associated with this memory-mapped Resource resides on a local storage device then when this method returns, it is guaranteed that all changes made to this mapped Resource since it was created, or since this method was last invoked, will have been written to that device.

        If the file descriptor associated with this memory-mapped Resource does not reside on a local device then no such guarantee is made.

        If this memory-mapped Resource was not mapped in read/write mode (FileChannel.MapMode.READ_WRITE) then invoking this method may have no effect. In particular, this method has no effect for files mapped in read-only or private mapping modes. This method may or may not have an effect for implementation-specific mapping modes.

        Throws:
        IllegalStateException - if this Resource is not alive.
        IllegalStateException - if this method is not accessed from the owning thread.
        UnsupportedOperationException - if this Resource is not memory-mapped, e.g. if isMapped() == false.
        ReadOnlyException - if this Resource is read-only.
        RuntimeException - if there is some other error writing the contents of this memory-mapped Resource to the associated storage device.

      • getCapacity

        long getCapacity()
        Gets the capacity of this object in bytes
        Returns:
        the capacity of this object in bytes

      • getCumulativeOffset

        default long getCumulativeOffset()
        Gets the cumulative offset in bytes of this object from the backing resource. This offset may also include other offset components such as the native off-heap memory address, DirectByteBuffer split offsets, region offsets, and unsafe arrayBaseOffsets.
        Returns:
        the cumulative offset in bytes of this object from the backing resource.

      • getCumulativeOffset

        long getCumulativeOffset​(long offsetBytes)
        Gets the cumulative offset in bytes of this object from the backing resource including the given offsetBytes. This offset may also include other offset components such as the native off-heap memory address, DirectByteBuffer split offsets, region offsets, and unsafe arrayBaseOffsets.
        Parameters:
        offsetBytes - offset to be added to the cumulative offset.
        Returns:
        the cumulative offset in bytes of this object from the backing resource including the given offsetBytes.

      • getRelativeOffset

        long getRelativeOffset()
        Returns the offset of address zero of this object relative to the base address of the backing resource. This does not include the object header for heap arrays nor the initial offset of a memory-mapped file.
        Returns:
        the offset of address zero of this object relative to the base address of the backing resource.

      • getTypeByteOrder

        ByteOrder getTypeByteOrder()
        Gets the current ByteOrder. This may be different from the ByteOrder of the backing resource and ByteOrder.nativeOrder()
        Returns:
        the current ByteOrder.

      • hasByteBuffer

        boolean hasByteBuffer()
        Returns true if this Memory is backed by a ByteBuffer.
        Returns:
        true if this Memory is backed by a ByteBuffer.

      • isByteOrderCompatible

        boolean isByteOrderCompatible​(ByteOrder byteOrder)
        Returns true if the Native ByteOrder is the same as the ByteOrder of the current Buffer or Memory and the same ByteOrder as the given byteOrder.
        Parameters:
        byteOrder - the given ByteOrder
        Returns:
        true if the Native ByteOrder is the same as the ByteOrder of the current Buffer or Memory and the same ByteOrder as the given byteOrder.

      • isDirect

        boolean isDirect()
        If true, the backing resource is off-heap memory. This is the case for allocated off-heap memory, memory-mapped files, or from a wrapped ByteBuffer that was allocated off-heap. If false, the backing resource is the Java heap.
        Returns:
        true if the backing resource is off-heap memory.

      • isDuplicate

        boolean isDuplicate()
        Returns true if this instance is a duplicate of a Buffer instance.
        Returns:
        true if this instance is a duplicate of a Buffer instance.

      • isHeap

        boolean isHeap()
        Returns true if this object is backed by an on-heap primitive array or an on-heap ByteBuffer.
        Returns:
        true if this object is backed by an on-heap primitive array or an on-heap ByteBuffer.

      • isLoaded

        boolean isLoaded()
        Tells whether or not the contents of this memory-mapped Resource is resident in physical memory.

        A return value of true implies that it is highly likely that all of the data in this memory-mapped Resource is resident in physical memory and may therefore be accessed without incurring any virtual-memory page faults or I/O operations.

        A return value of false does not necessarily imply that all of the data in this memory-mapped Resource is not resident in physical memory.

        The returned value is a hint, rather than a guarantee, because the underlying operating system may have paged out some of this Resource's data by the time that an invocation of this method returns.

        Returns:
        true if it is likely that all of the data in this memory-mapped Resource is resident in physical memory
        Throws:
        IllegalStateException - if this Resource is not alive.
        IllegalStateException - if this method is not accessed from the owning thread.
        UnsupportedOperationException - if this Resource is not memory-mapped, e.g. if isMapped() == false.

      • isMemory

        boolean isMemory()
        If true, this is a Memory or WritableMemory instance, which provides the Memory API. The Memory API is the principal API for this Memory Component. It provides a rich variety of direct manipulations of four types of resources: On-heap memory, off-heap memory, memory-mapped files, and ByteBuffers. If false, this is a Buffer or WritableBuffer instance, which provides the Buffer API.

        The Buffer API is largely parallel to the Memory API except that it adds a positional API similar to that in ByteBuffer. The positional API is a convenience when iterating over structured arrays, or buffering input or output streams (thus the name).

        Returns:
        true if this is a Memory or WritableMemory instance, which provides the Memory API, otherwise this is a Buffer or WritableBuffer instance, which provides the Buffer API.

      • isMapped

        boolean isMapped()
        Returns true if the backing resource is a memory-mapped file.
        Returns:
        true if the backing resource is a memory-mapped file.

      • isNonNativeOrder

        boolean isNonNativeOrder()
        If true, all put and get operations will assume the non-native ByteOrder. Otherwise, all put and get operations will assume the native ByteOrder.
        Returns:
        true, if all put and get operations will assume the non-native ByteOrder.

      • isReadOnly

        boolean isReadOnly()
        Returns true if this object or the backing resource is read-only.
        Returns:
        true if this object or the backing resource is read-only.

      • isRegionView

        boolean isRegionView()
        Returns true if this instance is a region view of another Memory or Buffer
        Returns:
        true if this instance is a region view of another Memory or Buffer

      • isSameResource

        boolean isSameResource​(Resource that)
        Returns true if the backing resource of this is identical with the backing resource of that. The capacities must be the same. If this is a region, the region offset must also be the same.
        Parameters:
        that - A different non-null Resource
        Returns:
        true if the backing resource of this is the same as the backing resource of that.

      • isAlive

        boolean isAlive()
        Returns true if this object is alive and has not been closed. This is relevant only for off-heap memory and memory-mapped Files.
        Returns:
        true if this object is alive and has not been closed.

      • load

        void load()
        Loads the contents of this memory-mapped Resource into physical memory.

        This method makes a best effort to ensure that, when it returns, this contents of the memory-mapped Resource is resident in physical memory. Invoking this method may cause some number of page faults and I/O operations to occur.

        Throws:
        IllegalStateException - if this Resource is not alive.
        IllegalStateException - if this method is not accessed from the owning thread.
        UnsupportedOperationException - if this Resource is not memory-mapped, e.g. if isMapped() == false.

      • toString

        String toString​(String header,
                long offsetBytes,
                int lengthBytes,
                boolean withData)
        Returns a description of this object with an optional formatted hex string of the data for the specified a range. Used primarily for testing.
        Parameters:
        header - a descriptive header
        offsetBytes - offset bytes relative to this object start
        lengthBytes - number of bytes to convert to a hex string
        withData - include output listing of byte data in the given range
        Returns:
        a formatted hex string in a human readable array

      • toString

        String toString()
        Returns a brief description of this object.
        Overrides:
        toString in class Object
        Returns:
        a brief description of this object.

      • xxHash64

        long xxHash64​(long offsetBytes,
              long lengthBytes,
              long seed)
        Returns the 64-bit hash of the sequence of bytes in this object specified by offsetBytes, lengthBytes and a seed. Note that the sequence of bytes is always processed in the same order independent of endianness.
        Parameters:
        offsetBytes - the given offset in bytes to the first byte of the byte sequence.
        lengthBytes - the given length in bytes of the byte sequence.
        seed - the given long seed.
        Returns:
        the 64-bit hash of the sequence of bytes in this object specified by offsetBytes and lengthBytes.

      • xxHash64

        long xxHash64​(long in,
              long seed)
        Returns a 64-bit hash from a single long. This method has been optimized for speed when only a single hash of a long is required.
        Parameters:
        in - A long.
        seed - A long valued seed.
        Returns:
        the hash.