org.apache.datasketches.memory (datasketches-memory-java17

package org.apache.datasketches.memory

This package defines the interfaces for the DataSketches-Memory project for Java 17, 21, and 25.

This project provides high performance primitive and primitive array access to direct (native) off-heap memory, and consistent views into ByteBuffer, and on-heap primitive arrays. It can be used as a more comprehensive and flexible replacement for ByteBuffer.

In addition, this project provides:

Two different access APIs: read-only Memory and WritableMemory for absolute offset access, and read-only Buffer and WritableBuffer for relative positional access that is similar to ByteBuffer.
Clean separation of Read-only API from Writable API, which makes writable versus read-only resources detectable at compile time.
The conversion from Writable to read-only is just a cast, so no unnecessary objects are created. For example:
```
     WritableMemory wMem = ...
     Memory mem = wMem;
 
```
AutoCloseable for the external resources that require it, which enables compile-time checks for non-closed resources.
Immediate invalidation of all downstream references of an AutoCloseable resource when that resource is closed, either manually or by the JVM. This virtually eliminates the possibility of accidentally writing into the memory space previously owned by a closed resource.
Improved performance over previous implementations.
No external runtime dependencies, which makes it simple to install in virtually any Java environment.

More specifically, this package provides access to three different types of resources using two different access APIs. These resources are contiguous blobs of bytes that provide at least byte-level read and write access. The three resources are:

Direct (a.k.a. Native) off-heap memory allocated by the user.
ByteBuffers, both heap-based and direct, writable and read-only.
Heap-based primitive arrays, which can be accessed as writable or read-only.

The two different access APIs are:

Memory, WritableMemory: Absolute offset addressing into a resource.
Buffer, WritableBuffer: Position relative addressing into a resource.

In addition, all combinations of access APIs and backing resources can be accessed via multibyte primitive methods (e.g. getLong(...), getLongArray(...), putLong(...), putLongArray(...)) as either ByteOrder.BIG_ENDIAN or ByteOrder.LITTLE_ENDIAN.

The resources don't know or care about the access APIs, and the access APIs don't really know or care what resource they are accessing.

An access API is joined with a resource with a static factory method.

Moving back and forth between Memory and Buffer:

    Memory mem = ...
    Buffer buf = mem.asBuffer();
    ...
    Memory mem2 = buf.asMemory();
    ...

Hierarchical memory regions can be easily created:

     WritableMemory wMem = ...
     WritableMemory wReg = wMem.writableRegion(offset, length); //OR
     Memory region = wMem.region(offset, length);

Author:: Lee Rhodes

Class

Description

Buffer

Defines the read-only API for relative positional access to a resource.

BufferPositionInvariantsException

Position operation violation.

DefaultMemoryRequestServer

This example MemoryRequestServer is simple but demonstrates one of many ways to manage continuous requests for larger memory.

Memory

Defines the read-only API for offset access to a resource.

MemoryBoundsException

Specific RuntimeException for bounds violations.

MemoryException

Specific RuntimeExceptions for the datasketches-memory component.

MemoryRequestServer

The MemoryRequestServer is a callback interface to provide a means for off-heap, heap and ByteBuffer backed resources to request more memory.

MurmurHash3v2

The MurmurHash3 is a fast, non-cryptographic, 128-bit hash function that has excellent avalanche and 2-way bit independence properties.

Positional

Defines the relative positional API.

ReadOnlyException

The exception thrown when attempting to write into a read-only Resource.

Resource

The base class for Memory and Buffer plus some common static variables and check methods.

WritableBuffer

Defines the writable API for relative positional access to a resource

WritableMemory

Defines the writable API for offset access to a resource.

XxHash

The XxHash is a fast, non-cryptographic, 64-bit hash function that has excellent avalanche and 2-way bit independence properties.

Package org.apache.datasketches.memory