Python

Socket module based TStorage communication channel.

class tstorage_client.channel.Channel(host: str, port: int, payload_type: PayloadType[T], timeout: float | None = None, memory_limit: int | None = None)

Bases: _ChannelMixin[T]

Channel provides TStorage communication facilities using socket module.

Channel can be used as context manager to open and close connection. All put and fetched data should be serializable by passed payload_type.

close() → Response

Close underlying socket.

Returns:: OK status if closed socket else ERROR status.

connect() → Response

Connect to specified host at port with set timeout.

Returns:: OK status if connected else ERROR status.

get(key_min: Key, key_max: Key, recv_buffer_size: int = 65536) → ResponseGet[T]

Get records from TStorage.

Records are get up to memory_limit and request fails if there is more data. If request fails at some point it automatically closes connection.

Parameters:

key_min – Lower end of keyrange (inclusive).
key_max – Upper end of keyrange (exclusive).
recv_buffer_size – Initial buffer size for receiving data from network capped by self.memory_limit.

Returns:

OK status, acq, data on success else status indicates error, data stores partial result and acq is invalid.

get_acq(key_min: Key, key_max: Key) → ResponseAcq

Get last acq value from TStorage.

Parameters:

key_min – Lower end of keyrange (inclusive).
key_max – Upper end of keyrange (exclusive).

Returns:

OK status, acq on success else status indicates error and acq is invalid.

get_iter(key_min: Key, key_max: Key, recv_buffer_size: int = 65536) → Iterator[Record[T] | ResponseAcq]

Get records from TStorage as iterator.

If request fails at some point it automatically closes connection. Records are yielded after whole batch parsing has been done. Remember to check for final response with OK status.

Parameters:

key_min – Lower end of keyrange (inclusive).
key_max – Upper end of keyrange (exclusive).
recv_buffer_size – Initial buffer size for receiving data from network capped by self.memory_limit.

Yields:

Record or response control data indicating error or successfully finished request.

get_stream(key_min: Key, key_max: Key, callback: Callable[[RecordsSet[T]], None], recv_buffer_size: int = 65536) → ResponseAcq

Get records from TStorage.

Records are get in memory_limit sized batches. If request fails at some point it automatically closes connection.

Parameters:

key_min – Lower end of keyrange (inclusive).
key_max – Upper end of keyrange (exclusive).
callback – Callable to call on each batch.
recv_buffer_size – Initial buffer size for receiving data from network capped by self.memory_limit.

Returns:

OK status, acq on success else status indicates error and acq is invalid.

property memory_limit: int | None: Get or set memory limit in bytes for GET requests.

put(data: RecordsSet[T], max_batch_size: int = 2147483647, skip_invalid: bool = False) → Response

Put records to TStorage without acq value.

Parameters:

data – Records to put.
max_batch_size – Controls maximal serialization buffer size. Must be lower than 2147483648 (2GiB). Despite this value always at least 1 record will be serialized.
skip_invalid – Skip invalid records. Otherwise finishes put immediately at invalid record.

Returns:

OK status on success else status indicates error.

puta(data: RecordsSet[T], max_batch_size: int = 2147483647, skip_invalid: bool = False) → Response

Put records to TStorage with acq value.

Parameters:

data – Records to put.
max_batch_size – Controls maximal serialization buffer size. Must be lower than 2147483648 (2GiB). Despite this value always at least 1 record will be serialized.
skip_invalid – Skip invalid records. Otherwise finishes puta immediately at invalid record.

Returns:

OK status on success else status indicates error.

property timeout: float | None: Get or set socket timeout

Asyncio module based TStorage communication channel.

class tstorage_client.channel_async.AsyncChannel(host: str, port: int, payload_type: PayloadType[T], memory_limit: int | None = None)

Bases: _ChannelMixin[T]

AsyncChannel provides TStorage communication facilities using asyncio module.

AsyncChannel can be used as async context manager to open and close connection. All put and fetched data should be serializable by passed payload_type. AsyncChannel doesn’t provide timeout. Please use functions from https://docs.python.org/3/library/asyncio-task.html#timeouts.

async close() → Response

Close underlying connection.

Returns:: OK status if closed connection else ERROR status.

async connect() → Response

Connect to specified host at port.

Returns:: OK status if connected else ERROR status.

async get(key_min: Key, key_max: Key, recv_buffer_size: int = 65536) → ResponseGet[T]

Get records from TStorage.

Records are get up to memory_limit and request fails if there is more data. If request fails at some point it automatically closes connection.

Parameters:

key_min – Lower end of keyrange (inclusive).
key_max – Upper end of keyrange (exclusive).
recv_buffer_size – Initial buffer size for receiving data from network capped by self.memory_limit.

Returns:

OK status, acq, data on success else status indicates error, data stores partial result and acq is invalid.

async get_acq(key_min: Key, key_max: Key) → ResponseAcq

Get last acq value from TStorage.

Parameters:

key_min – Lower end of keyrange (inclusive).
key_max – Upper end of keyrange (exclusive).

Returns:

OK status, acq on success else status indicates error and acq is invalid.

async get_iter(key_min: Key, key_max: Key, recv_buffer_size: int = 65536) → AsyncIterator[Record[T] | ResponseAcq]

Get records from TStorage as iterator.

If request fails at some point it automatically closes connection. Records are yielded after whole batch parsing has been done. Remember to check for final response with OK status.

Parameters:

key_min – Lower end of keyrange (inclusive).
key_max – Upper end of keyrange (exclusive).
recv_buffer_size – Initial buffer size for receiving data from network capped by self.memory_limit.

Yields:

Record or response control data indicating error or successfully finished request.

async get_stream(key_min: Key, key_max: Key, callback: Callable[[RecordsSet[T]], None], recv_buffer_size: int = 65536) → ResponseAcq

Get records from TStorage.

Records are get in memory_limit sized batches. If request fails at some point it automatically closes connection.

Parameters:

key_min – Lower end of keyrange (inclusive).
key_max – Upper end of keyrange (exclusive).
callback – Callable to call on each batch.
recv_buffer_size – Initial buffer size for receiving data from network capped by self.memory_limit.

Returns:

OK status, acq on success else status indicates error and acq is invalid.

property memory_limit: int | None: Get or set memory limit in bytes for GET requests.

async put(data: RecordsSet[T], max_batch_size: int = 2147483647, skip_invalid: bool = False) → Response

Put records to TStorage without acq value.

Parameters:

data – Records to put.
max_batch_size – Controls maximal serialization buffer size. Must be lower than 2147483648 (2GiB). Despite this value always at least 1 record will be serialized.
skip_invalid – Skip invalid records. Otherwise finishes put immediately at invalid record.

Returns:

OK status on success else status indicates error.

async puta(data: RecordsSet[T], max_batch_size: int = 2147483647, skip_invalid: bool = False) → Response

Put records to TStorage with acq value.

Parameters:

data – Records to put.
max_batch_size – Controls maximal serialization buffer size. Must be lower than 2147483648 (2GiB). Despite this value always at least 1 record will be serialized.
skip_invalid – Skip invalid records. Otherwise finishes puta immediately at invalid record.

Returns:

OK status on success else status indicates error.

Provides PayloadType interface and simple ready to use implementations

class tstorage_client.payload_type.BytesPayloadType

Bases: PayloadType[bytes]

Serialization of raw bytes-like payloads.

Builtin bytearray and memoryview are considered by mypy to be bytes compatible so other type than bytes can be returned. (https://mypy.readthedocs.io/en/latest/duck_type_compatibility.html).

from_bytes(buffer: bytes) → bytes: Returns buffer as is.

to_bytes(value: bytes) → bytes: Returns value as is.

class tstorage_client.payload_type.PayloadType

Bases: ABC, Generic[T]

Interface for payload serializers.

abstract from_bytes(buffer: bytes) → T | None: Converts bytes to value or None in case of failure.

abstract to_bytes(value: T) → bytes: Converts provided value to bytes to be send to TStorage.

class tstorage_client.payload_type.StructPayloadType(format: str)

Bases: PayloadType[T]

Class for simple struct module based serialization.

Only 1 value formats are supported.

from_bytes(buffer: bytes) → T | None: Converts bytes to value or None in case of failure.

to_bytes(value: T) → bytes: Converts provided value to bytes to be send to TStorage.

class tstorage_client.payload_type.UnitPayloadType

Bases: PayloadType[tuple[()]]

Serialization of empty payloads.

from_bytes(_: bytes) → tuple[()]: Returns empty tuple. It never fails so it never returns None.

to_bytes(_: Any) → Literal[b'']: Converts provided value to empty bytes object.

class tstorage_client.record.Key(cid: int = -1, mid: int = -1, moid: int = -1, cap: int = -1, acq: int = -1)

Bases: object

TStorage Key.

Keys must be unique amongst whole TStorage.

cid

Integer in range 0 to 2^31 - 1.

Type:: int

mid

Integer in range -2^63 to 2^63 - 1.

Type:: int

moid

Integer in range -2^31 to 2^31 - 1.

Type:: int

cap

Integer in range -2^63 to 2^63 - 1.

Type:: int

acq

Integer in range -2^63 to 2^63 - 1.

Type:: int

acq: int

classmethod bytes_count() → int

cap: int

cid: int

classmethod from_bytes(data: bytes) → Key | None

classmethod max() → Key

mid: int

classmethod min() → Key

moid: int

to_bytes(*, with_cid: bool = True, with_acq: bool = True) → bytes

valid() → bool

class tstorage_client.record.Record(key: Key, value: T)

Bases: Generic[T]

Record as stored by TStorage with type generic value.

key

Key idnetyfying this Record.

Type:: tstorage_client.record.Key

value

Payload as stored by TStorage.

Type:: tstorage_client.record.T

key: Key

value: T

tstorage_client.records_set.GetCallback

GetCallback[T] is an generic alias to any callable taking RecordsSet[T] as argument and returning nothing.

alias of Callable[[RecordsSet[T]], None]

class tstorage_client.records_set.RecordsSet(*args, **kwargs)

Bases: Iterable[Record[T]], Sized, Protocol

abstract append(value: Record[T]) → None

Module containing network response class hierarchy

class tstorage_client.response.Response(status: ResponseStatus)

Bases: object

Base response from network calls.

status

Error status of function.

Type:: tstorage_client.response.ResponseStatus

is_ok() → bool: Check if status does not represent error condition.

status: ResponseStatus

class tstorage_client.response.ResponseAcq(status: ResponseStatus, acq: int = -1)

Bases: Response

Base response from GET call.

status

Error status of function.

Type:: tstorage_client.response.ResponseStatus

acq

Returned acq value from TStorage.

Type:: int

acq: int

status: ResponseStatus

class tstorage_client.response.ResponseGet(status: ~tstorage_client.response.ResponseStatus, acq: int = -1, data: ~tstorage_client.records_set.RecordsSet[~tstorage_client.response.T] = <factory>)

Bases: ResponseAcq, Generic[T]

Response from GET call that carries records.

status

Error status of function.

Type:: tstorage_client.response.ResponseStatus

acq

Returned acq value from TStorage.

Type:: int

data

Returned records of requested type from TStorage.

Type:: tstorage_client.records_set.RecordsSet[tstorage_client.response.T]

acq: int

data: RecordsSet[T]

status: ResponseStatus

class tstorage_client.response.ResponseStatus(value)

Bases: IntEnum

Enumeration of possible return codes from various functions.

BAD_REQUEST = 129

DISCONNECTED = 128

ERROR = -1

NO_MEMORY = 131

OK = 0

UNPARSEABLE_ENTITY = 130

is_ok() → bool: Check if status does not represent error condition.

Utilities for storing and converting TStorage and unix timestamps

class tstorage_client.timestamp.Tstoragedatetime(datetime: datetime, nanoseconds: int)

Bases: object

Tstoragedatetime stores nanosecond precise time in friendly manner.

datetime

Python datetime part of nanosecond precise time.

Type:: datetime.datetime

nanoseconds

Remaining nanoseconds in range 0-999.

Type:: int

datetime: datetime

classmethod from_tstorage(timestamp: int | float, *, tzinfo: tzinfo = datetime.timezone.utc) → Tstoragedatetime: Create Tstoragedatetime from TStorage seconds.

classmethod from_tstorage_ns(timestamp: int, *, tzinfo: tzinfo = datetime.timezone.utc) → Tstoragedatetime

Create Tstoragedatetime from TStorage nanoseconds.

This function directly converts cap and acq values to user friendly format.

classmethod from_unix(timestamp: int | float, *, tzinfo: tzinfo = datetime.timezone.utc) → Tstoragedatetime: Create Tstoragedatetime from unix seconds.

classmethod from_unix_ns(timestamp: int, *, tzinfo: tzinfo = datetime.timezone.utc) → Tstoragedatetime: Create Tstoragedatetime from unix nanoseconds.

nanoseconds: int

to_tstorage() → float: Convert to TStorage seconds with nanosecond precise fractions.

to_tstorage_ns() → int: Convert to TStorage nanoseconds.

to_unix() → float: Convert to unix seconds with nanosecond precise fractions.

to_unix_ns() → int: Convert to unix nanoseconds.

tstorage_client.timestamp.from_unix(timestamp: int) → int

tstorage_client.timestamp.from_unix(timestamp: float) → float

Converts unix second timestamp to TStorage second timestamp.

Parameters:: timestamp – Unix timestamp in seconds.
Returns:: TStorage timestamp in seconds.

tstorage_client.timestamp.from_unix_ns(timestamp: int) → int

Converts unix nanosecond timestamp to TStorage nanosecond timestamp.

Parameters:: timestamp – Unix timestamp in nanoseconds.
Returns:: TStorage timestamp in nanoseconds.

tstorage_client.timestamp.now() → float: Get current TStorage timestamp in seconds

tstorage_client.timestamp.now_ns() → int: Get current TStorage timestamp in nanoseconds

tstorage_client.timestamp.to_unix(timestamp: int) → int

tstorage_client.timestamp.to_unix(timestamp: float) → float

Converts TStorage second timestamp to unix second timestamp.

Parameters:: timestamp – TStorage timestamp in seconds.
Returns:: Unix timestamp in seconds.

tstorage_client.timestamp.to_unix_ns(timestamp: int) → int

Converts TStorage nanosecond timestamp to unix nanosecond timestamp.

Parameters:: timestamp – TStorage timestamp in nanoseconds.
Returns:: Unix timestamp in nanoseconds.

TStorage client package.