allensdk.api.warehouse_cache.caching_utilities module

allensdk.api.warehouse_cache.caching_utilities.call_caching(fetch: Callable[[], Q], write: Callable[[Q], None], read: Callable[[], P], pre_write: Callable[[Q], Q] | None = None, cleanup: Callable[[], None] | None = None, lazy: bool = True, num_tries: int = 1, failure_message: str = '') → P

allensdk.api.warehouse_cache.caching_utilities.call_caching(fetch: Callable[[], Q], write: Callable[[Q], None], read: None = None, pre_write: Callable[[Q], Q] | None = None, cleanup: Callable[[], None] | None = None, lazy: bool = True, num_tries: int = 1, failure_message: str = '') → None

Access data, caching on a local store for future accesses.

Parameters:

fetch

Function which pulls data from a remote/expensive source.

write

Function which stores data in a local/inexpensive store.

read

Function which pulls data from a local/inexpensive store.

pre_write

Function applied to obtained data after fetching, but before writing.

cleanup

Function for fixing a failed fetch. e.g. unlinking a partially downloaded file. Exceptions raised by cleanup are not themselves handled

lazy

If True, attempt to read the data from the local/inexpensive store before fetching it. If False, forcibly fetch from the remote/expensive store.

num_tries

How many fetches to attempt before (re)raising an exception. A fetch is failed if reading the result raises an exception.

failure_message

Provides additional context in the event of a failed download. Emitted when retrying, and when a fetch failure occurs after tries are exhausted

Returns:

The result of calling read

allensdk.api.warehouse_cache.caching_utilities.one_file_call_caching(path: Path | str, fetch: Callable[[], Q], write: Callable[[Path | str, Q], None], read: Callable[[Path | str], P] | None = None, pre_write: Callable[[Q], Q] | None = None, cleanup: Callable[[], None] | None = None, lazy: bool = True, num_tries: int = 1, failure_message: str = '') → P | None

A call_caching variant where the local store is a single file. See call_caching for complete documentation.

Parameters:

path

Path at which the data will be stored