allensdk.api.warehouse_cache.caching_utilities module

allensdk.api.warehouse_cache.caching_utilities.call_caching(fetch: Callable[[], ~Q], write: Callable[[~Q], NoneType], read: Union[Callable[[], ~P], NoneType] = None, pre_write: Union[Callable[[~Q], ~Q], NoneType] = None, cleanup: Union[Callable[[], NoneType], NoneType] = None, lazy: bool = True, num_tries: int = 1, failure_message: str = '') → Union[~P, NoneType][source]

Access data, caching on a local store for future accesses.

Parameters:
fetch :

Function which pulls data from a remote/expensive source.

write :

Function which stores data in a local/inexpensive store.

read :

Function which pulls data from a local/inexpensive store.

pre_write :

Function applied to obtained data after fetching, but before writing.

cleanup :

Function for fixing a failed fetch. e.g. unlinking a partially downloaded file. Exceptions raised by cleanup are not themselves handled

lazy :

If True, attempt to read the data from the local/inexpensive store before fetching it. If False, forcibly fetch from the remote/expensive store.

num_tries :

How many fetches to attempt before (re)raising an exception. A fetch is failed if reading the result raises an exception.

failure_message :

Provides additional context in the event of a failed download. Emitted when retrying, and when a fetch failure occurs after tries are exhausted

Returns:
The result of calling read
allensdk.api.warehouse_cache.caching_utilities.one_file_call_caching(path: Union[pathlib.Path, str], fetch: Callable[[], ~Q], write: Callable[[Union[pathlib.Path, str], ~Q], NoneType], read: Union[Callable[[Union[pathlib.Path, str]], ~P], NoneType] = None, pre_write: Union[Callable[[~Q], ~Q], NoneType] = None, cleanup: Union[Callable[[], NoneType], NoneType] = None, lazy: bool = True, num_tries: int = 1, failure_message: str = '') → Union[~P, NoneType][source]

A call_caching variant where the local store is a single file. See call_caching for complete documentation.

Parameters:
path :

Path at which the data will be stored