ModelCacheClient

class ModelCacheClient(cache_client=None, expiration=3600)

Bases: object

Cache client to enable model sharing across whole process.

The client by default uses shared memory cache with expiration that allows storing models in memory, reuse them and free up space when the model is unused for a certain period of time. It also provides a lock to prevent downloading the same model by parallel executions.

Attributes Summary

LOADING_MODELS_CACHE_KEY
WAIT_FOR_LOADING_MODEL_ITERATIONS
WAIT_FOR_LOADING_MODEL_SLEEP_SECS

Methods Summary

get_and_save_model(name, load_func)
get_model(name)
save_model(name, model)

Attributes Documentation

LOADING_MODELS_CACHE_KEY = '__loading_models'

WAIT_FOR_LOADING_MODEL_ITERATIONS = 60

WAIT_FOR_LOADING_MODEL_SLEEP_SECS = 0.5

Methods Documentation

get_and_save_model(name, load_func)

Return type:

TypeVar(Model)

get_model(name)

Return type:

Optional[Any]

save_model(name, model)

Return type:

None