ModelCacheClient#

class ModelCacheClient(cache_client=None, expiration=3600)#

Bases: object

Cache client to enable model sharing across whole process.

The client by default uses shared memory cache with expiration that allows storing models in memory, reuse them and free up space when the model is unused for a certain period of time. It also provides a lock to prevent downloading the same model by parallel executions.

Attributes Summary

LOADING_MODELS_CACHE_KEY

WAIT_FOR_LOADING_MODEL_ITERATIONS

WAIT_FOR_LOADING_MODEL_SLEEP_SECS

Methods Summary

get_and_save_model(name, load_func)

rtype

~Model

get_model(name)

rtype

Optional[Any]

save_model(name, model)

rtype

None

Attributes Documentation

LOADING_MODELS_CACHE_KEY = '__loading_models'#
WAIT_FOR_LOADING_MODEL_ITERATIONS = 60#
WAIT_FOR_LOADING_MODEL_SLEEP_SECS = 0.5#

Methods Documentation

get_and_save_model(name, load_func)#
Return type

~Model

get_model(name)#
Return type

Optional[Any]

save_model(name, model)#
Return type

None