ModelCacheClient

ModelCacheClient#

class ModelCacheClient(cache_client=None, expiration=3600)#

Bases: object

Cache client to enable model sharing across whole process.

The client by default uses shared memory cache with expiration that allows storing models in memory, reuse them and free up space when the model is unused for a certain period of time. It also provides a lock to prevent downloading the same model by parallel executions.

Attributes Summary

`LOADING_MODELS_CACHE_KEY`
`WAIT_FOR_LOADING_MODEL_ITERATIONS`
`WAIT_FOR_LOADING_MODEL_SLEEP_SECS`

Methods Summary

get_and_save_model(name, load_func)

rtype: ~Model

get_model(name)

rtype: Optional[Any]

save_model(name, model)

rtype: None

Attributes Documentation

LOADING_MODELS_CACHE_KEY = '__loading_models'#

WAIT_FOR_LOADING_MODEL_ITERATIONS = 60#

WAIT_FOR_LOADING_MODEL_SLEEP_SECS = 0.5#

Methods Documentation

get_and_save_model(name, load_func)#

Return type: ~Model

get_model(name)#

Return type: Optional[Any]

save_model(name, model)#

Return type: None