mystic.cache module documentation

archive module

klepto archive readers and writers, for functions and data

_prep_dist(distances)

create a dict of mean and max graphical distance

Parameters:

distances (array[float]) – indicates the graphical distance

Returns:

dict(mean=distances.mean(), max=distances.max())

_read_func(name, keymap=None, type=None)

read function db with name ‘name’

Parameters:

  • name (string) – filename of the klepto db

  • keymap (klepto.keymap) – keymap used for key encoding

  • type (klepto.archive) – type of klepto archive

Returns:

klepto db object (has dictionary-like interface)

_write_func(archive, func, dist, keymap=None)

write stored function and distance information to archive

Parameters:

  • archive (klepto.archive) – function archive (output of _read_func)

  • func (function) – with interface y = f(x), x is a list of floats

  • dist (dict) – distance information

  • keymap (klepto.keymap) – keymap used for key encoding

Returns:

None

class dict_archive(name=None, dict=None, cached=True, **kwds)

Bases: dict_archive

initialize a dictionary archive

static __new__(dict_archive, name=None, dict=None, cached=True, **kwds)

initialize a dictionary with an in-memory dictionary archive backend

Parameters:

  • name (str, default=None) – (optional) identifier string

  • dict (dict, default={}) – initial dictionary to seed the archive

  • cached (bool, default=True) – interact through an in-memory cache

classmethod from_frame(dataframe)
class dir_archive(name=None, dict=None, cached=True, **kwds)

Bases: dir_archive

initialize a file folder with a synchronized dictionary interface

Parameters:

  • dirname (str, default='memo') – path of the archive root directory

  • serialized (bool, default=True) – save python objects in pickled files

  • compression (int, default=0) – compression level (0 to 9), 0 is None

  • permissions (octal, default=0o775) – read/write permission indicator

  • memmode (str, default=None) – mode, one of {None, 'r+', 'r', 'w+', 'c'}

  • memsize (int, default=100) – size (MB) of cache for in-memory compression

  • protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol

static __new__(dir_archive, name=None, dict=None, cached=True, **kwds)

initialize a dictionary with a file-folder archive backend

Parameters:

  • name (str, default='memo') – path of the archive root directory

  • dict (dict, default={}) – initial dictionary to seed the archive

  • cached (bool, default=True) – interact through an in-memory cache

  • serialized (bool, default=True) – save python objects in pickled files

  • compression (int, default=0) – compression level (0 to 9), 0 is None

  • permissions (octal, default=0o775) – read/write permission indicator

  • memmode (str, default=None) – mode, one of {None, 'r+', 'r', 'w+', 'c'}

  • memsize (int, default=100) – size (MB) of cache for in-memory compression

  • protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol

classmethod from_frame(dataframe)
class file_archive(name=None, dict=None, cached=True, **kwds)

Bases: file_archive

initialize a file with a synchronized dictionary interface

Parameters:

  • filename (str, default='memo.pkl') – path of the file archive

  • serialized (bool, default=True) – save python objects in pickled file

  • protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol

static __new__(file_archive, name=None, dict=None, cached=True, **kwds)

initialize a dictionary with a single file archive backend

Parameters:

  • name (str, default='memo.pkl') – path of the file archive

  • dict (dict, default={}) – initial dictionary to seed the archive

  • cached (bool, default=True) – interact through an in-memory cache

  • serialized (bool, default=True) – save python objects in pickled file

  • protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol

classmethod from_frame(dataframe)
get_dist(archive, func, keymap=None)

get the graphical distance of func from data in archive

Parameters:

  • archive (klepto.archive) – run archive (output of read)

  • func (function) – with interface y = f(x), x is a list of floats

  • keymap (klepto.keymap) – keymap used for key encoding

Returns:

array of floats, graphical distance from func to each point in archive

class hdf_archive(name=None, dict=None, cached=True, **kwds)

Bases: hdf_archive

initialize an archive

static __new__(hdf_archive, name=None, dict=None, cached=True, **kwds)

initialize a dictionary with a single hdf5 file archive backend

Parameters:

  • name (str, default='memo.hdf5') – path of the file archive

  • dict (dict, default={}) – initial dictionary to seed the archive

  • cached (bool, default=True) – interact through an in-memory cache

  • serialized (bool, default=True) – pickle saved python objects

  • protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol

  • meta (bool, default=False) – store in root metadata (not in dataset)

classmethod from_frame(dataframe)
class hdfdir_archive(name=None, dict=None, cached=True, **kwds)

Bases: hdfdir_archive

initialize an archive

static __new__(hdfdir_archive, name=None, dict=None, cached=True, **kwds)

initialize a dictionary with a hdf5 file-folder archive backend

Parameters:

  • name (str, default='memo') – path of the archive root directory

  • dict (dict, default={}) – initial dictionary to seed the archive

  • cached (bool, default=True) – interact through an in-memory cache

  • serialized (bool, default=True) – pickle saved python objects

  • permissions (octal, default=0o775) – read/write permission indicator

  • protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol

  • meta (bool, default=False) – store in root metadata (not in dataset)

classmethod from_frame(dataframe)
class null_archive(name=None, dict=None, cached=True, **kwds)

Bases: null_archive

initialize a permanently-empty dictionary

static __new__(null_archive, name=None, dict=None, cached=True, **kwds)

initialize a dictionary with a permanently-empty archive backend

Parameters:

  • name (str, default=None) – (optional) identifier string

  • dict (dict, default={}) – initial dictionary to seed the archive

  • cached (bool, default=True) – interact through an in-memory cache

classmethod from_frame(dataframe)
read(name, keys=None, keymap=None, type=None)

read klepto db with name ‘name’

Parameters:

  • name (string) – filename of the klepto db

  • keys (iterable) – keys to load; or boolean to load all/no keys

  • keymap (klepto.keymap) – used for key encoding

  • type (klepto.archive) – type of klepto archive

Returns:

klepto db object (has dictionary-like interface)

Notes

If keys is None, create a direct handle to the db. If a key in keys is not found, it will be ignored.

read_func(name, keymap=None, type=None, n=0)

read stored function from db with name ‘name’

Parameters:

  • name (string) – filename of the klepto db

  • keymap (klepto.keymap) – keymap used for key encoding

  • type (klepto.archive) – type of klepto archive

  • n (int) – db entry in reverse order (i.e. most recent is 0)

Returns:

tuple of (stored function, distance information)

Notes

If the db is empty, or n produces a bad index, returns None. Alternately, name can be the relevant klepto.archive instance.

class sql_archive(name=None, dict=None, cached=True, **kwds)

Bases: sqltable_archive

initialize a sql database with a synchronized dictionary interface

Connect to an existing database, or initialize a new database, at the selected database url. For example, to use a sqlite database ‘foo.db’ in the current directory, database=’sqlite:///foo.db’. To use a mysql or postgresql database, sqlalchemy must be installed. When connecting to sqlite, the default database is ‘:memory:’. Storable values are limited to strings, integers, floats, and other basic objects. To store functions, classes, and similar constructs, sqlalchemy must be installed.

Parameters:

  • database (str, default=None) – database url (see above note)

  • table (str, default='memo') – name of the associated database table

static __new__(sql_archive, name=None, dict=None, cached=True, **kwds)

initialize a dictionary with a sql database archive backend

Connect to an existing database, or initialize a new database, at the selected database url. For example, to use a sqlite database ‘foo.db’ in the current directory, database=’sqlite:///foo.db’. To use a mysql database ‘foo’ on localhost, database=’mysql://user:pass@localhost/foo’. For postgresql, use database=’postgresql://user:pass@localhost/foo’. When connecting to sqlite, the default database is ‘:memory:’; otherwise, the default database is ‘defaultdb’. If sqlalchemy is not installed, storable values are limited to strings, integers, floats, and other basic objects. If sqlalchemy is installed, additional keyword options can provide database configuration, such as connection pooling. To use a mysql or postgresql database, sqlalchemy must be installed.

Parameters:

  • name (str, default=None) – database url (see above note)

  • dict (dict, default={}) – initial dictionary to seed the archive

  • cached (bool, default=True) – interact through an in-memory cache

  • serialized (bool, default=True) – save objects as pickled strings

  • protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol

classmethod from_frame(dataframe)
class sqltable_archive(name=None, dict=None, cached=True, **kwds)

Bases: sqltable_archive

initialize a sql database with a synchronized dictionary interface

Connect to an existing database, or initialize a new database, at the selected database url. For example, to use a sqlite database ‘foo.db’ in the current directory, database=’sqlite:///foo.db’. To use a mysql or postgresql database, sqlalchemy must be installed. When connecting to sqlite, the default database is ‘:memory:’. Storable values are limited to strings, integers, floats, and other basic objects. To store functions, classes, and similar constructs, sqlalchemy must be installed.

Parameters:

  • database (str, default=None) – database url (see above note)

  • table (str, default='memo') – name of the associated database table

static __new__(sqltable_archive, name=None, dict=None, cached=True, **kwds)

initialize a dictionary with a sql database table archive backend

Connect to an existing database, or initialize a new database, at the selected database url. For example, to use a sqlite database ‘foo.db’ in the current directory, database=’sqlite:///foo.db’. To use a mysql database ‘foo’ on localhost, database=’mysql://user:pass@localhost/foo’. For postgresql, use database=’postgresql://user:pass@localhost/foo’. When connecting to sqlite, the default database is ‘:memory:’; otherwise, the default database is ‘defaultdb’. Connections should be given as database?table=tablename; for example, name=’sqlite:///foo.db?table=bar’. If not provided, the default tablename is ‘memo’. If sqlalchemy is not installed, storable values are limited to strings, integers, floats, and other basic objects. If sqlalchemy is installed, additional keyword options can provide database configuration, such as connection pooling. To use a mysql or postgresql database, sqlalchemy must be installed.

Parameters:

  • name (str, default=None) – url for database table (see above note)

  • dict (dict, default={}) – initial dictionary to seed the archive

  • cached (bool, default=True) – interact through an in-memory cache

  • serialized (bool, default=True) – save objects as pickled strings

  • protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol

classmethod from_frame(dataframe)
write(archive, entries, keymap=None)

write entries to klepto db instance ‘archive’

Parameters:

  • archive (klepto.archive) – archive instance

  • entries (iterable) – iterable/dict to update the archive with

  • keymap (klepto.keymap) – keymap used for key encoding

Returns:

None

Examples

>>> write(foo, {x0:y0, x1:y1, ...})
>>> write(foo, ((x0,y0), (x1,y1), ...))

Notes

Within entries, xi is a tuple of floats, and yi is a float.

function module

higher-level readers and writers for stored functions

db(name)

get function db with the name ‘name’

read(archives)

read stored functions from the list of dbs

Parameters:

archives (list[string]) – list of names of function archives

Returns:

a klepto.archive instance

Notes

The order of the dbs is important, with the index of archives corresponding to the desired axis. If a db is empty, returns None for the empty db. Also, a klepto.archive instance can be provided instead of the name of the db.

write(function, archives)

write function to corresponding archives