Caching in PyOP2¶
PyOP2 makes heavy use of caches to ensure performance is not adversely affected by too many runtime computations. The caching in PyOP2 takes a number of forms:
Disk-based caching of generated code
Since compiling a generated code module may be an expensive operation, PyOP2 caches the generated code on disk such that subsequent runs of the same simulation will not have to pay a compilation cost.
In memory caching of generated code function pointers
Once code has been generated and loaded into the running PyOP2 process, we cache the resulting callable function pointer for the lifetime of the process, such that subsequent calls to the same generated code are fast.
In memory caching of expensive to build objects
Some PyOP2 objects, in particular
Sparsityobjects, can be expensive to construct. Since a sparsity does not change if it is built again with the same arguments, we only construct the sparsity once for each unique set of arguments.
The caching strategies for PyOP2 follow from two axioms:
Maps, equality is identity
Caches of generated code should depend on metadata, but not data
The first axiom implies that two
Maps compare equal if and only if they are the same
object. The second implies that generated code must be independent
of the absolute size of the data the
generated it executed over. For example, the size of the iteration
set should not be part of the key, but the arity of any maps and size
and type of every data item should be.
On consequence of these rules is that there are effectively two separate types of cache in PyOP2, object and class caches, distinguished by where the cache itself lives.
These are used to cache objects that depend on metadata, but not
object instances, such are generated code. They are implemented by
the cacheable class inheriting from
There is currently no eviction strategy for class caches, should they grow too large, for example by executing many different parallel loops, an out of memory error can occur
These are used to cache objects that are built on top of
Maps. They are implemented by the
cacheable class inheriting from
ObjectCached and the
caching instance defining a
The motivation for these caches is that cache key for objects such as sparsities relies on an identical sparsity being built if the arguments are identical. So that users of the API do not have to worry too much about carrying around “temporary” objects forever such that they will hit caches, PyOP2 builds up a hierarchy of caches of transient objects on top of the immutable sets and maps.
So, for example, the user can build and throw away
DataSets as normal in their code. Internally, however,
these instances are cached on the set they are built on top of. Thus,
in the following snippet, we have that
ds2 are the same
s = op2.Set(1)
ds = op2.DataSet(s, 10)
ds2 = op2.DataSet(s, 10)
assert ds is ds2
The setup of these caches is such that the lifetime of objects in the
cache is tied to the lifetime of both the caching and the cached
object. In the above example, as long as the user program holds a
reference to one of
ds2 all three objects will
remain live. As soon as all references are lost, all three become
candidates for garbage collection.
The cache eviction strategy for these caches relies on the Python garbage collector, and hence on the user not holding onto references to some of either the cached or the caching objects for too long. Should the objects on which the caches live persist, an out of memory error may occur.
Debugging cache leaks¶
To debug potential problems with the cache, PyOP2 can be instructed to
print the size of both object and class caches at program exit. This
can be done by setting the environment variable
PYOP2_PRINT_CACHE_SIZE to 1 before running a PyOP2 program, or