API Reference

class loman.States

Possible states for a computation node

COMPUTABLE = 3
ERROR = 5
PINNED = 6
PLACEHOLDER = 0
STALE = 2
UNINITIALIZED = 1
UPTODATE = 4
class loman.Computation
add_map_node(result_node, input_node, subgraph, subgraph_input_node, subgraph_output_node)

Apply a graph to each element of iterable

In turn, each element in the input_node of this graph will be inserted in turn into the subgraph’s subgraph_input_node, then the subgraph’s subgraph_output_node calculated. The resultant list, with an element or each element in input_node, will be inserted into result_node of this graph. In this way add_map_node is similar to map in functional programming.

Parameters:
  • result_node – The node to place a list of results in this graph
  • input_node – The node to get a list input values from this graph
  • subgraph – The graph to use to perform calculation for each element
  • subgraph_input_node – The node in subgraph to insert each element in turn
  • subgraph_output_node – The node in subgraph to read the result for each element
add_named_tuple_expansion(name, namedtuple_type, group=None)

Automatically add nodes to extract each element of a named tuple type

It is often convenient for a calculation to return multiple values, and it is polite to do this a namedtuple rather than a regular tuple, so that later users have same name to identify elements of the tuple. It can also help make a computation clearer if a downstream computation depends on one element of such a tuple, rather than the entire tuple. This does not affect the computation per se, but it does make the intention clearer.

To avoid having to create many boiler-plate node definitions to expand namedtuples, the add_named_tuple_expansion method automatically creates new nodes for each element of a tuple. The convention is that an element called ‘element’, in a node called ‘node’ will be expanded into a new node called ‘node.element’, and that this will be applied for each element.

Example:

>>> from collections import namedtuple
>>> Coordinate = namedtuple('Coordinate', ['x', 'y'])
>>> comp = Computation()
>>> comp.add_node('c', value=Coordinate(1, 2))
>>> comp.add_named_tuple_expansion('c', Coordinate)
>>> comp.compute_all()
>>> comp.value('c.x')
1
>>> comp.value('c.y')
2
Parameters:
  • name – Node to cera
  • namedtuple_type (namedtuple class) – Expected type of the node
add_node(name, func=None, **kwargs)

Adds or updates a node in a computation

Parameters:
  • name – Name of the node to add. This may be any hashable object.
  • func (Function, default None) – Function to use to calculate the node if the node is a calculation node. By default, the input nodes to the function will be implied from the names of the function parameters. For example, a parameter called a would be taken from the node called a. This can be modified with the kwds parameter.
  • args (List, default None) – Specifies a list of nodes that will be used to populate arguments of the function positionally for a calculation node. e.g. If args is ['a', 'b', 'c'] then the function would be called with three parameters, taken from the nodes ‘a’, ‘b’ and ‘c’ respectively.
  • kwds (Dictionary, default None) – Specifies a mapping from parameter name to the node that should be used to populate that parameter when calling the function for a calculation node. e.g. If args is {'x': 'a', 'y': 'b'} then the function would be called with parameters named ‘x’ and ‘y’, and their values would be taken from nodes ‘a’ and ‘b’ respectively. Each entry in the dictionary can be read as “take parameter [key] from node [value]”.
  • value (default None) – If given, the value is inserted into the node, and the node state set to UPTODATE.
  • serialize (boolean, default True) – Whether the node should be serialized. Some objects cannot be serialized, in which case, set serialize to False
  • inspect (boolean, default True) – Whether to use introspection to determine the arguments of the function, which can be slow. If this is not set, kwds and args must be set for the function to obtain parameters.
  • group (default None) – Subgraph to render node in
  • tags (Iterable) – Set of tags to apply to node
clear_tag(name, tag)

Clear tag on a node or nodes. Ignored if tags are not set.

Parameters:
  • name – Node or nodes to clear tags for
  • tag – Tag to clear
compute(name, raise_exceptions=False)

Compute a node and all necessary predecessors

Following the computation, if successful, the target node, and all necessary ancestors that were not already UPTODATE will have been calculated and set to UPTODATE. Any node that did not need to be calculated will not have been recalculated.

If any nodes raises an exception, then the state of that node will be set to ERROR, and its value set to an object containing the exception object, as well as a traceback. This will not halt the computation, which will proceed as far as it can, until no more nodes that would be required to calculate the target are COMPUTABLE.

Parameters:
  • name – Name of the node to compute
  • raise_exceptions (Boolean, default False) – Whether to pass exceptions raised by node computations back to the caller
compute_all(raise_exceptions=False)

Compute all nodes of a computation that can be computed

Nodes that are already UPTODATE will not be recalculated. Following the computation, if successful, all nodes will have state UPTODATE, except UNINITIALIZED input nodes and PLACEHOLDER nodes.

If any nodes raises an exception, then the state of that node will be set to ERROR, and its value set to an object containing the exception object, as well as a traceback. This will not halt the computation, which will proceed as far as it can, until no more nodes are COMPUTABLE.

Parameters:raise_exceptions (Boolean, default False) – Whether to pass exceptions raised by node computations back to the caller
copy()

Create a copy of a computation

The copy is shallow. Any values in the new Computation’s DAG will be the same object as this Computation’s DAG. As new objects will be created by any further computations, this should not be an issue.

Return type:Computation
delete_node(name)

Delete a node from a computation

When nodes are explicitly deleted with delete_node, but are still depended on by other nodes, then they will be set to PLACEHOLDER status. In this case, if the nodes that depend on a PLACEHOLDER node are deleted, then the PLACEHOLDER node will also be deleted.

Parameters:name – Name of the node to delete. If the node does not exist, a NonExistentNodeException will be raised.
draw(colors='state', cmap=None, graph_attr=None, node_attr=None, edge_attr=None, show_expansion=False)

Draw a computation’s current state using the GraphViz utility

Parameters:
  • graph_attr – Mapping of (attribute, value) pairs for the graph. For example graph_attr={'size': '"10,8"'} can control the size of the output graph
  • node_attr – Mapping of (attribute, value) pairs set for all nodes.
  • edge_attr – Mapping of (attribute, value) pairs set for all edges.
  • show_expansion – Whether to show expansion nodes (i.e. named tuple expansion nodes) if they are not referenced by other nodes
get_inputs(name)

Get a list of the inputs for a node or set of nodes

Parameters:name – Name or names of nodes to get inputs for
Returns:If name is scalar, return a list of upstream nodes used as input. If name is a list, return a list of list of inputs.
get_timing(name)

Get the timing information for a node

Parameters:name – Name or names of the node to get the timing information of
Returns:
insert(name, value)

Insert a value into a node of a computation

Following insertation, the node will have state UPTODATE, and all its descendents will be COMPUTABLE or STALE.

If an attempt is made to insert a value into a node that does not exist, a NonExistentNodeException will be raised.

Parameters:
  • name – Name of the node to add.
  • value – The value to be inserted into the node.
insert_from(other, nodes=None)

Insert values into another Computation object into this Computation object

Parameters:
  • other – The computation object to take values from
  • nodes (List, default None) – Only populate the nodes with the names provided in this list. By default, all nodes from the other Computation object that have corresponding nodes in this Computation object will be inserted
insert_many(name_value_pairs)

Insert values into many nodes of a computation simultaneously

Following insertation, the nodes will have state UPTODATE, and all their descendents will be COMPUTABLE or STALE. In the case of inserting many nodes, some of which are descendents of others, this ensures that the inserted nodes have correct status, rather than being set as STALE when their ancestors are inserted.

If an attempt is made to insert a value into a node that does not exist, a NonExistentNodeException will be raised, and none of the nodes will be inserted.

Parameters:name_value_pairs (List of tuples) – Each tuple should be a pair (name, value), where name is the name of the node to insert the value into.
nodes()

Get a list of nodes in this computation :return: List of nodes

nodes_by_tag(tag)

Get the names of nodes with a particular tag or tags

Parameters:tag – Tag or tags for which to retrieve nodes
Returns:Names of the nodes with those tags
pin(name, value=None)

Set the state of a node to PINNED

Parameters:
  • name – Name of the node to set as PINNED.
  • value (default None) – Value to pin to the node, if provided.
static read_dill(file_)

Deserialize a computation from a file or file-like object

Parameters:file (File-like object, or string) – If string, writes to a file
set_stale(name)

Set the state of a node and all its dependencies to STALE

Parameters:name – Name of the node to set as STALE.
set_tag(name, tag)

Set tags on a node or nodes. Ignored if tags are already set.

Parameters:
  • name – Node or nodes to set tag for
  • tag – Tag to set
state(name)

Get the state of a node

This can also be accessed using the attribute-style accessor s if name is a valid Python attribute name:

>>> comp = Computation()
>>> comp.add_node('foo', value=1)
>>> comp.state('foo')
<States.UPTODATE: 4>
>>> comp.s.foo
<States.UPTODATE: 4>
Parameters:name (Key or [Keys]) – Name or names of the node to get state for
tags(name)

Get the tags associated with a node

>>> comp = Computation()
>>> comp.add_node('a', tags=['foo', 'bar'])
>>> comp.t.a
{'__serialize__', 'bar', 'foo'}
Parameters:name – Name or names of the node to get the tags of
Returns:
to_df()

Get a dataframe containing the states and value of all nodes of computation

>>> comp = loman.Computation()
>>> comp.add_node('foo', value=1)
>>> comp.add_node('bar', value=2)
>>> comp.to_df()
               state  value  is_expansion
bar  States.UPTODATE      2           NaN
foo  States.UPTODATE      1           NaN
to_dict()

Get a dictionary containing the values of all nodes of a computation

>>> comp = loman.Computation()
>>> comp.add_node('foo', value=1)
>>> comp.add_node('bar', value=2)
>>> comp.to_dict()
{'bar': 2, 'foo': 1}
unpin(name)

Unpin a node (state of node and all descendents will be set to STALE)

Parameters:name – Name of the node to set as PINNED.
value(name)

Get the current value of a node

This can also be accessed using the attribute-style accessor v if name is a valid Python attribute name:

>>> comp = Computation()
>>> comp.add_node('foo', value=1)
>>> comp.value('foo')
1
>>> comp.v.foo
1
Parameters:name (Key or [Keys]) – Name or names of the node to get the value of
write_dill(file_)

Serialize a computation to a file or file-like object

Parameters:file (File-like object, or string) – If string, writes to a file