Histories

The generic History class provides a general interface for storing data in various formats along with a time axis, along with an update function. The data structure is initialized empty, and filled as needed by calling the update function when a data point is requested.

A key assumption is that the update function is causal: it may depend on earlier data points (which should then be initialized before the first update call), but must not depend on the value at later points.

History tracks its current time with integer cur_tidx corresponding to its time axis: data values up to and included that time point are considered computed, and are simply retrieved from storage when requested. Accessing a data value later than cur_tidx triggers computation of all values up to cur_tidx. (The planned LaggedSeries subclasses would relax this.)

In general one does not instantiate the History class directly, but one of its subclasses. Currently the following are available:

  • Series: Store data in a Numpy array Useful for data and post mortem analysis, but gets really heavy when there are many histories.
  • Spiketrain: Store data in bool in a sparse array.
  • (Planned) LagFreeSeries: Store only the most recently computed time.
  • (Planned) LagSeries: Store only the last n time steps.

If none of these suit your need, you can also [define your own history type](#defining-your-own-history-type).

Usage tutorial

Todo

This section is completely out of date.

The History class, and the specialized classes that inherit from it, serves the purpose of keeping two data structures in sync:

  • A data structure data
  • A set of times time, stored as an Axis.

Time always corresponds to the data’s first dimension, such that len(data) == len(time_array). The data at a particular point t in time, i.e. data[t], is referred to as a timeslice within this documentation.

Histories provide methods for time-value and time-index access to the data, automatic computation, padding, convolutions, exporting to file-writable types, and other subtle conveniences that ultimately save you from error-prone indexing operations.

The basic use case is to use a history to generate data. To do this, we first create a history, providing the time_array as argument

spikes = Spiketrain(name = 'spikes',
                    time_array = np.arange(0, 10, 0.01),
                    pop_sizes = (200, 100, 400),
                    dtype = 'bool')

For the purpose of this example let’s define another history

rate = Series(spikes,
              name = 'rate',
              shape = (3,)
              dtype = 'float64')

Here we’ve used the first history as template to the second; this ensures they both share the time array.

We then define an update function. An update function always has the same signature: it expects one variable (t: time), and should be able to accept either a scalar t or an array of time values. The value of t may represent either an index or an actual time; if you need either quantity, use the get_t_idx() or get_time() methods to ensure it is of the right form. The update function may access any variable within the scope. This includes circular dependencies that loop back to the history itself, as long the it is indexed at an earlier time. For instance, the following are all acceptable update functions:

def rate_update1(t):
    return [spikes[t-0.1][slc].mean() for slc in spikes.pop_slices]

def rate_update2(t):
    tidx = spikes.get_t_idx(t)
    return [spikes[tidx-10][slc].mean() for slc in spikes.pop_slices]

def spike_update1(t):
    return np.concatenate( [ np.random.binomial(1, r, size)
                             for r, size in zip(rate[t], spikes.pop_sizes ] )

However, the following would introduce an unresolvable dependency when combined with spike_update1():

def rate_update3(t):
    return [spikes[t][slc].mean() for slc in spikes.pop_slices]

Note the use of both floating point and integer indexing in the examples above. Integers are interpreted as indices into the array, while floats are interpreted as times, which are converted to indices using the internal time_array. You may use whichever is most convenient in a particular situation. We recommend avoiding large numbers of time<->index conversions (e.g. at every simulation time point), as there is a small cost associated with it.

Accessing a history timeslice that hasn’t already been computed automatically triggers computation, so one can do:

sum_up_to_7 = spikes[:7.0].sum()

and be assured that this will return the correct value, triggering any required computation. This may take a long time on the first call, but is just as fast as indexing on any subsequent call. To compute a history at all time points, we can use any of the two following forms:

spikes._compute_up_to('end')
spikes.set()

(spikes.set() internally calls _compute_up_to('end') when called with no argument).

Retrieval vs evaluation

A history hist distinguishes between retrieval (indicated by square brackets []) and evaluation (indicated by round brackets ())

  • hist[tidx] will return the value of hist at position tidx. If that value has not already been computed, an IndexError is raised.
  • hist(tidx) will also return the value of hist. If the value has already been computed, it is simply returned – in this case hist(tidx) is equivalent to hist(tidx). However, in the opposite case, instead of raising an error, the history’s update_function() is called to fill it up to tidx, and then the value is retrieved. The round brackets are meant as a indicator that this may trigger an expensive function call.

In general, it is recommended using () when specifying a model’s update equations, and [] in post-simualtion analysis. This communicates intent, and avoids or catches errors.

Public API

The following attributes and methods are provided by History and thus guaranteed to be defined in all subclasses:

Attributes
name : str
Unique identifying string
shape : int tuple
Shape at a single time point. Full data shape should be (T,) + shape, where T is the number of time steps.
time : TimeAxis
Underlying Axis object describing the time axis.
idx_dtype : numpy integer dtype.
Type to use for indices within one time slice.
locked : bool
Whether modifications to history are allowed. Modify through method
Properties
trace :
Unpadded data. Calls self.get_data_trace() with default arguments.
time_stops : ndarray
Unpadded time array. Calls self.time.stops_array(padded=False).
Access to specified time properties:
t0 : floatX
Time at which history starts
tn : floatX
Time at which history ends
dt : floatX
Timestep size
dt64 : float64.
Timestep size; guaranteed double precision, useful for index calculations
tidx_dtype : numpy integer dtype
Type to use for time indices.
Methods
lock : Set the locked status to True unlock : Set the locked status to False

Defining your own history type

If you want to subclass one of existing History subclasses (e.g. Series), just proceed as usual by inheriting from the base class and and adding your desired methods. If you want to add initialization parameters, note that histories are implemented as Pydantic models, so additional parameters should be specified as class attributes (rather than by specializing __init__()).

If subclassing History directly, the following attributes and methods must be provided (see below) for method templates):

  • Attributes:
  • Methods:
    • initialized_data()
    • _getitem_internal()
    • update()
    • pad()
    • get_data_trace()
    • _compute_up_to()

The following methods may also be provided:

  • _compute_range()
  • _convolve_single_t() (Required for convolutions)
  • _convolve_batch()

Note

All methods which modify the history (update, set, clear, _compute_up_to) must raise a RuntimeError if lock is True.

In addition to the attributes listed in the public API, the following attributes and methods are also made available by the History base class:

  • Attributes:

    • _sym_tidx : time.tidx_dtype

      Tracker for the latest time bin for which we know history.

    • _num_tidx :
      For Numpy histories, same as _sym_tidx.
      For Symbolic histories, a handle to the tidx variable, which is to be updated with the new value of _sym_tidx. (See _num_data above, and Symbolic updates below.)
    • _sym_data: Where the actual data is stored.

      A shared variable. The type can be any NumPy variable, but NumPy arrays must be wrapped as a shim.shared variable. Some histories implement this as a tuple of arrays (e.g. Spiketrain)

    • _num_data: For NumPy histories, the same as _sym_data.

      For symbolic histories, an handle to the shared variable, which is to be updated with the new value of _sym_tidx. (See Symbolic updates.)

    • update_function : HistoryUpdateFunction

      Function taking a time and returning the history at that time

  • Methods:

    • __getitem__
      Calls self.retrieve()
    • __setitem__
      Calls self.update(), after converting times in a key to time (axis) indices, and None in a slice to the appropriate axis index. Thus update() only needs to implement operations on axis (not data !) indices.

Symbolic updates

TODO _num_tidx/_num_data always points to numeric data. Symbolic updates are accumulated in _sym_tidx/_sym_data.

Subclass method templates

The following method templates are intended as a guide; although I try to keep them up to date, if in doubt, always have a look at how these method are implemented in the existing history subclasses. Series is generally the most highly tested.