API reference

Experiment

class experimentator.Experiment(tree, data=None, has_started=False, has_finished=False, _children=None, filename=None, callback_by_level=None, callback_type_by_level=None, session_data=None, experiment_data=None, _callback_info=None)

Bases: experimentator.section.ExperimentSection

An ExperimentSection subclass that represents the largest ‘section’ of the experiment; that is, the entire experiment. Functionality added on top of ExperimentSection includes various constructors, saving to disk, and management of callbacks.

To create a new experiment, rather than instantiating directly it is recommended to use one of the constructor methods:

Attributes

tree (DesignTree) The DesignTree instance defining the experiment’s hierarchy.
filename (str) The file location where the Experiment will be pickled.
callback_by_level (dict) A dictionary, mapping level names to functions or With Statement Context Managers (e.g., generator functions decorated with contextlib.contextmanager). Defines behavior to run at each section (for functions) or before and/or after each section (for context managers) at the associated level.
callback_type_by_level (dict) A dictionary mapping level names to either the string 'context' or 'function'. This keeps track of which callbacks in Experiment.callback_by_level are context managers.
session_data (dict) A dictionary where temporary data can be stored, persistent only within one session of the Python interpreter. This is a good place to store external resources that aren’t picklable; external resources, for example, can be loaded in a context-manager callback and stored here. In addition, anything returned by the __exit__ method of a context-manager callback will be stored here, with the callback’s level name as the key. This dictionary is emptied before saving the Experiment to disk.
experiment_data (dict) A dictionary where data can be stored that is persistent across Python sessions. Everything stored here must be picklable.
add_callback(level, callback, *args, is_context=False, func_module=None, func_name=None, **kwargs)

Add a callback to run at a certain level.

A callback can be either a regular function, or a context-manager. The latter is useful for defining code to run at the start and end of every section at the level. For example, a block context manager could specify behavior that occurs before every trial in the block, and behavior that occurs after every trial in the block. See contextlib for various ways to create context managers, and experimentator’s context-manager docs for more details.

Any value returned by the __enter__ method of a context manager will be stored in Experiment.session_data under the key level.

If the callback is not a context manager, it should return a dictionary (or nothing), which is automatically passed to ExperimentSection.add_data. In theory, it should map dependent-variable names to results. See the callback docs for more details.

Parameters:

level : str

Which level of the hierarchy to manage.

callback : function or context-manager

The callback should have the signature callback(experiment, section, *args, **kwargs) where experiment and section are the current Experiment and ExperimentSection instances, respectively, and args and kwargs are arbitrary arguments passed to this method.

*args

Any arbitrary positional arguments to be passed to callback.

func_module : str, optional

func_name : str, optional

These two arguments specify where the given function should be imported from in future Python sessions (i.e., from <func_module> import <func_name>). Usually, this is figured out automatically by introspection; these arguments are provided for the rare situation where introspection fails.

**kwargs

Any arbitrary keyword arguments to be passed to callback.
add_data(data)

Update the ExperimentSection.data ChainMap. This data will apply to this section and all child sections. This can be used, for example, to manually record a participant’s age.

Parameters:

data : dict

Elements to be added to ExperimentSection.data.
all_subsections(**section_numbers)

Find all subsections in the experiment matching the given section numbers.

Yields specified ExperimentSection instances. The yielded sections will be at the lowest level given in section_numbers. If levels not in section_numbers are encountered before reaching its lowest level, all sections will be descended into.

Parameters:

**section_numbers

Keyword arguments describing what subsections to find. Keys are level names, values are ints or sequences of ints.

See also

experimentator.section.ExperimentSection.subsection

Examples

Assuming the levels of the experiment saved in 'example.exp' are ('participant', 'session', 'block', 'trial'):

>>> from experimentator import Experiment
>>> exp = Experiment.load('example.exp')

Get the first session of each participant:

>>> all_first_sessions = list(exp.all_subsections(session=1))

Get the second trial of the first block in each session:

>>> trials = list(exp.all_subsections(block=1, trial=2))

Get the first three trials of each block in the first session of each participant:

>>> more_trials = list(exp.all_subsections(session=1, trial=[1, 2, 3]))
append_child(data, tree=None, to_start=False, _renumber=True)

Create a new ExperimentSection (and its descendants) and append it as a child of the current ExperimentSection.

Parameters:

data : dict

Data to be included in the new section’s ExperimentSection.data ChainMap. Should include values of IVs at the section’s level, for example.

tree : DesignTree, optional

If given, the section will be appended from the top level of tree. If not passed, the tree of the current section will be used. Note that this does not affect IV values; IV values must still be included in data.

to_start : bool, optional

If True, the new ExperimentSection will be appended to the beginning of the current section. If False (the default), it will be appended to the end.

Notes

After calling this method, the section numbers in the children’s ExperimentSection.data attributes will be automatically replaced with the correct numbers.

append_design_tree(tree, to_start=False, _renumber=True)

Append all sections associated with the top level of a DesignTree (and therefore also create descendant sections) to the ExperimentSection.

Parameters:

tree : DesignTree

The tree to append.

to_start : bool, optional

If True, the sections will be inserted at the beginning of the section. If False (the default), they will be appended to the end.

Notes

After calling this method, the section numbers in the children’s ExperimentSection.data attributes will be automatically replaced with the correct numbers.

as_graph()

Build a networkx.DiGraph out of the experiment structure, starting at this section. Nodes are sections and graphs are parent-child relations. Node data are non-duplicated entries in ExperimentSection.data.

Returns:networkx.DiGraph
classmethod basic(levels, ivs_by_level, design_matrices_by_level=None, ordering_by_level=None, filename=None)

Construct a homogeneously-organized Experiment, with arbitrary levels but only one Design at each level, and the same structure throughout its hierarchy.

Parameters:

levels : sequence of str

Names of the levels of the experiment

ivs_by_level : dict

Dictionary specifying the IVs and their possible values at every level. The keys are be the level names, and the values are lists of the IVs at that level, specified in the form of tuples with the first element being the IV name and the second element a list of its possible values. Alternatively, the IVs at each level can be specified in a dictionary. See IV docs for more on specifying IVs.

design_matrices_by_level : dict, optional

Specify the design matrix for any levels. Keys are level names; values are design matrices. Any levels without a design matrix will be fully crossed. See design matrix docs for details.

ordering_by_level : dict, optional

Specify the ordering for each level. Keys are level names; values are instance objects from experimentator.order. For any levels without an order specified, Shuffle will be used.

filename : str, optional

File location to save the experiment.
Returns:

Experiment
classmethod blocked(trial_ivs, n_participants, design_matrices=None, orderings=None, block_ivs=None, filename=None)

Create a blocked within-subjects Experiment, in which all the IVs are at either the trial level or the block level.

Parameters:

trial_ivs : list or dict

A list of the IVs to define at the trial level, specified in the form of tuples with the first element being the IV name and the second element a list of its possible values. Alternatively, the IVs at each level can be specified in a dictionary. See the IV docs more on specifying IVs.

n_participants : int

Number of participants to initialize. If a NonAtomicOrdering is used, this is the number of participants per order.

design_matrices : dict, optional

Design matrices for the experiment. Keys are 'trial' and 'block'; values are the respective design matrices (if any). If not specified, IVs will be fully crossed. See the design matrix docs for details.

orderings : dict, optional

Dictionary with keys of 'trial' and 'block'. Each value should be an instance of the class Ordering or one of its subclasses, specifying how the trials will be ordered If not specified, Shuffle will be used.

block_ivs : list or dict, optional

IVs to define at the block level. See IV docs for more on specifying IVs.

filename : str, optional

File location to save the experiment.
Returns:

Experiment

Notes

For blocks to have any effect, you should either define at least one IV at the block level or use the ordering Ordering(n) to create n blocks for every participant.

Breadth-first search starting from here. Returns the entire search path.

Parameters:

key : func

Function that returns True or False when passed an ExperimentSection.
Returns:

list of ExperimentSection

Depth-first search starting from here. Returns the entire search path.

Parameters:

key : func

Function that returns True or False when passed an ExperimentSection.

path_key : func, optional

Function that returns True or False when passed an ExperimentSection. If given, the search will proceed only via sections for which path_key returns True.
Returns:

list of ExperimentSection
export_data(filename, skip_columns=None, **kwargs)

Export Experiment.dataframe in .csv format.

Parameters:

filename : str

A file location where the data should be saved.

skip_columns : list of str, optional

Columns to skip.

**kwargs

Arbitrary keyword arguments to pass to pandas.DataFrame.to_csv.

Notes

This method is not recommended for experiments with compound data types, for example an experiment which stores a time series for every trial. In those cases it is recommended to write a custom script that parses the Experiment.dataframe attribute as desired, or use the skip_columns option to skip any compound columns.

find_first_not_run(at_level, by_started=True)

Search the experimental hierarchy, and return the first descendant ExperimentSection at at_level that has not yet been run.

Parameters:

at_level : str

Which level to search.

by_started : bool, optional

If True (default), returns the first section that has not been started. Otherwise, finds the first section that has not finished.
Returns:

ExperimentSection
find_first_partially_run(at_level)

Search the experimental hierarchy, and return the first descendant ExperimentSection at at_level that has been started but not finished.

Parameters:

at_level : str

Which level to search.
Returns:

ExperimentSection
classmethod from_dict(spec)

Construct an Experiment based on a dictionary specification.

Parameters:

spec : dict

spec should have, at minimum, a key named 'design'. The value of this key specifies the DesignTree. See DesignTree.from_spec for details. The value of the key 'filename' or 'file', if one exists,is saved in Experiment.filename. All other fields are saved in Experiment.experiment_data.
Returns:

Experiment

See also

experimentator.Experiment.from_yaml_file

classmethod from_yaml_file(filename)

Construct an Experiment based on specification in a YAML file. Requires PyYAML.

Parameters:

filename : str

YAML file location. The YAML should specify a dictionary matching the specification of Experiment.from_dict.
Returns:

Experiment
get_next_tree()

Get a tree to use for creating child ExperimentSection instances.

Returns:DesignTree
static load(filename)

Load an experiment from disk.

Parameters:

filename : str

Path to a file generated by Experiment.save.
Returns:

Experiment
classmethod new(tree, filename=None)

Make a new Experiment.

Parameters:

tree : DesignTree

A DesignTree instance defining the experiment hierarchy.

filename : str, optional

A file location where the Experiment will be saved.
parent(section)

Find the parent of a section.

Parameters:

section : ExperimentSection

The section to find the parent of.
Returns:

ExperimentSection
parents(section)

Find all parents of a section, in top-to-bottom order.

Parameters:

section : ExperimentSection

The section to find the parents of.
Returns:

list of ExperimentSection
resume_section(section, **kwargs)

Rerun a section that has been started but not finished, starting where running last left off.

Parameters:

section : ExperimentSection

The section to resume.

**kwargs

Keyword arguments to pass to Experiment.run_section.

Notes

The wrapper function run_experiment_section should be used instead of this method, if possible.

run_section(section, demo=False, parent_callbacks=True, from_section=None)

Run a section and all its descendant sections. Saves the results in the data attribute of each lowest-level ExperimentSection.

Parameters:

section : ExperimentSection

The section to be run.

demo : bool, optional

Data will only be saved if demo is False (the default).

parent_callbacks : bool, optional

If True (the default), all parent callbacks will be called.

from_section : int or list of int, optional

Which section to start running from. If a list is passed, it specifies where to start running on multiple levels. For example, assuming the experiment hierarchy is ('participant', 'session', 'block', 'trial'), this would start from the fifth trial of the second block (of the first participant’s second session):

>>> exp = Experiment.load('example.exp')
>>> exp.run_section(exp.subsection(participant=1, session=2), from_section=[2, 5])

Notes

The wrapper function run_experiment_section should be used instead of this method, if possible.

save(filename=None)

Save the Experiment to disk.

Parameters:

filename : str, optional

If specified, overrides Experiment.filename.
subsection(**section_numbers)

Find a single, descendant ExperimentSection based on section numbers.

Parameters:

**section_numbers

Keyword arguments describing which subsection to find Must include every level higher than the desired section.
Returns:

ExperimentSection

See also

experimentator.section.ExperimentSection.all_subsections

Examples

Assuming the levels of the experiment saved in 'example.exp' are ('participant', 'session', 'block', 'trial'), this will return the third block of the second participant’s first session:

>>> from experimentator import Experiment
>>> exp = Experiment.load('example.exp')
>>> some_block = exp.subsection(participant=2, session=1, block=3)
walk()

Walk the tree depth-first, starting from here. Yields this section and every descendant section.

classmethod within_subjects(ivs, n_participants, design_matrix=None, ordering=None, filename=None)

Create a within-subjects Experiment, with all the IVs at the 'trial' level.

Parameters:

ivs : list or dict

A list of the experiment’s IVs, specified in the form of tuples with the first element being the IV name and the second element a list of its possible values. Alternatively, the IVs at each level can be specified in a dictionary. See the IV docs more on specifying IVs.

n_participants : int

Number of participants to initialize.

design_matrix : array-like, optional

Design matrix for the experiment. If not specified, IVs will be fully crossed. See the design matrix docs for more details.

ordering : Ordering, optional

An instance of the class Ordering or one of its subclasses, specifying how the trials will be ordered. If not specified, Shuffle will be used.

filename : str, optional

File location to save the experiment.
Returns:

Experiment

Helper functions

experimentator.run_experiment_section(experiment, section_obj=None, demo=False, resume=False, parent_callbacks=True, from_section=1, session_options='', **section_numbers)

Run an experiment from a file or an Experiment instance, and save it. If an exception is encountered, the Experiment will be backed up and saved.

Parameters:

experiment : str or Experiment

File location where an Experiment instance is pickled, or an Experiment instance.

demo : bool, optional

If True, data will not be saved and sections will not be marked as run.

resume: bool, optional

If True, the specified section will be resumed (started automatically where it left off).

parent_callbacks : bool, optional

If True (the default), all parent callbacks will be called.

section_obj : ExperimentSection, optional

The section of the experiment to run. Alternatively, the section can be specified using **section_numbers.

from_section : int or list of int, optional

Which section to start running from. If a list is passed, it specifies where to start running on multiple levels. See the example below.

session_options : str, optional

Pass an experiment-specific options string to be stored in Experiment.session_data under the key 'options'.

**section_numbers

Keyword arguments describing how to descend the experiment hierarchy to find the section to run. See the example below.

Examples

A simple example:
>>> exp = Experiment.load('example.exp')
>>> run_experiment_section(exp, exp.subsection(participant=1, session=2))

Equivalently:

>>> run_experiment_section(exp, participant=1, session=2)

To demonstrate from_section, assuming the experiment hierarchy is ('participant', 'session', 'block', 'trial'), this would start from the second block:

>>> run_experiment_section(exp, participant=1, session=2, from_section=2)

To start from the fifth trial of the second block:

>>> run_experiment_section(exp, participant=1, session=2, from_section=[2, 5])
experimentator.export_experiment_data(exp_filename, data_filename, **kwargs)

Reads a pickled Experiment instance and saves its data in .csv format.

Parameters:

exp_filename : str

The file location where an Experiment instance is pickled.

data_filename : str

The file location where the data will be written.

skip_columns : list of str, optional

Data columns to skip.

**kwargs

Arbitrary keyword arguments passed through to pandas.DataFrame.to_csv.

Notes

This shortcut function is not recommended for experiments with compound data types, for example an experiment which stores a time series for every trial. In such cases it is recommended to write a custom script that parses Experiment.dataframe as desired (or use the skip_columns option to ignore the compound data).

ExperimentSection

class experimentator.section.ExperimentSection(tree, data=None, has_started=False, has_finished=False, _children=None)

A section of the experiment, at any level of the hierarchy. Single trials and groups of trials (blocks, sessions, participants, etc.) are represented as ExperimentSection instances. A complete experiment consists of ExperimentSection instances arranged in a tree. The root element should be an Experiment (a subclass of ExperimentSection); the rest of the sections can be reached via its descendants (see below on the sequence protocol). A new ExperimentSection instance is automatically populated with ExperimentSection descendants according to the DesignTree passed to its constructor.

ExperimentSection implements Python’s sequence protocol; its contents are ExperimentSection instances at the level below. In other words, children can be accessed using the [index] notation, as well as with slices ([3:6]) or iteration (for section in experiment_section). However, ExperimentSection breaks the Python convention of 0-based indexing, using 1-based indexing to match the convention in experimental science.

The direct constructor is used to create an arbitrary ExperimentSection (i.e., possibly reloading an in-progress section), whereas ExperimentSection.new creates a section that hasn’t yet started.

Notes

Use 1-based indexing to refer to ExperimentSection children, both when when using indexing or slicing with an ExperimentSection, and when identifying sections in keyword arguments to methods such as ExperimentSection.subsection. This better corresponds to the language commonly used by scientists to identify participants, trials, etc.

Attributes

tree (DesignTree)
data (ChainMap)
description (str) The name and number of the section (e.g., 'trial 3').
dataframe (DataFrame) All data associated with the ExperimentSection and its descendants.
heterogeneous_design_iv_name (str) IV name determining which branch of the DesignTree to follow.
level (str) The level of the hierarchy at which this section lives.
levels (list of str) Level names below this section.
local_levels (set) Level names of this section’s children. Usually a single-element set.
is_bottom_level (bool) If true, this is the lowest level of the hierarchy.
is_top_level (bool) If true, this is the highest level of the hierarchy (likely an Experiment).
has_started: bool Whether this section has started to be run.
has_finished (bool) Whether this section has finished running.
add_data(data)

Update the ExperimentSection.data ChainMap. This data will apply to this section and all child sections. This can be used, for example, to manually record a participant’s age.

Parameters:

data : dict

Elements to be added to ExperimentSection.data.
all_subsections(**section_numbers)

Find all subsections in the experiment matching the given section numbers.

Yields specified ExperimentSection instances. The yielded sections will be at the lowest level given in section_numbers. If levels not in section_numbers are encountered before reaching its lowest level, all sections will be descended into.

Parameters:

**section_numbers

Keyword arguments describing what subsections to find. Keys are level names, values are ints or sequences of ints.

See also

experimentator.section.ExperimentSection.subsection

Examples

Assuming the levels of the experiment saved in 'example.exp' are ('participant', 'session', 'block', 'trial'):

>>> from experimentator import Experiment
>>> exp = Experiment.load('example.exp')

Get the first session of each participant:

>>> all_first_sessions = list(exp.all_subsections(session=1))

Get the second trial of the first block in each session:

>>> trials = list(exp.all_subsections(block=1, trial=2))

Get the first three trials of each block in the first session of each participant:

>>> more_trials = list(exp.all_subsections(session=1, trial=[1, 2, 3]))
append_child(data, tree=None, to_start=False, _renumber=True)

Create a new ExperimentSection (and its descendants) and append it as a child of the current ExperimentSection.

Parameters:

data : dict

Data to be included in the new section’s ExperimentSection.data ChainMap. Should include values of IVs at the section’s level, for example.

tree : DesignTree, optional

If given, the section will be appended from the top level of tree. If not passed, the tree of the current section will be used. Note that this does not affect IV values; IV values must still be included in data.

to_start : bool, optional

If True, the new ExperimentSection will be appended to the beginning of the current section. If False (the default), it will be appended to the end.

Notes

After calling this method, the section numbers in the children’s ExperimentSection.data attributes will be automatically replaced with the correct numbers.

append_design_tree(tree, to_start=False, _renumber=True)

Append all sections associated with the top level of a DesignTree (and therefore also create descendant sections) to the ExperimentSection.

Parameters:

tree : DesignTree

The tree to append.

to_start : bool, optional

If True, the sections will be inserted at the beginning of the section. If False (the default), they will be appended to the end.

Notes

After calling this method, the section numbers in the children’s ExperimentSection.data attributes will be automatically replaced with the correct numbers.

as_graph()

Build a networkx.DiGraph out of the experiment structure, starting at this section. Nodes are sections and graphs are parent-child relations. Node data are non-duplicated entries in ExperimentSection.data.

Returns:networkx.DiGraph

Breadth-first search starting from here. Returns the entire search path.

Parameters:

key : func

Function that returns True or False when passed an ExperimentSection.
Returns:

list of ExperimentSection

Depth-first search starting from here. Returns the entire search path.

Parameters:

key : func

Function that returns True or False when passed an ExperimentSection.

path_key : func, optional

Function that returns True or False when passed an ExperimentSection. If given, the search will proceed only via sections for which path_key returns True.
Returns:

list of ExperimentSection
find_first_not_run(at_level, by_started=True)

Search the experimental hierarchy, and return the first descendant ExperimentSection at at_level that has not yet been run.

Parameters:

at_level : str

Which level to search.

by_started : bool, optional

If True (default), returns the first section that has not been started. Otherwise, finds the first section that has not finished.
Returns:

ExperimentSection
find_first_partially_run(at_level)

Search the experimental hierarchy, and return the first descendant ExperimentSection at at_level that has been started but not finished.

Parameters:

at_level : str

Which level to search.
Returns:

ExperimentSection
get_next_tree()

Get a tree to use for creating child ExperimentSection instances.

Returns:DesignTree
classmethod new(tree, data=None)

Create a new ExperimentSection.

Parameters:

tree : DesignTree

Describes the design of the experiment hierarchy.

data : ChainMap

All data to be associated with the ExperimentSection, including the values of independent variables, the section numbers indicating the section’s location in the experiment, and any results associated with this section, arising from either the run callback of the Experiment or from the method ExperimentSection.add_data. data should be a collections.ChainMap, which behaves like a dictionary but has a hierarchical organization such that children can access values from the parent but not vice-versa.
parent(section)

Find the parent of a section.

Parameters:

section : ExperimentSection

The section to find the parent of.
Returns:

ExperimentSection
parents(section)

Find all parents of a section, in top-to-bottom order.

Parameters:

section : ExperimentSection

The section to find the parents of.
Returns:

list of ExperimentSection
subsection(**section_numbers)

Find a single, descendant ExperimentSection based on section numbers.

Parameters:

**section_numbers

Keyword arguments describing which subsection to find Must include every level higher than the desired section.
Returns:

ExperimentSection

See also

experimentator.section.ExperimentSection.all_subsections

Examples

Assuming the levels of the experiment saved in 'example.exp' are ('participant', 'session', 'block', 'trial'), this will return the third block of the second participant’s first session:

>>> from experimentator import Experiment
>>> exp = Experiment.load('example.exp')
>>> some_block = exp.subsection(participant=2, session=1, block=3)
walk()

Walk the tree depth-first, starting from here. Yields this section and every descendant section.

Design

class experimentator.Design(ivs=None, design_matrix=None, ordering=None, extra_data=None)

Design instances specify the experimental design at one level of the experimental hierarchy. They guide the creation of ExperimentSection instances by parsing design matrices or crossing independent variables (IVs).

Parameters:

ivs : dict or list of tuple, optional

Independent variables can be specified as a dictionary mapping names to possible values, or as a list of (name, values) tuples. If an IV takes continuous values, use None for its levels. This only works when specifying values using design_matrix. See the IV docs for more information.

design_matrix : array-like, optional

A numpy array (or convertible, e.g. a list-of-lists) representing a design matrix specifying how IV values should be grouped to form conditions. When no design_matrix is passed, IVs are fully crossed. See the design matrix docs for more details. Note that a design matrix may also specify the order of the conditions. For this reason, the default ordering changes from Shuffle to Ordering, preserving the order of the conditions.

ordering : Ordering, optional

An instance of Ordering or one of its subclasses defining the behavior for duplicating and ordering the conditions of the Design. The default is Shuffle unless a design_matrix is passed.

extra_data : dict, optional

Items from this dictionary will be included in the data attribute of any ExperimentSection instances created with this Design.

See also

experimentator.order, experimentator.DesignTree

Examples

>>> from experimentator.order import Shuffle
>>> design = Design(ivs={'side': ['left', 'right'], 'difficulty': ['easy', 'hard']}, ordering=Shuffle(2))
>>> design.first_pass()
IndependentVariable(name=(), values=())
>>> design.get_order()
[{'difficulty': 'easy', 'side': 'left'},
 {'difficulty': 'hard', 'side': 'left'},
 {'difficulty': 'easy', 'side': 'left'},
 {'difficulty': 'hard', 'side': 'right'},
 {'difficulty': 'easy', 'side': 'right'},
 {'difficulty': 'easy', 'side': 'right'},
 {'difficulty': 'hard', 'side': 'left'},
 {'difficulty': 'hard', 'side': 'right'}]

Attributes

iv_names (list of str)
iv_values (list of tuple)
design_matrix (array-like)
extra_data (dict)
ordering (Ordering)
heterogeneous_design_iv_name (str) The IV name that triggers a heterogeneous (i.e., branching) tree structure when it is encountered. 'design' by default.
is_heterogeneous (bool) True if this Design is the lowest level before the tree structure diverges.
branches (dict) The IV values corresponding to named heterogeneous branches in the tree structure following this Design.
first_pass()

Initialize design.

Initializes the design by parsing the design matrix or crossing the IVs If a NonAtomicOrdering is used, an additional IV will be returned which should be incorporated into the design one level up in the experimental hierarchy. For this reason, the first_pass methods in a hierarchy of Design instances should be called in reverse order, from bottom up. Use a DesignTree to ensure this occurs properly.

Returns:

iv_name : str or tuple

The name of the IV, for non-atomic orderings. Otherwise, an empty tuple.

iv_values : tuple

The possible values of the IV. Empty for atomic orderings.
classmethod from_dict(spec)

Construct a Design instance from a specification based on dictionaries (e.g., parsed from a YAML file).

Parameters:

spec : dict

A dictionary containing some of the following keys (all optional): 'name', the name of the level; 'ivs', 'design_matrix', 'extra_data', keyword arguments to the Design constructor; 'order' or 'ordering', a string, dictionary, or list determining the ordering method; and 'n' or 'number', the number argument to the specified ordering. A dictionary containing any fields not otherwise used is passed to the Design constructor as the extra_data argument. See the description in the docs for more information.
Returns:

name : str

Only returned if spec contains a field 'name'.

design : Design

See also

experimentator.DesignTree.from_spec

Examples

>>> design_spec = {
...'name': 'block',
...'ivs': {'speed': [1, 2, 3], 'size': [15, 30]},
...'ordering': 'Shuffle',
...'n': 3}
>>> Design.from_dict(design_spec)
Level(name='block', design=Design(ivs=[('speed', [1, 2, 3]), ('size', [15, 30])], design_matrix=None, ordering=Shuffle(number=3, avoid_repeats=False), extra_data={}))
static full_cross(iv_names, iv_values)

Perform a full factorial cross of the independent variables. Yields dictionaries, each describing one condition, a mapping from IV names to IV values. One dictionary is yielded for every possible combination of IV values.

Parameters:

iv_names : list of str

Names of IVs.

iv_values : list of list

Each element defines the possible values of an IV. Must be the same length as iv_names. Its elements must be hashable.
get_order(data=None)

Order the conditions.

Returns:

list of dict

A list of dictionaries, each specifying a condition (a mapping from IV names to values).
update(names, values)

Add additional independent variables to the Design. This will have no effect after Design.first_pass has been called.

Parameters:

names : list of str

Names of IVs to add.

values : list of list

For each IV, a list of possible values.

DesignTree

class experimentator.DesignTree(levels_and_designs=None, other_designs=None, branches=None)

A container for Design instances, describing the entire hierarchy of a basic Experiment. DesignTree instances are iterators; calling next on one will return another DesignTree with the top level removed. In this way, the entire experimental hierarchy can be created by recursively calling next.

Use DesignTree.new to create a new tree, the generic constructor is for instantiating trees whose attributes have already been processed (i.e., reloading already-created trees).

Notes

Calling next on the last level of a heterogeneous DesignTree will return a dictionary of named DesignTree instances (rather than a single DesignTree instance). The keys are the possible values of the IV 'design' and the values are the corresponding DesignTree instances.

Attributes

levels_and_designs (list of tuple)
other_designs (dict)
branches (dict) Only those items from other_designs that follow directly from this tree.
add_base_level()

Adds a section to the top of the tree called '_base'. This makes the DesignTree suitable for constructing an Experiment.

Notes

The Experiment constructor calls this automatically, and this shouldn’t be called when appending a tree to an existing Experiment, so there is no use case for manually calling this method.

static first_pass(levels_and_designs)

Make a first pass of all designs in a DesignTree, from bottom to top. This calls Design.first_pass on every Design instance in the tree in the proper order, updating designs when a new IV is returned. This is necessary for non-atomic orderings because they modify the parent Design.

classmethod from_spec(spec)

Constructs a DesignTree instance from a specification (e.g., parsed from a YAML file).

spec : dict or list of dict
The DesignTree specification. A dictionary with keys as tree names and values as lists of dictionaries. Each sub-dictionary should specify a Design according to Design.from_dict. The main tree should be named 'main'. Other names are used for generating heterogeneous trees (see DesignTree docs). A homogeneous tree can be specified as a dictionary with only a single key 'main', or directly as a list of dictionaries
Returns:DesignTree
classmethod new(levels_and_designs, **other_designs)

Create a new DesignTree.

Parameters:

levels_and_designs : OrderedDict or list of tuple

This input defines the structure of the tree, and is either an OrderedDict or a list of 2-tuples. Keys (or first element of each tuple) are level names. Values (or second element of each tuple) are design specifications, in the form of either a Design instance, or a list of Design instances to occur in sequence.

**other_designs

Named design trees, can be other DesignTree instances or suitable levels_and_designs inputs (i.e., OrderedDict or list of tuples). These designs allow for heterogeneous design structures (i.e. not every section at the same level has the same Design). To make a heterogeneous DesignTree, use an IV named 'design' at the level where the heterogeneity should occur. Values of this IV should be strings, each corresponding to the name of a DesignTree from` other_designs`. The value of the IV 'design' at each section determines which DesignTree is used for children of that section.

experimentator.order

This module contains the class Ordering and its descendants. These classes handle how unique conditions at a particular experimental level are ordered and duplicated. Ordering instances should be passed directly to the Design constructor; there is no reason to otherwise interact with them in normal use.

Of special note are non-atomic orderings: the class NonAtomicOrdering and its descendants. ‘’Non-atomic’’ here means that the orderings between sections are not independent. A Shuffle ordering is atomic; the order in one section is independent of the order in another. However, if one wants to ensure, for example, that possible block orders are evenly distributed among participants (a counterbalanced design), the block orders within each participants are not independent. Each participant can decide its order of blocks only in the context of the other participants’ block orders. This means that the parent section must handle orderings (in the example of counterbalanced blocks, the Experiment.base_section–the experiment itself, essentially–must tell each participant what block order to use).

class experimentator.order.Ordering(number=1)

Bases: object

The base ordering class. It will keep conditions in the order they are defined by the Design instance (either the order of rows in the design matrix, or a full factorial cross–the output of calling itertools.product on the IV levels). Remember not to rely on the order of dictionary items. Therefore, if a specific order is desired, it is recommended to use a design matrix or an OrderedDict to define the IVs. See the IV docs for more information.

Parameters:

number : int, optional

The number of times each unique condition should appear. The default is 1. If number > 1, the entire order will be cycled (as opposed to repeating each condition within the order).
first_pass(conditions)

Handle operations that should only be performed once, initializing the object before ordering conditions. For Ordering, the only operation is duplication of the list of conditions (if Ordering.number > 1). This methods should not be called manually.

Parameters:

conditions : sequence of dict

A list of conditions, where each condition is a dictionary mapping IV names to IV values.
Returns:

iv_name : str or tuple

The name of the IV, for non-atomic orderings. Otherwise, an empty tuple.

iv_values : tuple

The possible values of the IV. Empty for atomic orderings.
get_order(data=None)

Get an order of conditions. For Ordering, always returns the same order.

Parameters:

data : dict, optional

A dictionary describing the data of the parent section. Unused for atomic orderings.
Returns:

list of dict

A list of conditions, where each condition is a dictionary mapping IV names to IV values.
static possible_orders(conditions, unique=True)

Yield all possible orders of the conditions. Each order is a list of dictionaries, with each dictionary representing a condition.

Parameters:

conditions : sequence of dict

A list of conditions, where each condition is a dictionary mapping IV names to IV values.

unique : bool, optional

If true (the default), will only return unique orders. If false, some identical orders will be generated if conditions contains identical elements.
class experimentator.order.Shuffle(number=1, avoid_repeats=False)

Bases: experimentator.order.Ordering

This ordering randomly shuffles the conditions.

Parameters:

number : int, optional

Number of times each condition should appear (default=1). Conditions are duplicated before shuffling.

avoid_repeats : bool, optional

If True (default is False), no identical conditions will appear back-to-back.
get_order(data=None)

Get an order of conditions. For Shuffle, returns the conditions in a random order.

Parameters:

data : dict, optional

A dictionary describing the data of the parent section. Unused for atomic orderings.
Returns:

list of dict

A list of conditions, where each condition is a dictionary mapping IV names to IV values.
class experimentator.order.NonAtomicOrdering(number=1)

Bases: experimentator.order.Ordering

This is a base class for non-atomic orderings, and is not meant to be directly instantiated. Non-atomic orderings work by creating a new independent variable one level up. The IV name will start with an underscore, a convention to avoid name clashes.

get_order(data=None)

Get an order of conditions.

Parameters:

data : dict, optional

A dictionary describing the data of the parent section.
Returns:

list of dict

A list of conditions, where each condition is a dictionary mapping IV names to IV values.
iv

The IV associated with the non-atomic ordering. It will be added to the design one level up in the experiment hierarchy.

Returns:

iv_name : str or tuple

The name of the IV, for non-atomic orderings. Otherwise, an empty tuple.

iv_values : seq

The possible values of the IV. Empty for atomic orderings.
class experimentator.order.CompleteCounterbalance(number=1)

Bases: experimentator.order.NonAtomicOrdering

In a complete counterbalance design, every unique ordering of the conditions appears the same numbers of times.

Parameters:

number : int, optional

The number of times each condition should be duplicated. Note that conditions are duplicated before determining the possible orderings.

Notes

The number of possible orderings can get very large very quickly. Therefore, a complete counterbalance is not recommended for more than 3 conditions. The number of unique orderings can be determined by factorial(number * k) // number**k, where k is the number of conditions (assuming all conditions are unique). For example, with 5 conditions there are 120 possible orders; with 3 conditions and number==2, there are 90 unique orders.

first_pass(conditions)

Handle operations that should only be performed once, initializing the object before ordering conditions. For CompleteCounterbalance, all possible orders are determined. This method should not be called manually.

Parameters:

conditions : sequence of dict

A list of conditions, where each condition is a dictionary mapping IV names to IV values.
Returns:

iv_name : str or tuple

The name of the IV to be created one level up, 'counterbalance_order'.

iv_values : tuple

Values of the IV to be created one level up, integers each associated with an order of the conditions.
class experimentator.order.Sorted(number=1, order='both')

Bases: experimentator.order.NonAtomicOrdering

Sorts the conditions based on the value of the IV defined at its level. This ordering can be non-atomic (if order == 'both'), creating an IV with levels of 'ascending' and 'descending'. If order is 'ascending' or 'descending', the ordering will be atomic (each section will be ordered the same way).

Parameters:

order : {‘both’, ‘ascending’, ‘descending’}, optional

The order to sort the sections. If 'both' (the default), half the sections will be created in ascending order, and half in descending order, depending on the value of the new IV 'sorted_order'.

number : int, optional

The number of times each condition should appear.

Notes

To avoid ambiguity, Sorted can only be used at levels containing only one IV.

first_pass(conditions)

Handle operations that should only be performed once, initializing the object before ordering conditions. For Sorted, the conditions are sorted on IV values.

Parameters:

conditions : sequence of dict

A list of conditions, where each condition is a dictionary mapping IV names to IV values.
Returns:

iv_name : str or tuple

If order is 'both', the name of the IV to be created one level up, 'sorted_order'. Otherwise, an empty tuple (denoting that no IV will be created).

iv_values : tuple

If order is 'both', values of the IV to be created one level up, integers each associated with an order of the conditions. Otherwise, empty tuple.
get_order(data=None)

Get an order of conditions.

Parameters:

data : dict, optional

A dictionary describing the data of the parent section.
Returns:

list of dict

A list of conditions, where each condition is a dictionary mapping IV names to IV values.
class experimentator.order.LatinSquare(number=1, balanced=True, uniform=False)

Bases: experimentator.order.NonAtomicOrdering

Orders the conditions by constructing an NxN Latin square, where N is the number of unique conditions. A Latin square is a matrix with each element appearing exactly once in each row and column. Each row represents a different potential ordering of the conditions. This allows for balanced counterbalancing, in designs too large to accommodate a complete counterbalance.

Parameters:

number : int, optional

The number of times the Latin square should be repeated (default=1). Duplication occurs after constructing the square.

balanced : bool, optional

If True (the default), first-order order effects will be balanced Each condition will appear the same number of times immediately before and immediately after every other condition. Balanced latin squares can only be constructed with an even number of conditions.

uniform : bool, optional

If True (default is False), the Latin square will be randomly sampled from a uniform distribution of Latin squares of size NxN. Otherwise, the sampling will be biased. The construction of balanced, uniform Latin squares is not implemented.

Notes

The algorithm for computing unbalanced Latin squares is not very efficient. It is not recommended to construct unbalanced, uniform Latin squares of order above 5; for non-uniform, unbalanced Latin squares it is safe to go up to an order of 10. Higher than that, computation times increase rapidly.

The algorithm for computing balanced Latin squares is fast only because it is not robust; it is very biased and only samples from the same limited set of balanced Latin squares. However, this is usually not an issue. For more implementation details, see latin_square and balanced_latin_square.

first_pass(conditions)

Handle operations that should only be performed once, initializing the object before ordering conditions. For LatinSquare, the square is constructed.

Parameters:

conditions : sequence of dict

A list of conditions, where each condition is a dictionary mapping IV names to IV values.
Returns:

iv_name : str or tuple

The name of the IV to be created one level up, 'latin_square_row'.

iv_values : tuple

Values of the IV to be created one level up, integers each corresponding to one row of the Latin square.