API reference¶
Experiment¶
-
class
experimentator.
Experiment
(tree, data=None, has_started=False, has_finished=False, _children=None, filename=None, callback_by_level=None, callback_type_by_level=None, session_data=None, experiment_data=None, _callback_info=None)¶ Bases:
experimentator.section.ExperimentSection
An
ExperimentSection
subclass that represents the largest ‘section’ of the experiment; that is, the entire experiment. Functionality added on top ofExperimentSection
includes various constructors, saving to disk, and management of callbacks.To create a new experiment, rather than instantiating directly it is recommended to use one of the constructor methods:
Attributes
tree ( DesignTree
) TheDesignTree
instance defining the experiment’s hierarchy.filename (str) The file location where the Experiment
will be pickled.callback_by_level (dict) A dictionary, mapping level names to functions or With Statement Context Managers (e.g., generator functions decorated with contextlib.contextmanager
). Defines behavior to run at each section (for functions) or before and/or after each section (for context managers) at the associated level.callback_type_by_level (dict) A dictionary mapping level names to either the string 'context'
or'function'
. This keeps track of which callbacks inExperiment.callback_by_level
are context managers.session_data (dict) A dictionary where temporary data can be stored, persistent only within one session of the Python interpreter. This is a good place to store external resources that aren’t picklable; external resources, for example, can be loaded in a context-manager callback and stored here. In addition, anything returned by the __exit__
method of a context-manager callback will be stored here, with the callback’s level name as the key. This dictionary is emptied before saving theExperiment
to disk.experiment_data (dict) A dictionary where data can be stored that is persistent across Python sessions. Everything stored here must be picklable. -
add_callback
(level, callback, *args, is_context=False, func_module=None, func_name=None, **kwargs)¶ Add a callback to run at a certain level.
A callback can be either a regular function, or a context-manager. The latter is useful for defining code to run at the start and end of every section at the level. For example, a block context manager could specify behavior that occurs before every trial in the block, and behavior that occurs after every trial in the block. See
contextlib
for various ways to create context managers, and experimentator’s context-manager docs for more details.Any value returned by the
__enter__
method of a context manager will be stored inExperiment.session_data
under the key level.If the callback is not a context manager, it should return a dictionary (or nothing), which is automatically passed to
ExperimentSection.add_data
. In theory, it should map dependent-variable names to results. See the callback docs for more details.Parameters: level : str
Which level of the hierarchy to manage.
callback : function or context-manager
The callback should have the signature
callback(experiment, section, *args, **kwargs)
where experiment and section are the currentExperiment
andExperimentSection
instances, respectively, and args and kwargs are arbitrary arguments passed to this method.*args
Any arbitrary positional arguments to be passed to callback.
func_module : str, optional
func_name : str, optional
These two arguments specify where the given function should be imported from in future Python sessions (i.e.,
from <func_module> import <func_name>
). Usually, this is figured out automatically by introspection; these arguments are provided for the rare situation where introspection fails.**kwargs
Any arbitrary keyword arguments to be passed to callback.
-
add_data
(data)¶ Update the
ExperimentSection.data
ChainMap
. This data will apply to this section and all child sections. This can be used, for example, to manually record a participant’s age.Parameters: data : dict
Elements to be added to
ExperimentSection.data
.
-
all_subsections
(**section_numbers)¶ Find all subsections in the experiment matching the given section numbers.
Yields specified
ExperimentSection
instances. The yielded sections will be at the lowest level given in section_numbers. If levels not in section_numbers are encountered before reaching its lowest level, all sections will be descended into.Parameters: **section_numbers
Keyword arguments describing what subsections to find. Keys are level names, values are ints or sequences of ints.
Examples
Assuming the levels of the experiment saved in
'example.exp'
are('participant', 'session', 'block', 'trial')
:>>> from experimentator import Experiment >>> exp = Experiment.load('example.exp')
Get the first session of each participant:
>>> all_first_sessions = list(exp.all_subsections(session=1))
Get the second trial of the first block in each session:
>>> trials = list(exp.all_subsections(block=1, trial=2))
Get the first three trials of each block in the first session of each participant:
>>> more_trials = list(exp.all_subsections(session=1, trial=[1, 2, 3]))
-
append_child
(data, tree=None, to_start=False, _renumber=True)¶ Create a new
ExperimentSection
(and its descendants) and append it as a child of the currentExperimentSection
.Parameters: data : dict
Data to be included in the new section’s
ExperimentSection.data
ChainMap
. Should include values of IVs at the section’s level, for example.tree :
DesignTree
, optionalIf given, the section will be appended from the top level of tree. If not passed, the tree of the current section will be used. Note that this does not affect IV values; IV values must still be included in data.
to_start : bool, optional
If True, the new
ExperimentSection
will be appended to the beginning of the current section. If False (the default), it will be appended to the end.Notes
After calling this method, the section numbers in the children’s
ExperimentSection.data
attributes will be automatically replaced with the correct numbers.
-
append_design_tree
(tree, to_start=False, _renumber=True)¶ Append all sections associated with the top level of a
DesignTree
(and therefore also create descendant sections) to theExperimentSection
.Parameters: tree :
DesignTree
The tree to append.
to_start : bool, optional
If True, the sections will be inserted at the beginning of the section. If False (the default), they will be appended to the end.
Notes
After calling this method, the section numbers in the children’s
ExperimentSection.data
attributes will be automatically replaced with the correct numbers.
-
as_graph
()¶ Build a
networkx.DiGraph
out of the experiment structure, starting at this section. Nodes are sections and graphs are parent-child relations. Node data are non-duplicated entries inExperimentSection.data
.Returns: networkx.DiGraph
-
classmethod
basic
(levels, ivs_by_level, design_matrices_by_level=None, ordering_by_level=None, filename=None)¶ Construct a homogeneously-organized
Experiment
, with arbitrary levels but only oneDesign
at each level, and the same structure throughout its hierarchy.Parameters: levels : sequence of str
Names of the levels of the experiment
ivs_by_level : dict
Dictionary specifying the IVs and their possible values at every level. The keys are be the level names, and the values are lists of the IVs at that level, specified in the form of tuples with the first element being the IV name and the second element a list of its possible values. Alternatively, the IVs at each level can be specified in a dictionary. See IV docs for more on specifying IVs.
design_matrices_by_level : dict, optional
Specify the design matrix for any levels. Keys are level names; values are design matrices. Any levels without a design matrix will be fully crossed. See design matrix docs for details.
ordering_by_level : dict, optional
Specify the ordering for each level. Keys are level names; values are instance objects from
experimentator.order
. For any levels without an order specified,Shuffle
will be used.filename : str, optional
File location to save the experiment.
Returns:
-
classmethod
blocked
(trial_ivs, n_participants, design_matrices=None, orderings=None, block_ivs=None, filename=None)¶ Create a blocked within-subjects
Experiment
, in which all the IVs are at either the trial level or the block level.Parameters: trial_ivs : list or dict
A list of the IVs to define at the trial level, specified in the form of tuples with the first element being the IV name and the second element a list of its possible values. Alternatively, the IVs at each level can be specified in a dictionary. See the IV docs more on specifying IVs.
n_participants : int
Number of participants to initialize. If a
NonAtomicOrdering
is used, this is the number of participants per order.design_matrices : dict, optional
Design matrices for the experiment. Keys are
'trial'
and'block'
; values are the respective design matrices (if any). If not specified, IVs will be fully crossed. See the design matrix docs for details.orderings : dict, optional
block_ivs : list or dict, optional
IVs to define at the block level. See IV docs for more on specifying IVs.
filename : str, optional
File location to save the experiment.
Returns: Notes
For blocks to have any effect, you should either define at least one IV at the block level or use the ordering
Ordering(n)
to createn
blocks for every participant.
-
breadth_first_search
(key)¶ Breadth-first search starting from here. Returns the entire search path.
Parameters: key : func
Function that returns True or False when passed an
ExperimentSection
.Returns: list of
ExperimentSection
-
depth_first_search
(key, path_key=None, _path=None)¶ Depth-first search starting from here. Returns the entire search path.
Parameters: key : func
Function that returns True or False when passed an
ExperimentSection
.path_key : func, optional
Function that returns True or False when passed an
ExperimentSection
. If given, the search will proceed only via sections for which path_key returns True.Returns: list of
ExperimentSection
-
export_data
(filename, skip_columns=None, **kwargs)¶ Export
Experiment.dataframe
in.csv
format.Parameters: filename : str
A file location where the data should be saved.
skip_columns : list of str, optional
Columns to skip.
**kwargs
Arbitrary keyword arguments to pass to
pandas.DataFrame.to_csv
.Notes
This method is not recommended for experiments with compound data types, for example an experiment which stores a time series for every trial. In those cases it is recommended to write a custom script that parses the
Experiment.dataframe
attribute as desired, or use the skip_columns option to skip any compound columns.
-
find_first_not_run
(at_level, by_started=True)¶ Search the experimental hierarchy, and return the first descendant
ExperimentSection
at at_level that has not yet been run.Parameters: at_level : str
Which level to search.
by_started : bool, optional
If True (default), returns the first section that has not been started. Otherwise, finds the first section that has not finished.
Returns:
-
find_first_partially_run
(at_level)¶ Search the experimental hierarchy, and return the first descendant
ExperimentSection
at at_level that has been started but not finished.Parameters: at_level : str
Which level to search.
Returns:
-
classmethod
from_dict
(spec)¶ Construct an
Experiment
based on a dictionary specification.Parameters: spec : dict
spec should have, at minimum, a key named
'design'
. The value of this key specifies theDesignTree
. SeeDesignTree.from_spec
for details. The value of the key'filename'
or'file'
, if one exists,is saved inExperiment.filename
. All other fields are saved inExperiment.experiment_data
.Returns:
-
classmethod
from_yaml_file
(filename)¶ Construct an
Experiment
based on specification in a YAML file. Requires PyYAML.Parameters: filename : str
YAML file location. The YAML should specify a dictionary matching the specification of
Experiment.from_dict
.Returns:
-
get_next_tree
()¶ Get a tree to use for creating child
ExperimentSection
instances.Returns: DesignTree
-
static
load
(filename)¶ Load an experiment from disk.
Parameters: filename : str
Path to a file generated by
Experiment.save
.Returns:
-
classmethod
new
(tree, filename=None)¶ Make a new
Experiment
.Parameters: tree :
DesignTree
A
DesignTree
instance defining the experiment hierarchy.filename : str, optional
A file location where the
Experiment
will be saved.
-
parent
(section)¶ Find the parent of a section.
Parameters: section :
ExperimentSection
The section to find the parent of.
Returns:
-
parents
(section)¶ Find all parents of a section, in top-to-bottom order.
Parameters: section :
ExperimentSection
The section to find the parents of.
Returns: list of
ExperimentSection
-
resume_section
(section, **kwargs)¶ Rerun a section that has been started but not finished, starting where running last left off.
Parameters: section :
ExperimentSection
The section to resume.
**kwargs
Keyword arguments to pass to
Experiment.run_section
.Notes
The wrapper function
run_experiment_section
should be used instead of this method, if possible.
-
run_section
(section, demo=False, parent_callbacks=True, from_section=None)¶ Run a section and all its descendant sections. Saves the results in the
data
attribute of each lowest-levelExperimentSection
.Parameters: section :
ExperimentSection
The section to be run.
demo : bool, optional
Data will only be saved if demo is False (the default).
parent_callbacks : bool, optional
If True (the default), all parent callbacks will be called.
from_section : int or list of int, optional
Which section to start running from. If a list is passed, it specifies where to start running on multiple levels. For example, assuming the experiment hierarchy is
('participant', 'session', 'block', 'trial')
, this would start from the fifth trial of the second block (of the first participant’s second session):>>> exp = Experiment.load('example.exp') >>> exp.run_section(exp.subsection(participant=1, session=2), from_section=[2, 5])
Notes
The wrapper function
run_experiment_section
should be used instead of this method, if possible.
-
save
(filename=None)¶ Save the
Experiment
to disk.Parameters: filename : str, optional
If specified, overrides
Experiment.filename
.
-
subsection
(**section_numbers)¶ Find a single, descendant
ExperimentSection
based on section numbers.Parameters: **section_numbers
Keyword arguments describing which subsection to find Must include every level higher than the desired section.
Returns: Examples
Assuming the levels of the experiment saved in
'example.exp'
are('participant', 'session', 'block', 'trial')
, this will return the third block of the second participant’s first session:>>> from experimentator import Experiment >>> exp = Experiment.load('example.exp') >>> some_block = exp.subsection(participant=2, session=1, block=3)
-
walk
()¶ Walk the tree depth-first, starting from here. Yields this section and every descendant section.
-
classmethod
within_subjects
(ivs, n_participants, design_matrix=None, ordering=None, filename=None)¶ Create a within-subjects
Experiment
, with all the IVs at the'trial'
level.Parameters: ivs : list or dict
A list of the experiment’s IVs, specified in the form of tuples with the first element being the IV name and the second element a list of its possible values. Alternatively, the IVs at each level can be specified in a dictionary. See the IV docs more on specifying IVs.
n_participants : int
Number of participants to initialize.
design_matrix : array-like, optional
Design matrix for the experiment. If not specified, IVs will be fully crossed. See the design matrix docs for more details.
ordering :
Ordering
, optionalfilename : str, optional
File location to save the experiment.
Returns:
-
Helper functions¶
-
experimentator.
run_experiment_section
(experiment, section_obj=None, demo=False, resume=False, parent_callbacks=True, from_section=1, session_options='', **section_numbers)¶ Run an experiment from a file or an
Experiment
instance, and save it. If an exception is encountered, theExperiment
will be backed up and saved.Parameters: experiment : str or
Experiment
File location where an
Experiment
instance is pickled, or anExperiment
instance.demo : bool, optional
If True, data will not be saved and sections will not be marked as run.
resume: bool, optional
If True, the specified section will be resumed (started automatically where it left off).
parent_callbacks : bool, optional
If True (the default), all parent callbacks will be called.
section_obj :
ExperimentSection
, optionalThe section of the experiment to run. Alternatively, the section can be specified using **section_numbers.
from_section : int or list of int, optional
Which section to start running from. If a list is passed, it specifies where to start running on multiple levels. See the example below.
session_options : str, optional
Pass an experiment-specific options string to be stored in
Experiment.session_data
under the key'options'
.**section_numbers
Keyword arguments describing how to descend the experiment hierarchy to find the section to run. See the example below.
Examples
- A simple example:
>>> exp = Experiment.load('example.exp') >>> run_experiment_section(exp, exp.subsection(participant=1, session=2))
Equivalently:
>>> run_experiment_section(exp, participant=1, session=2)
To demonstrate from_section, assuming the experiment hierarchy is
('participant', 'session', 'block', 'trial')
, this would start from the second block:>>> run_experiment_section(exp, participant=1, session=2, from_section=2)
To start from the fifth trial of the second block:
>>> run_experiment_section(exp, participant=1, session=2, from_section=[2, 5])
-
experimentator.
export_experiment_data
(exp_filename, data_filename, **kwargs)¶ Reads a pickled
Experiment
instance and saves its data in.csv
format.Parameters: exp_filename : str
The file location where an
Experiment
instance is pickled.data_filename : str
The file location where the data will be written.
skip_columns : list of str, optional
Data columns to skip.
**kwargs
Arbitrary keyword arguments passed through to
pandas.DataFrame.to_csv
.Notes
This shortcut function is not recommended for experiments with compound data types, for example an experiment which stores a time series for every trial. In such cases it is recommended to write a custom script that parses
Experiment.dataframe
as desired (or use the skip_columns option to ignore the compound data).
ExperimentSection¶
-
class
experimentator.section.
ExperimentSection
(tree, data=None, has_started=False, has_finished=False, _children=None)¶ A section of the experiment, at any level of the hierarchy. Single trials and groups of trials (blocks, sessions, participants, etc.) are represented as
ExperimentSection
instances. A complete experiment consists ofExperimentSection
instances arranged in a tree. The root element should be anExperiment
(a subclass ofExperimentSection
); the rest of the sections can be reached via its descendants (see below on the sequence protocol). A newExperimentSection
instance is automatically populated withExperimentSection
descendants according to theDesignTree
passed to its constructor.ExperimentSection
implements Python’s sequence protocol; its contents areExperimentSection
instances at the level below. In other words, children can be accessed using the[index]
notation, as well as with slices ([3:6]
) or iteration (for section in experiment_section
). However,ExperimentSection
breaks the Python convention of 0-based indexing, using 1-based indexing to match the convention in experimental science.The direct constructor is used to create an arbitrary
ExperimentSection
(i.e., possibly reloading an in-progress section), whereasExperimentSection.new
creates a section that hasn’t yet started.Notes
Use 1-based indexing to refer to
ExperimentSection
children, both when when using indexing or slicing with anExperimentSection
, and when identifying sections in keyword arguments to methods such asExperimentSection.subsection
. This better corresponds to the language commonly used by scientists to identify participants, trials, etc.Attributes
tree ( DesignTree
)data ( ChainMap
)description (str) The name and number of the section (e.g., 'trial 3'
).dataframe ( DataFrame
) All data associated with theExperimentSection
and its descendants.heterogeneous_design_iv_name (str) IV name determining which branch of the DesignTree
to follow.level (str) The level of the hierarchy at which this section lives. levels (list of str) Level names below this section. local_levels (set) Level names of this section’s children. Usually a single-element set. is_bottom_level (bool) If true, this is the lowest level of the hierarchy. is_top_level (bool) If true, this is the highest level of the hierarchy (likely an Experiment
).has_started: bool Whether this section has started to be run. has_finished (bool) Whether this section has finished running. -
add_data
(data)¶ Update the
ExperimentSection.data
ChainMap
. This data will apply to this section and all child sections. This can be used, for example, to manually record a participant’s age.Parameters: data : dict
Elements to be added to
ExperimentSection.data
.
-
all_subsections
(**section_numbers)¶ Find all subsections in the experiment matching the given section numbers.
Yields specified
ExperimentSection
instances. The yielded sections will be at the lowest level given in section_numbers. If levels not in section_numbers are encountered before reaching its lowest level, all sections will be descended into.Parameters: **section_numbers
Keyword arguments describing what subsections to find. Keys are level names, values are ints or sequences of ints.
Examples
Assuming the levels of the experiment saved in
'example.exp'
are('participant', 'session', 'block', 'trial')
:>>> from experimentator import Experiment >>> exp = Experiment.load('example.exp')
Get the first session of each participant:
>>> all_first_sessions = list(exp.all_subsections(session=1))
Get the second trial of the first block in each session:
>>> trials = list(exp.all_subsections(block=1, trial=2))
Get the first three trials of each block in the first session of each participant:
>>> more_trials = list(exp.all_subsections(session=1, trial=[1, 2, 3]))
-
append_child
(data, tree=None, to_start=False, _renumber=True)¶ Create a new
ExperimentSection
(and its descendants) and append it as a child of the currentExperimentSection
.Parameters: data : dict
Data to be included in the new section’s
ExperimentSection.data
ChainMap
. Should include values of IVs at the section’s level, for example.tree :
DesignTree
, optionalIf given, the section will be appended from the top level of tree. If not passed, the tree of the current section will be used. Note that this does not affect IV values; IV values must still be included in data.
to_start : bool, optional
If True, the new
ExperimentSection
will be appended to the beginning of the current section. If False (the default), it will be appended to the end.Notes
After calling this method, the section numbers in the children’s
ExperimentSection.data
attributes will be automatically replaced with the correct numbers.
-
append_design_tree
(tree, to_start=False, _renumber=True)¶ Append all sections associated with the top level of a
DesignTree
(and therefore also create descendant sections) to theExperimentSection
.Parameters: tree :
DesignTree
The tree to append.
to_start : bool, optional
If True, the sections will be inserted at the beginning of the section. If False (the default), they will be appended to the end.
Notes
After calling this method, the section numbers in the children’s
ExperimentSection.data
attributes will be automatically replaced with the correct numbers.
-
as_graph
()¶ Build a
networkx.DiGraph
out of the experiment structure, starting at this section. Nodes are sections and graphs are parent-child relations. Node data are non-duplicated entries inExperimentSection.data
.Returns: networkx.DiGraph
-
breadth_first_search
(key)¶ Breadth-first search starting from here. Returns the entire search path.
Parameters: key : func
Function that returns True or False when passed an
ExperimentSection
.Returns: list of
ExperimentSection
-
depth_first_search
(key, path_key=None, _path=None)¶ Depth-first search starting from here. Returns the entire search path.
Parameters: key : func
Function that returns True or False when passed an
ExperimentSection
.path_key : func, optional
Function that returns True or False when passed an
ExperimentSection
. If given, the search will proceed only via sections for which path_key returns True.Returns: list of
ExperimentSection
-
find_first_not_run
(at_level, by_started=True)¶ Search the experimental hierarchy, and return the first descendant
ExperimentSection
at at_level that has not yet been run.Parameters: at_level : str
Which level to search.
by_started : bool, optional
If True (default), returns the first section that has not been started. Otherwise, finds the first section that has not finished.
Returns:
-
find_first_partially_run
(at_level)¶ Search the experimental hierarchy, and return the first descendant
ExperimentSection
at at_level that has been started but not finished.Parameters: at_level : str
Which level to search.
Returns:
-
get_next_tree
()¶ Get a tree to use for creating child
ExperimentSection
instances.Returns: DesignTree
-
classmethod
new
(tree, data=None)¶ Create a new
ExperimentSection
.Parameters: tree :
DesignTree
Describes the design of the experiment hierarchy.
data :
ChainMap
All data to be associated with the
ExperimentSection
, including the values of independent variables, the section numbers indicating the section’s location in the experiment, and any results associated with this section, arising from either the run callback of theExperiment
or from the methodExperimentSection.add_data
. data should be acollections.ChainMap
, which behaves like a dictionary but has a hierarchical organization such that children can access values from the parent but not vice-versa.
-
parent
(section)¶ Find the parent of a section.
Parameters: section :
ExperimentSection
The section to find the parent of.
Returns:
-
parents
(section)¶ Find all parents of a section, in top-to-bottom order.
Parameters: section :
ExperimentSection
The section to find the parents of.
Returns: list of
ExperimentSection
-
subsection
(**section_numbers)¶ Find a single, descendant
ExperimentSection
based on section numbers.Parameters: **section_numbers
Keyword arguments describing which subsection to find Must include every level higher than the desired section.
Returns: Examples
Assuming the levels of the experiment saved in
'example.exp'
are('participant', 'session', 'block', 'trial')
, this will return the third block of the second participant’s first session:>>> from experimentator import Experiment >>> exp = Experiment.load('example.exp') >>> some_block = exp.subsection(participant=2, session=1, block=3)
-
walk
()¶ Walk the tree depth-first, starting from here. Yields this section and every descendant section.
-
Design¶
-
class
experimentator.
Design
(ivs=None, design_matrix=None, ordering=None, extra_data=None)¶ Design
instances specify the experimental design at one level of the experimental hierarchy. They guide the creation ofExperimentSection
instances by parsing design matrices or crossing independent variables (IVs).Parameters: ivs : dict or list of tuple, optional
Independent variables can be specified as a dictionary mapping names to possible values, or as a list of
(name, values)
tuples. If an IV takes continuous values, useNone
for its levels. This only works when specifying values using design_matrix. See the IV docs for more information.design_matrix : array-like, optional
A
numpy array
(or convertible, e.g. a list-of-lists) representing a design matrix specifying how IV values should be grouped to form conditions. When no design_matrix is passed, IVs are fully crossed. See the design matrix docs for more details. Note that a design matrix may also specify the order of the conditions. For this reason, the default ordering changes fromShuffle
toOrdering
, preserving the order of the conditions.ordering :
Ordering
, optionalextra_data : dict, optional
Items from this dictionary will be included in the
data
attribute of anyExperimentSection
instances created with thisDesign
.Examples
>>> from experimentator.order import Shuffle >>> design = Design(ivs={'side': ['left', 'right'], 'difficulty': ['easy', 'hard']}, ordering=Shuffle(2)) >>> design.first_pass() IndependentVariable(name=(), values=()) >>> design.get_order() [{'difficulty': 'easy', 'side': 'left'}, {'difficulty': 'hard', 'side': 'left'}, {'difficulty': 'easy', 'side': 'left'}, {'difficulty': 'hard', 'side': 'right'}, {'difficulty': 'easy', 'side': 'right'}, {'difficulty': 'easy', 'side': 'right'}, {'difficulty': 'hard', 'side': 'left'}, {'difficulty': 'hard', 'side': 'right'}]
Attributes
iv_names (list of str) iv_values (list of tuple) design_matrix (array-like) extra_data (dict) ordering ( Ordering
)heterogeneous_design_iv_name (str) The IV name that triggers a heterogeneous (i.e., branching) tree structure when it is encountered. 'design'
by default.is_heterogeneous (bool) True if this Design
is the lowest level before the tree structure diverges.branches (dict) The IV values corresponding to named heterogeneous branches in the tree structure following this Design
.-
first_pass
()¶ Initialize design.
Initializes the design by parsing the design matrix or crossing the IVs If a
NonAtomicOrdering
is used, an additional IV will be returned which should be incorporated into the design one level up in the experimental hierarchy. For this reason, thefirst_pass
methods in a hierarchy ofDesign
instances should be called in reverse order, from bottom up. Use aDesignTree
to ensure this occurs properly.Returns: iv_name : str or tuple
The name of the IV, for
non-atomic orderings
. Otherwise, an empty tuple.iv_values : tuple
The possible values of the IV. Empty for atomic orderings.
-
classmethod
from_dict
(spec)¶ Construct a
Design
instance from a specification based on dictionaries (e.g., parsed from a YAML file).Parameters: spec : dict
A dictionary containing some of the following keys (all optional):
'name'
, the name of the level;'ivs'
,'design_matrix'
,'extra_data'
, keyword arguments to theDesign
constructor;'order'
or'ordering'
, a string, dictionary, or list determining the ordering method; and'n'
or'number'
, thenumber
argument to the specified ordering. A dictionary containing any fields not otherwise used is passed to theDesign
constructor as theextra_data
argument. See the description in the docs for more information.Returns: name : str
Only returned if spec contains a field
'name'
.design :
Design
See also
Examples
>>> design_spec = { ...'name': 'block', ...'ivs': {'speed': [1, 2, 3], 'size': [15, 30]}, ...'ordering': 'Shuffle', ...'n': 3} >>> Design.from_dict(design_spec) Level(name='block', design=Design(ivs=[('speed', [1, 2, 3]), ('size', [15, 30])], design_matrix=None, ordering=Shuffle(number=3, avoid_repeats=False), extra_data={}))
-
static
full_cross
(iv_names, iv_values)¶ Perform a full factorial cross of the independent variables. Yields dictionaries, each describing one condition, a mapping from IV names to IV values. One dictionary is yielded for every possible combination of IV values.
Parameters: iv_names : list of str
Names of IVs.
iv_values : list of list
Each element defines the possible values of an IV. Must be the same length as iv_names. Its elements must be hashable.
-
get_order
(data=None)¶ Order the conditions.
Returns: list of dict
A list of dictionaries, each specifying a condition (a mapping from IV names to values).
-
update
(names, values)¶ Add additional independent variables to the
Design
. This will have no effect afterDesign.first_pass
has been called.Parameters: names : list of str
Names of IVs to add.
values : list of list
For each IV, a list of possible values.
-
DesignTree¶
-
class
experimentator.
DesignTree
(levels_and_designs=None, other_designs=None, branches=None)¶ A container for
Design
instances, describing the entire hierarchy of a basicExperiment
.DesignTree
instances are iterators; callingnext
on one will return anotherDesignTree
with the top level removed. In this way, the entire experimental hierarchy can be created by recursively callingnext
.Use
DesignTree.new
to create a new tree, the generic constructor is for instantiating trees whose attributes have already been processed (i.e., reloading already-created trees).Notes
Calling
next
on the last level of a heterogeneousDesignTree
will return a dictionary of namedDesignTree
instances (rather than a singleDesignTree
instance). The keys are the possible values of the IV'design'
and the values are the correspondingDesignTree
instances.Attributes
levels_and_designs (list of tuple) other_designs (dict) branches (dict) Only those items from other_designs that follow directly from this tree. -
add_base_level
()¶ Adds a section to the top of the tree called
'_base'
. This makes theDesignTree
suitable for constructing anExperiment
.Notes
The
Experiment
constructor calls this automatically, and this shouldn’t be called when appending a tree to an existingExperiment
, so there is no use case for manually calling this method.
-
static
first_pass
(levels_and_designs)¶ Make a first pass of all designs in a
DesignTree
, from bottom to top. This callsDesign.first_pass
on everyDesign
instance in the tree in the proper order, updating designs when a new IV is returned. This is necessary fornon-atomic orderings
because they modify the parentDesign
.
-
classmethod
from_spec
(spec)¶ Constructs a
DesignTree
instance from a specification (e.g., parsed from a YAML file).- spec : dict or list of dict
- The
DesignTree
specification. A dictionary with keys as tree names and values as lists of dictionaries. Each sub-dictionary should specify aDesign
according toDesign.from_dict
. The main tree should be named'main'
. Other names are used for generating heterogeneous trees (seeDesignTree
docs). A homogeneous tree can be specified as a dictionary with only a single key'main'
, or directly as a list of dictionaries
Returns: DesignTree
-
classmethod
new
(levels_and_designs, **other_designs)¶ Create a new
DesignTree
.Parameters: levels_and_designs :
OrderedDict
or list of tupleThis input defines the structure of the tree, and is either an
OrderedDict
or a list of 2-tuples. Keys (or first element of each tuple) are level names. Values (or second element of each tuple) are design specifications, in the form of either aDesign
instance, or a list ofDesign
instances to occur in sequence.**other_designs
Named design trees, can be other
DesignTree
instances or suitable levels_and_designs inputs (i.e.,OrderedDict
or list of tuples). These designs allow for heterogeneous design structures (i.e. not every section at the same level has the sameDesign
). To make a heterogeneousDesignTree
, use an IV named'design'
at the level where the heterogeneity should occur. Values of this IV should be strings, each corresponding to the name of aDesignTree
from` other_designs`. The value of the IV'design'
at each section determines whichDesignTree
is used for children of that section.
-
experimentator.order¶
This module contains the class Ordering
and its descendants.
These classes handle how unique conditions at a particular experimental level are ordered and duplicated.
Ordering
instances should be passed directly to the Design
constructor;
there is no reason to otherwise interact with them in normal use.
Of special note are non-atomic orderings:
the class NonAtomicOrdering
and its descendants.
‘’Non-atomic’’ here means that the orderings between sections are not independent.
A Shuffle
ordering is atomic;
the order in one section is independent of the order in another.
However, if one wants to ensure, for example, that possible block orders are evenly distributed among participants
(a counterbalanced design
),
the block orders within each participants are not independent.
Each participant can decide its order of blocks only in the context of the other participants’ block orders.
This means that the parent section must handle orderings
(in the example of counterbalanced blocks,
the Experiment.base_section
–the experiment itself, essentially–must tell each participant what block order to use).
-
class
experimentator.order.
Ordering
(number=1)¶ Bases:
object
The base ordering class. It will keep conditions in the order they are defined by the
Design
instance (either the order of rows in the design matrix, or a full factorial cross–the output of callingitertools.product
on the IV levels). Remember not to rely on the order of dictionary items. Therefore, if a specific order is desired, it is recommended to use a design matrix or anOrderedDict
to define the IVs. See the IV docs for more information.Parameters: number : int, optional
The number of times each unique condition should appear. The default is 1. If
number > 1
, the entire order will be cycled (as opposed to repeating each condition within the order).-
first_pass
(conditions)¶ Handle operations that should only be performed once, initializing the object before ordering conditions. For
Ordering
, the only operation is duplication of the list of conditions (ifOrdering.number
> 1). This methods should not be called manually.Parameters: conditions : sequence of dict
A list of conditions, where each condition is a dictionary mapping IV names to IV values.
Returns: iv_name : str or tuple
The name of the IV, for non-atomic orderings. Otherwise, an empty tuple.
iv_values : tuple
The possible values of the IV. Empty for atomic orderings.
-
get_order
(data=None)¶ Get an order of conditions. For
Ordering
, always returns the same order.Parameters: data : dict, optional
A dictionary describing the data of the parent section. Unused for atomic orderings.
Returns: list of dict
A list of conditions, where each condition is a dictionary mapping IV names to IV values.
-
static
possible_orders
(conditions, unique=True)¶ Yield all possible orders of the conditions. Each order is a list of dictionaries, with each dictionary representing a condition.
Parameters: conditions : sequence of dict
A list of conditions, where each condition is a dictionary mapping IV names to IV values.
unique : bool, optional
If true (the default), will only return unique orders. If false, some identical orders will be generated if conditions contains identical elements.
-
-
class
experimentator.order.
Shuffle
(number=1, avoid_repeats=False)¶ Bases:
experimentator.order.Ordering
This ordering randomly shuffles the conditions.
Parameters: number : int, optional
Number of times each condition should appear (default=1). Conditions are duplicated before shuffling.
avoid_repeats : bool, optional
If True (default is False), no identical conditions will appear back-to-back.
-
get_order
(data=None)¶ Get an order of conditions. For
Shuffle
, returns the conditions in a random order.Parameters: data : dict, optional
A dictionary describing the data of the parent section. Unused for atomic orderings.
Returns: list of dict
A list of conditions, where each condition is a dictionary mapping IV names to IV values.
-
-
class
experimentator.order.
NonAtomicOrdering
(number=1)¶ Bases:
experimentator.order.Ordering
This is a base class for non-atomic orderings, and is not meant to be directly instantiated. Non-atomic orderings work by creating a new independent variable one level up. The IV name will start with an underscore, a convention to avoid name clashes.
-
get_order
(data=None)¶ Get an order of conditions.
Parameters: data : dict, optional
A dictionary describing the data of the parent section.
Returns: list of dict
A list of conditions, where each condition is a dictionary mapping IV names to IV values.
-
iv
¶ The IV associated with the non-atomic ordering. It will be added to the design one level up in the experiment hierarchy.
Returns: iv_name : str or tuple
The name of the IV, for non-atomic orderings. Otherwise, an empty tuple.
iv_values : seq
The possible values of the IV. Empty for atomic orderings.
-
-
class
experimentator.order.
CompleteCounterbalance
(number=1)¶ Bases:
experimentator.order.NonAtomicOrdering
In a complete counterbalance design, every unique ordering of the conditions appears the same numbers of times.
Parameters: number : int, optional
The number of times each condition should be duplicated. Note that conditions are duplicated before determining the possible orderings.
Notes
The number of possible orderings can get very large very quickly. Therefore, a complete counterbalance is not recommended for more than 3 conditions. The number of unique orderings can be determined by
factorial(number * k) // number**k
, where k is the number of conditions (assuming all conditions are unique). For example, with 5 conditions there are 120 possible orders; with 3 conditions andnumber==2
, there are 90 unique orders.-
first_pass
(conditions)¶ Handle operations that should only be performed once, initializing the object before ordering conditions. For
CompleteCounterbalance
, all possible orders are determined. This method should not be called manually.Parameters: conditions : sequence of dict
A list of conditions, where each condition is a dictionary mapping IV names to IV values.
Returns: iv_name : str or tuple
The name of the IV to be created one level up,
'counterbalance_order'
.iv_values : tuple
Values of the IV to be created one level up, integers each associated with an order of the conditions.
-
-
class
experimentator.order.
Sorted
(number=1, order='both')¶ Bases:
experimentator.order.NonAtomicOrdering
Sorts the conditions based on the value of the IV defined at its level. This ordering can be non-atomic (if order
== 'both'
), creating an IV with levels of'ascending'
and'descending'
. If order is'ascending'
or'descending'
, the ordering will be atomic (each section will be ordered the same way).Parameters: order : {‘both’, ‘ascending’, ‘descending’}, optional
The order to sort the sections. If
'both'
(the default), half the sections will be created in ascending order, and half in descending order, depending on the value of the new IV'sorted_order'
.number : int, optional
The number of times each condition should appear.
Notes
To avoid ambiguity,
Sorted
can only be used at levels containing only one IV.-
first_pass
(conditions)¶ Handle operations that should only be performed once, initializing the object before ordering conditions. For
Sorted
, the conditions are sorted on IV values.Parameters: conditions : sequence of dict
A list of conditions, where each condition is a dictionary mapping IV names to IV values.
Returns: iv_name : str or tuple
If order is
'both'
, the name of the IV to be created one level up,'sorted_order'
. Otherwise, an empty tuple (denoting that no IV will be created).iv_values : tuple
If order is
'both'
, values of the IV to be created one level up, integers each associated with an order of the conditions. Otherwise, empty tuple.
-
get_order
(data=None)¶ Get an order of conditions.
Parameters: data : dict, optional
A dictionary describing the data of the parent section.
Returns: list of dict
A list of conditions, where each condition is a dictionary mapping IV names to IV values.
-
-
class
experimentator.order.
LatinSquare
(number=1, balanced=True, uniform=False)¶ Bases:
experimentator.order.NonAtomicOrdering
Orders the conditions by constructing an NxN Latin square, where N is the number of unique conditions. A Latin square is a matrix with each element appearing exactly once in each row and column. Each row represents a different potential ordering of the conditions. This allows for balanced counterbalancing, in designs too large to accommodate a complete counterbalance.
Parameters: number : int, optional
The number of times the Latin square should be repeated (default=1). Duplication occurs after constructing the square.
balanced : bool, optional
If True (the default), first-order order effects will be balanced Each condition will appear the same number of times immediately before and immediately after every other condition. Balanced latin squares can only be constructed with an even number of conditions.
uniform : bool, optional
If True (default is False), the Latin square will be randomly sampled from a uniform distribution of Latin squares of size NxN. Otherwise, the sampling will be biased. The construction of balanced, uniform Latin squares is not implemented.
Notes
The algorithm for computing unbalanced Latin squares is not very efficient. It is not recommended to construct unbalanced, uniform Latin squares of order above 5; for non-uniform, unbalanced Latin squares it is safe to go up to an order of 10. Higher than that, computation times increase rapidly.
The algorithm for computing balanced Latin squares is fast only because it is not robust; it is very biased and only samples from the same limited set of balanced Latin squares. However, this is usually not an issue. For more implementation details, see
latin_square
andbalanced_latin_square
.-
first_pass
(conditions)¶ Handle operations that should only be performed once, initializing the object before ordering conditions. For
LatinSquare
, the square is constructed.Parameters: conditions : sequence of dict
A list of conditions, where each condition is a dictionary mapping IV names to IV values.
Returns: iv_name : str or tuple
The name of the IV to be created one level up,
'latin_square_row'
.iv_values : tuple
Values of the IV to be created one level up, integers each corresponding to one row of the Latin square.
-