allensdk.brain_observatory.behavior.behavior_session module

class allensdk.brain_observatory.behavior.behavior_session.BehaviorSession(behavior_session_id: BehaviorSessionId, stimulus_timestamps: StimulusTimestamps, running_acquisition: RunningAcquisition, raw_running_speed: RunningSpeed, running_speed: RunningSpeed, licks: Licks, rewards: Rewards, stimuli: Stimuli, task_parameters: TaskParameters, trials: Trials, metadata: BehaviorMetadata, date_of_acquisition: DateOfAcquisition, eye_tracking_table: EyeTrackingTable | None = None, eye_tracking_rig_geometry: RigGeometry | None = None)

Bases: DataObject, LimsReadableInterface, NwbReadableInterface, JsonReadableInterface, NwbWritableInterface

Represents data from a single Visual Behavior behavior session. Initialize by using class methods from_lims or from_nwb_path.

property behavior_session_id: int

Unique identifier for a behavioral session. :rtype: int

property eye_tracking: DataFrame | None

A dataframe containing ellipse fit parameters for the eye, pupil and corneal reflection (cr). Fits are derived from tracking points from a DeepLabCut model applied to video frames of a subject’s right eye. Raw tracking points and raw video frames are not exposed by the SDK.

Notes: - All columns starting with ‘pupil_’ represent ellipse fit parameters

relating to the pupil.

• All columns starting with ‘eye_’ represent ellipse fit parameters relating to the eyelid.

• All columns starting with ‘cr_’ represent ellipse fit parameters relating to the corneal reflection, which is caused by an infrared LED positioned near the eye tracking camera.

• All positions are in units of pixels.

• All areas are in units of pixels^2

• All values are in the coordinate space of the eye tracking camera, NOT the coordinate space of the stimulus display (i.e. this is not gaze location), with (0, 0) being the upper-left corner of the eye-tracking image.

• The ‘likely_blink’ column is True for any row (frame) where the pupil fit failed OR eye fit failed OR an outlier fit was identified on the pupil or eye fit.

• The pupil_area, cr_area, eye_area, and pupil/eye_width, height, phi columns are set to NaN wherever ‘likely_blink’ == True.

• The pupil_area_raw, cr_area_raw, eye_area_raw columns contains all pupil fit values (including where ‘likely_blink’ == True).

• All ellipse fits are derived from tracking points that were output by a DeepLabCut model that was trained on hand-annotated data from a subset of imaging sessions on optical physiology rigs.

• Raw DeepLabCut tracking points are not publicly available.

Return type:

pandas.DataFrame

property eye_tracking_rig_geometry: dict

the eye tracking equipment geometry associated with a given behavior session.

Returns:

dict

dictionary with the following keys:

camera_eye_position_mm (array of float) camera_rotation_deg (array of float) equipment (string) led_position (array of float) monitor_position_mm (array of float) monitor_rotation_deg (array of float)

classmethod from_json(session_data: dict, read_stimulus_presentations_table_from_file=False, stimulus_presentation_columns: List[str] | None = None, stimulus_presentation_exclude_columns: List[str] | None = None, eye_tracking_z_threshold: float = 3.0, eye_tracking_dilation_frames: int = 2, eye_tracking_drop_frames: bool = False, sync_file_permissive: bool = False, running_speed_load_from_multiple_stimulus_files: bool = False) → BehaviorSession

Parameters:

session_data

Dict of input data necessary to construct a session

read_stimulus_presentations_table_from_file

Whether to read the stimulus table from a file rather than construct it here

stimulus_presentation_columns

Columns to include in the stimulus presentation table. This also specifies the order of the columns.

stimulus_presentation_exclude_columns

Optional list of columns to exclude from stimulus presentations table

eye_tracking_z_threshold

See BehaviorSession.from_nwb

eye_tracking_dilation_frames

See BehaviorSession.from_nwb

eye_tracking_drop_frames

See drop_frames arg in allensdk.brain_observatory.behavior. data_objects.eye_tracking.eye_tracking_table.EyeTrackingTable. from_data_file

sync_file_permissive

See permissive arg in SyncFile constructor

running_speed_load_from_multiple_stimulus_files

Whether to load running speed from multiple stimulus files If False, will just load from a single behavior stimulus file

Returns:

BehaviorSession instance

classmethod from_lims(behavior_session_id: int, lims_db: PostgresQueryMixin | None = None, sync_file: SyncFile | None = None, monitor_delay: float | None = None, date_of_acquisition: DateOfAcquisition | None = None, eye_tracking_z_threshold: float = 3.0, eye_tracking_dilation_frames: int = 2) → BehaviorSession

Parameters:

behavior_session_idint

Behavior session id

lims_dbPostgresQueryMixin, Optional

Database connection. If not provided will create a new one.

sync_fileSyncFile, Optional

If provided, will be used to compute the stimulus timestamps associated with this session. Otherwise, the stimulus timestamps will be computed from the stimulus file.

monitor_delayfloat, Optional

Monitor delay. If not provided, will use an estimate. To provide this value, see for example allensdk.brain_observatory.behavior.data_objects.stimuli.util. calculate_monitor_delay

date_of_acquisitionDateOfAcquisition, Optional

Date of acquisition. If not provided, will read from behavior_sessions table.

eye_tracking_z_thresholdfloat

See BehaviorSession.from_nwb, default 3.0

eye_tracking_dilation_framesint

See BehaviorSession.from_nwb, default 2

Returns:

BehaviorSession instance

classmethod from_nwb(nwbfile: NWBFile, add_is_change_to_stimulus_presentations_table=True, eye_tracking_z_threshold: float = 3.0, eye_tracking_dilation_frames: int = 2) → BehaviorSession

Parameters:

nwbfile

add_is_change_to_stimulus_presentations_table: Whether to add a column

denoting whether the stimulus presentation represented a change event. May not be needed in case this column is precomputed

eye_tracking_z_thresholdfloat, optional

The z-threshold when determining which frames likely contain outliers for eye or pupil areas. Influences which frames are considered ‘likely blinks’. By default 3.0

eye_tracking_dilation_framesint, optional

Determines the number of adjacent frames that will be marked as ‘likely_blink’ when performing blink detection for eye_tracking data, by default 2

Returns:

classmethod from_nwb_path(nwb_path: str, **kwargs) → BehaviorSession

Parameters:

nwb_path

Path to nwb file

kwargs

Kwargs to be passed to from_nwb

Returns:

An instantiation of a BehaviorSession

get_performance_metrics(engaged_trial_reward_rate_threshold: float = 2.0) → dict

Get a dictionary containing a subject’s behavior response summary data.

Parameters:

engaged_trial_reward_rate_thresholdfloat, optional

The number of rewards per minute that needs to be attained before a subject is considered ‘engaged’, by default 2.0

Returns:

dict

Returns a dict of performance metrics with the following fields:

trial_count: (int)

The length of the trial dataframe (including all ‘go’, ‘catch’, and ‘aborted’ trials)

go_trial_count: (int)

Number of ‘go’ trials in a behavior session

catch_trial_count: (int)

Number of ‘catch’ trial types during a behavior session

hit_trial_count: (int)

Number of trials with a hit behavior response type in a behavior session

miss_trial_count: (int)

Number of trials with a miss behavior response type in a behavior session

false_alarm_trial_count: (int)

Number of trials where the mouse had a false alarm behavior response

correct_reject_trial_count: (int)

Number of trials with a correct reject behavior response during a behavior session

auto_reward_count:

Number of trials where the mouse received an auto reward of water.

earned_reward_count:

Number of trials where the mouse was eligible to receive a water reward (‘go’ trials) and did receive an earned water reward

total_reward_count:

Number of trials where the mouse received a water reward (earned or auto rewarded)

total_reward_volume: (float)

Volume of all water rewards received during a behavior session (earned and auto rewarded)

maximum_reward_rate: (float)

The peak of the rolling reward rate (rewards/minute)

engaged_trial_count: (int)

Number of trials where the mouse is engaged (reward rate > 2 rewards/minute)

mean_hit_rate: (float)

The mean of the rolling hit_rate

mean_hit_rate_uncorrected:

The mean of the rolling hit_rate_raw

mean_hit_rate_engaged: (float)

The mean of the rolling hit_rate, excluding epochs when the rolling reward rate was below 2 rewards/minute

mean_false_alarm_rate: (float)

The mean of the rolling false_alarm_rate, excluding epochs when the rolling reward rate was below 2 rewards/minute

mean_false_alarm_rate_uncorrected: (float)

The mean of the rolling false_alarm_rate_raw

mean_false_alarm_rate_engaged: (float)

The mean of the rolling false_alarm_rate, excluding epochs when the rolling reward rate was below 2 rewards/minute

mean_dprime: (float)

The mean of the rolling d_prime

mean_dprime_engaged: (float)

The mean of the rolling d_prime, excluding epochs when the rolling reward rate was below 2 rewards/minute

max_dprime: (float)

The peak of the rolling d_prime

max_dprime_engaged: (float)

The peak of the rolling d_prime, excluding epochs when the rolling reward rate was below 2 rewards/minute

get_reward_rate() → ndarray

Get the reward rate of the subject for the task calculated over a 25 trial rolling window and provides a measure of the rewards earned per unit time (in units of rewards/minute).

Returns:

np.ndarray

The reward rate (rewards/minute) of the subject for the task calculated over a 25 trial rolling window.

get_rolling_performance_df() → DataFrame

Return a DataFrame containing trial by trial behavior response performance metrics.

Returns:

pd.DataFrame

A pandas DataFrame containing:

trials_id [index]: (int)

Index of the trial. All trials, including aborted trials, are assigned an index starting at 0 for the first trial.

reward_rate: (float)

Rewards earned in the previous 25 trials, normalized by the elapsed time of the same 25 trials. Units are rewards/minute.

hit_rate_raw: (float)

Fraction of go trials where the mouse licked in the response window, calculated over the previous 100 non-aborted trials. Without trial count correction applied.

hit_rate: (float)

Fraction of go trials where the mouse licked in the response window, calculated over the previous 100 non-aborted trials. With trial count correction applied.

false_alarm_rate_raw: (float)

Fraction of catch trials where the mouse licked in the response window, calculated over the previous 100 non-aborted trials. Without trial count correction applied.

false_alarm_rate: (float)

Fraction of catch trials where the mouse licked in the response window, calculated over the previous 100 non-aborted trials. Without trial count correction applied.

rolling_dprime: (float)

d prime calculated using the rolling hit_rate and rolling false_alarm _rate.

property licks: DataFrame

A dataframe containing lick timestmaps and frames, sampled at 60 Hz.

NOTE: For BehaviorSessions, returned timestamps are not aligned to external ‘synchronization’ reference timestamps. Synchronized timestamps are only available for BehaviorOphysExperiments.

Returns:

np.ndarray

A dataframe containing lick timestamps. dataframe columns:

timestamps: (float)

time of lick, in seconds

frame: (int)

frame of lick

list_data_attributes_and_methods() → List[str]

Convenience method for end-users to list attributes and methods that can be called to access data for a BehaviorSession.

NOTE: Because BehaviorOphysExperiment inherits from BehaviorSession, this method will also be available there.

Returns:

List[str]

A list of attributes and methods that end-users can access or call to get data.

property metadata: Dict[str, Any]

metadata for a given session

Returns:

Dict

A dictionary containing behavior session specific metadata dictionary keys:

age_in_days: (int)

age of mouse in days

behavior_session_uuid: (int)

unique identifier for a behavior session

behavior_session_id: (int)

unique identifier for a behavior session

cre_line: (string)

cre driver line for a transgenic mouse

date_of_acquisition: (date time object)

date and time of experiment acquisition, yyyy-mm-dd hh:mm:ss

driver_line: (list of string)

all driver lines for a transgenic mouse

equipment_name: (string)

identifier for equipment data was collected on

full_genotype: (string)

full genotype of transgenic mouse

mouse_id: (int)

unique identifier for a mouse

project_code: (string)

String of project session is associated with.

reporter_line: (string)

reporter line for a transgenic mouse

session_type: (string)

visual stimulus type displayed during behavior session

sex: (string)

sex of the mouse

stimulus_frame_rate: (float)

frame rate (Hz) at which the visual stimulus is displayed

property raw_running_speed: DataFrame

Get unfiltered running speed data. Sampled at 60Hz.

NOTE: For BehaviorSessions, returned timestamps are not aligned to external ‘synchronization’ reference timestamps. Synchronized timestamps are only available for BehaviorOphysExperiments.

Returns:

pd.DataFrame

Dataframe containing unfiltered running speed and timestamps dataframe columns:

timestamps: (float)

time in seconds

speed: (float)

speed in cm/sec

property rewards: DataFrame

Retrieves rewards from data file saved at the end of the behavior session.

NOTE: For BehaviorSessions, returned timestamps are not aligned to external ‘synchronization’ reference timestamps. Synchronized timestamps are only available for BehaviorOphysExperiments.

Returns:

pd.DataFrame

A dataframe containing timestamps of delivered rewards. Timestamps are sampled at 60Hz.

dataframe columns:

volume: (float)

volume of individual water reward in ml. 0.007 if earned reward, 0.005 if auto reward.

timestamps: (float)

time in seconds

auto_rewarded: (bool)

True if free reward was delivered for that trial. Occurs during the first 5 trials of a session and throughout as needed

property running_speed: DataFrame

Running speed and timestamps, sampled at 60Hz. By default applies a 10Hz low pass filter to the data. To get the running speed without the filter, use raw_running_speed.

NOTE: For BehaviorSessions, returned timestamps are not aligned to external ‘synchronization’ reference timestamps. Synchronized timestamps are only available for BehaviorOphysExperiments.

Returns:

pd.DataFrame

Dataframe containing running speed and timestamps dataframe columns:

timestamps: (float)

time in seconds

speed: (float)

speed in cm/sec

property stimulus_presentations: DataFrame

Table whose rows are stimulus presentations (e.g. a given image, for a given duration, typically 250 ms) and whose columns are presentation characteristics.

For VBO, the stimulus presentations tables now contain data from all stimuli not just the active/passive image stimulus. These new stimuli are delineated by stimulus_block and stimulus_block_name. The previously released image behavior stimulus is accessible as the block with a name containing “change_detection”. Use the following example code snippet to retrieve the original stimulus block from the pandas table:

`stimulus_presentations[

stimulus_presentations.stimulus_block_name.str.contains(

‘change_detection’

)

]`

Returns:

pd.DataFrame

Table whose rows are stimulus presentations (e.g. a given image, for a given duration, typically 250 ms) and whose columns are presentation characteristics.

dataframe columns:

stimulus_presentations_id [index]: (int)

identifier for a stimulus presentation (presentation of an image)

stimulus_block: (int)

Index of the different stimuli presented to the mouse in order.

stimulus_block_name [VBO only]: (int)

Name of the individual stimulus blocks describing briefly what was shown to the mouse.

image_index [VBO only]: (int)

image index (0-7) for a given session, corresponding to each image name

image_name: (string)

Name of the image presented to the mouse if applicable. This indexes in to the stimulus_templates object.

movie_frame_index [VBO only]: (int)

Index into the natural movie presented. use with the method get_raw_natural_movie or if you need the movie frames stretched and warped to the monitor use get_natural_movie_template. (Warning this is a very compute intensive process).

duration: (float)

duration of an image presentation (flash) in seconds (stop_time - start_time). NaN if omitted

start_time: (float)

image presentation start time in seconds

end_time: (float)

image presentation end time in seconds

start_frame: (int)

image presentation start frame

end_frame: (int)

image presentation end frame

is_change: (bool)

True if the image stimulus has changed. <NA> if not an image stimulus block.

is_image_novel: (bool)

True if the image stimulus is novel. <NA> if not an image stimulus block.

omitted: (bool)

True if no image was shown for this stimulus presentation. <NA> if not an image stimulus block.

movie_repeat [VBO only]: (int)

Number of times the movie has been repeated during the natural movie block. -99 if not a movie block.

flashes_since_change: (bool)

Number of times the image has been flashed in between when it has changed.

trials_id: (int)

Id to match to the table Index of the trials table.

active: (bool)

Stimulus block overlaps with the trials.

is_sham_change: (bool)

True if the stimulus overlaps with a catch trial.

stimulus_index [VBN only]: (int)

0 for gabor_20_deg_250ms, 1 for flash_250ms. -99 otherwise

position_x [VBN only]: (float)

X position of gabor grating when shown. NaN if not applicable.

position_y [VBN only]: (float)

Y position of gabor grating when shown. NaN if not applicable.

color [VBN only]: (float)

Value of the color int flash_250ms stimulus. Either -1 or 1. NaN if not applicable.

rewarded [VBN only]: (bool)

Merge of trials data. True if the mouse was rewarded during the stimulus. <NA> if not applicable to stimulus block.

orientation [VBN only]: (float)

Orientation of the gabor grating in degrees.

temporal_frequency [VBN only]: (float)

Temporal frequency of the gabor grating. NaN if not applicable.

spatial_frequency [VBN only]: (float)

Spatial frequency of the gabor grating. NaN if not applicable.

contrast [VBN only]: float

Contrast value of the gabor grating. NaN if not applicable.

property stimulus_templates: DataFrame | None

Get stimulus templates (scenes) for behavior session.

Returns:

pd.DataFrame

A pandas DataFrame object containing the stimulus images for the experiment.

dataframe columns:

image_name [index]: (string)

name of image presented, if ‘omitted’ then no image was presented

unwarped: (array of int)

image array of unwarped stimulus image

warped: (array of int)

image array of warped stimulus image

property stimulus_timestamps: ndarray

Timestamps associated with the stimulus presetntation on the monitor retrieveddata file saved at the end of the behavior session. Sampled at 60Hz.

NOTE: For BehaviorSessions, returned timestamps are not aligned to external ‘synchronization’ reference timestamps. Synchronized timestamps are only available for BehaviorOphysExperiments.

Returns:

np.ndarray

Timestamps associated with stimulus presentations on the monitor

property task_parameters: dict

Get task parameters from data file saved at the end of the behavior session file.

Returns:

dict

A dictionary containing parameters used to define the task runtime behavior.

auto_reward_volume: (float)

Volume of auto rewards in ml.

blank_duration_sec(list of floats)

Duration in seconds of inter stimulus interval. Inter-stimulus interval chosen as a uniform random value. between the range defined by the two values. Values are ignored if stimulus_duration_sec is null.

response_window_sec: (list of floats)

Range of period following an image change, in seconds, where mouse response influences trial outcome. First value represents response window start. Second value represents response window end. Values represent time before display lag is accounted for and applied.

n_stimulus_frames: (int)

Total number of visual stimulus frames presented during a behavior session.

task: (string)

Type of visual stimulus task.

session_type: (string)

Visual stimulus type run during behavior session.

omitted_flash_fraction: (float)

Probability that a stimulus image presentations is omitted. Change stimuli, and the stimulus immediately preceding the change, are never omitted.

stimulus_distribution: (string)

Distribution for drawing change times. Either ‘exponential’ or ‘geometric’.

stimulus_duration_sec: (float)

Duration in seconds of each stimulus image presentation

reward_volume: (float)

Volume of earned water reward in ml.

stimulus: (string)

Stimulus type (‘gratings’ or ‘images’).

to_nwb(add_metadata=True, include_experiment_description=True, stimulus_presentations_stimulus_column_name: str = 'stimulus_name') → NWBFile

Parameters:

add_metadata

Set this to False to prevent adding metadata to the nwb instance.

include_experiment_description: Whether to include a description of the

experiment in the nwbfile

stimulus_presentations_stimulus_column_name: Name of the column

denoting the stimulus name in the presentations table

property trials: DataFrame

Get trials from data file saved at the end of the behavior session.

Returns:

pd.DataFrame

A dataframe containing trial and behavioral response data.

dataframe columns:

trials_id [index]: (int)

trial identifier

lick_times: (array of float)

array of lick times in seconds during that trial. Empty array if no licks occured during the trial.

reward_time: (NaN or float)

Time the reward is delivered following a correct response or on auto rewarded trials.

reward_volume: (float)

volume of reward in ml. 0.005 for auto reward 0.007 for earned reward

hit: (bool)

Behavior response type. On catch trial mouse licks within reward window.

false_alarm: (bool)

Behavior response type. On catch trial mouse licks within reward window.

miss: (bool)

Behavior response type. On a go trial, mouse either does not lick at all, or licks after reward window

is_change: (bool)

True if an image change occurs during the trial (if the trial was both a ‘go’ trial and the trial was not aborted)

aborted: (bool)

Behavior response type. True if the mouse licks before the scheduled change time.

go: (bool)

Trial type. True if there was a change in stimulus image identity on this trial

catch: (bool)

Trial type. True if there was not a change in stimulus identity on this trial

auto_rewarded: (bool)

True if free reward was delivered for that trial. Occurs during the first 5 trials of a session and throughout as needed.

correct_reject: (bool)

Behavior response type. On a catch trial, mouse either does not lick at all or licks after reward window

start_time: (float)

start time of the trial in seconds

stop_time: (float)

end time of the trial in seconds

trial_length: (float)

duration of trial in seconds (stop_time -start_time)

response_time: (float)

time of first lick in trial in seconds and NaN if trial aborted

initial_image_name: (string)

name of image presented at start of trial

change_image_name: (string)

name of image that is changed to at the change time, on go trials

change_time: (float)

Time in the session at which the change occurred.

response_latency [VBO only]: (float)

Delay between change and first lick in seconds. NaN if no change occurred.

change_frame: (int)

Frame in the session at which the change occurred. -99 if no change.

change_time_no_display_delay [VBN only]: (float)

Time of the change in seconds, before the display lag is accounted for and applied.