GETTING STARTED

Prerequisites

• install or update the AllenSDK,

  our Python based toolkit for accessing and working with Allen Institute datasets.

• Pandas familiarity

Data is provided in in NWB format and can be downloaded using the AllenSDK, or accessed directly via an S3 bucket (instructions provided in notebook #1 below). Regardless of which method of file download you choose, we recommend that you load and interact with the data using the tools provided in the AllenSDK, which have been designed to simplify data access and subsequent analysis. No knowledge of the NWB file format is required.

Specific information about how Visual Behavior Optical Physiology data is stored in NWB files and how AllenSDK accesses NWB files can be found here.

TUTORIALS

To get started, check out these jupyter notebooks to learn how to:

1. Download data using the AllenSDK or directly from our Amazon S3 bucket (download .ipynb) (Open in Colab)

2. Identify experiments of interest using the dataset manifest (download .ipynb) (Open in Colab)

3. Load and visualize data from a 2-photon imaging experiment (download .ipynb) (Open in Colab)

4. Examine the full training history of one mouse (download .ipynb) (Open in Colab)

5. Compare behavior and neural activity across different trial types in the task (download .ipynb) (Open in Colab)

DOCUMENTATION

To learn more about the experimental design and available data, read through the Visual Behavior and Visual Behavior Ophys chapters in the SWDB Databook.

For a description of available AllenSDK methods and attributes for data access, see this further documentation.

For detailed information about the experimental design, data acquisition, and informatics methods, please refer to our technical whitepaper.

If you have questions about the dataset that aren’t addressed by the whitepaper or any of our tutorials, please reach out by posting at https://community.brain-map.org/

VISUAL BEHAVIOR OPTICAL PHYSIOLOGY DATASETS

The Visual Behavior 2P project used in vivo 2-photon calcium imaging (also called optical physiology, or “ophys”) to measure the activity of genetically identified neurons in the visual cortex of mice performing a go/no-go visual change detection task. This dataset can be used to evaluate the influence of experience, expectation, and task engagement on neural coding and dynamics in excitatory and inhibitory cell populations. A description of the experimental design and available data is provided below.

We used single- and multi-plane imaging approaches to record the activity of populations of neurons across multiple cortical depths and visual areas during change detection behavior. Each population of neurons was imaged repeatedly over multiple days under different sensory and behavioral contexts, including familiar and novel stimuli, as well as active behavior and passive viewing conditions.

Different imaging configurations and stimulus sets were used in different groups of mice, resulting in four unique datasets (indicated by their project_code in SDK metadata tables). Two single-plane 2-photon datasets were acquired in the primary visual cortex (VISp). In the VisualBehavior dataset, mice were trained with image set A and tested with image set B which was novel to the mice. In the VisualBehaviorTask1B dataset, mice were trained with image set B and tested with image set A as the novel image set. One multi-plane dataset (VisualBehahviorMultiscope) was acquired at 4 cortical depths in 2 visual areas (VISp & VISl) using image set A for training and image set B for novelty. Another multi-plane dataset (VisualBehaviorMultiscope4areasx2d) was acquired at 2 cortical depths in 4 visual areas (VISp, VISl, VISal, VISam). In this dataset, two of the images that became highly familiar during training with image set G were interleaved among novel images in image set H.

For each dataset, we imaged the activity of GCaMP6 expressing cells in populations of excitatory (Slc17a7-IRES2-Cre;Camk2a-tTA;Ai93(TITL-GCaMP6f) or Ai94(TITL-GCaMP6s)), Vip inhibitory (Vip-IRES-Cre;Ai148(TIT2L-GCaMP6f-ICL-tTA2)), and Sst inhibitory (Sst-IRES-Cre;Ai148(TIT2L-GCaMP6f-ICL-tTA2)) neurons. Imaging took place between 75-400um below the cortical surface.

The full dataset includes neural and behavioral measurements from 107 mice during 703 in vivo 2-photon imaging sessions from 326 unique fields of view, resulting in longitudinal recordings from 50,476 cortical neurons. The table below describes the numbers of mice, sessions, and unique recorded neurons for each transgenic line and experimental configuration:

BEHAVIORAL TRAINING

Prior to 2-photon imaging, mice were trained to perform a go/no-go visual change detection task in which they learned to lick a spout in response to changes in stimulus identity to earn a water reward. The full behavioral training history of all imaged mice is provided as part of the dataset, allowing investigation into task learning, behavioral strategy, and inter-animal variability. There are a total of 4,787 behavior sessions available for analysis.

We used a standardized procedure to progress mice through a series of training stages, with transitions between stages determined by specific advancement criteria. First, mice learned to detect changes in the orientation of full field static grating stimuli. Next, a 500ms inter stimulus interval period with mean luminance gray screen was added between the 250ms stimulus presentations, incorporating a short-term memory component to the task. Once mice successfully and consistently performed orientation change detection with flashed gratings, they moved to the image change detection version of the task. During image change detection, 8 natural scene images were presented during each behavioral session, for a total of 64 possible image transitions. When behavioral performance again reached criterion (d-prime >1 for 2 out of 3 consecutive days), mice were transitioned to the 2-photon imaging stage in which they performed the task under a microscope to allow simultaneous measurement of neural activity and behavior.

Behavioral training data for mice progressing through these stages of task learning is accessible via the BehaviorSession class of the AllenSDK or the get_behavior_session() method of the VisualBehaviorOphysProjectCache. Each BehaviorSession contains the following data streams, event times, and metadata:

• Running speed

• Lick times

• Reward times

• Stimulus presentations

• Behavioral trial information

• Mouse metadata (age, sex, genotype, etc)

2-PHOTON IMAGING DURING BEHAVIOR

Once mice are well-trained on the image change detection task, they transition to performing the behavior under a 2-photon microscope. Each 2-photon field of view is imaged across multiple session types, allowing measurement of neural activity across different sensory and behavioral contexts.

Mice initially perform the task under the microscope with the same set of images they observed during training, which have become highly familiar (each image is viewed thousands of times during training). Mice also undergo several sessions with a novel image set that they had not seen prior to the 2-photon imaging portion of the experiment. Passive viewing sessions are interleaved between active behavior sessions. On passive days, mice are given their daily water before the session (and are thus satiated) and view the stimulus in open loop mode, with the lick spout retracted to indicate that rewards are not available. This allows investigation of the impact of motivation and attention on patterns of neural activity.

During imaging sessions (but not during training), stimulus presentations are randomly omitted with a 5% probability, resulting in an extended gray screen period between two presentations of the same stimulus and disrupting the expected cadence of stimulus presentations. The change and pre-change stimulus presentations are never omitted. Running speed, pupil diameter, licking, and reward delivery are measured and aligned to neural activity traces.

The BehaviorOphysExperiment class in the AllenSDK (or the get_behavior_ophys_experiment() method of the VisualBehaviorOphysProjectCache) provides all data for a single imaging plane, recorded in a single session, and contains the following data and metadata:

• Maximum intensity image

• Average intensity image

• Segmentation masks and ROI metadata

• dF/F traces (baseline corrected, normalized fluorescence traces)

• Corrected fluorescence traces (neuropil subtracted and demixed, but not normalized)

• Events (detected with an L0 event detection algorithm)

• Pupil position, diameter, and area

• Running speed (in cm/second)

• Lick times

• Reward times

• Stimulus presentation times

• Behavioral trial information

• Mouse metadata (age, sex, genotype, etc)

The data collected in a single continuous recording is defined as a session and receives a unique ophys_session_id. Each imaging plane in a given session is referred to as an experiment and receives a unique ophys_experiment_id. For single-plane imaging, there is only one imaging plane (i.e. one experiment) per session. For multi-plane imaging, there can be up to 8 imaging planes (i.e. 8 experiments) per session. Due to our strict QC process, described below, not all multi-plane imaging sessions have exactly 8 experiments, as some imaging planes may not meet our data quality criteria.

We aimed to track the activity of single neurons across the session types described above by targeting the same population of neurons over multiple recording sessions, with only one session recorded per day for a given mouse. The collection of imaging sessions for a given population of cells, belonging to a single imaging plane measured across days, is called a container and receives a unique ophys_container_id. A container can include between 3 and 11 separate sessions for that imaging plane. Mice imaged with the multi-plane 2-photon microscope can have multiple containers, one for each imaging plane recorded across multiple sessions. The session types available for a given container can vary, due to our selection criteria to ensure data quality (described below).

Thus, each mouse can have one or more containers, each representing a unique imaging plane (experiment) that has been targeted on multiple recording days (sessions), under different behavioral and sensory conditions (session types).

SESSION STRUCTURE

During behavioral training, sessions consist of 60 minutes of change detection behavior (other than the TRAINING_0 sessions, which are 15 minutes of associative reward pairing).

During ophys sessions (session_type starting with OPHYS), there is a 5 minute period where no stimulus is shown before the change detection task begins, as well as 5 minutes of gray screen after the task ends. This allows evaluation of spontaneous activity in the absence of stimulus or task. After the second 5 minute gray screen period, a 30 second natural movie clip is shown 10 times. This movie clip is the same as the Visual Coding 2P stimulus called natural_movie_one. This allows evaluation of stimulus driven activity that is independent of the task.

Ophys session structure:

DATA PROCESSING

Each 2-photon movie is processed through a series of steps to obtain single cell traces of baseline-corrected fluorescence (dF/F) and detected events, and packaged into the NWB file format along with stimulus and behavioral information, as well as other metadata.

Detailed descriptions of data processing steps can be found in the technical white paper, as well as our data processing repository.

QUALITY CONTROL

Every 2-photon imaging session was carefully evaluated for a variety of quality control criteria to ensure that the final dataset is of the highest quality possible. Sessions or imaging planes that do not meet our criteria are excluded from the released dataset. These are a few of the key aspects of the data that are evaluated:

• intensity drift

• image saturation or bleaching

• z-drift over the course of a session

• accuracy of session-to-session field of view matching

• excessive or uncorrectable motion in the image

• uncorrectable crosstalk between simultaneously recorded multiscope planes

• errors affecting temporal alignment of data streams

• hardware or software failures

• brain health

• animal stress

SUMMARY OF AVAILABLE DATA

Behavior	Physiology	Metadata
Running speed	Max intensity projection image	Mouse genotype, age, sex
Licks	Average projection image	Date of acquisition
Rewards	Segmentation mask image	Imaging parameters
Pupil area	Cell specimen table	Task parameters
Pupil position	Cell ROI masks	Session type
Stimulus presentations table	Corrected fluorescence traces	Stimulus images
Trials table	dF/F activity traces	Performance metrics
Stimulus timestamps	Detected events
	Ophys timestamps

DATA FILE CHANGELOG

v1.1.0

Removed data

• Removed behavior session with incorrect image presentations: 931566300.

  • This results in the removal several other subordinate data:

    • ophys session removed: 931566300

    • ophys experiments removed: 932372699, 932372701, 932372705, 932372707, 932372711

• Removed ophys session 875259383 from the ophys metadata table. The behavior component of this session is available as behavior session id=875471358. No ophys data for this session was previously released.

• Removed truncated behavior sessions with ids: 934610593, 958310218, 975358131, 1011688792

• The above identifiers have also been removed from their respective metadata tables. Columns such as prior_exposures however, are correctly calculated with knowledge of these sessions.

• The cell ROIs associated with the above experiments have also been removed from the cell ROI metadata table.

• Current data counts

  • 107 mice (same as v1.0.0)

  • 4079 behavior training session (down from 4082)

  • 703 in vivo 2-photon imaging sessions (down from 704 sessions, previous releases erroneously included session 875259383 in their metadata tables and claimed 705 sessions.)

  • 50,476 logitudinal recordings (down from 50,489)

Metadata Changes

• Additions to multiple tables

  • Added project_code and behavior_type (active/passive) value to all tables.

  • Added imaging_plane_group_count, num_depths_per_area, num_targeted_structures, experience_level to Behavior and Ophys session tables.

• Behavior session table

  • Added trials summary columns: catch_trial_count, correct_reject_trial_count, engaged_trial_count, false_alarm_trial_count, miss_trial_count, trial_count.

  • Added image_set column.

• Ophys experiment table

  • Added targeted_imaging_depth to experiment table, representing the average depth of all experiments in the published container.

• Better consistency of integer typing throughout.

NWB Data Changes

• The value for Age in the metadata, Session/Experiment objects now consistent. NWBs now reflect the age of the animal at the time the session/experiment was taken.

• Enforced better and more consistent typing between the metadata tables and the session metadata.

• All date/times in NWBs and metadata tables are now explicitly UTC timezone.

• Stimulus presentations tables now contain information for additional stimulus conditions, including 10 repeats of a 30 second movie clip at the end of session_types starting with OPHYS, and 5 minute gray screen period before and after change detection behavior in session_types starting with OPHYS. These new stimuli are delineated by stimulus_block and stimulus_block_name. The previously released image behavior stimulus is accessible as the block with a name containing “change_detection”. Use the following example code snippet to retrieve the original stimulus block from the pandas table:

  stimulus_presentations[stimulus_presentations.stimulus_block_name.str.contains(‘change_detection’)]

  See “SESSION STRUCTURE” section above for more details.

• New columns in the stimulus_presentations table:

  • is_image_novel, is_sham_change, movie_frame_index, movie_repeat, stimulus_block, stimulus_block_name, stimulus_name, active

  • See in code documentation for stimulus_presentations table for definitions of these new columns.

• New trials columns:

  • change_time, change_frame, response_latency

  • See in code documentation for trials table for definitions of these new columns.

• Include additional traces in the ophys experiment object:

  • corrected_fluorescence_traces, demixed traces, neuropil traces, df/f traces

  • corrected_fluorescence_traces are now the correct trace.

Supplemental cache data

• New accessors in VisualBehaviorOphysProjectCache to download and cache the natural movie presented to the mice. Additional accessor to convert the movie to the format as shown on the screen during the session. (Warning this conversion is compute intensive). Methods are:

  • get_raw_natural_movie: Downloads the raw movie if needed and returns as an ndarray.

  • get_natural_movie_template: Converts the raw movie to the format as shown on the screen during the session. Return a pandas dataframe in a similar formate to the image templates. Warning this conversion is compute intensive.

v1.0.1

Metadata corrections - ophys_container_id columns contained extra IDs of incorrect containers.

v1.0.0

New Data

• 107 mice, up from 82

• 4082 behavior training sessions, up from 3021.

• 705 in vivo 2-photon imaging sessions, up from 551.

• 50,489 logitudinal recordings from cortical cells, up from 34,619

Metadata changes

• A new metadata table is present: ophys_cells_table. This table has a project-wide aggregate of cell_specimen_id, cell_roi_id, and ophys_experiment_id.

• Added ‘experience_level’, ‘passive’ and ‘image_set’ columns to ophys_experiments_table

Data Corrections

• 196 BehaviorOphysExperiments had excess invalid ROIs in the dataframe returned by the events field. These have been corrected to remove these invalid ROIs.

v0.3.0

13 sessions were labeled with the wrong session_type in v0.2.0. We have corrected that error. The offending sessions were

behavior_session_id	ophys_session_id	session_type_v0.2.0	session_type_v0.3.0
875020233		OPHYS_3_images_A	OPHYS_2_images_A_passive
902810506		TRAINING_4_images_B_training	TRAINING_3_images_B_10uL_reward
914219174		OPHYS_0_images_B_habituation	TRAINING_5_images_B_handoff_ready
863571063		TRAINING_5_images_A_handoff_ready	TRAINING_1_gratings
974330793		OPHYS_0_images_B_habituation	TRAINING_5_images_B_handoff_ready
863571072		OPHYS_5_images_B_passive	TRAINING_4_images_A_training
1010972317		OPHYS_4_images_A	OPHYS_3_images_B
1011659817		OPHYS_5_images_A_passive	OPHYS_4_images_A
1003302686	1003277121	OPHYS_6_images_A	OPHYS_5_images_A_passive
863571054		OPHYS_7_receptive_field_mapping	TRAINING_5_images_A_epilogue
974282914	974167263	OPHYS_6_images_B	OPHYS_5_images_B_passive
885418521		OPHYS_1_images_A	TRAINING_5_images_A_handoff_lapsed
915739774		OPHYS_1_images_A	OPHYS_0_images_A_habituation