Organization of a Visual Behavior NWB file

The Allen Institute’s Brain Observatory data sets include data from 2-photon microscope recordings including image segmentation, motion correction and fluorescence traces. These data are separated into two different types of files - files with just data about specimen behavior and files which include this _and_ optical physiology data. It also includes information about the visual stimuli presented during the recording. This information is packaged in the NWB format. Here is a brief description how the data is stored in the NWB file.

Browsing the file

The NWB format is a self-documenting file format that is designed to store a wide range of neurophysiology data. Structurally, NWB is nothing more than an HDF5 file that has data organized in a particular way. For example, the format has a dedicated location for storing all data that is acquired during an experiment, and another place to store stimulus data that was presented. Data is usually stored in a time series, and each time series has information to document the type of data that it stores. Importantly, all HDF5 tools and libraries are able to read the NWB file.

When using NWB files, the most important tool to become familiar with is HDFView. HDFView is an open-source tool that is published by the HDF Group. It allows you to browse the contents of an HDF5 (and NWB) file in an intuitive way, and also to view some of the data stored within it. HDFView can be downloaded at www.hdfgroup.org. It is highly recommended that anyone using an NWB file download and install this tool.

NWB file organization

Each NWB file is organized into seven sections. These sections store acquired data (‘acquisition’), stimulus data (‘stimulus’), general metadata about the experiment (‘general’), experiment organization (‘intervals’), processed data (‘processing’), metadata about data types found in the NWB (‘specifications’), and a free-form area for storing analysis data (‘analysis’).

_static/behavior_ophys_nwb_fig-1.png

Figure 1. An NWB file, viewed in HDFView

At the left of HDFView shows the organization of data in an NWB file. It is most useful to imagine an HDF5 file as a container that stores many files and folders within it. Because of this structure, you can browse an HDF5 file, and an NWB file, in the same way that you browse the files and folders on your computer. Folders can be opened by double-clicking on the folder icon. The contents of a ‘file’ within HDF5, which called a dataset, can be seen by double-clicking it. An NWB file has seven top-level folders and five top-level datasets. In Figure 1, the contents of the ‘session_description’ dataset is displayed. An NWB file is designed to store data from a single experimental session on a single animal. The session_description dataset provides a quick summary of the information available in the file. The NWB specification, describing the format and how data is stored, can be downloaded here.

Stimulus data

Within an NWB file, stimulus data is organized into ‘templates’, which are stimulus descriptions that can be used one or more times, and ‘presentation’, which stores the time-series data about what was presented and when.

_static/behavior_ophys_nwb_fig-2.png

Figure 2. Stimuli presented during the experiment

In this experiment shown in Figure 2, there was only one stimulus presented - a set of natural images. When browsing the file, note the panel to the right. Clicking on a folder or dataset will cause information about that object to be displayed there. This will display a table of information related to that dataset, including comments, descriptions, and other relevant metadata. Double clicking one of these will further reveal the value stored inside.

Processing

In the NWB file, the processing folder is designed to store the contents of the different levels of intermediate processing of data that are necessary to convert raw data into something that can be used for scientific analysis, as well as information about the behavior of the specimen during data collection. In the Allen Institute’s NWB file, these levels of processing include motion correction of the acquired 2-photon movie frames, image segmentation into regions of interest, the fluorescence signal for each of these regions, and the dF/F signal that is useful for correlating cell activity with presented stimuli.

This is also where you will find the main difference between the two types on NWB files produced for the Allen Institute Brain Observatory dataset. In the processing directory, you will notice behavior data information like licking, running, rewards, etc. If this is an NWB strictly containing behavior data, then that will be all. If the NWB file is one that includes optical physiology data as well then there wil also be a directory in processing called ophys, where you will find the intermediate processing data.

_static/behavior_ophys_nwb_fig-3.png

Figure 3. Location of dF/F data

HDFView provides a very efficient way to browse the contents of an NWB file.

Below is a brief description and location for several types of data in the NWB file.

Category Data Description NWB Location SDK Method(s), Property, or Properties
Acquired/Processed Data Frame by frame ellipse fits of subject corneal reflection /acquisition/EyeTracking/corneal_reflection_tracking BehaviorOphysExperiment().eye_tracking
Frame by frame ellipse fits of subject eye /acquisition/EyeTracking/eye_tracking
Frame by frame ellipse fits of subject pupil /acquisition/EyeTracking/pupil_tracking
Video frames containing likely blinks /acquisition/EyeTracking/likely_blink
Running wheel reference voltage /acquisition/v_in  
Running wheel reference signal /acquisition/v_sig  
Metadata (Subject) Subject age /general/subject/age
BehaviorOphysExperiment().metadata
BehaviorSession().metadata
Subject description /general/subject/description
Subject genotype /general/subject/genotype
Subject sex /general/subject/sex
Subject species /general/subject/species
Subject identifier /general/subject/subject_id
Metadata (General) Date of NWB file creation /file_create_date
Equipment identifier /general/devices
A description of the behavior task that the subject is performing /general/experiment_description
Institution /general/institution
Dataset keywords /general/keywords

Additional metadata

May include (depending on if NWB file also contains optical physiology data):

  • field_of_view_width
  • field_of_width_height
  • imaging_depth
  • stimulus_frame_rate
/general/metadata
Description of session ‘stage’ /session_description
Date of session acquisition /session_start_time
Behavior Session or Behavior Ophys Experiment identifier /identifier
Metadata (Optical Physiology) Optical physiology imaging parameters /general/optophysiology
Behavior Data Parameters pertaining to the behavior task /general/task_parameters
BehaviorOphysExperiment().task_parameters
BehaviorSession().task_parameters
Stimuli presentation information and timings for behavior task
/intervals
/stimulus/presentation
BehaviorOphysExperiment().stimulus_presentations
BehaviorSession().stimulus_presentations
Stimulus templates /stimulus/templates
BehaviorOphysExperiment().stimulus_templates
BehaviorSession().stimulus_templates
Synchronized timestamps of stimuli /processing/stimulus/timestamps
BehaviorOphysExperiment().stimulus_timestamps
BehaviorSession().stimulus_timestamps
Trial data for behavior task /intervals/trials
BehaviorOphysExperiment().trials
BehaviorSession().trials
Data about subject licking during behavior task /processing/licking/licks
BehaviorOphysExperiment().licks
BehaviorSession().licks
Information about water rewards during behavior task /processing/rewards
BehaviorOphysExperiment().rewards
BehaviorSession().rewards

Running data during behavior task containing:

  • Change in distance
  • Speed
  • Unfiltered speed
/processing/running/dx
/processing/running/speed
/processing/running/speed_unfiltered
BehaviorOphysExperiment().running_speed
BehaviorSession().running_speed
BehaviorOphysExperiment().raw_running_speed
BehaviorSession().raw_running_speed
Optical physiology Data Corrected fluorescence traces /processing/ophys/corrected_fluorescence BehaviorOphysExperiment().corrected_fluorescence_traces
dF/F values /processing/ophys/dff BehaviorOphysExperiment().dff_traces
Detected events /processing/ophys/event_detection BehaviorOphysExperiment().events
Table of segmented regions of interest (ROI) /processing/ophys/image_segmentation/cell_specimen_table BehaviorOphysExperiment().cell_specimen_table
Average projection image of optical physiology imaging plane over time /processing/ophys/images/average_image BehaviorOphysExperiment().average_projection
Maximum projection image of optical physiology imaging plane over time /processing/ophys/images/max_projection BehaviorOphysExperiment().max_projection
Segmented ROI image /processing/ophys/images/segmentation_mask_image BehaviorOphysExperiment().segmentation_mask_image
Imaging motion correction in x dimension /processing/ophys/ophys_motion_correction_x BehaviorOphysExperiment().motion_correction
Imaging motion correction in y dimension /processing/ophys/ophys_motion_correction_y  
NWB Information NWB schema and extension information /specifications