Import libraries¶
This notebook shows how to load ophys data from Visual Behavior Project using AllenSDK tools. It briefly describes the type of data available and shows a few simple ways of plotting ophys traces along with animal's behavior.
We will first install allensdk into your environment by running the appropriate commands below.
Install AllenSDK into your local environment¶
You can install AllenSDK locally with:
!pip install allensdk
Requirement already satisfied: allensdk in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (2.16.2) Requirement already satisfied: psycopg2-binary in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.9.9) Requirement already satisfied: hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.* in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.11.0) Requirement already satisfied: h5py in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.10.0) Requirement already satisfied: matplotlib in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.7.4) Requirement already satisfied: numpy<1.24 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.23.5) Requirement already satisfied: pandas==1.5.3 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.5.3) Requirement already satisfied: jinja2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.1.2) Requirement already satisfied: scipy<1.11 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.10.1) Requirement already satisfied: six in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.16.0) Requirement already satisfied: pynrrd in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.0.0) Requirement already satisfied: future in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.18.3) Requirement already satisfied: requests in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.31.0) Requirement already satisfied: requests-toolbelt in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.0.0) Requirement already satisfied: simplejson in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.19.2) Requirement already satisfied: scikit-image in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.21.0) Requirement already satisfied: scikit-build in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.17.6) Requirement already satisfied: statsmodels in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.14.0) Requirement already satisfied: simpleitk in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.3.1) Requirement already satisfied: argschema in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.0.4) Requirement already satisfied: glymur in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.12.9.post1) Requirement already satisfied: xarray<2023.2.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2023.1.0) Requirement already satisfied: pynwb in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.5.0) Requirement already satisfied: tables in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.8.0) Requirement already satisfied: seaborn in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.13.0) Requirement already satisfied: aiohttp in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.9.1) Requirement already satisfied: nest-asyncio in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.5.8) Requirement already satisfied: tqdm in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (4.66.1) Requirement already satisfied: ndx-events in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.2.0) Requirement already satisfied: boto3 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.33.4) Requirement already satisfied: semver in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.0.2) Requirement already satisfied: cachetools in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (5.3.2) Requirement already satisfied: sqlalchemy in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.0.23) Requirement already satisfied: python-dateutil in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.8.2) Requirement already satisfied: pytz>=2020.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pandas==1.5.3->allensdk) (2023.3.post1) Requirement already satisfied: importlib-resources in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (6.1.1) Requirement already satisfied: jsonschema>=2.6.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (4.20.0) Requirement already satisfied: ruamel-yaml>=0.16 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.18.5) Requirement already satisfied: packaging>=21.3 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from xarray<2023.2.0->allensdk) (23.2) Requirement already satisfied: attrs>=17.3.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (23.1.0) Requirement already satisfied: multidict<7.0,>=4.5 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (6.0.4) Requirement already satisfied: yarl<2.0,>=1.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (1.9.3) Requirement already satisfied: frozenlist>=1.1.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (1.4.0) Requirement already satisfied: aiosignal>=1.1.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (1.3.1) Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (4.0.3) Requirement already satisfied: marshmallow<4.0,>=3.0.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from argschema->allensdk) (3.20.1) Requirement already satisfied: pyyaml in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from argschema->allensdk) (6.0.1) Requirement already satisfied: botocore<1.34.0,>=1.33.4 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from boto3->allensdk) (1.33.4) Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from boto3->allensdk) (1.0.1) Requirement already satisfied: s3transfer<0.9.0,>=0.8.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from boto3->allensdk) (0.8.2) Requirement already satisfied: lxml in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from glymur->allensdk) (4.9.3) Requirement already satisfied: MarkupSafe>=2.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jinja2->allensdk) (2.1.3) Requirement already satisfied: contourpy>=1.0.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (1.1.1) Requirement already satisfied: cycler>=0.10 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (0.12.1) Requirement already satisfied: fonttools>=4.22.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (4.45.1) Requirement already satisfied: kiwisolver>=1.0.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (1.4.5) Requirement already satisfied: pillow>=6.2.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (10.1.0) Requirement already satisfied: pyparsing>=2.3.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (3.1.1) Requirement already satisfied: setuptools in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pynwb->allensdk) (56.0.0) Requirement already satisfied: nptyping in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pynrrd->allensdk) (2.5.0) Requirement already satisfied: typing-extensions in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pynrrd->allensdk) (4.8.0) Requirement already satisfied: charset-normalizer<4,>=2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (3.6) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (1.26.18) Requirement already satisfied: certifi>=2017.4.17 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (2023.11.17) Requirement already satisfied: distro in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-build->allensdk) (1.8.0) Requirement already satisfied: tomli in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-build->allensdk) (2.0.1) Requirement already satisfied: wheel>=0.32.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-build->allensdk) (0.42.0) Requirement already satisfied: networkx>=2.8 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (3.1) Requirement already satisfied: imageio>=2.27 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (2.33.0) Requirement already satisfied: tifffile>=2022.8.12 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (2023.7.10) Requirement already satisfied: PyWavelets>=1.1.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (1.4.1) Requirement already satisfied: lazy_loader>=0.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (0.3) Requirement already satisfied: greenlet!=0.4.17 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from sqlalchemy->allensdk) (3.0.1) Requirement already satisfied: patsy>=0.5.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from statsmodels->allensdk) (0.5.3) Requirement already satisfied: cython>=0.29.21 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (3.0.6) Requirement already satisfied: numexpr>=2.6.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (2.8.6) Requirement already satisfied: blosc2~=2.0.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (2.0.0) Requirement already satisfied: py-cpuinfo in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (9.0.0) Requirement already satisfied: msgpack in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from blosc2~=2.0.0->tables->allensdk) (1.0.7) Requirement already satisfied: zipp>=3.1.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from importlib-resources->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (3.17.0) Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (2023.11.1) Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (1.3.10) Requirement already satisfied: referencing>=0.28.4 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.31.1) Requirement already satisfied: rpds-py>=0.7.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.13.2) Requirement already satisfied: ruamel.yaml.clib>=0.2.7 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from ruamel-yaml>=0.16->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.2.8)
Install AllenSDK into your notebook environment (good for Google Colab)¶
You can install AllenSDK into your notebook environment by executing the cell below.
If using Google Colab, click on the RESTART RUNTIME button that appears at the end of the output when this cell is complete,. Note that running this cell will produce a long list of outputs and some error messages. Clicking RESTART RUNTIME at the end will resolve these issues. You can minimize the cell after you are done to hide the output.
!pip install --upgrade pip
!pip install allensdk
Requirement already satisfied: pip in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (23.3.1) Requirement already satisfied: allensdk in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (2.16.2) Requirement already satisfied: psycopg2-binary in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.9.9) Requirement already satisfied: hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.* in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.11.0) Requirement already satisfied: h5py in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.10.0) Requirement already satisfied: matplotlib in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.7.4) Requirement already satisfied: numpy<1.24 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.23.5) Requirement already satisfied: pandas==1.5.3 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.5.3) Requirement already satisfied: jinja2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.1.2) Requirement already satisfied: scipy<1.11 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.10.1) Requirement already satisfied: six in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.16.0) Requirement already satisfied: pynrrd in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.0.0) Requirement already satisfied: future in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.18.3) Requirement already satisfied: requests in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.31.0) Requirement already satisfied: requests-toolbelt in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.0.0) Requirement already satisfied: simplejson in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.19.2) Requirement already satisfied: scikit-image in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.21.0) Requirement already satisfied: scikit-build in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.17.6) Requirement already satisfied: statsmodels in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.14.0) Requirement already satisfied: simpleitk in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.3.1) Requirement already satisfied: argschema in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.0.4) Requirement already satisfied: glymur in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.12.9.post1) Requirement already satisfied: xarray<2023.2.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2023.1.0) Requirement already satisfied: pynwb in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.5.0) Requirement already satisfied: tables in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.8.0) Requirement already satisfied: seaborn in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.13.0) Requirement already satisfied: aiohttp in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.9.1) Requirement already satisfied: nest-asyncio in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.5.8) Requirement already satisfied: tqdm in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (4.66.1) Requirement already satisfied: ndx-events in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.2.0) Requirement already satisfied: boto3 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.33.4) Requirement already satisfied: semver in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.0.2) Requirement already satisfied: cachetools in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (5.3.2) Requirement already satisfied: sqlalchemy in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.0.23) Requirement already satisfied: python-dateutil in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.8.2) Requirement already satisfied: pytz>=2020.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pandas==1.5.3->allensdk) (2023.3.post1) Requirement already satisfied: importlib-resources in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (6.1.1) Requirement already satisfied: jsonschema>=2.6.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (4.20.0) Requirement already satisfied: ruamel-yaml>=0.16 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.18.5) Requirement already satisfied: packaging>=21.3 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from xarray<2023.2.0->allensdk) (23.2) Requirement already satisfied: attrs>=17.3.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (23.1.0) Requirement already satisfied: multidict<7.0,>=4.5 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (6.0.4) Requirement already satisfied: yarl<2.0,>=1.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (1.9.3) Requirement already satisfied: frozenlist>=1.1.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (1.4.0) Requirement already satisfied: aiosignal>=1.1.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (1.3.1) Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (4.0.3) Requirement already satisfied: marshmallow<4.0,>=3.0.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from argschema->allensdk) (3.20.1) Requirement already satisfied: pyyaml in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from argschema->allensdk) (6.0.1) Requirement already satisfied: botocore<1.34.0,>=1.33.4 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from boto3->allensdk) (1.33.4) Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from boto3->allensdk) (1.0.1) Requirement already satisfied: s3transfer<0.9.0,>=0.8.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from boto3->allensdk) (0.8.2) Requirement already satisfied: lxml in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from glymur->allensdk) (4.9.3) Requirement already satisfied: MarkupSafe>=2.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jinja2->allensdk) (2.1.3) Requirement already satisfied: contourpy>=1.0.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (1.1.1) Requirement already satisfied: cycler>=0.10 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (0.12.1) Requirement already satisfied: fonttools>=4.22.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (4.45.1) Requirement already satisfied: kiwisolver>=1.0.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (1.4.5) Requirement already satisfied: pillow>=6.2.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (10.1.0) Requirement already satisfied: pyparsing>=2.3.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (3.1.1) Requirement already satisfied: setuptools in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pynwb->allensdk) (56.0.0) Requirement already satisfied: nptyping in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pynrrd->allensdk) (2.5.0) Requirement already satisfied: typing-extensions in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pynrrd->allensdk) (4.8.0) Requirement already satisfied: charset-normalizer<4,>=2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (3.6) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (1.26.18) Requirement already satisfied: certifi>=2017.4.17 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (2023.11.17) Requirement already satisfied: distro in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-build->allensdk) (1.8.0) Requirement already satisfied: tomli in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-build->allensdk) (2.0.1) Requirement already satisfied: wheel>=0.32.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-build->allensdk) (0.42.0) Requirement already satisfied: networkx>=2.8 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (3.1) Requirement already satisfied: imageio>=2.27 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (2.33.0) Requirement already satisfied: tifffile>=2022.8.12 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (2023.7.10) Requirement already satisfied: PyWavelets>=1.1.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (1.4.1) Requirement already satisfied: lazy_loader>=0.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (0.3) Requirement already satisfied: greenlet!=0.4.17 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from sqlalchemy->allensdk) (3.0.1) Requirement already satisfied: patsy>=0.5.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from statsmodels->allensdk) (0.5.3) Requirement already satisfied: cython>=0.29.21 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (3.0.6) Requirement already satisfied: numexpr>=2.6.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (2.8.6) Requirement already satisfied: blosc2~=2.0.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (2.0.0) Requirement already satisfied: py-cpuinfo in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (9.0.0) Requirement already satisfied: msgpack in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from blosc2~=2.0.0->tables->allensdk) (1.0.7) Requirement already satisfied: zipp>=3.1.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from importlib-resources->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (3.17.0) Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (2023.11.1) Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (1.3.10) Requirement already satisfied: referencing>=0.28.4 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.31.1) Requirement already satisfied: rpds-py>=0.7.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.13.2) Requirement already satisfied: ruamel.yaml.clib>=0.2.7 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from ruamel-yaml>=0.16->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.2.8)
Import required libraries¶
We need to import libraries for plotting and manipulating data
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_context('notebook', font_scale=1.5, rc={'lines.markeredgewidth': 2})
# prefered magic functions for jupyter notebook
%load_ext autoreload
%autoreload 2
%matplotlib inline
# confirm that you are currently using the newest version of SDK (2.10.0 for now)
import allensdk
allensdk.__version__
'2.16.2'
# import behavior projet cache class from SDK to be able to load the data
import allensdk.brain_observatory.behavior.behavior_project_cache as bpc
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm
Load data tables¶
This code block allows you to use behavior_project_cache
(bpc) class to get behavior and ophys tables.
output_dir = r'\Data\visual_behavior_ophys_cache_dir'
bc = bpc.VisualBehaviorOphysProjectCache.from_s3_cache(cache_dir=output_dir)
behavior_session_table = bc.get_behavior_session_table()
ophys_session_table = bc.get_ophys_session_table()
experiment_table = bc.get_ophys_experiment_table()
#print number of items in each table for all imaging and behavioral sessions
print('Number of behavior sessions = {}'.format(len(behavior_session_table)))
print('Number of ophys sessions = {}'.format(len(ophys_session_table)))
print('Number of ophys experiments = {}'.format(len(experiment_table)))
#print number of items in each table with Mesoscope imaging
print('Number of behavior sessions with Mesoscope = {}'.format(len(behavior_session_table[behavior_session_table.project_code.isin(['VisualBehaviorMultiscope'])])))
print('Number of ophys sessions with Mesoscope = {}'.format(len(ophys_session_table[ophys_session_table.project_code.isin(['VisualBehaviorMultiscope'])])))
print('Number of ophys experiments with Mesoscope = {}'.format(len(experiment_table[experiment_table.project_code.isin(['VisualBehaviorMultiscope'])])))
/home/runner/work/AllenSDK/AllenSDK/allensdk/api/cloud_cache/cloud_cache.py:439: MissingLocalManifestWarning: This cache directory appears to contain data files, but it has no record of what those files are. You might want to consider running VisualBehaviorOphysProjectCache.construct_local_manifest() to avoid needlessly downloading duplicates of data files that did not change between data releases. NOTE: running this method will require hashing every data file you have currently downloaded and could be very time consuming. To avoid this warning in the future, make sure that /tmp/tmp6lx6q1i0/_downloaded_data.json is not deleted between instantiations of this cache warnings.warn(msg, MissingLocalManifestWarning) ophys_session_table.csv: 100%|██████████| 247k/247k [00:00<00:00, 1.32MMB/s] behavior_session_table.csv: 100%|██████████| 1.59M/1.59M [00:00<00:00, 7.87MMB/s] ophys_experiment_table.csv: 100%|██████████| 657k/657k [00:00<00:00, 3.74MMB/s] ophys_cells_table.csv: 100%|██████████| 4.28M/4.28M [00:00<00:00, 10.7MMB/s]
Number of behavior sessions = 4782 Number of ophys sessions = 703 Number of ophys experiments = 1936 Number of behavior sessions with Mesoscope = 1305 Number of ophys sessions with Mesoscope = 162 Number of ophys experiments with Mesoscope = 862
/home/runner/work/AllenSDK/AllenSDK/allensdk/brain_observatory/behavior/behavior_project_cache/behavior_project_cache.py:135: UpdatedStimulusPresentationTableWarning: As of AllenSDK version 2.16.0, the latest Visual Behavior Ophys data has been significantly updated from previous releases. Specifically the user will need to update all processing of the stimulus_presentations tables. These tables now include multiple stimulus types delineated by the columns `stimulus_block` and `stimulus_block_name`. The data that was available in previous releases are stored in the block name containing 'change_detection' and can be accessed in the pandas table by using: `stimulus_presentations[stimulus_presentations.stimulus_block_name.str.contains('change_detection')]` warnings.warn(
- Experiment table contains ophys experiment ids as well as associated metadata with them (
cre_line
,session_type
,project_code
, etc). This table gives you an overview of what data at the level of each experiment is available. The term experiment is used to describe one imaging plane during one session. For sessions that are imaged using mesoscope (equipment_name
= MESO.1), there will be up to 4 experiments associated with that sessions (2 imaging depths by 2 visual areas). Across sessions, the same imaging planes or experiments are linked usingophys_container_id
. For sessions that are imaged using scientifica (equipment_name
= CAMP#.#), there will be only 1 experiment which are similarly linked across different session types usingophys_container_id
. - Ophys session table is similar to experiment table but it is a higher level overview of the data details. It groups imaging sessions using
ophys_session_id
and provides metadata associated with those sessions. - Behavior session table contains metadata related to animals' training history as well as their behavior during ophys imaging sessions. The table is organized using
behavior_session_id
. Behavior sessions that were also imaging sessions have ophys ids assosiated with them.
In this notebook, we will use experiment_table
to select experiments of interest and look at them in a greater detail.
# let's print a sample of 5 rows to see what's in the table
experiment_table.sample(5, random_state=42)
behavior_session_id | ophys_session_id | ophys_container_id | mouse_id | indicator | full_genotype | driver_line | cre_line | reporter_line | sex | ... | passive | experience_level | prior_exposures_to_session_type | prior_exposures_to_image_set | prior_exposures_to_omissions | date_of_acquisition | equipment_name | published_at | isi_experiment_id | file_id | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ophys_experiment_id | |||||||||||||||||||||
1057725798 | 1056253022 | 1056238781 | 1052096554 | 528097 | GCaMP6f | Vip-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | [Vip-IRES-Cre] | Vip-IRES-Cre | Ai148(TIT2L-GC6f-ICL-tTA2) | F | ... | False | Novel >1 | 0 | 7 | 10 | 2020-10-13 10:09:00.366000+00:00 | MESO.1 | 2021-03-25 | 1031852674 | 1932 |
891108758 | 890157940 | 889944877 | 1018028370 | 435431 | GCaMP6f | Vip-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | [Vip-IRES-Cre] | Vip-IRES-Cre | Ai148(TIT2L-GC6f-ICL-tTA2) | M | ... | True | Novel >1 | 1 | 6 | 10 | 2019-06-19 10:45:53.800000+00:00 | MESO.1 | 2021-03-25 | 815343876 | 887 |
891108773 | 890157940 | 889944877 | 1018028377 | 435431 | GCaMP6f | Vip-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | [Vip-IRES-Cre] | Vip-IRES-Cre | Ai148(TIT2L-GC6f-ICL-tTA2) | M | ... | True | Novel >1 | 1 | 6 | 10 | 2019-06-19 10:45:53.800000+00:00 | MESO.1 | 2021-03-25 | 815343876 | 893 |
1088351100 | 1088263239 | 1088247037 | 1086191444 | 550603 | GCaMP6f | Vip-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | [Vip-IRES-Cre] | Vip-IRES-Cre | Ai148(TIT2L-GC6f-ICL-tTA2) | F | ... | True | Familiar | 0 | 26 | 4 | 2021-03-05 13:19:43.064000+00:00 | MESO.1 | 2021-08-12 | 1064781085 | 1740 |
975608392 | 975498864 | 975452945 | 1018028199 | 482853 | GCaMP6f | Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | [Sst-IRES-Cre] | Sst-IRES-Cre | Ai148(TIT2L-GC6f-ICL-tTA2) | M | ... | False | Familiar | 0 | 25 | 0 | 2019-11-01 14:53:10.259000+00:00 | MESO.1 | 2021-03-25 | 937755139 | 1450 |
5 rows × 30 columns
You can get any experiment ids from the experiment table by subsetting the table using various conditions (aka columns) in the table. Here, we can select experiments from Sst mice only, novel Ophys session 4, with 0 prior exposures to the stimulus (meaning the session was not a relake).
# get all Sst experiments for ophys session 4
selected_experiment_table = experiment_table[(experiment_table.cre_line=='Sst-IRES-Cre')&
(experiment_table.session_number==4) &
(experiment_table.prior_exposures_to_image_set==0)]
print('Number of experiments: {}'.format(len(selected_experiment_table)))
Number of experiments: 58
Remember that any given experiment contains data from only one imaging plane. Some of these experiments come from the same imaging session. Here, we can check how many unique imaging sessions are associated with experiments selected above.
print('Number of unique sessions: {}'.format(len(selected_experiment_table['ophys_session_id'].unique())))
Number of unique sessions: 19
Load an experiment¶
Let's pick a random experiment from the table and plot example ophys and behavioral data.
# select first experiment from the table to look at in more detail.
# Note that python enumeration starts at 0.
ophys_experiment_id = selected_experiment_table.index[0]
dataset = bc.get_behavior_ophys_experiment(ophys_experiment_id)
behavior_ophys_experiment_957759562.nwb: 100%|██████████| 241M/241M [00:09<00:00, 25.7MMB/s]
show metadata for this experiment¶
dataset.metadata
{'equipment_name': 'MESO.1', 'sex': 'F', 'age_in_days': 213, 'stimulus_frame_rate': 60.0, 'session_type': 'OPHYS_4_images_B', 'date_of_acquisition': datetime.datetime(2019, 9, 27, 8, 58, 37, 5000, tzinfo=tzlocal()), 'reporter_line': 'Ai148(TIT2L-GC6f-ICL-tTA2)', 'cre_line': 'Sst-IRES-Cre', 'behavior_session_uuid': UUID('40897cd4-3279-4a2d-b65d-b3f984e34e17'), 'driver_line': ['Sst-IRES-Cre'], 'mouse_id': '457841', 'project_code': 'VisualBehaviorMultiscope', 'full_genotype': 'Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt', 'behavior_session_id': 957032492, 'indicator': 'GCaMP6f', 'emission_lambda': 520.0, 'excitation_lambda': 910.0, 'ophys_container_id': 1018028342, 'field_of_view_height': 512, 'field_of_view_width': 512, 'imaging_depth': 150, 'targeted_imaging_depth': 150, 'imaging_plane_group': 0, 'imaging_plane_group_count': 4, 'ophys_experiment_id': 957759562, 'ophys_frame_rate': 11.0, 'ophys_session_id': 957020350, 'targeted_structure': 'VISp'}
You can get additional information about this experiment from the metadata field of the dataset class. Here, you can see that this experiment was in Sst Cre line, in a female mouse at 233 days old, recorded using mesoscope (this is one of four imaging planes), at imaging depth of 150 microns, in primary visual cortex (VISp). This experiment is also from OPHYS 1 session using image set A.
plot max projection from this experiment¶
plt.imshow(dataset.max_projection, cmap='gray')
<matplotlib.image.AxesImage at 0x7fe1f3557460>
Max projection plots an average image from the movie recorded during an imaging session. Plotting max projection can give you a sense of how many neurons were visible during imaging and how clear and stable the imaging session was.
load cell specimen table with cells' imaging metrics¶
dataset.cell_specimen_table.sample(3, random_state=42)
cell_roi_id | height | mask_image_plane | max_correction_down | max_correction_left | max_correction_right | max_correction_up | valid_roi | width | x | y | roi_mask | |
---|---|---|---|---|---|---|---|---|---|---|---|---|
cell_specimen_id | ||||||||||||
1086614819 | 1080740886 | 22 | 0 | 5.0 | 25.0 | 3.0 | 30.0 | True | 16 | 443 | 275 | [[False, False, False, False, False, False, Fa... |
1086615620 | 1080740955 | 17 | 0 | 5.0 | 25.0 | 3.0 | 30.0 | True | 19 | 280 | 104 | [[False, False, False, False, False, False, Fa... |
1086614149 | 1080740882 | 16 | 0 | 5.0 | 25.0 | 3.0 | 30.0 | True | 17 | 216 | 456 | [[False, False, False, False, False, False, Fa... |
cell_specimen_table
includes information about x
and y
coordinates of the cell in the imaging plane as well as how much correction was applied during motion correction process.
cell_roi_id
is a unique id assigned to each ROI during segmentation.
cell_specimen_id
is a unique id assigned to each cell after cell matching, which means that if we were able to identify and match the same cell across multiple sessions, it can be identified by its unique cell specimen id.
roi_mask
is a boolean array that can be used to visualize where any given cell is in the imaging field.
plt.imshow(dataset.cell_specimen_table.iloc[1]['roi_mask'])
<matplotlib.image.AxesImage at 0x7fe1f5c14160>
show dff traces for the first 10 cells this experiment¶
dataset.dff_traces.head(10)
cell_roi_id | dff | |
---|---|---|
cell_specimen_id | ||
1086614149 | 1080740882 | [0.3417784869670868, 0.1917504370212555, 0.169... |
1086614819 | 1080740886 | [0.0, -2.010002374649048, -1.3567286729812622,... |
1086614512 | 1080740888 | [0.07121878117322922, 0.10929209738969803, 0.0... |
1086613265 | 1080740947 | [0.3090711534023285, 0.02156120352447033, -0.0... |
1086616398 | 1080740952 | [0.0, 0.15438319742679596, 0.37885594367980957... |
1086615620 | 1080740955 | [0.15867245197296143, 0.15752384066581726, 0.2... |
1086615201 | 1080740967 | [0.42869994044303894, 0.23723232746124268, -0.... |
1086616101 | 1080741008 | [0.727859377861023, 0.20604123175144196, 0.333... |
dff_traces
dataframe contains traces for all neurons in this experiment, unaligned to any events in the task.
You can select rows by their enumerated number using .iloc[]
method:
dataset.dff_traces.iloc[4]
cell_roi_id 1080740952 dff [0.0, 0.15438319742679596, 0.37885594367980957... Name: 1086616398, dtype: object
Alternatively, you can use cell_specimen_id
as index to select cells with .loc[]
method:
cell_specimen_id = dataset.dff_traces.index[0]
dataset.dff_traces.loc[cell_specimen_id]
cell_roi_id 1080740882 dff [0.3417784869670868, 0.1917504370212555, 0.169... Name: 1086614149, dtype: object
If you don't want dff in a pandas dataframe format, you can load dff traces as an array, using np.vstack
function to format the data into cell by time array and .values
to only grab values in dff column:
dff_array = np.vstack(dataset.dff_traces.dff.values)
print('This array contrains dff traces from {} neurons and it is {} samples long.'.format(dff_array.shape[0], dff_array.shape[1]))
This array contrains dff traces from 8 neurons and it is 48284 samples long.
show events traces for the first 10 cells in this experiment¶
dataset.events.head(10)
cell_roi_id | events | filtered_events | lambda | noise_std | |
---|---|---|---|---|---|
cell_specimen_id | |||||
1086614149 | 1080740882 | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | 0.0753 | 0.085235 |
1086614819 | 1080740886 | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | 5.3130 | 0.714571 |
1086614512 | 1080740888 | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | 0.0324 | 0.055955 |
1086613265 | 1080740947 | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | 0.0272 | 0.051073 |
1086616398 | 1080740952 | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | 0.0570 | 0.074011 |
1086615620 | 1080740955 | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | 0.0769 | 0.086979 |
1086615201 | 1080740967 | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | 0.0872 | 0.091952 |
1086616101 | 1080741008 | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... | 0.1596 | 0.123868 |
events
table is similar to dff_traces
but the output provides traces of extrapolated events. Events are computed on unmixed dff traces for each cell as described in Giovannucci et al. 2019. The magnitude of events approximates the firing rate of neurons with the resolusion of about 200 ms. The biggest advantage of using events over dff traces is they exclude prolonged Ca transients that may conteminate neural responses to subsequent stimuli. You can also use filtered_events
which are events convolved with a filter created using stats.halfnorm
method.
lambda
is computed from Poisson distribution of events in the trace (think of it as a center of mass of the distribution, larger lambda == higher "firing rate").
noise_std
is a measure of variability in the events trace.
load ophys timestamps¶
The timestamps are the same for dff_traces
and events
, in seconds
dataset.ophys_timestamps
array([ 8.72468, 8.81788, 8.91108, ..., 4509.98089, 4510.07412, 4510.16735])
Pick a cell and plot the traces¶
We can select a random cell from the experiment and plot its dff and events traces along with other behavioral and stimulus data.
cell_specimen_ids = dataset.cell_specimen_table.index.values # a list of all cell ids
cell_specimen_id = cell_specimen_ids[5] # let's pick 6th cell
print('Cell specimen id = {}'.format(cell_specimen_id)) # print id
Cell specimen id = 1086615620
# plot dff and events traces overlaid from the cell selected above
fig, ax = plt.subplots(1,1, figsize = (20,10))
ax.plot(dataset.ophys_timestamps, dataset.dff_traces.loc[cell_specimen_id, 'dff'])
ax.plot(dataset.ophys_timestamps, dataset.events.loc[cell_specimen_id, 'events'])
ax.set_xlabel('time (seconds)')
ax.set_ylabel('trace magnitude')
ax.set_title('Cell specimen id = {}'.format(cell_specimen_id), fontsize = 20)
ax.legend(['dff', 'events'], fontsize = 20)
<matplotlib.legend.Legend at 0x7fe1f32fef70>
We can see that as expected, events trace is much cleaner than dff and it generally follows big Ca transients really well. We can also see that this cell was not very active during our experiment. Each experiment has a 5 minute movie at the end, which often drives neural activity really well. We can see a notable increase in cell's activity at the end of this experiment as well.
plot mouse running speed from this experiment¶
fig, ax = plt.subplots(1,1, figsize = (20,5))
ax.plot(dataset.stimulus_timestamps, dataset.running_speed['speed'], color='gray', linestyle='--')
ax.set_xlabel('time (seconds)')
ax.set_ylabel('running speed (cm/s)')
ax.set_title('Ophys experiment {}'.format(ophys_experiment_id), fontsize = 20)
Text(0.5, 1.0, 'Ophys experiment 957759562')
plot pupil area for the same experiment¶
fig, ax = plt.subplots(1,1, figsize = (20,5))
ax.plot(dataset.eye_tracking.timestamps, dataset.eye_tracking.pupil_area, color='gray')
ax.set_xlabel('time (seconds)')
ax.set_ylabel('pupil area (pixels^2)')
ax.set_title('Ophys experiment {}'.format(ophys_experiment_id), fontsize = 20)
Text(0.5, 1.0, 'Ophys experiment 957759562')
You can find all attributes and methods that belong to dataset class using this helpful method:
dataset.list_data_attributes_and_methods()
['average_projection', 'behavior_session_id', 'cell_specimen_table', 'corrected_fluorescence_traces', 'demixed_traces', 'dff_traces', 'events', 'eye_tracking', 'eye_tracking_rig_geometry', 'get_cell_specimen_ids', 'get_cell_specimen_indices', 'get_dff_traces', 'get_performance_metrics', 'get_reward_rate', 'get_rolling_performance_df', 'get_segmentation_mask_image', 'licks', 'max_projection', 'metadata', 'motion_correction', 'neuropil_traces', 'ophys_experiment_id', 'ophys_session_id', 'ophys_timestamps', 'raw_running_speed', 'rewards', 'roi_masks', 'running_speed', 'segmentation_mask_image', 'stimulus_presentations', 'stimulus_templates', 'stimulus_timestamps', 'task_parameters', 'trials', 'update_targeted_imaging_depth']
You can learn more about them by calling help
on them:
help(dataset.get_segmentation_mask_image)
Help on method get_segmentation_mask_image in module allensdk.brain_observatory.behavior.behavior_ophys_experiment: get_segmentation_mask_image() -> allensdk.brain_observatory.behavior.image_api.Image method of allensdk.brain_observatory.behavior.behavior_ophys_experiment.BehaviorOphysExperiment instance a 2D binary image of all valid cell masks Returns ---------- allensdk.brain_observatory.behavior.image_api.Image: array-like interface to segmentation_mask image data and metadata
Get information about visual stimuli presented on each trial¶
get stimulus information for this experiment and assign it to a table called stimulus_table
stimulus_table = dataset.stimulus_presentations
stimulus_table.head(10)
stimulus_block | stimulus_block_name | image_index | image_name | movie_frame_index | duration | start_time | end_time | start_frame | end_frame | is_change | is_image_novel | omitted | movie_repeat | flashes_since_change | trials_id | active | stimulus_name | is_sham_change | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
stimulus_presentations_id | |||||||||||||||||||
0 | 0 | initial_gray_screen_5min | -99 | NaN | -99 | 308.772448 | 0.000000 | 308.772448 | 0 | 17985 | False | <NA> | <NA> | -99 | 0 | -99 | False | spontaneous | False |
1 | 1 | change_detection_behavior | 0 | im000 | -99 | 0.250210 | 308.772448 | 309.022658 | 17985 | 18000 | False | True | False | -99 | 1 | 0 | True | Natural_Images_Lum_Matched_set_ophys_6_2017 | False |
2 | 1 | change_detection_behavior | 0 | im000 | -99 | 0.250180 | 309.523058 | 309.773238 | 18030 | 18045 | False | True | False | -99 | 2 | 0 | True | Natural_Images_Lum_Matched_set_ophys_6_2017 | False |
3 | 1 | change_detection_behavior | 0 | im000 | -99 | 0.250160 | 310.273688 | 310.523848 | 18075 | 18090 | False | True | False | -99 | 3 | 1 | True | Natural_Images_Lum_Matched_set_ophys_6_2017 | False |
4 | 1 | change_detection_behavior | 0 | im000 | -99 | 0.250220 | 311.024258 | 311.274478 | 18120 | 18135 | False | True | False | -99 | 4 | 1 | True | Natural_Images_Lum_Matched_set_ophys_6_2017 | False |
5 | 1 | change_detection_behavior | 0 | im000 | -99 | 0.250210 | 311.774888 | 312.025098 | 18165 | 18180 | False | True | False | -99 | 5 | 1 | True | Natural_Images_Lum_Matched_set_ophys_6_2017 | False |
6 | 1 | change_detection_behavior | 0 | im000 | -99 | 0.250150 | 312.525538 | 312.775688 | 18210 | 18225 | False | True | False | -99 | 6 | 1 | True | Natural_Images_Lum_Matched_set_ophys_6_2017 | False |
7 | 1 | change_detection_behavior | 0 | im000 | -99 | 0.250210 | 313.276118 | 313.526328 | 18255 | 18270 | False | True | False | -99 | 7 | 1 | True | Natural_Images_Lum_Matched_set_ophys_6_2017 | False |
8 | 1 | change_detection_behavior | 6 | im031 | -99 | 0.250220 | 314.026728 | 314.276948 | 18300 | 18315 | True | True | False | -99 | 0 | 1 | True | Natural_Images_Lum_Matched_set_ophys_6_2017 | False |
9 | 1 | change_detection_behavior | 6 | im031 | -99 | 0.250190 | 314.777358 | 315.027548 | 18345 | 18360 | False | True | False | -99 | 1 | 1 | True | Natural_Images_Lum_Matched_set_ophys_6_2017 | False |
This table provides helpful information like image name, start, duration and stop of image presentation, and whether the image was omitted.
print('This experiment had {} stimuli.'.format(len(stimulus_table)))
print('Out of all stimuli presented, {} were omitted.'.format(len(stimulus_table[stimulus_table['image_name']=='omitted'])))
This experiment had 13799 stimuli. Out of all stimuli presented, 160 were omitted.
You can also use keys()
method to see the names of the columns in any pandas dataframe table:
stimulus_table.keys()
Index(['stimulus_block', 'stimulus_block_name', 'image_index', 'image_name', 'movie_frame_index', 'duration', 'start_time', 'end_time', 'start_frame', 'end_frame', 'is_change', 'is_image_novel', 'omitted', 'movie_repeat', 'flashes_since_change', 'trials_id', 'active', 'stimulus_name', 'is_sham_change'], dtype='object')
Get task and behavioral data for each trial¶
get behavioral trial information and assign it to trials_table
trials_table = dataset.trials
trials_table.head(5)
start_time | stop_time | initial_image_name | change_image_name | is_change | change_time | go | catch | lick_times | response_time | ... | reward_time | reward_volume | hit | false_alarm | miss | correct_reject | aborted | auto_rewarded | change_frame | trial_length | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
trials_id | |||||||||||||||||||||
0 | 308.73646 | 309.85402 | im000 | im000 | False | NaN | False | False | [309.12011, 309.30359, 309.53711] | NaN | ... | NaN | 0.000 | False | False | False | False | True | False | -99 | 1.11756 |
1 | 310.23770 | 318.26089 | im000 | im031 | True | 314.026728 | False | False | [314.37439, 314.50783, 314.64127, 314.79139, 3... | 314.37439 | ... | 314.14086 | 0.005 | False | False | False | False | False | True | 18300 | 8.02319 |
2 | 318.49442 | 319.37846 | im031 | im031 | False | NaN | False | False | [318.6779, 318.86138, 319.06156, 319.39515, 31... | NaN | ... | NaN | 0.000 | False | False | False | False | True | False | -99 | 0.88404 |
3 | 319.99565 | 323.19826 | im031 | im031 | False | NaN | False | False | [322.58109, 322.71452, 322.88134, 323.46514, 3... | NaN | ... | NaN | 0.000 | False | False | False | False | True | False | -99 | 3.20261 |
4 | 323.73205 | 331.02131 | im031 | im000 | True | 326.787158 | False | False | [327.1515, 327.26826, 327.38503, 327.51846, 32... | 327.15150 | ... | 326.91797 | 0.005 | False | False | False | False | False | True | 19065 | 7.28926 |
5 rows × 21 columns
trials_table.keys()
Index(['start_time', 'stop_time', 'initial_image_name', 'change_image_name', 'is_change', 'change_time', 'go', 'catch', 'lick_times', 'response_time', 'response_latency', 'reward_time', 'reward_volume', 'hit', 'false_alarm', 'miss', 'correct_reject', 'aborted', 'auto_rewarded', 'change_frame', 'trial_length'], dtype='object')
This table has information about experiment trials. go
trials are change trials when the animal was supposed to lick. If the animal licked, hit
is set to True for that trial. If the animal was rewarded, reward_time
will have time in seconds. If this was an auto rewarded trial (regardless of whether the animal got it right), auto_rewarded
is set to True. The trials table also includes response_latency
which can be used as reaction time of the animal during the experiment.
Plot an example of one selected cell¶
Now, we will put together a plotting functions that utilizes data in the dataset class to plot ophys traces and behavioral data from an experiment.
# pull the image stimuli from the stimulus table
stimulus_presentations = dataset.stimulus_presentations[
dataset.stimulus_presentations.stimulus_block_name.str.contains('change_detection')]
# create a list of all unique stimuli presented in this experiment
unique_stimuli = [stimulus for stimulus in stimulus_presentations['image_name'].unique()]
# create a colormap with each unique image having its own color
colormap = {image_name: sns.color_palette()[image_number] for image_number, image_name in enumerate(np.sort(unique_stimuli))}
colormap['omitted'] = (1,1,1) # set omitted stimulus to white color
# add the colors for each image to the stimulus presentations table in the dataset
stimulus_presentations['color'] = stimulus_presentations['image_name'].map(lambda image_name: colormap[image_name])
# function to plot dff traces
def plot_dff_trace(ax, cell_specimen_id, initial_time, final_time):
'''
ax: axis on which to plot
cell_specimen_id: id of the cell to plot
intial_time: initial time to plot from
final_time: final time to plot to
'''
#create a dataframe using dff trace from one seleted cell
data = {'dff': dataset.dff_traces.loc[cell_specimen_id].dff,
'timestamps': dataset.ophys_timestamps}
df = pd.DataFrame(data)
dff_trace_sample = df.query('timestamps >= @initial_time and timestamps <= @final_time')
ax.plot(
dff_trace_sample['timestamps'],
dff_trace_sample['dff']/dff_trace_sample['dff'].max()
)
# function to plot events traces
def plot_events_trace(ax, cell_specimen_id, initial_time, final_time):
# create a dataframe using events trace from one seleted cell
data = {'events': dataset.events.loc[cell_specimen_id].events,
'timestamps': dataset.ophys_timestamps}
df = pd.DataFrame(data)
events_trace_sample = df.query('timestamps >= @initial_time and timestamps <= @final_time')
ax.plot(
events_trace_sample['timestamps'],
events_trace_sample['events']/events_trace_sample['events'].max()
)
# function to plot running speed
def plot_running(ax, initial_time, final_time):
running_sample = dataset.running_speed.query('timestamps >= @initial_time and timestamps <= @final_time')
ax.plot(
running_sample['timestamps'],
running_sample['speed']/running_sample['speed'].max(),
'--',
color = 'gray',
linewidth = 1
)
# function to plot pupil diameter
def plot_pupil(ax, initial_time, final_time):
pupil_sample = dataset.eye_tracking.query('timestamps >= @initial_time and timestamps <= @final_time')
ax.plot(
pupil_sample['timestamps'],
pupil_sample['pupil_width']/pupil_sample['pupil_width'].max(),
color = 'gray',
linewidth = 1
)
# function to plot licks
def plot_licks(ax, initial_time, final_time):
licking_sample = dataset.licks.query('timestamps >= @initial_time and timestamps <= @final_time')
ax.plot(
licking_sample['timestamps'],
np.zeros_like(licking_sample['timestamps']),
marker = 'o',
markersize = 3,
color = 'black',
linestyle = 'none'
)
# function to plot rewards
def plot_rewards(ax, initial_time, final_time):
rewards_sample = dataset.rewards.query('timestamps >= @initial_time and timestamps <= @final_time')
ax.plot(
rewards_sample['timestamps'],
np.zeros_like(rewards_sample['timestamps']),
marker = 'd',
color = 'blue',
linestyle = 'none',
markersize = 12,
alpha = 0.5
)
def plot_stimuli(ax, initial_time, final_time):
stimulus_presentations_sample = stimulus_presentations.query('end_time >= @initial_time and start_time <= @final_time')
for idx, stimulus in stimulus_presentations_sample.iterrows():
ax.axvspan(stimulus['start_time'], stimulus['end_time'], color=stimulus['color'], alpha=0.25)
initial_time = 820 # start time in seconds
final_time = 860 # stop time in seconds
fig, ax = plt.subplots(2,1,figsize = (15,7))
plot_dff_trace(ax[0], cell_specimen_ids[3], initial_time, final_time)
plot_events_trace(ax[0], cell_specimen_ids[3], initial_time, final_time)
plot_stimuli(ax[0], initial_time, final_time)
ax[0].set_ylabel('normalized response magnitude')
ax[0].set_yticks([])
ax[0].legend(['dff trace', 'events trace'])
plot_running(ax[1], initial_time, final_time)
plot_pupil(ax[1], initial_time, final_time)
plot_licks(ax[1], initial_time, final_time)
plot_rewards(ax[1], initial_time, final_time)
plot_stimuli(ax[1], initial_time, final_time)
ax[1].set_yticks([])
ax[1].legend(['running speed', 'pupil','licks', 'rewards'])
<matplotlib.legend.Legend at 0x7fe1f25ecb80>
From looking at the activity of this neuron, we can see that it was very active during our experiment but its activity does not appear to be reliably locked to image presentations. It does seem to vaguely follow animal's running speed, thus it might be modulated by running.
Vip cell example¶
We can get a different, Vip experiment from Ophys session 1 and plot it to compare response traces. This gives us a similar plot from a different inhibitory neuron to compare their neural dynamics.
selected_experiment_table = experiment_table[(experiment_table.cre_line=='Vip-IRES-Cre')&
(experiment_table.session_number==1)]
dataset = bc.get_behavior_ophys_experiment(selected_experiment_table.index.values[1])
cell_specimen_ids = dataset.cell_specimen_table.index.values # a list of all cell ids
behavior_ophys_experiment_826587940.nwb: 100%|██████████| 370M/370M [00:14<00:00, 25.2MMB/s]
# pull the image stimuli from the stimulus table
stimulus_presentations = dataset.stimulus_presentations[
dataset.stimulus_presentations.stimulus_block_name.str.contains('change_detection')]
# create a list of all unique stimuli presented in this experiment
unique_stimuli = [stimulus for stimulus in stimulus_presentations['image_name'].unique()]
# create a colormap with each unique image having its own color
colormap = {image_name: sns.color_palette()[image_number] for image_number, image_name in enumerate(np.sort(unique_stimuli))}
colormap['omitted'] = (1,1,1)
# add the colors for each image to the stimulus presentations table in the dataset
stimulus_presentations['color'] = stimulus_presentations['image_name'].map(lambda image_name: colormap[image_name])
# we can plot the same information for a different cell in the dataset
initial_time = 580 # start time in seconds
final_time = 620 # stop time in seconds
fig, ax = plt.subplots(2,1,figsize = (15,7))
plot_dff_trace(ax[0], cell_specimen_ids[5], initial_time, final_time)
plot_events_trace(ax[0], cell_specimen_ids[5], initial_time, final_time)
plot_stimuli(ax[0], initial_time, final_time)
ax[0].set_ylabel('normalized response magnitude')
ax[0].set_yticks([])
ax[0].legend(['dff trace', 'events trace'])
plot_running(ax[1], initial_time, final_time)
plot_pupil(ax[1], initial_time, final_time)
plot_licks(ax[1], initial_time, final_time)
plot_rewards(ax[1], initial_time, final_time)
plot_stimuli(ax[1], initial_time, final_time)
ax[1].set_yticks([])
ax[1].legend(['running speed', 'pupil','licks', 'rewards'])
<matplotlib.legend.Legend at 0x7fe1f2eff640>
We can see that the dynamics of a Vip neuron are also not driven by the visual stimuli. Aligning neural activity to different behavioral or experimental events might reveal what this neuron is driven by.