{ "cells": [ { "cell_type": "markdown", "id": "8432ad9b", "metadata": { "papermill": { "duration": 0.009347, "end_time": "2023-11-30T06:07:50.765611", "exception": false, "start_time": "2023-11-30T06:07:50.756264", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "# Accessing Visual Behavior Ophys Data" ] }, { "cell_type": "markdown", "id": "86cb4f05", "metadata": { "papermill": { "duration": 0.008018, "end_time": "2023-11-30T06:07:50.781877", "exception": false, "start_time": "2023-11-30T06:07:50.773859", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "## Tutorial overview\n", "\n", "This Jupyter notebook covers the various methods for accessing the Allen Institute Visual Behavior Ophys dataset. We will go over how to request data, where it's stored, and what the various files contain. If you're having trouble downloading the data, or you just want to know more about what's going on under the hood, this is a good place to start.\n", "\n", "This data release will not have a web interface for browsing through the released data, as with the [two-photon imaging Visual Coding dataset](http://observatory.brain-map.org/visualcoding). Instead, the data must be retrieved through the AllenSDK (Python 3.6+) or via requests sent to the **Amazon Web Services (AWS)** **Simple Storage Service (S3)** bucket (name: [visual-behavior-ophys-data](https://s3.console.aws.amazon.com/s3/buckets/visual-behavior-ophys-data)) for this project.\n", "\n", "Functions related to data analysis as well as descriptions of metadata table columns will be covered in other tutorials. For a full list of available tutorials for this project, see the [SDK documentation](https://allensdk.readthedocs.io/en/latest/visual_behavior_optical_physiology.html)." ] }, { "cell_type": "markdown", "id": "1be4fb02", "metadata": { "papermill": { "duration": 0.008037, "end_time": "2023-11-30T06:07:50.797922", "exception": false, "start_time": "2023-11-30T06:07:50.789885", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "## Options for data access\n", "\n", "The `VisualBehaviorOphysProjectCache` object in the AllenSDK is the easiest way to interact with the released data. This object abstracts away the details of on-disk file storage, and delivers the data to you as ready-to-analyze Python objects. The cache will automatically keep track of which files are stored locally, and will download additional files on an as-needed basis. Usually you won't need to worry about the organization of these files, but this tutorial will cover those details in case you want to analyze them without using the AllenSDK (e.g., in Matlab). This tutorial begins with an introduction to this approach.\n", "\n", "Another option is to directly download the data using an S3 URL. This should be used if the other options are broken or are not available to you. Instructions for this can be found at the end of this tutorial." ] }, { "cell_type": "markdown", "id": "4e9c2097", "metadata": { "papermill": { "duration": 0.008011, "end_time": "2023-11-30T06:07:50.814799", "exception": false, "start_time": "2023-11-30T06:07:50.806788", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "## Using the AllenSDK to retrieve data\n", "\n", "Most users will want to access data via the AllenSDK. This requires nothing more than a Python interpreter and some free disk space to store the data locally.\n", "\n", "How much data is there? If you want to download the complete dataset (3021 Behavior Sessions, 551 Behavior Ophys Sessions containing 1165 Behavior Ophys Experiments), you'll need 1000.8 GB of space, split across the following files:\n", "\n", "1. CSV files containing information about behavior sessions, behavior ophys sessions, and behavior ophys experiments (1.3 MB)\n", "2. NWB files containing data for behavior sessions (437.6 GB total, min file size = 0.049 GB, max file size = 0.194 GB)\n", "3. NWB files containing data for behavior ophys experiments (563.2 GB total, min file size = 0.231 GB, max file size = 2.96 GB)\n", "\n", "Before downloading the data, you must decide on a cache directory where you would like downloaded data to be stored. This directory is where the `VisualBehaviorOphysProjectCache` object will look first when you request a metadata table or a data file.\n", "\n", "When you initialize a local cache for the first time, it will create the manifest file at the path that you specify. This file lives one directory up from the rest of the data, so make sure you put it somewhere that has enough space available.\n", "\n", "When you need to access the data in subsequent analysis sessions, you should point the `VisualBehaviorOphysProjectCache` object to an existing cache directory; otherwise, it will try to re-download the data in a new location.\n", "\n", "To get started with this approach, first take care of the necessary imports:\n", "\n", "We will first install allensdk into your environment by running the appropriate commands below. " ] }, { "cell_type": "markdown", "id": "fe5057d6", "metadata": { "papermill": { "duration": 0.008147, "end_time": "2023-11-30T06:07:50.831099", "exception": false, "start_time": "2023-11-30T06:07:50.822952", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "## Instal AllenSDK into your local environment" ] }, { "cell_type": "markdown", "id": "68b8cb24", "metadata": { "papermill": { "duration": 0.008024, "end_time": "2023-11-30T06:07:50.847189", "exception": false, "start_time": "2023-11-30T06:07:50.839165", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "You can install AllenSDK with:" ] }, { "cell_type": "code", "execution_count": 1, "id": "5e8afbc8", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:07:50.865078Z", "iopub.status.busy": "2023-11-30T06:07:50.864537Z", "iopub.status.idle": "2023-11-30T06:07:52.950408Z", "shell.execute_reply": "2023-11-30T06:07:52.949654Z" }, "papermill": { "duration": 2.096916, "end_time": "2023-11-30T06:07:52.952183", "exception": false, "start_time": "2023-11-30T06:07:50.855267", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: allensdk in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (2.16.2)\r\n", "Requirement already satisfied: psycopg2-binary in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.9.9)\r\n", "Requirement already satisfied: hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.* in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.11.0)\r\n", "Requirement already satisfied: h5py in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.10.0)\r\n", "Requirement already satisfied: matplotlib in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.7.4)\r\n", "Requirement already satisfied: numpy<1.24 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.23.5)\r\n", "Requirement already satisfied: pandas==1.5.3 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.5.3)\r\n", "Requirement already satisfied: jinja2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.1.2)\r\n", "Requirement already satisfied: scipy<1.11 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.10.1)\r\n", "Requirement already satisfied: six in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.16.0)\r\n", "Requirement already satisfied: pynrrd in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.0.0)\r\n", "Requirement already satisfied: future in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.18.3)\r\n", "Requirement already satisfied: requests in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.31.0)\r\n", "Requirement already satisfied: requests-toolbelt in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.0.0)\r\n", "Requirement already satisfied: simplejson in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.19.2)\r\n", "Requirement already satisfied: scikit-image in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.21.0)\r\n", "Requirement already satisfied: scikit-build in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.17.6)\r\n", "Requirement already satisfied: statsmodels in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.14.0)\r\n", "Requirement already satisfied: simpleitk in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.3.1)\r\n", "Requirement already satisfied: argschema in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.0.4)\r\n", "Requirement already satisfied: glymur in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.12.9.post1)\r\n", "Requirement already satisfied: xarray<2023.2.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2023.1.0)\r\n", "Requirement already satisfied: pynwb in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.5.0)\r\n", "Requirement already satisfied: tables in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.8.0)\r\n", "Requirement already satisfied: seaborn in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.13.0)\r\n", "Requirement already satisfied: aiohttp in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.9.1)\r\n", "Requirement already satisfied: nest-asyncio in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.5.8)\r\n", "Requirement already satisfied: tqdm in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (4.66.1)\r\n", "Requirement already satisfied: ndx-events in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.2.0)\r\n", "Requirement already satisfied: boto3 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.33.4)\r\n", "Requirement already satisfied: semver in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.0.2)\r\n", "Requirement already satisfied: cachetools in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (5.3.2)\r\n", "Requirement already satisfied: sqlalchemy in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.0.23)\r\n", "Requirement already satisfied: python-dateutil in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.8.2)\r\n", "Requirement already satisfied: pytz>=2020.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pandas==1.5.3->allensdk) (2023.3.post1)\r\n", "Requirement already satisfied: importlib-resources in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (6.1.1)\r\n", "Requirement already satisfied: jsonschema>=2.6.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (4.20.0)\r\n", "Requirement already satisfied: ruamel-yaml>=0.16 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.18.5)\r\n", "Requirement already satisfied: packaging>=21.3 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from xarray<2023.2.0->allensdk) (23.2)\r\n", "Requirement already satisfied: attrs>=17.3.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (23.1.0)\r\n", "Requirement already satisfied: multidict<7.0,>=4.5 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (6.0.4)\r\n", "Requirement already satisfied: yarl<2.0,>=1.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (1.9.3)\r\n", "Requirement already satisfied: frozenlist>=1.1.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (1.4.0)\r\n", "Requirement already satisfied: aiosignal>=1.1.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (1.3.1)\r\n", "Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (4.0.3)\r\n", "Requirement already satisfied: marshmallow<4.0,>=3.0.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from argschema->allensdk) (3.20.1)\r\n", "Requirement already satisfied: pyyaml in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from argschema->allensdk) (6.0.1)\r\n", "Requirement already satisfied: botocore<1.34.0,>=1.33.4 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from boto3->allensdk) (1.33.4)\r\n", "Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from boto3->allensdk) (1.0.1)\r\n", "Requirement already satisfied: s3transfer<0.9.0,>=0.8.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from boto3->allensdk) (0.8.2)\r\n", "Requirement already satisfied: lxml in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from glymur->allensdk) (4.9.3)\r\n", "Requirement already satisfied: MarkupSafe>=2.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jinja2->allensdk) (2.1.3)\r\n", "Requirement already satisfied: contourpy>=1.0.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (1.1.1)\r\n", "Requirement already satisfied: cycler>=0.10 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (0.12.1)\r\n", "Requirement already satisfied: fonttools>=4.22.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (4.45.1)\r\n", "Requirement already satisfied: kiwisolver>=1.0.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (1.4.5)\r\n", "Requirement already satisfied: pillow>=6.2.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (10.1.0)\r\n", "Requirement already satisfied: pyparsing>=2.3.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (3.1.1)\r\n", "Requirement already satisfied: setuptools in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pynwb->allensdk) (56.0.0)\r\n", "Requirement already satisfied: nptyping in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pynrrd->allensdk) (2.5.0)\r\n", "Requirement already satisfied: typing-extensions in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pynrrd->allensdk) (4.8.0)\r\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (3.3.2)\r\n", "Requirement already satisfied: idna<4,>=2.5 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (3.6)\r\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (1.26.18)\r\n", "Requirement already satisfied: certifi>=2017.4.17 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (2023.11.17)\r\n", "Requirement already satisfied: distro in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-build->allensdk) (1.8.0)\r\n", "Requirement already satisfied: tomli in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-build->allensdk) (2.0.1)\r\n", "Requirement already satisfied: wheel>=0.32.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-build->allensdk) (0.42.0)\r\n", "Requirement already satisfied: networkx>=2.8 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (3.1)\r\n", "Requirement already satisfied: imageio>=2.27 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (2.33.0)\r\n", "Requirement already satisfied: tifffile>=2022.8.12 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (2023.7.10)\r\n", "Requirement already satisfied: PyWavelets>=1.1.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (1.4.1)\r\n", "Requirement already satisfied: lazy_loader>=0.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (0.3)\r\n", "Requirement already satisfied: greenlet!=0.4.17 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from sqlalchemy->allensdk) (3.0.1)\r\n", "Requirement already satisfied: patsy>=0.5.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from statsmodels->allensdk) (0.5.3)\r\n", "Requirement already satisfied: cython>=0.29.21 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (3.0.6)\r\n", "Requirement already satisfied: numexpr>=2.6.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (2.8.6)\r\n", "Requirement already satisfied: blosc2~=2.0.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (2.0.0)\r\n", "Requirement already satisfied: py-cpuinfo in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (9.0.0)\r\n", "Requirement already satisfied: msgpack in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from blosc2~=2.0.0->tables->allensdk) (1.0.7)\r\n", "Requirement already satisfied: zipp>=3.1.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from importlib-resources->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (3.17.0)\r\n", "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (2023.11.1)\r\n", "Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (1.3.10)\r\n", "Requirement already satisfied: referencing>=0.28.4 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.31.1)\r\n", "Requirement already satisfied: rpds-py>=0.7.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.13.2)\r\n", "Requirement already satisfied: ruamel.yaml.clib>=0.2.7 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from ruamel-yaml>=0.16->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.2.8)\r\n" ] } ], "source": [ "!pip install allensdk" ] }, { "cell_type": "markdown", "id": "05e83dd7", "metadata": { "papermill": { "duration": 0.008992, "end_time": "2023-11-30T06:07:52.970712", "exception": false, "start_time": "2023-11-30T06:07:52.961720", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "## Install AllenSDK into your notebook environment (good for Google Colab)" ] }, { "cell_type": "markdown", "id": "ff3f2f34", "metadata": { "papermill": { "duration": 0.008945, "end_time": "2023-11-30T06:07:52.988588", "exception": false, "start_time": "2023-11-30T06:07:52.979643", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "You can install AllenSDK into your notebook environment by executing the cell below.\n", "\n", "If using Google Colab, click on the RESTART RUNTIME button that appears at the end of the output when this cell is complete,. Note that running this cell will produce a long list of outputs and some error messages. Clicking RESTART RUNTIME at the end will resolve these issues.\n", "You can minimize the cell after you are done to hide the output." ] }, { "cell_type": "code", "execution_count": 2, "id": "03a52ea5", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:07:53.008012Z", "iopub.status.busy": "2023-11-30T06:07:53.007432Z", "iopub.status.idle": "2023-11-30T06:07:56.799491Z", "shell.execute_reply": "2023-11-30T06:07:56.798722Z" }, "papermill": { "duration": 3.803832, "end_time": "2023-11-30T06:07:56.801197", "exception": false, "start_time": "2023-11-30T06:07:52.997365", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: pip in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (23.3.1)\r\n", "Requirement already satisfied: allensdk in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (2.16.2)\r\n", "Requirement already satisfied: psycopg2-binary in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.9.9)\r\n", "Requirement already satisfied: hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.* in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.11.0)\r\n", "Requirement already satisfied: h5py in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.10.0)\r\n", "Requirement already satisfied: matplotlib in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.7.4)\r\n", "Requirement already satisfied: numpy<1.24 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.23.5)\r\n", "Requirement already satisfied: pandas==1.5.3 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.5.3)\r\n", "Requirement already satisfied: jinja2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.1.2)\r\n", "Requirement already satisfied: scipy<1.11 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.10.1)\r\n", "Requirement already satisfied: six in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.16.0)\r\n", "Requirement already satisfied: pynrrd in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.0.0)\r\n", "Requirement already satisfied: future in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.18.3)\r\n", "Requirement already satisfied: requests in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.31.0)\r\n", "Requirement already satisfied: requests-toolbelt in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.0.0)\r\n", "Requirement already satisfied: simplejson in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.19.2)\r\n", "Requirement already satisfied: scikit-image in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.21.0)\r\n", "Requirement already satisfied: scikit-build in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.17.6)\r\n", "Requirement already satisfied: statsmodels in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.14.0)\r\n", "Requirement already satisfied: simpleitk in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.3.1)\r\n", "Requirement already satisfied: argschema in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.0.4)\r\n", "Requirement already satisfied: glymur in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.12.9.post1)\r\n", "Requirement already satisfied: xarray<2023.2.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2023.1.0)\r\n", "Requirement already satisfied: pynwb in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.5.0)\r\n", "Requirement already satisfied: tables in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.8.0)\r\n", "Requirement already satisfied: seaborn in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.13.0)\r\n", "Requirement already satisfied: aiohttp in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.9.1)\r\n", "Requirement already satisfied: nest-asyncio in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.5.8)\r\n", "Requirement already satisfied: tqdm in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (4.66.1)\r\n", "Requirement already satisfied: ndx-events in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (0.2.0)\r\n", "Requirement already satisfied: boto3 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (1.33.4)\r\n", "Requirement already satisfied: semver in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (3.0.2)\r\n", "Requirement already satisfied: cachetools in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (5.3.2)\r\n", "Requirement already satisfied: sqlalchemy in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.0.23)\r\n", "Requirement already satisfied: python-dateutil in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from allensdk) (2.8.2)\r\n", "Requirement already satisfied: pytz>=2020.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pandas==1.5.3->allensdk) (2023.3.post1)\r\n", "Requirement already satisfied: importlib-resources in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (6.1.1)\r\n", "Requirement already satisfied: jsonschema>=2.6.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (4.20.0)\r\n", "Requirement already satisfied: ruamel-yaml>=0.16 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.18.5)\r\n", "Requirement already satisfied: packaging>=21.3 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from xarray<2023.2.0->allensdk) (23.2)\r\n", "Requirement already satisfied: attrs>=17.3.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (23.1.0)\r\n", "Requirement already satisfied: multidict<7.0,>=4.5 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (6.0.4)\r\n", "Requirement already satisfied: yarl<2.0,>=1.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (1.9.3)\r\n", "Requirement already satisfied: frozenlist>=1.1.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (1.4.0)\r\n", "Requirement already satisfied: aiosignal>=1.1.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (1.3.1)\r\n", "Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from aiohttp->allensdk) (4.0.3)\r\n", "Requirement already satisfied: marshmallow<4.0,>=3.0.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from argschema->allensdk) (3.20.1)\r\n", "Requirement already satisfied: pyyaml in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from argschema->allensdk) (6.0.1)\r\n", "Requirement already satisfied: botocore<1.34.0,>=1.33.4 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from boto3->allensdk) (1.33.4)\r\n", "Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from boto3->allensdk) (1.0.1)\r\n", "Requirement already satisfied: s3transfer<0.9.0,>=0.8.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from boto3->allensdk) (0.8.2)\r\n", "Requirement already satisfied: lxml in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from glymur->allensdk) (4.9.3)\r\n", "Requirement already satisfied: MarkupSafe>=2.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jinja2->allensdk) (2.1.3)\r\n", "Requirement already satisfied: contourpy>=1.0.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (1.1.1)\r\n", "Requirement already satisfied: cycler>=0.10 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (0.12.1)\r\n", "Requirement already satisfied: fonttools>=4.22.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (4.45.1)\r\n", "Requirement already satisfied: kiwisolver>=1.0.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (1.4.5)\r\n", "Requirement already satisfied: pillow>=6.2.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (10.1.0)\r\n", "Requirement already satisfied: pyparsing>=2.3.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from matplotlib->allensdk) (3.1.1)\r\n", "Requirement already satisfied: setuptools in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pynwb->allensdk) (56.0.0)\r\n", "Requirement already satisfied: nptyping in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pynrrd->allensdk) (2.5.0)\r\n", "Requirement already satisfied: typing-extensions in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from pynrrd->allensdk) (4.8.0)\r\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (3.3.2)\r\n", "Requirement already satisfied: idna<4,>=2.5 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (3.6)\r\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (1.26.18)\r\n", "Requirement already satisfied: certifi>=2017.4.17 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from requests->allensdk) (2023.11.17)\r\n", "Requirement already satisfied: distro in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-build->allensdk) (1.8.0)\r\n", "Requirement already satisfied: tomli in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-build->allensdk) (2.0.1)\r\n", "Requirement already satisfied: wheel>=0.32.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-build->allensdk) (0.42.0)\r\n", "Requirement already satisfied: networkx>=2.8 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (3.1)\r\n", "Requirement already satisfied: imageio>=2.27 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (2.33.0)\r\n", "Requirement already satisfied: tifffile>=2022.8.12 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (2023.7.10)\r\n", "Requirement already satisfied: PyWavelets>=1.1.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (1.4.1)\r\n", "Requirement already satisfied: lazy_loader>=0.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from scikit-image->allensdk) (0.3)\r\n", "Requirement already satisfied: greenlet!=0.4.17 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from sqlalchemy->allensdk) (3.0.1)\r\n", "Requirement already satisfied: patsy>=0.5.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from statsmodels->allensdk) (0.5.3)\r\n", "Requirement already satisfied: cython>=0.29.21 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (3.0.6)\r\n", "Requirement already satisfied: numexpr>=2.6.2 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (2.8.6)\r\n", "Requirement already satisfied: blosc2~=2.0.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (2.0.0)\r\n", "Requirement already satisfied: py-cpuinfo in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from tables->allensdk) (9.0.0)\r\n", "Requirement already satisfied: msgpack in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from blosc2~=2.0.0->tables->allensdk) (1.0.7)\r\n", "Requirement already satisfied: zipp>=3.1.0 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from importlib-resources->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (3.17.0)\r\n", "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (2023.11.1)\r\n", "Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (1.3.10)\r\n", "Requirement already satisfied: referencing>=0.28.4 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.31.1)\r\n", "Requirement already satisfied: rpds-py>=0.7.1 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from jsonschema>=2.6.0->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.13.2)\r\n", "Requirement already satisfied: ruamel.yaml.clib>=0.2.7 in /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages (from ruamel-yaml>=0.16->hdmf!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*->allensdk) (0.2.8)\r\n" ] } ], "source": [ "!pip install --upgrade pip\n", "!pip install allensdk" ] }, { "cell_type": "markdown", "id": "b26fd439", "metadata": { "papermill": { "duration": 0.009539, "end_time": "2023-11-30T06:07:56.820822", "exception": false, "start_time": "2023-11-30T06:07:56.811283", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "## Import required packages" ] }, { "cell_type": "code", "execution_count": 3, "id": "396e1034", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:07:56.841158Z", "iopub.status.busy": "2023-11-30T06:07:56.840862Z", "iopub.status.idle": "2023-11-30T06:08:01.676763Z", "shell.execute_reply": "2023-11-30T06:08:01.676103Z" }, "papermill": { "duration": 4.847863, "end_time": "2023-11-30T06:08:01.678185", "exception": false, "start_time": "2023-11-30T06:07:56.830322", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n", " from .autonotebook import tqdm as notebook_tqdm\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Your allensdk version is: 2.16.2\n" ] } ], "source": [ "from pathlib import Path\n", "import matplotlib.pyplot as plt\n", "\n", "import allensdk\n", "from allensdk.brain_observatory.behavior.behavior_project_cache import VisualBehaviorOphysProjectCache\n", "\n", "# Confirming your allensdk version\n", "print(f\"Your allensdk version is: {allensdk.__version__}\")" ] }, { "cell_type": "code", "execution_count": 4, "id": "781412bf", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:08:01.699452Z", "iopub.status.busy": "2023-11-30T06:08:01.699018Z", "iopub.status.idle": "2023-11-30T06:08:01.702521Z", "shell.execute_reply": "2023-11-30T06:08:01.701964Z" }, "papermill": { "duration": 0.01549, "end_time": "2023-11-30T06:08:01.703832", "exception": false, "start_time": "2023-11-30T06:08:01.688342", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [ "parameters" ] }, "outputs": [], "source": [ "# Update this to a valid directory in your filesystem\n", "# Remember to choose a location that has plenty of free space available.\n", "output_dir = \"/local1/visual_behavior_ophys_cache_dir\"\n", "DOWNLOAD_COMPLETE_DATASET = True " ] }, { "cell_type": "code", "execution_count": 6, "id": "22738ce0", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:08:01.749802Z", "iopub.status.busy": "2023-11-30T06:08:01.749338Z", "iopub.status.idle": "2023-11-30T06:08:04.929033Z", "shell.execute_reply": "2023-11-30T06:08:04.928292Z" }, "papermill": { "duration": 3.191765, "end_time": "2023-11-30T06:08:04.930417", "exception": false, "start_time": "2023-11-30T06:08:01.738652", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/runner/work/AllenSDK/AllenSDK/allensdk/api/cloud_cache/cloud_cache.py:439: MissingLocalManifestWarning: This cache directory appears to contain data files, but it has no record of what those files are. You might want to consider running\n", "\n", "VisualBehaviorOphysProjectCache.construct_local_manifest()\n", "\n", "to avoid needlessly downloading duplicates of data files that did not change between data releases. NOTE: running this method will require hashing every data file you have currently downloaded and could be very time consuming.\n", "\n", "To avoid this warning in the future, make sure that\n", "\n", "/tmp/tmpvwwlk2_m/_downloaded_data.json\n", "\n", "is not deleted between instantiations of this cache\n", " warnings.warn(msg, MissingLocalManifestWarning)\n", "ophys_session_table.csv: 100%|██████████| 247k/247k [00:00<00:00, 1.57MMB/s] \n", "behavior_session_table.csv: 100%|██████████| 1.59M/1.59M [00:00<00:00, 8.87MMB/s]\n", "ophys_experiment_table.csv: 100%|██████████| 657k/657k [00:00<00:00, 3.66MMB/s] \n", "ophys_cells_table.csv: 100%|██████████| 4.28M/4.28M [00:00<00:00, 11.0MMB/s]\n", "/home/runner/work/AllenSDK/AllenSDK/allensdk/brain_observatory/behavior/behavior_project_cache/behavior_project_cache.py:135: UpdatedStimulusPresentationTableWarning: \n", "\tAs of AllenSDK version 2.16.0, the latest Visual Behavior Ophys data has been significantly updated from previous releases. Specifically the user will need to update all processing of the stimulus_presentations tables. These tables now include multiple stimulus types delineated by the columns `stimulus_block` and `stimulus_block_name`.\n", "\n", "The data that was available in previous releases are stored in the block name containing 'change_detection' and can be accessed in the pandas table by using: \n", "\t`stimulus_presentations[stimulus_presentations.stimulus_block_name.str.contains('change_detection')]`\n", " warnings.warn(\n" ] } ], "source": [ "output_dir = Path(output_dir)\n", "\n", "cache = VisualBehaviorOphysProjectCache.from_s3_cache(cache_dir=output_dir)" ] }, { "cell_type": "markdown", "id": "789b2c29", "metadata": { "papermill": { "duration": 0.010418, "end_time": "2023-11-30T06:08:04.951897", "exception": false, "start_time": "2023-11-30T06:08:04.941479", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "Instantiating the cache will have it to download 3 project metadata files:\n", "\n", "1. `behavior_session_table.csv` (879 kB)\n", "2. `ophys_session_table.csv` (165.1 kB)\n", "3. `ophys_experiment_table.csv` (335.6 kB)\n", "\n", "Each one contains a table of information related to its file name. If you're using the AllenSDK, you won't have to worry about how these files are formatted. Instead, you'll load the relevant data using specific accessor method: `get_behavior_session_table()`, `get_ophys_session_table()`, and `get_ophys_experiment_table()`. These functions return a pandas DataFrame containing a row for each item and a column for each metric.\n", "\n", "If you are analyzing data without using the AllenSDK, you can load the data using your CSV file reader of choice. However, please be aware the columns in the original file do not necessarily match what's returned by the AllenSDK, which may combine information from multiple files to produce the final DataFrame." ] }, { "cell_type": "markdown", "id": "2c174b0a", "metadata": { "papermill": { "duration": 0.010248, "end_time": "2023-11-30T06:08:04.972553", "exception": false, "start_time": "2023-11-30T06:08:04.962305", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "### Managing versions of the dataset\n", "\n", "Over time, updates may be made to the released dataset. These updates will result in new versions of the dataset being available in the S3 bucket. The versions of the dataset are managed through distinct data manifests stored on S3." ] }, { "cell_type": "markdown", "id": "df8231e7", "metadata": { "papermill": { "duration": 0.010383, "end_time": "2023-11-30T06:08:04.993345", "exception": false, "start_time": "2023-11-30T06:08:04.982962", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "#### Discovering manifests\n", "\n", "To see all of the manifest files available for this dataset online, run" ] }, { "cell_type": "code", "execution_count": 7, "id": "90f5ce47", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:08:05.015899Z", "iopub.status.busy": "2023-11-30T06:08:05.015313Z", "iopub.status.idle": "2023-11-30T06:08:05.022146Z", "shell.execute_reply": "2023-11-30T06:08:05.021467Z" }, "papermill": { "duration": 0.019802, "end_time": "2023-11-30T06:08:05.023555", "exception": false, "start_time": "2023-11-30T06:08:05.003753", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "data": { "text/plain": [ "['visual-behavior-ophys_project_manifest_v0.1.0.json',\n", " 'visual-behavior-ophys_project_manifest_v0.2.0.json',\n", " 'visual-behavior-ophys_project_manifest_v0.3.0.json',\n", " 'visual-behavior-ophys_project_manifest_v1.0.0.json',\n", " 'visual-behavior-ophys_project_manifest_v1.0.1.json',\n", " 'visual-behavior-ophys_project_manifest_v1.1.0.json']" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cache.list_manifest_file_names()" ] }, { "cell_type": "markdown", "id": "58652284", "metadata": { "papermill": { "duration": 0.01038, "end_time": "2023-11-30T06:08:05.044462", "exception": false, "start_time": "2023-11-30T06:08:05.034082", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "To see the most up-to-date available manifest, run" ] }, { "cell_type": "code", "execution_count": 8, "id": "376665f9", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:08:05.066760Z", "iopub.status.busy": "2023-11-30T06:08:05.066254Z", "iopub.status.idle": "2023-11-30T06:08:05.070647Z", "shell.execute_reply": "2023-11-30T06:08:05.069995Z" }, "papermill": { "duration": 0.016917, "end_time": "2023-11-30T06:08:05.071973", "exception": false, "start_time": "2023-11-30T06:08:05.055056", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "data": { "text/plain": [ "'visual-behavior-ophys_project_manifest_v1.1.0.json'" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cache.latest_manifest_file()" ] }, { "cell_type": "markdown", "id": "3ea84f59", "metadata": { "papermill": { "duration": 0.010406, "end_time": "2023-11-30T06:08:05.092994", "exception": false, "start_time": "2023-11-30T06:08:05.082588", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "To see the name of the most up-to-date manifest that you have already downloaded to your system run (note: this just means that the manifest file has been downloaded; it does not necessarily mean that any data has been downloaded)" ] }, { "cell_type": "code", "execution_count": 9, "id": "a546e49f", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:08:05.115184Z", "iopub.status.busy": "2023-11-30T06:08:05.114761Z", "iopub.status.idle": "2023-11-30T06:08:05.119164Z", "shell.execute_reply": "2023-11-30T06:08:05.118638Z" }, "papermill": { "duration": 0.016976, "end_time": "2023-11-30T06:08:05.120442", "exception": false, "start_time": "2023-11-30T06:08:05.103466", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "data": { "text/plain": [ "'visual-behavior-ophys_project_manifest_v1.1.0.json'" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cache.latest_downloaded_manifest_file()" ] }, { "cell_type": "markdown", "id": "3659c8d0", "metadata": { "papermill": { "duration": 0.010461, "end_time": "2023-11-30T06:08:05.141569", "exception": false, "start_time": "2023-11-30T06:08:05.131108", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "You can list all of the manifest files currently downloaded to your system with" ] }, { "cell_type": "code", "execution_count": 10, "id": "27a9e239", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:08:05.163986Z", "iopub.status.busy": "2023-11-30T06:08:05.163595Z", "iopub.status.idle": "2023-11-30T06:08:05.167950Z", "shell.execute_reply": "2023-11-30T06:08:05.167320Z" }, "papermill": { "duration": 0.01707, "end_time": "2023-11-30T06:08:05.169302", "exception": false, "start_time": "2023-11-30T06:08:05.152232", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "data": { "text/plain": [ "['visual-behavior-ophys_project_manifest_v1.1.0.json']" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cache.list_all_downloaded_manifests()" ] }, { "cell_type": "markdown", "id": "533ed358", "metadata": { "papermill": { "duration": 0.010686, "end_time": "2023-11-30T06:08:05.190817", "exception": false, "start_time": "2023-11-30T06:08:05.180131", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "#### Loading manifests/dataset versions\n", "\n", "The `VisualBehaviorOphysProjectCache` determines which version of the dataset to use by loading one of these manifests. By default, the `VisualBehaviorProjectCache` loads either\n", "\n", "- the most up-to-date available data manifest, if you are instaniating it on an empty `cache_dir`\n", "\n", "- the data manifest you were last using, if you are instantiating it on a pre-existing `cache_dir` (in this case, the `VisualBehaviorOphysProjectCache` will emit a warning if a more up-to-data data manifest exists online letting you know that you can, if you choose, move to the more up-to-date data manifest)\n", "\n", "To see the manifest that you currently have loaded, run" ] }, { "cell_type": "code", "execution_count": 11, "id": "1e63398b", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:08:05.213214Z", "iopub.status.busy": "2023-11-30T06:08:05.212814Z", "iopub.status.idle": "2023-11-30T06:08:05.217074Z", "shell.execute_reply": "2023-11-30T06:08:05.216426Z" }, "papermill": { "duration": 0.017192, "end_time": "2023-11-30T06:08:05.218608", "exception": false, "start_time": "2023-11-30T06:08:05.201416", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "data": { "text/plain": [ "'visual-behavior-ophys_project_manifest_v1.1.0.json'" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cache.current_manifest()" ] }, { "cell_type": "markdown", "id": "6e94f4ce", "metadata": { "papermill": { "duration": 0.010633, "end_time": "2023-11-30T06:08:05.240010", "exception": false, "start_time": "2023-11-30T06:08:05.229377", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "To load a particular data manifest by hand, run (note: because we are intentionally loading an out-of-date manifest, this will emit an error alerting us to the existence of the most up-to-date manifest). We then reload the latest manifest." ] }, { "cell_type": "code", "execution_count": 12, "id": "17232af7", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:08:05.262582Z", "iopub.status.busy": "2023-11-30T06:08:05.262046Z", "iopub.status.idle": "2023-11-30T06:08:05.702258Z", "shell.execute_reply": "2023-11-30T06:08:05.701497Z" }, "papermill": { "duration": 0.453105, "end_time": "2023-11-30T06:08:05.703806", "exception": false, "start_time": "2023-11-30T06:08:05.250701", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/runner/work/AllenSDK/AllenSDK/allensdk/api/cloud_cache/cloud_cache.py:492: OutdatedManifestWarning: \n", "\n", "The manifest file you are loading is not the most up to date manifest file available for this dataset. The most up to data manifest file available for this dataset is \n", "\n", "visual-behavior-ophys_project_manifest_v1.1.0.json\n", "\n", "To see the differences between these manifests,run\n", "\n", "VisualBehaviorOphysProjectCache.compare_manifests('visual-behavior-ophys_project_manifest_v0.1.0.json', 'visual-behavior-ophys_project_manifest_v1.1.0.json')\n", "\n", "To see all of the manifest files currently downloaded onto your local system, run\n", "\n", "self.list_all_downloaded_manifests()\n", "\n", "If you just want to load the latest manifest, run\n", "\n", "self.load_latest_manifest()\n", "\n", "\n", " warnings.warn(msg, OutdatedManifestWarning)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "the manifest has manifest_version 0.1.0 but this version of AllenSDK is compatible only with manifest versions 1.0.0 <= X < 2.0.0. \n", "Consider using a version of AllenSDK closer to the version used to release the data: 2.10.0\n" ] } ], "source": [ "from allensdk.brain_observatory.behavior.behavior_project_cache.utils import \\\n", " BehaviorCloudCacheVersionException\n", "\n", "try:\n", " cache.load_manifest('visual-behavior-ophys_project_manifest_v0.1.0.json')\n", "except BehaviorCloudCacheVersionException as e:\n", " print(e)\n", " cache.load_manifest(cache.latest_manifest_file())" ] }, { "cell_type": "code", "execution_count": 13, "id": "e945c2da", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:08:05.727653Z", "iopub.status.busy": "2023-11-30T06:08:05.727216Z", "iopub.status.idle": "2023-11-30T06:08:05.731629Z", "shell.execute_reply": "2023-11-30T06:08:05.730975Z" }, "papermill": { "duration": 0.017571, "end_time": "2023-11-30T06:08:05.732981", "exception": false, "start_time": "2023-11-30T06:08:05.715410", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "data": { "text/plain": [ "'visual-behavior-ophys_project_manifest_v1.1.0.json'" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cache.current_manifest()" ] }, { "cell_type": "markdown", "id": "3c7a1db1", "metadata": { "papermill": { "duration": 0.010993, "end_time": "2023-11-30T06:08:05.755035", "exception": false, "start_time": "2023-11-30T06:08:05.744042", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "As the earlier warning informed us, we can see the difference between an two versions of the dataset by running" ] }, { "cell_type": "code", "execution_count": 14, "id": "2c109ecc", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:08:05.778449Z", "iopub.status.busy": "2023-11-30T06:08:05.778189Z", "iopub.status.idle": "2023-11-30T06:08:06.170272Z", "shell.execute_reply": "2023-11-30T06:08:06.169605Z" }, "papermill": { "duration": 0.405504, "end_time": "2023-11-30T06:08:06.171591", "exception": false, "start_time": "2023-11-30T06:08:05.766087", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Changes going from\n", "visual-behavior-ophys_project_manifest_v0.1.0.json\n", "to\n", "visual-behavior-ophys_project_manifest_v0.2.0.json\n", "\n", "project_metadata/behavior_session_table.csv changed\n", "project_metadata/ophys_experiment_table.csv changed\n", "project_metadata/ophys_session_table.csv changed\n", "\n" ] } ], "source": [ "msg = cache.compare_manifests('visual-behavior-ophys_project_manifest_v0.1.0.json',\n", " 'visual-behavior-ophys_project_manifest_v0.2.0.json')\n", "print(msg)" ] }, { "cell_type": "markdown", "id": "23787efa", "metadata": { "papermill": { "duration": 0.011041, "end_time": "2023-11-30T06:08:06.194536", "exception": false, "start_time": "2023-11-30T06:08:06.183495", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "In the case we just examined, only the metadata files have changed.\n", "\n", "The `VisualBehaviorOphysProjectCache` is smart enough to know that, if a file has not changed between version `A` and version `B` of the dataset, and you have already downloaded the file while version `A` of the manifest was loaded, when you move to version `B`, it does not need to download the data again. It will simply construct a symlink where version `B` of the data should exist on your system, pointing to version `A` of the file.\n", "\n", "Because only metadata files changed between `v0.1.0` and `v0.2.0` of the dataset, we could move freely between the two versions without having to worry about downloading a bunch of new data files. This may not be the case for future dataset updates, so you should keep that in mind before moving from an older to a newer version out of hand." ] }, { "cell_type": "markdown", "id": "3796bdf0", "metadata": { "papermill": { "duration": 0.011184, "end_time": "2023-11-30T06:08:06.216815", "exception": false, "start_time": "2023-11-30T06:08:06.205631", "status": "completed" }, "pycharm": { "name": "#%% md\n" }, "tags": [] }, "source": [ "### Using the AllenSDK to access Visual Behavior Ophys metadata\n", "\n", "Let's take a closer look at what's in the `behavior_session_table.csv` file:" ] }, { "cell_type": "code", "execution_count": 15, "id": "5f22ddda", "metadata": { "execution": { "iopub.execute_input": "2023-11-30T06:08:06.240814Z", "iopub.status.busy": "2023-11-30T06:08:06.240248Z", "iopub.status.idle": "2023-11-30T06:08:06.258659Z", "shell.execute_reply": "2023-11-30T06:08:06.257988Z" }, "papermill": { "duration": 0.032041, "end_time": "2023-11-30T06:08:06.260049", "exception": false, "start_time": "2023-11-30T06:08:06.228008", "status": "completed" }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Total number of behavior sessions: 4782\n" ] }, { "data": { "text/html": [ "
\n", " | ophys_session_id | \n", "ophys_container_id | \n", "mouse_id | \n", "indicator | \n", "full_genotype | \n", "driver_line | \n", "cre_line | \n", "reporter_line | \n", "sex | \n", "age_in_days | \n", "... | \n", "num_targeted_structures | \n", "catch_trial_count | \n", "engaged_trial_count | \n", "correct_reject_trial_count | \n", "trial_count | \n", "miss_trial_count | \n", "go_trial_count | \n", "file_id | \n", "false_alarm_trial_count | \n", "hit_trial_count | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
behavior_session_id | \n", "\n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " |
870987812 | \n", "<NA> | \n", "NaN | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "80 | \n", "... | \n", "<NA> | \n", "0 | \n", "70 | \n", "0 | \n", "121 | \n", "0 | \n", "0 | \n", "1941 | \n", "0 | \n", "0 | \n", "
872953842 | \n", "<NA> | \n", "NaN | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "84 | \n", "... | \n", "<NA> | \n", "49 | \n", "27 | \n", "45 | \n", "564 | \n", "275 | \n", "293 | \n", "1942 | \n", "4 | \n", "18 | \n", "
873774125 | \n", "<NA> | \n", "NaN | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "85 | \n", "... | \n", "<NA> | \n", "41 | \n", "0 | \n", "38 | \n", "548 | \n", "300 | \n", "314 | \n", "1943 | \n", "3 | \n", "14 | \n", "
874790261 | \n", "<NA> | \n", "NaN | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "86 | \n", "... | \n", "<NA> | \n", "42 | \n", "0 | \n", "42 | \n", "492 | \n", "312 | \n", "326 | \n", "1944 | \n", "0 | \n", "14 | \n", "
875207834 | \n", "<NA> | \n", "NaN | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "87 | \n", "... | \n", "<NA> | \n", "44 | \n", "5 | \n", "41 | \n", "504 | \n", "300 | \n", "322 | \n", "1945 | \n", "3 | \n", "22 | \n", "
5 rows × 34 columns
\n", "\n", " | behavior_session_id | \n", "ophys_container_id | \n", "mouse_id | \n", "indicator | \n", "full_genotype | \n", "driver_line | \n", "cre_line | \n", "reporter_line | \n", "sex | \n", "age_in_days | \n", "... | \n", "behavior_type | \n", "experience_level | \n", "prior_exposures_to_session_type | \n", "prior_exposures_to_image_set | \n", "prior_exposures_to_omissions | \n", "date_of_acquisition | \n", "equipment_name | \n", "num_depths_per_area | \n", "ophys_experiment_id | \n", "num_targeted_structures | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ophys_session_id | \n", "\n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " |
951410079 | \n", "951520319 | \n", "[1018028339, 1018028342, 1018028345, 101802835... | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "206 | \n", "... | \n", "active_behavior | \n", "Familiar | \n", "0 | \n", "65 | \n", "0 | \n", "2019-09-20 09:59:38.837000+00:00 | \n", "MESO.1 | \n", "4 | \n", "[951980471, 951980473, 951980475, 951980479, 9... | \n", "2 | \n", "
952430817 | \n", "952554548 | \n", "[1018028339, 1018028345, 1018028354, 1018028357] | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "209 | \n", "... | \n", "passive_viewing | \n", "Familiar | \n", "0 | \n", "66 | \n", "1 | \n", "2019-09-23 08:45:38.490000+00:00 | \n", "MESO.1 | \n", "4 | \n", "[953659743, 953659745, 953659749, 953659752] | \n", "2 | \n", "
954954402 | \n", "953982960 | \n", "[1018028339, 1018028342, 1018028345, 101802835... | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "210 | \n", "... | \n", "active_behavior | \n", "Familiar | \n", "0 | \n", "67 | \n", "2 | \n", "2019-09-24 09:01:31.582000+00:00 | \n", "MESO.1 | \n", "4 | \n", "[958527464, 958527471, 958527474, 958527479, 9... | \n", "2 | \n", "
955775716 | \n", "956010809 | \n", "[1018028339, 1018028342, 1018028345] | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "212 | \n", "... | \n", "active_behavior | \n", "Familiar | \n", "1 | \n", "68 | \n", "3 | \n", "2019-09-26 09:22:21.772000+00:00 | \n", "MESO.1 | \n", "4 | \n", "[956941841, 956941844, 956941846] | \n", "2 | \n", "
957020350 | \n", "957032492 | \n", "[1018028339, 1018028342, 1018028345, 101802835... | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "213 | \n", "... | \n", "active_behavior | \n", "Novel 1 | \n", "0 | \n", "0 | \n", "4 | \n", "2019-09-27 08:58:37.005000+00:00 | \n", "MESO.1 | \n", "4 | \n", "[957759562, 957759564, 957759566, 957759570, 9... | \n", "2 | \n", "
5 rows × 25 columns
\n", "\n", " | behavior_session_id | \n", "ophys_session_id | \n", "ophys_container_id | \n", "mouse_id | \n", "indicator | \n", "full_genotype | \n", "driver_line | \n", "cre_line | \n", "reporter_line | \n", "sex | \n", "... | \n", "passive | \n", "experience_level | \n", "prior_exposures_to_session_type | \n", "prior_exposures_to_image_set | \n", "prior_exposures_to_omissions | \n", "date_of_acquisition | \n", "equipment_name | \n", "published_at | \n", "isi_experiment_id | \n", "file_id | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ophys_experiment_id | \n", "\n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " |
951980471 | \n", "951520319 | \n", "951410079 | \n", "1018028342 | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "... | \n", "False | \n", "Familiar | \n", "0 | \n", "65 | \n", "0 | \n", "2019-09-20 09:59:38.837000+00:00 | \n", "MESO.1 | \n", "2021-03-25 | \n", "848974280 | \n", "0 | \n", "
951980473 | \n", "951520319 | \n", "951410079 | \n", "1018028345 | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "... | \n", "False | \n", "Familiar | \n", "0 | \n", "65 | \n", "0 | \n", "2019-09-20 09:59:38.837000+00:00 | \n", "MESO.1 | \n", "2021-03-25 | \n", "848974280 | \n", "1 | \n", "
951980475 | \n", "951520319 | \n", "951410079 | \n", "1018028339 | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "... | \n", "False | \n", "Familiar | \n", "0 | \n", "65 | \n", "0 | \n", "2019-09-20 09:59:38.837000+00:00 | \n", "MESO.1 | \n", "2021-03-25 | \n", "848974280 | \n", "2 | \n", "
951980479 | \n", "951520319 | \n", "951410079 | \n", "1018028354 | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "... | \n", "False | \n", "Familiar | \n", "0 | \n", "65 | \n", "0 | \n", "2019-09-20 09:59:38.837000+00:00 | \n", "MESO.1 | \n", "2021-03-25 | \n", "848974280 | \n", "3 | \n", "
951980481 | \n", "951520319 | \n", "951410079 | \n", "1018028357 | \n", "457841 | \n", "GCaMP6f | \n", "Sst-IRES-Cre/wt;Ai148(TIT2L-GC6f-ICL-tTA2)/wt | \n", "[Sst-IRES-Cre] | \n", "Sst-IRES-Cre | \n", "Ai148(TIT2L-GC6f-ICL-tTA2) | \n", "F | \n", "... | \n", "False | \n", "Familiar | \n", "0 | \n", "65 | \n", "0 | \n", "2019-09-20 09:59:38.837000+00:00 | \n", "MESO.1 | \n", "2021-03-25 | \n", "848974280 | \n", "4 | \n", "
5 rows × 30 columns
\n", "\n", " | unwarped | \n", "warped | \n", "
---|---|---|
image_name | \n", "\n", " | \n", " |
gratings_0.0 | \n", "[[127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 12... | \n", "[[255, 255, 255, 255, 255, 255, 255, 255, 255,... | \n", "
gratings_90.0 | \n", "[[127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 12... | \n", "[[255, 255, 255, 255, 255, 255, 255, 255, 255,... | \n", "
gratings_180.0 | \n", "[[127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 12... | \n", "[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,... | \n", "
gratings_270.0 | \n", "[[127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 12... | \n", "[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,... | \n", "