Metadata-Version: 2.1
Name: npc-sessions
Version: 0.0.124
Summary: Tools and interfaces for working with behavior and epyhys sessions from the Mindscope Neuropixels team, in the cloud.
Author-email: Ben Hardcastle <ben.hardcastle@alleninstitute.org>, Arjun Sridhar <arjun.sridhar@alleninstitute.org>, Ethan McBride <ethan.mcbride@alleninstitute.org>, Corbett Bennett <corbettb@alleninstitute.org>
License: MIT
Project-URL: Repository, https://github.com/AllenInstitute/npc_sessions
Project-URL: Issues, https://github.com/AllenInstitute/npc_sessions/issues
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: npc-session>=0.1.26
Requires-Dist: npc-lims>=0.1.86
Requires-Dist: polars[all]>=0.18.14
Requires-Dist: h5py>=3.9.0
Requires-Dist: pandas[aws,excel,fss,hdf5,performance,plot]>=1.1.5
Requires-Dist: universal-pathlib>=0.1.1
Requires-Dist: s3fs>=2023.6.0
Requires-Dist: pynwb
Requires-Dist: crc32c>=2.3.post0
Requires-Dist: rich>=13.5.2
Requires-Dist: zarr>=2.16.1
Requires-Dist: wavpack-numcodecs>=0.1.4
Requires-Dist: matplotlib>=3.7.2
Requires-Dist: scipy>=1.9.3
Requires-Dist: DynamicRoutingTask>=0.1.14
Requires-Dist: numba>=0.57.1
Requires-Dist: ipywidgets>=8.1.0
Requires-Dist: ipympl>=0.9.3
Requires-Dist: opencv-python-headless>=4.8.0.76
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: ndx-events>=0.2.0
Requires-Dist: SimpleITK>=2.2.1
Requires-Dist: pyarrow>=13.0.0
Requires-Dist: spikeinterface>=0.98.2
Requires-Dist: ipykernel>=6.25.2
Requires-Dist: loky>=3.4.1
Requires-Dist: hdmf>=3.5.4
Requires-Dist: nbconvert>=7.9.2
Requires-Dist: hdmf-zarr>=0.4.0
Requires-Dist: traitlets<=5.9.0

# npc_sessions
**n**euro**p**ixels **c**loud **sessions**
	
Tools for accessing data and metadata for behavior and epyhys sessions from the
Mindscope Neuropixels team - in the cloud.

[![PyPI](https://img.shields.io/pypi/v/npc-sessions.svg?label=PyPI&color=blue)](https://pypi.org/project/npc-sessions/)
[![Python version](https://img.shields.io/pypi/pyversions/npc-sessions)](https://pypi.org/project/npc-sessions/)

[![Coverage](https://img.shields.io/codecov/c/github/alleninstitute/npc_sessions?logo=codecov)](https://app.codecov.io/github/AllenInstitute/npc_sessions)
[![CI/CD](https://img.shields.io/github/actions/workflow/status/alleninstitute/npc_sessions/publish.yml?label=CI/CD&logo=github)](https://github.com/alleninstitute/npc_sessions/actions/workflows/publish.yml)
[![GitHub issues](https://img.shields.io/github/issues/alleninstitute/npc_sessions?logo=github)](https://github.com/alleninstitute/npc_sessions/issues)


## quickstart
Make a conda environment with python>=3.9 and simply pip install the npc_sessions package:

```bash
conda create -n npc_sessions python>=3.9
conda activate npc_sessions
pip install npc_sessions
```

```python
>>> from npc_sessions import DynamicRoutingSession, get_sessions;

# each object is used to get metadata and paths for a session:         
>>> session = DynamicRoutingSession('668755_2023-08-31')  
>>> session.is_ephys                                
True
>>> session.stim_paths[0].stem                      
'DynamicRouting1_668755_20230831_131418'
# data is processed on-demand to generate individual pynwb modules:
>>> session.subject                                 # doctest: +SKIP
  subject pynwb.file.Subject at 0x1999418231888
  Fields:
    age: P205D
    age__reference: birth
    date_of_birth: 2023-02-06 20:23:02-08:00
    genotype: wt/wt
    sex: M
    species: Mus musculus
    strain: C57BL6J(NP)
    subject_id: 668755

# a full NWBFile instance can also be generated with all currently-available data:
>>> session.nwb                                     # doctest: +SKIP
root pynwb.file.NWBFile at 0x...
Fields:
  acquisition: {
    lick spout <class 'ndx_events.events.Events'>
  }
  devices: {
    18005102491 <class 'pynwb.device.Device'>,
    18005114452 <class 'pynwb.device.Device'>,
    18005123131 <class 'pynwb.device.Device'>,
    18194810652 <class 'pynwb.device.Device'>,
    19192719021 <class 'pynwb.device.Device'>,
    19192719061 <class 'pynwb.device.Device'>
  }
   ...  

# loop over all currently-tracked sessions using the session-generator:
>>> all(s.session_start_time.year >= 2022 for s in get_sessions()) # doctest: +SKIP
True
>>> trials_dfs = {}
>>> for session in get_sessions():                  # doctest: +SKIP
...     trials_dfs[session.id] = session.trials[:]

```

## to develop with conda
To install with the intention of contributing to this package:

1) create a conda environment:
```bash
conda create -n npc_sessions python>=3.9
conda activate npc_sessions
```
2) clone npc_sessions from github:
```bash
git clone git@github.com:AllenInstitute/npc_sessions.git
```
3) pip install all dependencies:
```bash
cd npc_sessions
pip install -e .
```

# Current NWB components

#### key data types

- `DynamicTable`: for general tabular data. Cells can contain arrays (e.g.
  `spike_times` in a table of units)
  - can be accessed as a pandas dataframe

- `TimeIntervals`: subclass of `DynamicTable` which always has a `start_time` and `stop_time`
column, plus other user-defined columns

- `TimeSeries`: has a vector array of `data`, a `units` string, and either:
  - `timestamps` (same length as `data`)
  - or `starting_time` and `rate`

- `ElectricalSeries`: subclass of `TimeSeries` with units fixed as volts

- `Events`: (added via an NWB extension) like `TimeSeries` with timestamps only, without values for `data` (think lick times)

all of the above also have a `description`, as well as other type-specific attributes

---
- *session metadata* (multiple attributes)
- subject (multiple attributes)
- devices: `DynamicTable`	
  - physical probes (model, serial number)
  - currently only neuropixels probes
- electrode_groups: `DynamicTable`
  - represents the group of channels on one probe inserted in the brain
  - has session-specific info, like position relative to other probes or stereotactic coords
- electrodes: `DynamicTable`
  - individual channels on a probe
  - has CCF coords
- units: `DynamicTable`
  - metrics, links to `electrodes` via `peak_channel`

- epochs: `TimeIntervals` 		
  - start/stop time of each stim block
  - has a list of tags (includes `TaskControl` subclass name)
- intervals: `Mapping[str, TimeIntervals]`
  - 1x table per stim epoch with trials
  - behavior performance table (each block an interval)
- trials: `TimeIntervals` 
  - same as `intervals[DynamicRouting1]`
- invalid_times: `TimeIntervals` 

- **acquisition**: `Mapping[str, Any]` *raw data*
  - if is_ephys:
    - raw AP: `Mapping[str, ElectricalSeries]`
    - raw LFP: `Mapping[str, ElectricalSeries]`
  - if is_sync:
    - lick_sensor_rising_edges: `Events`    
    - lick_sensor_falling_edges: `Events`   
  - if is_task:
    - rewards: `Events`
  - if is_video:   	
    - video frame times: 1x `Events` per camera
	
- **processing**: `Mapping[str, Any]` *processed/filtered data*
  - behavior: `Mapping[str, Any]`
      - licks: `Events`		
        - from sync or stim file
      - running_speed: `TimeSeries` 		
        - from stim file, enhanced with sync info if available
  - ecephys: `Mapping[str, Any]`

- **analysis**: `Mapping[str, Any]` *derived data, results*
  - if is_ephys:
    - all_spike_histograms: 1x `TimeSeries` per probe
    - drift_maps: `ImageSeries`
  - if is_task:
    - performance: `TimeIntervals`

Todo:
- video feature extraction: pupil size, gaze tracking
- video processing: facemap
- filtered LFP
- stimulus templates (vis, aud, opto)
- OptogeneticStimulusSite
- analysis -> RFMaps
- per-unit response metric for each stim modality
