Metadata-Version: 2.1
Name: ndsampler
Version: 0.7.7
Summary: Fast sampling from large images
Home-page: https://gitlab.kitware.com/computer-vision/ndsampler
Author: Jon Crall
Author-email: jon.crall@kitware.com
License: Apache 2
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Utilities
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.6
Description-Content-Type: text/x-rst
License-File: LICENSE
Requires-Dist: ubelt
Requires-Dist: parse
Requires-Dist: xarray
Requires-Dist: sortedcontainers
Requires-Dist: fasteners
Requires-Dist: atomicwrites
Requires-Dist: pyqtree
Requires-Dist: kwimage
Requires-Dist: kwarray
Requires-Dist: kwcoco
Requires-Dist: numpy ; python_version < "3.10" and python_version >= "3.6"
Requires-Dist: Pillow ; python_version < "3.10" and python_version >= "3.6"
Requires-Dist: scipy ; python_version < "3.10" and python_version >= "3.9"
Requires-Dist: scikit-learn ; python_version < "3.10" and python_version >= "3.9"
Requires-Dist: numpy ; python_version < "3.11" and python_version >= "3.10"
Requires-Dist: scipy ; python_version < "3.11" and python_version >= "3.10"
Requires-Dist: scipy ; python_version < "3.7" and python_version >= "3.6"
Requires-Dist: scikit-learn ; python_version < "3.7" and python_version >= "3.6"
Requires-Dist: networkx ; python_version < "3.7.0" and python_version >= "3.6.0"
Requires-Dist: networkx ; python_version < "3.8" and python_version >= "3.7"
Requires-Dist: scipy ; python_version < "3.8" and python_version >= "3.7"
Requires-Dist: scikit-learn ; python_version < "3.8" and python_version >= "3.7"
Requires-Dist: scipy ; python_version < "3.9" and python_version >= "3.8"
Requires-Dist: scikit-learn ; python_version < "3.9" and python_version >= "3.8"
Requires-Dist: numpy ; python_version < "4.0" and python_version >= "3.11"
Requires-Dist: scipy ; python_version < "4.0" and python_version >= "3.11"
Requires-Dist: scikit-learn ; python_version >= "3.10"
Requires-Dist: Pillow ; python_version >= "3.10"
Requires-Dist: networkx ; python_version >= "3.8"
Provides-Extra: all
Requires-Dist: ubelt ; extra == 'all'
Requires-Dist: parse ; extra == 'all'
Requires-Dist: xarray ; extra == 'all'
Requires-Dist: sortedcontainers ; extra == 'all'
Requires-Dist: fasteners ; extra == 'all'
Requires-Dist: atomicwrites ; extra == 'all'
Requires-Dist: pyqtree ; extra == 'all'
Requires-Dist: kwimage ; extra == 'all'
Requires-Dist: kwarray ; extra == 'all'
Requires-Dist: kwcoco ; extra == 'all'
Requires-Dist: pytest ; extra == 'all'
Requires-Dist: coverage ; extra == 'all'
Requires-Dist: xdoctest ; extra == 'all'
Requires-Dist: kwplot ; extra == 'all'
Requires-Dist: lru-dict ; extra == 'all'
Provides-Extra: all-strict
Requires-Dist: ubelt ==1.3.3 ; extra == 'all-strict'
Requires-Dist: parse ==1.14.0 ; extra == 'all-strict'
Requires-Dist: xarray ==0.16.0 ; extra == 'all-strict'
Requires-Dist: sortedcontainers ==2.3.0 ; extra == 'all-strict'
Requires-Dist: fasteners ==0.14.1 ; extra == 'all-strict'
Requires-Dist: atomicwrites ==1.3.0 ; extra == 'all-strict'
Requires-Dist: pyqtree ==1.0.0 ; extra == 'all-strict'
Requires-Dist: kwimage ==0.9.19 ; extra == 'all-strict'
Requires-Dist: kwarray ==0.6.7 ; extra == 'all-strict'
Requires-Dist: kwcoco ==0.5.6 ; extra == 'all-strict'
Requires-Dist: pytest ==6.2.4 ; extra == 'all-strict'
Requires-Dist: coverage ==5.2.1 ; extra == 'all-strict'
Requires-Dist: xdoctest ==1.1.2 ; extra == 'all-strict'
Requires-Dist: kwplot ==0.4.10 ; extra == 'all-strict'
Requires-Dist: lru-dict ==1.1.7 ; extra == 'all-strict'
Requires-Dist: numpy ==1.19.3 ; (python_version < "3.10" and python_version >= "3.6") and extra == 'all-strict'
Requires-Dist: Pillow ==8.0.0 ; (python_version < "3.10" and python_version >= "3.6") and extra == 'all-strict'
Requires-Dist: scipy ==1.8.0 ; (python_version < "3.10" and python_version >= "3.9") and extra == 'all-strict'
Requires-Dist: scikit-learn ==1.0.2 ; (python_version < "3.10" and python_version >= "3.9") and extra == 'all-strict'
Requires-Dist: numpy ==1.21.6 ; (python_version < "3.11" and python_version >= "3.10") and extra == 'all-strict'
Requires-Dist: scipy ==1.8.0 ; (python_version < "3.11" and python_version >= "3.10") and extra == 'all-strict'
Requires-Dist: pytest-cov <2.6.0,==2.0.0 ; (python_version < "3.6.0") and extra == 'all-strict'
Requires-Dist: scipy ==1.5.4 ; (python_version < "3.7" and python_version >= "3.6") and extra == 'all-strict'
Requires-Dist: scikit-learn ==0.24.1 ; (python_version < "3.7" and python_version >= "3.6") and extra == 'all-strict'
Requires-Dist: networkx <=2.5.1,==2.2.0 ; (python_version < "3.7.0" and python_version >= "3.6.0") and extra == 'all-strict'
Requires-Dist: networkx <=2.4.0,==2.2.0 ; (python_version < "3.7.0" and python_version >= "3.6.0") and extra == 'all-strict'
Requires-Dist: networkx ==2.6.2 ; (python_version < "3.8" and python_version >= "3.7") and extra == 'all-strict'
Requires-Dist: scipy ==1.6.0 ; (python_version < "3.8" and python_version >= "3.7") and extra == 'all-strict'
Requires-Dist: scikit-learn ==0.24.1 ; (python_version < "3.8" and python_version >= "3.7") and extra == 'all-strict'
Requires-Dist: scipy ==1.8.0 ; (python_version < "3.9" and python_version >= "3.8") and extra == 'all-strict'
Requires-Dist: scikit-learn ==1.0.2 ; (python_version < "3.9" and python_version >= "3.8") and extra == 'all-strict'
Requires-Dist: numpy ==1.23.2 ; (python_version < "4.0" and python_version >= "3.11") and extra == 'all-strict'
Requires-Dist: scipy ==1.9.2 ; (python_version < "4.0" and python_version >= "3.11") and extra == 'all-strict'
Requires-Dist: scikit-learn ==1.1.0 ; (python_version >= "3.10") and extra == 'all-strict'
Requires-Dist: Pillow ==9.1.0 ; (python_version >= "3.10") and extra == 'all-strict'
Requires-Dist: pytest-cov ==2.12.1 ; (python_version >= "3.6.0") and extra == 'all-strict'
Requires-Dist: networkx ==2.7 ; (python_version >= "3.8") and extra == 'all-strict'
Requires-Dist: numpy ; (python_version < "3.10" and python_version >= "3.6") and extra == 'all'
Requires-Dist: Pillow ; (python_version < "3.10" and python_version >= "3.6") and extra == 'all'
Requires-Dist: scipy ; (python_version < "3.10" and python_version >= "3.9") and extra == 'all'
Requires-Dist: scikit-learn ; (python_version < "3.10" and python_version >= "3.9") and extra == 'all'
Requires-Dist: numpy ; (python_version < "3.11" and python_version >= "3.10") and extra == 'all'
Requires-Dist: scipy ; (python_version < "3.11" and python_version >= "3.10") and extra == 'all'
Requires-Dist: pytest-cov ; (python_version < "3.6.0") and extra == 'all'
Requires-Dist: scipy ; (python_version < "3.7" and python_version >= "3.6") and extra == 'all'
Requires-Dist: scikit-learn ; (python_version < "3.7" and python_version >= "3.6") and extra == 'all'
Requires-Dist: networkx ; (python_version < "3.7.0" and python_version >= "3.6.0") and extra == 'all'
Requires-Dist: networkx ; (python_version < "3.8" and python_version >= "3.7") and extra == 'all'
Requires-Dist: scipy ; (python_version < "3.8" and python_version >= "3.7") and extra == 'all'
Requires-Dist: scikit-learn ; (python_version < "3.8" and python_version >= "3.7") and extra == 'all'
Requires-Dist: scipy ; (python_version < "3.9" and python_version >= "3.8") and extra == 'all'
Requires-Dist: scikit-learn ; (python_version < "3.9" and python_version >= "3.8") and extra == 'all'
Requires-Dist: numpy ; (python_version < "4.0" and python_version >= "3.11") and extra == 'all'
Requires-Dist: scipy ; (python_version < "4.0" and python_version >= "3.11") and extra == 'all'
Requires-Dist: scikit-learn ; (python_version >= "3.10") and extra == 'all'
Requires-Dist: Pillow ; (python_version >= "3.10") and extra == 'all'
Requires-Dist: pytest-cov ; (python_version >= "3.6.0") and extra == 'all'
Requires-Dist: networkx ; (python_version >= "3.8") and extra == 'all'
Provides-Extra: graphics
Provides-Extra: graphics-strict
Requires-Dist: opencv-python ==3.4.15.55 ; (python_version < "3.10" and python_version >= "3.9") and extra == 'graphics-strict'
Requires-Dist: opencv-python ==4.5.4.58 ; (python_version < "3.11" and python_version >= "3.10") and extra == 'graphics-strict'
Requires-Dist: opencv-python ==3.1.0.0 ; (python_version < "3.4" and python_version >= "2.7") and extra == 'graphics-strict'
Requires-Dist: opencv-python ==3.1.0.5 ; (python_version < "3.5" and python_version >= "3.4") and extra == 'graphics-strict'
Requires-Dist: opencv-python ==3.1.0.2 ; (python_version < "3.6" and python_version >= "3.5") and extra == 'graphics-strict'
Requires-Dist: opencv-python ==3.4.13.47 ; (python_version < "3.7" and python_version >= "3.6") and extra == 'graphics-strict'
Requires-Dist: opencv-python ==3.4.15.55 ; (python_version < "3.8" and python_version >= "3.7") and extra == 'graphics-strict'
Requires-Dist: opencv-python ==3.4.15.55 ; (python_version < "3.9" and python_version >= "3.8") and extra == 'graphics-strict'
Requires-Dist: opencv-python ==4.5.5.64 ; (python_version < "4.0" and python_version >= "3.11") and extra == 'graphics-strict'
Requires-Dist: opencv-python ; (python_version < "3.10" and python_version >= "3.9") and extra == 'graphics'
Requires-Dist: opencv-python ; (python_version < "3.11" and python_version >= "3.10") and extra == 'graphics'
Requires-Dist: opencv-python ; (python_version < "3.4" and python_version >= "2.7") and extra == 'graphics'
Requires-Dist: opencv-python ; (python_version < "3.5" and python_version >= "3.4") and extra == 'graphics'
Requires-Dist: opencv-python ; (python_version < "3.6" and python_version >= "3.5") and extra == 'graphics'
Requires-Dist: opencv-python ; (python_version < "3.7" and python_version >= "3.6") and extra == 'graphics'
Requires-Dist: opencv-python ; (python_version < "3.8" and python_version >= "3.7") and extra == 'graphics'
Requires-Dist: opencv-python ; (python_version < "3.9" and python_version >= "3.8") and extra == 'graphics'
Requires-Dist: opencv-python ; (python_version < "4.0" and python_version >= "3.11") and extra == 'graphics'
Provides-Extra: headless
Provides-Extra: headless-strict
Requires-Dist: opencv-python-headless ==3.4.15.55 ; (python_version < "3.10" and python_version >= "3.9") and extra == 'headless-strict'
Requires-Dist: opencv-python-headless ==4.5.4.58 ; (python_version < "3.11" and python_version >= "3.10") and extra == 'headless-strict'
Requires-Dist: opencv-python-headless ==3.4.2.16 ; (python_version < "3.4" and python_version >= "2.7") and extra == 'headless-strict'
Requires-Dist: opencv-python-headless ==3.4.2.16 ; (python_version < "3.5" and python_version >= "3.4") and extra == 'headless-strict'
Requires-Dist: opencv-python-headless ==3.4.2.16 ; (python_version < "3.6" and python_version >= "3.5") and extra == 'headless-strict'
Requires-Dist: opencv-python-headless ==3.4.13.47 ; (python_version < "3.7" and python_version >= "3.6") and extra == 'headless-strict'
Requires-Dist: opencv-python-headless ==3.4.15.55 ; (python_version < "3.8" and python_version >= "3.7") and extra == 'headless-strict'
Requires-Dist: opencv-python-headless ==3.4.15.55 ; (python_version < "3.9" and python_version >= "3.8") and extra == 'headless-strict'
Requires-Dist: opencv-python-headless ==4.5.5.64 ; (python_version < "4.0" and python_version >= "3.11") and extra == 'headless-strict'
Requires-Dist: opencv-python-headless ; (python_version < "3.10" and python_version >= "3.9") and extra == 'headless'
Requires-Dist: opencv-python-headless ; (python_version < "3.11" and python_version >= "3.10") and extra == 'headless'
Requires-Dist: opencv-python-headless ; (python_version < "3.4" and python_version >= "2.7") and extra == 'headless'
Requires-Dist: opencv-python-headless ; (python_version < "3.5" and python_version >= "3.4") and extra == 'headless'
Requires-Dist: opencv-python-headless ; (python_version < "3.6" and python_version >= "3.5") and extra == 'headless'
Requires-Dist: opencv-python-headless ; (python_version < "3.7" and python_version >= "3.6") and extra == 'headless'
Requires-Dist: opencv-python-headless ; (python_version < "3.8" and python_version >= "3.7") and extra == 'headless'
Requires-Dist: opencv-python-headless ; (python_version < "3.9" and python_version >= "3.8") and extra == 'headless'
Requires-Dist: opencv-python-headless ; (python_version < "4.0" and python_version >= "3.11") and extra == 'headless'
Provides-Extra: optional
Requires-Dist: kwplot ; extra == 'optional'
Requires-Dist: lru-dict ; extra == 'optional'
Provides-Extra: optional-strict
Requires-Dist: kwplot ==0.4.10 ; extra == 'optional-strict'
Requires-Dist: lru-dict ==1.1.7 ; extra == 'optional-strict'
Provides-Extra: runtime-strict
Requires-Dist: ubelt ==1.3.3 ; extra == 'runtime-strict'
Requires-Dist: parse ==1.14.0 ; extra == 'runtime-strict'
Requires-Dist: xarray ==0.16.0 ; extra == 'runtime-strict'
Requires-Dist: sortedcontainers ==2.3.0 ; extra == 'runtime-strict'
Requires-Dist: fasteners ==0.14.1 ; extra == 'runtime-strict'
Requires-Dist: atomicwrites ==1.3.0 ; extra == 'runtime-strict'
Requires-Dist: pyqtree ==1.0.0 ; extra == 'runtime-strict'
Requires-Dist: kwimage ==0.9.19 ; extra == 'runtime-strict'
Requires-Dist: kwarray ==0.6.7 ; extra == 'runtime-strict'
Requires-Dist: kwcoco ==0.5.6 ; extra == 'runtime-strict'
Requires-Dist: numpy ==1.19.3 ; (python_version < "3.10" and python_version >= "3.6") and extra == 'runtime-strict'
Requires-Dist: Pillow ==8.0.0 ; (python_version < "3.10" and python_version >= "3.6") and extra == 'runtime-strict'
Requires-Dist: scipy ==1.8.0 ; (python_version < "3.10" and python_version >= "3.9") and extra == 'runtime-strict'
Requires-Dist: scikit-learn ==1.0.2 ; (python_version < "3.10" and python_version >= "3.9") and extra == 'runtime-strict'
Requires-Dist: numpy ==1.21.6 ; (python_version < "3.11" and python_version >= "3.10") and extra == 'runtime-strict'
Requires-Dist: scipy ==1.8.0 ; (python_version < "3.11" and python_version >= "3.10") and extra == 'runtime-strict'
Requires-Dist: scipy ==1.5.4 ; (python_version < "3.7" and python_version >= "3.6") and extra == 'runtime-strict'
Requires-Dist: scikit-learn ==0.24.1 ; (python_version < "3.7" and python_version >= "3.6") and extra == 'runtime-strict'
Requires-Dist: networkx <=2.5.1,==2.2.0 ; (python_version < "3.7.0" and python_version >= "3.6.0") and extra == 'runtime-strict'
Requires-Dist: networkx <=2.4.0,==2.2.0 ; (python_version < "3.7.0" and python_version >= "3.6.0") and extra == 'runtime-strict'
Requires-Dist: networkx ==2.6.2 ; (python_version < "3.8" and python_version >= "3.7") and extra == 'runtime-strict'
Requires-Dist: scipy ==1.6.0 ; (python_version < "3.8" and python_version >= "3.7") and extra == 'runtime-strict'
Requires-Dist: scikit-learn ==0.24.1 ; (python_version < "3.8" and python_version >= "3.7") and extra == 'runtime-strict'
Requires-Dist: scipy ==1.8.0 ; (python_version < "3.9" and python_version >= "3.8") and extra == 'runtime-strict'
Requires-Dist: scikit-learn ==1.0.2 ; (python_version < "3.9" and python_version >= "3.8") and extra == 'runtime-strict'
Requires-Dist: numpy ==1.23.2 ; (python_version < "4.0" and python_version >= "3.11") and extra == 'runtime-strict'
Requires-Dist: scipy ==1.9.2 ; (python_version < "4.0" and python_version >= "3.11") and extra == 'runtime-strict'
Requires-Dist: scikit-learn ==1.1.0 ; (python_version >= "3.10") and extra == 'runtime-strict'
Requires-Dist: Pillow ==9.1.0 ; (python_version >= "3.10") and extra == 'runtime-strict'
Requires-Dist: networkx ==2.7 ; (python_version >= "3.8") and extra == 'runtime-strict'
Provides-Extra: tests
Requires-Dist: pytest ; extra == 'tests'
Requires-Dist: coverage ; extra == 'tests'
Requires-Dist: xdoctest ; extra == 'tests'
Provides-Extra: tests-strict
Requires-Dist: pytest ==6.2.4 ; extra == 'tests-strict'
Requires-Dist: coverage ==5.2.1 ; extra == 'tests-strict'
Requires-Dist: xdoctest ==1.1.2 ; extra == 'tests-strict'
Requires-Dist: pytest-cov <2.6.0,==2.0.0 ; (python_version < "3.6.0") and extra == 'tests-strict'
Requires-Dist: pytest-cov ==2.12.1 ; (python_version >= "3.6.0") and extra == 'tests-strict'
Requires-Dist: pytest-cov ; (python_version < "3.6.0") and extra == 'tests'
Requires-Dist: pytest-cov ; (python_version >= "3.6.0") and extra == 'tests'

ndsampler
=========

|GitlabCIPipeline| |GitlabCICoverage| |Pypi| |Downloads|


+------------------+---------------------------------------------------------+
| Read the docs    | https://ndsampler.readthedocs.io                        |
+------------------+---------------------------------------------------------+
| Gitlab (main)    | https://gitlab.kitware.com/computer-vision/ndsampler    |
+------------------+---------------------------------------------------------+
| Github (mirror)  | https://github.com/Kitware/ndsampler                    |
+------------------+---------------------------------------------------------+
| Pypi             | https://pypi.org/project/ndsampler                      |
+------------------+---------------------------------------------------------+

The main webpage for this project is: https://gitlab.kitware.com/computer-vision/ndsampler

Fast random access to small regions in large images.

Random access is amortized by converting images into an efficient backend
format (current backends include cloud-optimized geotiffs (cog) or numpy array
files (npy)). If images are already in COG format, then no conversion is
needed.

The ndsampler module was built with detection, segmentation, and classification
tasks in mind, but it is not limited to these use cases.

The basic idea is to ensure your data is in MS-coco format, and then the
CocoSampler class will let you sample positive and negative regions.

For classification tasks the MS-COCO data could just be that every image has an
annotation that takes up the entire image.

The aspiration of this module is to allow you to access your data in-situ (i.e.
no pre-processing), although the cost of that may depend on how efficient your
data is to access. However a faster cache can be built at the cost of disk
space. Currently we have a "cog" and "npy" backend. Help is wanted to integrate
backends for hdf5 and other medical / domain-specific formats.

Installation
------------

The `ndsampler <https://pypi.org/project/ndsampler/>`_.  package can be installed via pip:

.. code-block:: bash

    pip install ndsampler


Note that ndsampler depends on `kwimage <https://pypi.org/project/kwimage/>`_,
where there is a known compatibility issue between `opencv-python <https://pypi.org/project/opencv-python/>`_
and `opencv-python-headless <https://pypi.org/project/opencv-python-headless/>`_. Please ensure that one
or the other (but not both) are installed as well:

.. code-block:: bash

    pip install opencv-python-headless

    # OR

    pip install opencv-python


Lastly, to fully leverage ndsampler's features GDAL must be installed (although
much of ndsampler can work without it).  Kitware has a pypi index that hosts
GDAL wheels for linux systems, but other systems will need to find some way of
installing gdal (conda is safe choice).

.. code-block:: bash

    pip install --find-links https://girder.github.io/large_image_wheels GDAL


Features
--------

* CocoDataset for managing and manipulating annotated image datasets
* Amortized O(1) sampling of N-dimension space-time data (wrt to constant window size) (e.g. images and video).
* Hierarchical or mutually exclusive category management.
* Random negative window sampling.
* Coverage-based positive sampling.
* Dynamic toydata generator.


Also installs the kwcoco package and CLI tool.


Usage
-----

The main pattern of usage is:

1. Use kwcoco to load a json-based COCO dataset (or create a ``kwcoco.CocoDataset``
   programatically).

2. Pass that dataset to an ``ndsampler.CocoSampler`` object, and that
   effectively wraps the json structure that holds your images and annotations
   and it allows you to sample patches from those images efficiently.

3. You can either manually specify image + region or you can specify an
   annotation id, in which case it loads the region corresponding to the
   annotation.


Example
--------

This example shows how you can efficiently load subregions from images.

.. code-block:: python

    >>> # Imagine you have some images
    >>> import kwimage
    >>> image_paths = [
    >>>     kwimage.grab_test_image_fpath('astro'),
    >>>     kwimage.grab_test_image_fpath('carl'),
    >>>     kwimage.grab_test_image_fpath('airport'),
    >>> ]  # xdoc: +IGNORE_WANT
    ['~/.cache/kwimage/demodata/KXhKM72.png',
     '~/.cache/kwimage/demodata/flTHWFD.png',
     '~/.cache/kwimage/demodata/Airport.jpg']
    >>> # And you want to randomly load subregions of them in O(1) time
    >>> import ndsampler
    >>> import kwcoco
    >>> # First make a COCO dataset that refers to your images (and possibly annotations)
    >>> dataset = {
    >>>     'images': [{'id': i, 'file_name': fpath} for i, fpath in enumerate(image_paths)],
    >>>     'annotations': [],
    >>>     'categories': [],
    >>> }
    >>> coco_dset = kwcoco.CocoDataset(dataset)
    >>> print(coco_dset)
    <CocoDataset(tag=None, n_anns=0, n_imgs=3, n_cats=0)>
    >>> # Now pass the dataset to a sampler and tell it where it can store temporary files
    >>> workdir = ub.Path.appdir('ndsampler/demo').ensuredir()
    >>> sampler = ndsampler.CocoSampler(coco_dset, workdir=workdir)
    >>> # Now you can load arbirary samples by specifing a target dictionary
    >>> # with an image_id (gid) center location (cx, cy) and width, height.
    >>> target = {'gid': 0, 'cx': 200, 'cy': 200, 'width': 100, 'height': 100}
    >>> sample = sampler.load_sample(target)
    >>> # The sample contains the image data, any visible annotations, a reference
    >>> # to the original target, and params of the transform used to sample this
    >>> # patch
    >>> print(sorted(sample.keys()))
    ['annots', 'im', 'params', 'tr']
    >>> im = sample['im']
    >>> print(im.shape)
    (100, 100, 3)
    >>> # The load sample function is at the core of what ndsampler does
    >>> # There are other helper functions like load_positive / load_negative
    >>> # which deal with annotations. See those for more details.
    >>> # For random negative sampling see coco_regions.

A Note On COGs
--------------
COGs (cloud optimized geotiffs) are the backbone efficient sampling in the
ndsampler library.

To preform deep learning efficiently you need to be able to effectively
randomly sample cropped regions from images, so when ``ndsampler.Sampler``
(more acurately the ``FramesSampler`` belonging to the base ``Sampler`` object)
is in "cog" mode, it caches all images larger than 512x512 in cog format.

I've noticed a significant speedups even for "small" 1024x1024 images.  I
haven't made effective use of the overviews feature yet, but in the future I
plan to, as I want to allow ndsampler to sample in scale as well as in space.

Its possible to obtain this speedup with the "npy" backend, which supports true
random sampling, but this is an uncompressed format, which can require a large
amount of disk space. Using the "None" backend, means that loading a small
windowed region requires loading the entire image first (which can be ok for
some applications).

Using COGs requires that GDAL is installed.  Installing GDAL is a pain though.

https://gist.github.com/cspanring/5680334

Using conda is relatively simple

.. code-block:: bash

    conda install gdal

    # Test that this works
    python -c "from osgeo import gdal; print(gdal)"


Also possible to use system packages

.. code-block:: bash

    # References:
    # https://gis.stackexchange.com/questions/28966/python-gdal-package-missing-header-file-when-installing-via-pip
    # https://gist.github.com/cspanring/5680334


    # Install GDAL system libs
    sudo apt install libgdal-dev

    GDAL_VERSION=`gdal-config --version`
    echo "GDAL_VERSION = $GDAL_VERSION"
    pip install --global-option=build_ext --global-option="-I/usr/include/gdal" GDAL==$GDAL_VERSION


    # Test that this works
    python -c "from osgeo import gdal; print(gdal)"


Kitware also has a pypi index that hosts GDAL wheels for linux systems:

.. code-block:: bash

    pip install --find-links https://girder.github.io/large_image_wheels GDAL


TODO
----

- [ ] Currently only supports image-based detection tasks, but not much work is
  needed to extend to video. The code was originally based on sampling code for
  video, so ndimensions is builtin to most places in the code. However, there are
  currently no test cases that demonstrate that this library does work with video.
  So we should (a) port the video toydata code from irharn to test ndcases and (b)
  fix the code to work for both still images and video where things break.

- [ ] Currently we are good at loading many small objects in 2d images.
  However, we are bad at loading images with one single large object that needs
  to be downsampled (e.g. loading an entire 1024x1024 image and downsampling it
  to 224x224). We should find a way to mitigate this using pyramid overviews in
  the backend COG files.


.. |Pypi| image:: https://img.shields.io/pypi/v/ndsampler.svg
   :target: https://pypi.python.org/pypi/ndsampler

.. |Downloads| image:: https://img.shields.io/pypi/dm/ndsampler.svg
   :target: https://pypistats.org/packages/ndsampler

.. |ReadTheDocs| image:: https://readthedocs.org/projects/ndsampler/badge/?version=latest
    :target: http://ndsampler.readthedocs.io/en/latest/

.. # See: https://ci.appveyor.com/project/jon.crall/ndsampler/settings/badges
.. .. |Appveyor| image:: https://ci.appveyor.com/api/projects/status/py3s2d6tyfjc8lm3/branch/master?svg=true
.. :target: https://ci.appveyor.com/project/jon.crall/ndsampler/branch/master

.. |GitlabCIPipeline| image:: https://gitlab.kitware.com/computer-vision/ndsampler/badges/master/pipeline.svg
   :target: https://gitlab.kitware.com/computer-vision/ndsampler/-/jobs

.. |GitlabCICoverage| image:: https://gitlab.kitware.com/computer-vision/ndsampler/badges/master/coverage.svg?job=coverage
    :target: https://gitlab.kitware.com/computer-vision/ndsampler/commits/master
