Metadata-Version: 2.1
Name: xbitdiff
Version: 0.1
Summary: TODO.
Maintainer-email: Timothy Hodson <thodson@usgs.gov>
Project-URL: homepage, https://github.com/thodson-usgs/xbitdiff
Keywords: xarray,bitwise diff
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: xarray
Provides-Extra: test
Requires-Dist: pytest>5.0.0; extra == "test"
Requires-Dist: pytest-cov[all]; extra == "test"

# xbitdiff

WARNING: This package does not work yet.

We store lot of redundant information, and in the era of big data, those costs add up.
`xbitdiff` is a simple wrapper around `xarray` for creating data diffs that can help reduce storage costs.
The name `xbitdiff` is an homage to `xbitinfo` but the packages are not directly related.

Here's a typical scenario:
You host several large datasets.
Perhaps one is the output of a climate model along with a bias-corrected version.
Or perhaps a large ensemble of models forced with different climate scenarios, like CMIP.
Each of those datasets contains a substantial amount of redundant information.
A simple strategy to reduce the storage footprint is to compute the difference or "diff" of one dataset and another,
then store the original and the diff,
which compresses better than the full dataset.
`xbitdiff` helps abstract diff management behind the scenes,
so the user can work with datasets, not diffs.
