Metadata-Version: 2.1
Name: binpickle
Version: 0.3.2
Summary: Efficient binary storage of ML models
Home-page: https://binpickle.lenskit.org
Author: Michael Ekstrand
Author-email: michaelekstrand@boisestate.edu
License: MIT License
Project-URL: GitHub, https://github.com/lenskit/binpickle
Platform: UNKNOWN
Classifier: License :: OSI Approved :: MIT License
Description-Content-Type: text/markdown
Requires-Dist: msgpack (>=1.0)
Requires-Dist: pickle5 ; python_version < "3.8"
Provides-Extra: blosc
Requires-Dist: blosc ; extra == 'blosc'
Provides-Extra: dev
Requires-Dist: flake8 ; extra == 'dev'
Requires-Dist: rstcheck ; extra == 'dev'
Requires-Dist: twine ; extra == 'dev'
Provides-Extra: doc
Requires-Dist: sphinx ; extra == 'doc'
Provides-Extra: numcodecs
Requires-Dist: numcodecs ; extra == 'numcodecs'
Provides-Extra: test
Requires-Dist: pytest ; extra == 'test'
Requires-Dist: pytest-cov ; extra == 'test'
Requires-Dist: hypothesis ; extra == 'test'
Requires-Dist: pandas (>=1.0) ; extra == 'test'
Requires-Dist: numpy (>=1.17) ; extra == 'test'

# BinPickle - efficient binary pickled data

[![PyPI version](https://badge.fury.io/py/binpickle.svg)](https://badge.fury.io/py/binpickle)
![Test and Build](https://github.com/lenskit/binpickle/workflows/Test%20and%20Package/badge.svg)
[![codecov](https://codecov.io/gh/lenskit/binpickle/branch/master/graph/badge.svg)](https://codecov.io/gh/lenskit/binpickle)

This package uses the new Pickle Protocol 5 in Python 3.8 (or its `pickle5` backport)
to efficiently serialize large objects, particularly from scientific Python packages,
to an on-disk format.  This format is designed to support two use cases:

1.  Serializing data-intensive statistical models in a memory-mappable format so
    multiple processes can share the same (read-only) model memory.
2.  Serializing data-intensive statistical models with good compression for long-term
    storage and cross-machine transportation.

BinPickle does this by using Pickle 5's out-of-band buffer serialization support to
write buffers uncompressed and page-aligned for memory mapping (use case 1) or with
per-buffer efficient compression with libraries like Blosc (use case 2).

## Format Stability

We do **not** yet guarantee the stability of the BinPickle format.  We will avoid gratuitous changes,
but BinPickle 1.0 will be the first with a stability guarantee.

## Acknowledgements

This material is based upon work supported by the National Science Foundation under
Grant No. IIS 17-51278. Any opinions, findings, and conclusions or recommendations
expressed in this material are those of the author(s) and do not necessarily reflect
the views of the National Science Foundation.  This page has not been approved by
Boise State University and does not reflect official university positions.


