Rating is available when the video has been rented.
This feature is not available right now. Please try again later.
Published on Jul 27, 2014
PyData Berlin 2014 Bloscpack  is a reference implementation and file-format for fast serialization of numerical data. It features lightweight, chunked and compressed storage, based on the extremely fast Blosc  metacodec and supports serialization of Numpy arrays out-of-the-box. Recently, Blosc -- being the metacodec that it is -- has received support for using the popular and widely used Snappy , LZ4 , and ZLib  codecs, and so, now Bloscpack supports serializing Numpy arrays easily with those codecs! In this talk I will present recent benchmarks of Bloscpack performance on a variety of artificial and real-world datasets with a special focus on the newly available codecs. In these benchmarks I will compare Bloscpack, both performance and usability wise, to alternatives such as Numpy's native offerings (NPZ and NPY), HDF5/PyTables , and if time permits, to novel bleeding edge solutions. Lastly I will argue that compressed and chunked storage format such as Bloscpack can be and somewhat already is a useful substrate on which to build more powerful applications such as online analytical processing engines and distributed computing frameworks. : https://github.com/Blosc/bloscpack : https://github.com/Blosc/c-blosc/ : http://code.google.com/p/snappy/ : http://code.google.com/p/lz4/ : http://www.zlib.net/ : http://www.pytables.org/moin