Anton Dubrau - Using small data in the client instead of big data in the cloud





The interactive transcript could not be loaded.


Rating is available when the video has been rented.
This feature is not available right now. Please try again later.
Published on May 31, 2016

PyData Berlin 2016

Rather than using big data in the cloud, at Transit App we send data sets to the client and process it there. To keep the data small, we’ve switched to using binary files instead of plain text formats like ᴊsᴏɴ. In this talk we’ll discuss how you can turn data into binary, how it works, and our own approach to store complete transit schedules on the device and show their data instantly.

Rather than using big data in the cloud, we at Transit App found it useful to to give as much data to client as possible and process it client-side. In order to keep transfer volume and performance reasonable, it makes sense to move from text based interchange formats like ᴊsᴏɴ or xᴍʟ back back in time to using binary file formats. In this talk we will introduce some common libraries to make data compact, to make it fast, and how these libraries work. We will also discuss the approach of our own library for creating schema-less binary file formats, and how it helped us store complete transit schedules in small files on the phone and display their data instantly.

Comments are turned off. Learn more
When autoplay is enabled, a suggested video will automatically play next.

Up next

to add this to Watch Later

Add to

Loading playlists...