Jeff Galper » feather http://jeffgalper.com/blog Not as bad as rabies. Fri, 26 Aug 2011 05:17:53 +0000 en hourly 1 http://wordpress.org/?v=3.3 Tar, meet Feather http://jeffgalper.com/blog/2009/12/11/tar-meet-feather/ http://jeffgalper.com/blog/2009/12/11/tar-meet-feather/#comments Fri, 11 Dec 2009 09:03:51 +0000 Jeff http://jeffgalper.com/blog/?p=14 Due to my endless frustrations with the Tar Library we use at work and some limitations of the format in general, I decided to create my own file format and library. There is now Feather. The Feather format can be described as a simple, extensible storage format. By using optional fields, a feather file can include all the information normally stored in a tar, or as little as the name, size, and data of a file.

While UStar formatted Tar files have a maximum filename length of 255 (with some restrictions), Feather has a maximum filename length of 23,767.  Without the GNU extensions, Tar has a maximum file size limit of 8 GB.  Feather, on the other hand, has a maximum file size of 8 EB, or 8 GB * 1024³.  Finally, Feather has native support for UTF-16, which Tar lacks completely.

The current state of the Feather repository is a bit of a mess, because it was originally programmed solely using ByteBuffers and FileChannels, but a need arose for Input and Output Streams.  I will likely remove the use of ByteBuffers and FileChannels so that there is only one official implementation of Feather.

]]> http://jeffgalper.com/blog/2009/12/11/tar-meet-feather/feed/ 0