Due to my endless frustrations with the Tar Library we use at work and some limitations of the format in general, I decided to create my own file format and library. There is now Feather. The Feather format can be described as a simple, extensible storage format. By using optional fields, a feather file can include all the information normally stored in a tar, or as little as the name, size, and data of a file.
While UStar formatted Tar files have a maximum filename length of 255 (with some restrictions), Feather has a maximum filename length of 23,767. Without the GNU extensions, Tar has a maximum file size limit of 8 GB. Feather, on the other hand, has a maximum file size of 8 EB, or 8 GB * 1024³. Finally, Feather has native support for UTF-16, which Tar lacks completely.
The current state of the Feather repository is a bit of a mess, because it was originally programmed solely using ByteBuffers and FileChannels, but a need arose for Input and Output Streams. I will likely remove the use of ByteBuffers and FileChannels so that there is only one official implementation of Feather.
Comments Off
Entries (RSS)