Archive for mars, 2009

Quick Tip: using the best of gz and lzma

Vendredi, mars 27th, 2009

I come back with this small compression tip:

lzma can compress gz archives too, in some cases.

On one of my files (binary dump from one of my applications), I obtain these ratios:

1.9 Gb (raw) -> 2.4 Mb (raw.gz) -> 304 Kb (raw.gz.lzma)

My original file contained a lot of redondant data (basically, a huge matrix), so it was an easy job for gz to compress it to a very small file size (791 x !!), but suprisingly even this “compressed” archive could be compressed a lot more (x 8 !!) by lzma.

Simply awesome, because operating lzma on big Gb files is very slow, so gzip is more suited for these, especially if you call it directly from a program, to read/write a compressed stream.

Just try it yourself, depending on your data, it may worth it!