Hi, As part of a cron job I have running, I'm using gzip -c to append compressed data to the end of an existing gzip file. According to the gzip man page this should work, and for the most part it does. It seems, however, that in some cases this doesn't work. While gunzip and winzip are both happy opening the file, WinRAR complains about a failed CRC in the file, and PKUnzip only unzips part of the file. If I gzip -dc|gzip the file, WinRAR and PKUnzip are perfectly happy to open the file until next time a new portion is appended to it. Does anyone have an explaination for this? I don't really want to have to unzip and re-zip each file every night, as the computational cost would be far too high. Any advice would be most appreciated :) A -- Andy Furnell
![]() |
0 |
![]() |
Andy wrote: ) As part of a cron job I have running, I'm using gzip -c to append ) compressed data to the end of an existing gzip file. According to the ) gzip man page this should work, and for the most part it does. It seems, ) however, that in some cases this doesn't work. While gunzip and winzip ) are both happy opening the file, WinRAR complains about a failed CRC in ) the file, and PKUnzip only unzips part of the file. ) ) If I gzip -dc|gzip the file, WinRAR and PKUnzip are perfectly happy to ) open the file until next time a new portion is appended to it. ) ) Does anyone have an explaination for this? I don't really want to have ) to unzip and re-zip each file every night, as the computational cost ) would be far too high. The simplest explanation would be that both WinRAR and PKUnzip don't fully comply with the gzip standard, and can therefore not read all valid gzipped files. Complain to the authors of WinRAR and/or PKUnzip. And ask yourself why you have to use one of those programs to open those files, and/or simply use gzip to open them. (Or look for another program that correctly opens such files, probably any program using zlib.) SaSW, Willem (at stack dot nl) -- Disclaimer: I am in no way responsible for any of the statements made in the above text. For all I know I might be drugged or something.. No I'm not paranoid. You all think I'm paranoid, don't you ! #EOT
![]() |
0 |
![]() |
In article <slrnbfj0dq.224f.no@toad.stack.nl>, Willem wrote: > Andy wrote: > ) As part of a cron job I have running, I'm using gzip -c to append > ) compressed data to the end of an existing gzip file. According to the > ) gzip man page this should work, and for the most part it does. It seems, > ) however, that in some cases this doesn't work. While gunzip and winzip > ) are both happy opening the file, WinRAR complains about a failed CRC in > ) the file, and PKUnzip only unzips part of the file. > ) > ) If I gzip -dc|gzip the file, WinRAR and PKUnzip are perfectly happy to > ) open the file until next time a new portion is appended to it. > ) > ) Does anyone have an explaination for this? I don't really want to have > ) to unzip and re-zip each file every night, as the computational cost > ) would be far too high. > > The simplest explanation would be that both WinRAR and PKUnzip don't fully > comply with the gzip standard, and can therefore not read all valid gzipped > files. Complain to the authors of WinRAR and/or PKUnzip. Will do. I just figured that this might have been something people had come across before. > And ask yourself why you have to use one of those programs to open those > files, and/or simply use gzip to open them. (Or look for another program > that correctly opens such files, probably any program using zlib.) Believe me, if it was up to me I'd ban all customers from using anything other than gzip to open these files. Unfortunately they seem to have a mind of their own these days ;) Thanks for the clarification, anyhow :) -- Andy Furnell
![]() |
0 |
![]() |
Andy Furnell <andy@ipng.org.uk> wrote in message news:<slrnbfirdk.s26.andy.furnell@penfold.noc.clara.net>... > Hi, > > As part of a cron job I have running, I'm using gzip -c to append > compressed data to the end of an existing gzip file. According to the > gzip man page this should work, and for the most part it does. It seems, > however, that in some cases this doesn't work. While gunzip and winzip > are both happy opening the file, WinRAR complains about a failed CRC in > the file, and PKUnzip only unzips part of the file. I surprised it was even possible to concatenate compressed files. Maybe you could compress each file separately and add it to an archive.
![]() |
0 |
![]() |
In article <8a1ed69a.0306250659.217acf79@posting.google.com>, Matt Mahoney wrote: > Andy Furnell <andy@ipng.org.uk> wrote in message news:<slrnbfirdk.s26.andy.furnell@penfold.noc.clara.net>... >> Hi, >> >> As part of a cron job I have running, I'm using gzip -c to append >> compressed data to the end of an existing gzip file. According to the >> gzip man page this should work, and for the most part it does. It seems, >> however, that in some cases this doesn't work. While gunzip and winzip >> are both happy opening the file, WinRAR complains about a failed CRC in >> the file, and PKUnzip only unzips part of the file. > > I surprised it was even possible to concatenate compressed files. > Maybe you could compress each file separately and add it to an > archive. The FreeBSD man page would seem to imply that you can: Multiple compressed files can be concatenated. In this case, gunzip will extract all members at once. For exam- ple: gzip -c file1 > foo.gz gzip -c file2 >> foo.gz Then gunzip -c foo is equivalent to cat file1 file2 AIUI the only disadvantage to doing this is that you lose compression compared to concatenating the file, then gzipping it as a whole. I did consider storing the gzipped files in an archive, but the object of the exercise is to produce a single contiguous compressed log that customers can download and make appreciative noises at rather than coming up with a technically sound solution :) A -- Andy Furnell
![]() |
0 |
![]() |
Andy Furnell <andy@ipng.org.uk> wrote in message news:<slrnbfirdk.s26.andy.furnell@penfold.noc.clara.net>... > While gunzip and winzip > are both happy opening the file, WinRAR complains about a failed CRC in > the file, and PKUnzip only unzips part of the file. gzip (and winzip as well) are written to continue decompressing gzip streams as long as there is still data in the input stream. So you can concatenate the output of independent gzip compressions and expect gzip to decompress them in a single step, which is what you're doing. However it appears that WinRAR is looking to the end of the file for the CRC (which it shouldn't), and PKUnzip is stopping after the first gzip segement. Either you can keep the output of the separate gzip runs in separate files and decompress them that way, or you can suggest to your users that they use gzip, which compiles on a large number of platforms. mark
![]() |
0 |
![]() |