I hope someone in the group can tell me this is a simple example.

I have a pretty basic Ruby script where I am trying to decode and
string then inflate it.  However Zlib is throwing a string error that I
can't track down the source.


The error when running this script is:

Ruby Version: 1.8.2
Zlib Version: 1.2.1

The error Message is:

"invalid distance too far back"


#########################################################
require 'base64'
require 'zlib'


begin
    f =3D File.open("c:/temp/string.txt","r")
    fout =3D File.open("c:/temp/decodedString","w")


    while (ln =3D f.gets)
         begin
        decodeString =3D Base64.decode64(ln)
        fout.puts("Here comes the decoded string \n")
        fout.puts(decodeString)
        inflatedString  =3D Zlib::Inflate.inflate decodeString


        fout.puts("here comes the inflated string \n\n")
        fout.puts(inflatedString)
        rescue StandardError =3D> err
           fout.puts("an exception occured \n #{err}\n")
           print "an exception occured #{err}\n"
       end #block under while loops
    end #while loop


ensure
    f.close
    fout.puts("closing program from ensure")
    fout.close


end #top begin section


#########################################################


The string in the file I am operating on is:


eJztmF1vmzAUhv9KlJteZdgQCEjElWNcghSc1JiqubKiLpoqZd1Uomn79zMfTY1Jq1VJ14ty5/O=
=AD+PseHk0dWILz8/X03+LV9LO5/PEwv4BdwcYlClkvBMcsIT1ZC3ixzMqdcZmtGUHh1v919FX9=
+bo=ADtmH5M3ZM5lhMXgbrcpiumQD1FI09UiiQZ7tXE6JHPMh5YSb4Vc8lh2DFWo9AjmlPLjnqq=
PJZs9u=ADSxPZ7TyM3qtgo781PWxswinWFB5tcBx11zxhFXH0cYrl6VXG7NciCXrptkskpxe5zQ=
ThtmoMhJG=ADQZVVLuUqny2SbG64ljFd1UBGcIr1KteS8wi3McU0nMp8R5lAiJiTBbneGMypfz0=
iyWaqJqNuZoV=ADnxJaJbJhGUCM0I7E28eQ/J8QbtjV4XZMqIMp9RwrPq51EJnKy2+RZv9BoXic=
fNQbO72CtB/IA4B=ADACAIAjcIrUbRsEPABtWGsXKf1TZ/KI95VI1f00wMETzsOEgNiqXVrDQKU=
QB91ZMmGCSidVVRV9o=AD4ItW7O4JwBOqzNafFZjnRVtimE7EqW1c0RFWyHhiIli146vwRtKsiL=
e+txCLK4tDSBY1aVU0PTG=ADrr3I6q46sqtCKTXwTKORmaCXE5S1PRSK7P0KIDyi1q347w2HE9F=
x5HGMLxGDi+/54Ig6MIO14wO=ADQPCYDyy7R7hz8wv9GznA/idlFdnz2/P76n82q7vfsj96/f89=
vyezq8T2O/6F/jF+xf0/Pb8nsav=AD+gGgM/Em//8VTt2/53iFO53f9av81of4I9vrST4zydbhs=
4T1ykeyv+QRmHM=3D

The zlib stream you sent, ending in base64 with "QRmHM=" is incomplete.
 It stops in the middle of the compressed data.

mark

On 2006-01-13, chris.r.malek@gmail.com <chris.r.malek@gmail.com> wrote:
> I hope someone in the group can tell me this is a simple example.
>
> I have a pretty basic Ruby script where I am trying to decode and
> string then inflate it.  However Zlib is throwing a string error that I
> can't track down the source.

how is the data compressed. zlib ssupports three formats,

raw deflate, zlib deflate, and gzip.

AIUI you need to initialise the inflater (you do in C)
and you don't seem to be doing that.


Bye.
   Jasen

>Mark Adler: The zlib stream you sent, ending in base64 with "QRmHM=" is incomplete. It stops in the middle of the compressed data.

>Jasen Betts: how is the data compressed. zlib ssupports three formats,

This string is actually coming from a PeopleSoft Application Messaging
module.  So I really don't have any way of knowing there exact method.
According to the documentation on thier website they send messages in
be "deflated and encoded in base64".

I did not see in the ruby documentation of where to set the compression
method.   Is there a standard method that is mostly used?  PeopleSoft
in general follows most web standards.




This is troubling.

The data you sent is a zlib stream, as identified by the first two
bytes.  Ruby's zlib interface does allow you to select zlib, gzip, or
raw through the specification of window_bits to inflate.new (if it's
hooked up to zlib 1.2.x), but you don't need to mess with that for this
application, since the default assumes zlib streams.

Chris Malek wrote:
> This is troubling.

As I look at it more closely, there is an error in the compressed
stream.  There may have a transmission or translation error at some
point.

mark

On 2006-01-14, Chris Malek <chris.r.malek@gmail.com> wrote:
>>Mark Adler: The zlib stream you sent, ending in base64 with "QRmHM=" is incomplete. It stops in the middle of the compressed data.
>
>>Jasen Betts: how is the data compressed. zlib ssupports three formats,
>
> This string is actually coming from a PeopleSoft Application Messaging
> module.  So I really don't have any way of knowing there exact method.
> According to the documentation on thier website they send messages in
> be "deflated and encoded in base64".
>
> I did not see in the ruby documentation of where to set the compression
> method.   Is there a standard method that is mostly used?  PeopleSoft
> in general follows most web standards.

I don't have ruby installed here, neither do I have access to the
documentation of the ruby-zlib API... as for web standards there's
standards and then there's the microsoft way.

post or email me some of the base 64 encoded data (3 or 4 lines would be
plenty) and I'll see which flavour it is.

Bye.
   Jasen

Here is the XML message that I am really try to parse.

The <data> element is really the deflated and encoded.

Thanks for you help.


<?xml version="1.0"?><request version="1.1"><from node="PS_SAIDP"/><to
node="PSFT_EP"/><operations namespace="PublishSubscribe"
interface="PublishSubscribeSystem"><invoke member="Publish"><variable
type="object"
interface="Publication"><publication><publishingnode>PS_SAIDP</publishingnode><channel>NU_TRANSCRIPT_VOUCHER_SYNC</channel><publicationid>4</publicationid><subject>NU_TRANSCRIPT_VOUCHER_SYNC</subject><class>PSAPMSG</class><originatingnode>PS_SAIDP</originatingnode><publisher>CMALEK</publisher><publicationprocess>NU_SR0021</publicationprocess><publishtimestamp>2006-01-12T17:32:16.973000-0800</publishtimestamp><status>4</status><defaultdataversion>VERSION_1</defaultdataversion><dataversions><publicationdataversion><version>VERSION_1</version><data
encoding="base64(deflate)" encodedlength="808(605)"
length="4950">eJztmF1vmzAUhv9KlJteZdgQCEjElWNcghSc1JiqubKiLpoqZd1Uomn79zMfTY1Jq1VJ14ty5/O+PseHk0dWILz8/X03+LV9LO5/PEwv4BdwcYlClkvBMcsIT1ZC3ixzMqdcZmtGUHh1v919FX9+botmH5M3ZM5lhMXgbrcpiumQD1FI09UiiQZ7tXE6JHPMh5YSb4Vc8lh2DFWo9AjmlPLjnqqPJZs9uSxPZ7TyM3qtgo781PWxswinWFB5tcBx11zxhFXH0cYrl6VXG7NciCXrptkskpxe5zQThtmoMhJGQZVVLuUqny2SbG64ljFd1UBGcIr1KteS8wi3McU0nMp8R5lAiJiTBbneGMypfz0iyWaqJqNuZoVnxJaJbJhGUCM0I7E28eQ/J8QbtjV4XZMqIMp9RwrPq51EJnKy2+RZv9BoXicfNQbO72CtB/IA4BACAIAjcIrUbRsEPABtWGsXKf1TZ/KI95VI1f00wMETzsOEgNiqXVrDQKUQB91ZMmGCSidVVRV9o4ItW7O4JwBOqzNafFZjnRVtimE7EqW1c0RFWyHhiIli146vwRtKsiLe+txCLK4tDSBY1aVU0PTGrr3I6q46sqtCKTXwTKORmaCXE5S1PRSK7P0KIDyi1q347w2HE9Fx5HGMLxGDi+/54Ig6MIO14wOQPCYDyy7R7hz8wv9GznA/idlFdnz2/P76n82q7vfsj96/f89vyezq8T2O/6F/jF+xf0/Pb8nsav+gGgM/Em//8VTt2/53iFO53f9av81of4I9vrST4zydbhs4T1ykeyv+QRmHM=</data></publicationdataversion></dataversions></publication></variable></invoke></operations></request>

On 2006-01-17, Chris Malek <chris.r.malek@gmail.com> wrote:
> Here is the XML message that I am really try to parse.
>
> The <data> element is really the deflated and encoded.

> <data encoding="base64(deflate)" encodedlength="808(605)" length="4950">eJzt

length isn't 808, I count 813

anyway your problem, could it be that the decoder needs to be primed.



Bye.
   Jasen

Chris Malek wrote:
> Here is the XML message that I am really try to parse.
....
<data encoding="base64(deflate)" encodedlength="808(605)"
length="4950">
eJztmF1vmzAUhv9KlJteZdgQCEjElWNcghSc1JiqubKiLpoqZd1Uomn79zMfTY1Jq1VJ14ty5/O+PseHk0dWILz8/X03+LV9LO5/PEwv4BdwcYlClkvBMcsIT1ZC3ixzMqdcZmtGUHh1v919FX9+botmH5M3ZM5lhMXgbrcpiumQD1FI09UiiQZ7tXE6JHPMh5YSb4Vc8lh2DFWo9AjmlPLjnqqPJZs9uSxPZ7TyM3qtgo781PWxswinWFB5tcBx11zxhFXH0cYrl6VXG7NciCXrptkskpxe5zQThtmoMhJGQZVVLuUqny2SbG64ljFd1UBGcIr1KteS8wi3McU0nMp8R5lAiJiTBbneGMypfz0iyWaqJqNuZoVnxJaJbJhGUCM0I7E28eQ/J8QbtjV4XZMqIMp9RwrPq51EJnKy2+RZv9BoXicfNQbO72CtB/IA4BACAIAjcIrUbRsEPABtWGsXKf1TZ/KI95VI1f00wMETzsOEgNiqXVrDQKUQB91ZMmGCSidVVRV9o4ItW7O4JwBOqzNafFZjnRVtimE7EqW1c0RFWyHhiIli146vwRtKsiLe+txCLK4tDSBY1aVU0PTGrr3I6q46sqtCKTXwTKORmaCXE5S1PRSK7P0KIDyi1q347w2HE9Fx5HGMLxGDi+/54Ig6MIO14wOQPCYDyy7R7hz8wv9GznA/idlFdnz2/P76n82q7vfsj96/f89vyezq8T2O/6F/jF+xf0/Pb8nsav+gGgM/Em//8VTt2/53iFO53f9av81of4I9vrST4zydbhs4T1ykeyv+QRmHM=
</data>

There are several lines of evidence that the quoted data is in error.
First, there are supposed to be 808 base64 characters in the data, but
there are 810.  Furthermore, the single pad ("=") base64 character says
that there are supposed to be 4n+3 base64 characters before the pad,
but in fact there are 4n'+1, agreeing with the count mismatch.
Finally, zlib finds a compressed data error in the middle.  One
hypothesis that matches the data is that two base64 characters were
inserted somewhere in the data (not necessarily next to each other),
which is causing the zlib detected error.  The error is detected
somewhere around the "McU0", but the actual error is likely to be at
some point before that.

mark