Dear all,

I am using saszipam to read zipped csv files (code at the end of email). I
am getting an error message because there is an extra empty line in the csv
file. If I remove this line then I can unzip and read the csv, otherwise I
get an error message. I don't mind removing the line but I have over 1000
files to unzip. Is there a different way to remove the extra line from the
zipped csv file?

Thanks for the time.
Elvira

filename in saszipam 'F:\Cash\EuronextAmsterdam-BBO-200404-1.zip';
data test;
infile in(EuronextAmsterdam-BBO-200404-1.csv)
   dlm = ';' MISSOVER DSD lrecl=32767 firstobs=2  ;
   informat Internal_code best32. ;
   informat ISIN_code  $9.;
   informat Instrument_name $35. ;
   informat Quotation_place $9.;
   informat BBO_date yymmdd10.;
   informat BBO_time time12.;
   informat BBO_number best32.;
   informat Best_bid_price best32.;
   informat Best_ask_price best32.;
   informat Size_best_bid best32. ;
   informat Size_best_ask best32. ;
   informat Number_orders_best_bid best32. ;
   informat Number_orders_best_ask best32. ;

input
   Internal_code
      ISIN_code $
      Instrument_name $
         Quotation_place $
      BBO_date $
      BBO_time $
   BBO_number
      Best_bid_price
      Best_ask_price
      Size_best_bid
      Size_best_ask
      Number_orders_best_bid
      Number_orders_best_ask
   ;
run;

Doesn't sound like a problem with the compression but with the file.
What is the actual error message? What happens if you unzip one of the
files and read the unzipped file directly.  Do you get the same error
message?
Debug the program using the unzipped file and then try again reading
from the zip file.


On Jan 7, 11:12=A0am, eso...@RSM.NL (Elvira Sojli) wrote:
> Dear all,
>
> I am using saszipam to read zipped csv files (code at the end of email). =
I
> am getting an error message because there is an extra empty line in the c=
sv
> file. If I remove this line then I can unzip and read the csv, otherwise =
I
> get an error message. I don't mind removing the line but I have over 1000
> files to unzip. Is there a different way to remove the extra line from th=
e
> zipped csv file?
>
> Thanks for the time.
> Elvira
>
> filename in saszipam 'F:\Cash\EuronextAmsterdam-BBO-200404-1.zip';
> data test;
> infile in(EuronextAmsterdam-BBO-200404-1.csv)
> =A0 =A0dlm =3D ';' MISSOVER DSD lrecl=3D32767 firstobs=3D2 =A0;
> =A0 =A0informat Internal_code best32. ;
> =A0 =A0informat ISIN_code =A0$9.;
> =A0 =A0informat Instrument_name $35. ;
> =A0 =A0informat Quotation_place $9.;
> =A0 =A0informat BBO_date yymmdd10.;
> =A0 =A0informat BBO_time time12.;
> =A0 =A0informat BBO_number best32.;
> =A0 =A0informat Best_bid_price best32.;
> =A0 =A0informat Best_ask_price best32.;
> =A0 =A0informat Size_best_bid best32. ;
> =A0 =A0informat Size_best_ask best32. ;
> =A0 =A0informat Number_orders_best_bid best32. ;
> =A0 =A0informat Number_orders_best_ask best32. ;
>
> input
> =A0 =A0Internal_code
> =A0 =A0 =A0 ISIN_code $
> =A0 =A0 =A0 Instrument_name $
> =A0 =A0 =A0 =A0 =A0Quotation_place $
> =A0 =A0 =A0 BBO_date $
> =A0 =A0 =A0 BBO_time $
> =A0 =A0BBO_number
> =A0 =A0 =A0 Best_bid_price
> =A0 =A0 =A0 Best_ask_price
> =A0 =A0 =A0 Size_best_bid
> =A0 =A0 =A0 Size_best_ask
> =A0 =A0 =A0 Number_orders_best_bid
> =A0 =A0 =A0 Number_orders_best_ask
> =A0 =A0;
> run;

Elvira,

saszipam is an undocumented feature thus, if this is production code, the
best advice would be to use an external zip package.

However, if it isn't production code, then why not simply read the file,
with saszipam, as lines of text and write all but the last line of each
file to a new csv file .. which you should then be able to use as input
without confronting the current error.

Art
--------
On Thu, 7 Jan 2010 11:12:41 -0500, Elvira Sojli <esojli@RSM.NL> wrote:

>Dear all,
>
>I am using saszipam to read zipped csv files (code at the end of email). I
>am getting an error message because there is an extra empty line in the
csv
>file. If I remove this line then I can unzip and read the csv, otherwise I
>get an error message. I don't mind removing the line but I have over 1000
>files to unzip. Is there a different way to remove the extra line from the
>zipped csv file?
>
>Thanks for the time.
>Elvira
>
>filename in saszipam 'F:\Cash\EuronextAmsterdam-BBO-200404-1.zip';
>data test;
>infile in(EuronextAmsterdam-BBO-200404-1.csv)
>   dlm = ';' MISSOVER DSD lrecl=32767 firstobs=2  ;
>   informat Internal_code best32. ;
>   informat ISIN_code  $9.;
>   informat Instrument_name $35. ;
>   informat Quotation_place $9.;
>   informat BBO_date yymmdd10.;
>   informat BBO_time time12.;
>   informat BBO_number best32.;
>   informat Best_bid_price best32.;
>   informat Best_ask_price best32.;
>   informat Size_best_bid best32. ;
>   informat Size_best_ask best32. ;
>   informat Number_orders_best_bid best32. ;
>   informat Number_orders_best_ask best32. ;
>
>input
>   Internal_code
>      ISIN_code $
>      Instrument_name $
>         Quotation_place $
>      BBO_date $
>      BBO_time $
>   BBO_number
>      Best_bid_price
>      Best_ask_price
>      Size_best_bid
>      Size_best_ask
>      Number_orders_best_bid
>      Number_orders_best_ask
>   ;
>run;

Dear Tom and Art,

Thank you for your prompt replies. Indeed the problem is with the csv =
file itself, i.e. if I try to just open the csv without using the upzip =
procedure the same error message occurs. If I manually delete the last =
line then SAS can read the file easily.

Art, thanks for the suggestion but I am not sure how to implement it. I =
need some help on how to do the following: 'simply read the file, with =
saszipam, as lines of text and write all but the last line of each file =
to a new csv file'. To complicate things further, I don't know where the =
last line of each file occurs, and each file could have more than 3 =
million lines.

Thanks for the time :).

Kind regards,
Elvira

-----Original Message-----
From: SAS(r) Discussion on behalf of Tom Abernathy
Sent: Thu 07/01/2010 18:35
To: SAS-L@LISTSERV.UGA.EDU
Subject:      Re: Error in reading compressed file
=20
Doesn't sound like a problem with the compression but with the file.
What is the actual error message? What happens if you unzip one of the
files and read the unzipped file directly.  Do you get the same error
message?
Debug the program using the unzipped file and then try again reading
from the zip file.


On Jan 7, 11:12 am, eso...@RSM.NL (Elvira Sojli) wrote:
> Dear all,
>
> I am using saszipam to read zipped csv files (code at the end of =
email). I
> am getting an error message because there is an extra empty line in =
the csv
> file. If I remove this line then I can unzip and read the csv, =
otherwise I
> get an error message. I don't mind removing the line but I have over =
1000
> files to unzip. Is there a different way to remove the extra line from =
the
> zipped csv file?
>
> Thanks for the time.
> Elvira
>
> filename in saszipam 'F:\Cash\EuronextAmsterdam-BBO-200404-1.zip';
> data test;
> infile in(EuronextAmsterdam-BBO-200404-1.csv)
>    dlm =3D ';' MISSOVER DSD lrecl=3D32767 firstobs=3D2  ;
>    informat Internal_code best32. ;
>    informat ISIN_code  $9.;
>    informat Instrument_name $35. ;
>    informat Quotation_place $9.;
>    informat BBO_date yymmdd10.;
>    informat BBO_time time12.;
>    informat BBO_number best32.;
>    informat Best_bid_price best32.;
>    informat Best_ask_price best32.;
>    informat Size_best_bid best32. ;
>    informat Size_best_ask best32. ;
>    informat Number_orders_best_bid best32. ;
>    informat Number_orders_best_ask best32. ;
>
> input
>    Internal_code
>       ISIN_code $
>       Instrument_name $
>          Quotation_place $
>       BBO_date $
>       BBO_time $
>    BBO_number
>       Best_bid_price
>       Best_ask_price
>       Size_best_bid
>       Size_best_ask
>       Number_orders_best_bid
>       Number_orders_best_ask
>    ;
> run;


=0A--------------------------------Disclaimer------------------------------=
--=0ADe informatie  verzonden in dit e-mail bericht  inclusief de bijlage(n)=
 is=0Avertrouwelijk  en is  uitsluitend  bestemd  voor de geadresseerde  van=
 dit=0Abericht. Lees verder: http://www.eur.nl/email-disclaimer=0A=0AThe inf=
ormation in this e-mail message  is confidential and may be legally=0Aprivil=
eged. Read more: http://www.eur.nl/english/email-disclaimer=0A--------------=
------------------------------------------------------------=0A

Elvira,

I'd be glad to show an example of my suggestion but, before doing so, I have
a comment, a question and an alternative approach (although you could end up
still needing to do what I had suggested or something else).

I put together and compressed a small csv file and, to my surprise, saszipam
wasn't able to read it regardless of whether it contained an extra line or
not.

For some alternative command line possibilities, including freeware and
shareware compression routines, take a look at:

www2.sas.com/proceedings/sugi31/155-31.pdf

i.e., before suggesting how you might skip the last line (which, by the way,
may only entail using something like eof=lastrec in your infile statement),
what kind of error message are you receiving.  It may be that saszipam
simply no longer works.

Art
-------
On Thu, 7 Jan 2010 21:26:46 +0100, Elvira Sojli <ESojli@RSM.NL> wrote:

>Dear Tom and Art,
>
>Thank you for your prompt replies. Indeed the problem is with the csv file
itself, i.e. if I try to just open the csv without using the upzip procedure
the same error message occurs. If I manually delete the last line then SAS
can read the file easily.
>
>Art, thanks for the suggestion but I am not sure how to implement it. I
need some help on how to do the following: 'simply read the file, with
saszipam, as lines of text and write all but the last line of each file to a
new csv file'. To complicate things further, I don't know where the last
line of each file occurs, and each file could have more than 3 million
lines.
>
>Thanks for the time :).
>
>Kind regards,
>Elvira
>
>-----Original Message-----
>From: SAS(r) Discussion on behalf of Tom Abernathy
>Sent: Thu 07/01/2010 18:35
>To: SAS-L@LISTSERV.UGA.EDU
>Subject:      Re: Error in reading compressed file
>
>Doesn't sound like a problem with the compression but with the file.
>What is the actual error message? What happens if you unzip one of the
>files and read the unzipped file directly.  Do you get the same error
>message?
>Debug the program using the unzipped file and then try again reading
>from the zip file.
>
>
>On Jan 7, 11:12 am, eso...@RSM.NL (Elvira Sojli) wrote:
>> Dear all,
>>
>> I am using saszipam to read zipped csv files (code at the end of email).
I
>> am getting an error message because there is an extra empty line in the
csv
>> file. If I remove this line then I can unzip and read the csv, otherwise
I
>> get an error message. I don't mind removing the line but I have over 1000
>> files to unzip. Is there a different way to remove the extra line from
the
>> zipped csv file?
>>
>> Thanks for the time.
>> Elvira
>>
>> filename in saszipam 'F:\Cash\EuronextAmsterdam-BBO-200404-1.zip';
>> data test;
>> infile in(EuronextAmsterdam-BBO-200404-1.csv)
>>    dlm = ';' MISSOVER DSD lrecl=32767 firstobs=2  ;
>>    informat Internal_code best32. ;
>>    informat ISIN_code  $9.;
>>    informat Instrument_name $35. ;
>>    informat Quotation_place $9.;
>>    informat BBO_date yymmdd10.;
>>    informat BBO_time time12.;
>>    informat BBO_number best32.;
>>    informat Best_bid_price best32.;
>>    informat Best_ask_price best32.;
>>    informat Size_best_bid best32. ;
>>    informat Size_best_ask best32. ;
>>    informat Number_orders_best_bid best32. ;
>>    informat Number_orders_best_ask best32. ;
>>
>> input
>>    Internal_code
>>       ISIN_code $
>>       Instrument_name $
>>          Quotation_place $
>>       BBO_date $
>>       BBO_time $
>>    BBO_number
>>       Best_bid_price
>>       Best_ask_price
>>       Size_best_bid
>>       Size_best_ask
>>       Number_orders_best_bid
>>       Number_orders_best_ask
>>    ;
>> run;
>
>
>
--------------------------------Disclaimer--------------------------------
De informatie  verzonden in dit e-mail bericht  inclusief de bijlage(n) is
vertrouwelijk  en is  uitsluitend  bestemd  voor de geadresseerde  van dit
bericht. Lees verder: http://www.eur.nl/email-disclaimer

The information in this e-mail message  is confidential and may be legally
privileged. Read more: http://www.eur.nl/english/email-disclaimer
--------------------------------------------------------------------------