Dear all,

I am recently interested in network programming in IDL because I want to download geomagnetic Kp indices from a NOAA FTP server. The data files are under:
ftp://ftp.ngdc.noaa.gov/STP/GEOMAGNETIC_DATA/INDICES/KP_AP

I tried both SOCKET and IDLnetURL, but didn't get much of luck in either case. The platforms I used include 1) OS X Lion + IDL 8.2 and 2) Snow Leopard + IDL 7.1/8.1.

First, for socket, I used a solution, *read_ftp*, from the IDL official website:
http://www.exelisvis.com/language/en-US/Support/HelpArticleDetail/articleid/3537.aspx?dnnprintmode=true&mid=6229&SkinSrc=%5BG%5DSkins%2F_default%2FNo+Skin&ContainerSrc=%5BG%5DContainers%2F_default%2FNo+Container

Here is my code:
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
url = 'ftp://ftp.ngdc.noaa.gov/STP/GEOMAGNETIC_DATA/INDICES/KP_AP/2008'
read_ftp, url, file = '2008'
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
Below between the two star lines is what happened in IDL:
***********************************************************************************************
IDL> url = 'ftp://ftp.ngdc.noaa.gov/STP/GEOMAGNETIC_DATA/INDICES/KP_AP/2008'
IDL> read_ftp, url, file = '2008'

<220-                     ----- Notice -----
<220-
<220- This is a United States Department of Commerce computer system, which
<220- may be accessed and used only for official Government business by
<220- authorized personnel. Unauthorized access or use of this computer
<220- system may subject violators to criminal, civil, and/or administrative
<220- action.
<220-
<220- All information on this computer system may be intercepted, recorded,
<220- read, copied, and disclosed by and to authorized personnel for
<220- official purposes, including criminal investigations. Access or use of
<220- this computer system by any person whether authorized or unauthorized,
<220- constitutes consent to these terms.
<220-
<220-                     ----- Notice -----
<220-
<220- Questions/Problems should be directed to ngdc.linux-admin@noaa.gov
<220 
>USER anonymous
**************************************************************************************************
IDL became frozen after I got above output.

Second, for IDLnetURL, I used a solution described in this article:
http://www.exelisvis.com/Support/HelpArticleDetail/ArticleId/4246.aspx

Here is my code:
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
oUrl = OBJ_NEW('IDLnetUrl', URL_SCHEME='ftp', $
    URL_HOST='ftp.ngdc.noaa.gov', $
    URL_Port=21, $
    URL_USERNAME='anonymous', URL_PASSWORD='', $
    URL_PATH='STP/GEOMAGNETIC_DATA/INDICES/KP_AP/2008')
downloadLoc = oURL->Get(FILENAME='2008')
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
I got this output, after I waited more than 10 minutes:
***********************************************************************************
% IDLNETURL::GET:  CCurlException:  Error: Ftp Get Request Failed. Error = select/poll error, Curl
              Error Code = 27. Last Ftp Response = 250 Directory successfully changed.
***********************************************************************************

So, does anyone know a working solution to download a file from an FTP server, please? Your help is much appreciated!

Best regards,
JBT

Le samedi 21 juillet 2012 22:50:05 UTC+2, JBT a =E9crit=A0:
> Dear all,
> ...
> So, does anyone know a working solution to download a file from an FTP se=
rver, please? Your help is much appreciated!
>=20
> Best regards,
> JBT

Your last code (using IDLnetURL) works perfectly on my machine (Windows 7, =
IDL8.2). Did you authorize IDL to communicate through the firewall of your =
machine ?
alain.

JBT writes: 

> So, does anyone know a working solution to download a file from an FTP server, please? Your help is much appreciated!

I had no trouble whatsoever accessing this file. Here is
the code I used. Windows 7 64-bit OS, IDL 7.1.2.

oUrl = OBJ_NEW('IDLnetUrl', URL_SCHEME='ftp', $
    URL_HOST='ftp.ngdc.noaa.gov', $
    URL_Port=21, $
    URL_USERNAME='anonymous', URL_PASSWORD='', $
    URL_PATH='STP/GEOMAGNETIC_DATA/INDICES/KP_AP/2008')
void = oURL -> Get(FILENAME='C:\IDL\test\file_2008')

The file was retrieved nearly instantaneously.

I've gotten this CCurlException error quite a bit working
with Google static maps. It is essential that the
directory where the file will be saved actually 
exists and is writeable when the Get method is
called. In my case, the problem has almost always been
writing to a particular directory. I always give it
an absolute file name, but I'm not sure if that is
a learned behavior or not. :-)

Cheers,

David


-- 
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")

On Saturday, July 21, 2012 2:22:36 PM UTC-7, alx wrote:
> Le samedi 21 juillet 2012 22:50:05 UTC+2, JBT a =E9crit=A0:
> > Dear all,
> > ...
> > So, does anyone know a working solution to download a file from an F=
TP server, please? Your help is much appreciated!
> >=20
> > Best regards,
> > JBT
>=20
> Your last code (using IDLnetURL) works perfectly on my machine (Windows 7=
, IDL8.2). Did you authorize IDL to communicate through the firewall of you=
r machine ?
> alain.

Thanks, Alain, David.=20

I turned off the firewall on my computer and tried the IDLnetURL code again=
 with an absolute file path for the local filename, but still got no luck. =
So, I guess it has something do with the difference between the Windows ver=
sion and the Mac version of IDL. Does anybody have a clue on this, please?

JBT

Hello,

On 07/21/12 19:01, JBT wrote:
> On Saturday, July 21, 2012 2:22:36 PM UTC-7, alx wrote:
>> Le samedi 21 juillet 2012 22:50:05 UTC+2, JBT a �crit : > Dear all, > ... > So, does anyone know a working
>> solution to download a file from an FTP server, please? Your help is much appreciated! > > Best regards, >
>> JBT
>> 
>> Your last code (using IDLnetURL) works perfectly on my machine (Windows 7, IDL8.2). Did you authorize IDL to
>> communicate through the firewall of your machine ? alain.
> 
> Thanks, Alain, David.
> 
> I turned off the firewall on my computer and tried the IDLnetURL code again with an absolute file path for the local
> filename, but still got no luck. So, I guess it has something do with the difference between the Windows version and
> the Mac version of IDL. Does anybody have a clue on this, please?

No joy on my linux setup either:

IDL> print, !version
{ x86 linux unix linux 8.1 Mar  9 2011      32      64}
IDL> oUrl = OBJ_NEW('IDLnetUrl', URL_SCHEME='ftp', $
IDL>     URL_HOST='ftp.ngdc.noaa.gov', $
IDL>     URL_Port=21, $
IDL>     URL_USERNAME='anonymous', URL_PASSWORD='', $
IDL>     URL_PATH='STP/GEOMAGNETIC_DATA/INDICES/KP_AP/2008')
% Loaded DLM: URL.
IDL> downloadLoc = oURL->Get(FILENAME='2008')
% IDLNETURL::GET:  CCurlException:  Error: Ftp Get Request Failed. Error = RETR response: 425,
              Curl Error Code = 19. Last Ftp Response = 425 Failed to establish connection.

Different error.

When I just do the ftp access on the command line everything worked fine.

Maybe it's a unix-related thing. I.e. IDL can't do this properly on a non-windows platform?

cheers,

paulv

On 7/21/12 3:50 PM, JBT wrote:
> Dear all,
>
> So, does anyone know a working solution to download a file from an FTP server, please? Your help is much appreciated!
>

Both socket and IDLnetUrl methods worked on my OS X Lion + IDL 8.2:

IDL> help,!version,/st
** Structure !VERSION, 8 tags, length=104, data length=100:
    ARCH            STRING    'x86_64'
    OS              STRING    'darwin'
    OS_FAMILY       STRING    'unix'
    OS_NAME         STRING    'Mac OS X'
    RELEASE         STRING    '8.2'
    BUILD_DATE      STRING    'Apr 10 2012'
    MEMORY_BITS     INT             64
    FILE_OFFSET_BITS
                    INT             64
IDL>
IDL> url = 'ftp://ftp.ngdc.noaa.gov/STP/GEOMAGNETIC_DATA/INDICES/KP_AP/2008'
IDL> read_ftp, url, file = '2008'
<220-                     ----- Notice -----
<220-
<220- This is a United States Department of Commerce computer system, which
<220- may be accessed and used only for official Government business by
<220- authorized personnel. Unauthorized access or use of this computer
<220- system may subject violators to criminal, civil, and/or administrative
<220- action.
<220-
<220- All information on this computer system may be intercepted, recorded,
<220- read, copied, and disclosed by and to authorized personnel for
<220- official purposes, including criminal investigations. Access or use of
<220- this computer system by any person whether authorized or unauthorized,
<220- constitutes consent to these terms.
<220-
<220-                     ----- Notice -----
<220-
<220- Questions/Problems should be directed to ngdc.linux-admin@noaa.gov
<220
 >USER anonymous
 >PASS test@test.com
 >TYPE I
 >CWD /STP/GEOMAGNETIC_DATA/INDICES/KP_AP
 >SIZE 2008
 >PASV
 >RETR 2008
 >QUIT
IDL>

And as David noted, the IDLnetUrl method was almost instantaneous. The 
socket method took 45 sec.

-Caesar

On 7/23/12 8:58 AM,  C.E. Ordonez wrote:
> On 7/21/12 3:50 PM, JBT wrote:
>> Dear all,
>>
>> So, does anyone know a working solution to download a file from an FTP
>> server, please? Your help is much appreciated!
>>
>
> Both socket and IDLnetUrl methods worked on my OS X Lion + IDL 8.2:
>
> IDL> help,!version,/st
> ** Structure !VERSION, 8 tags, length=104, data length=100:
>     ARCH            STRING    'x86_64'
>     OS              STRING    'darwin'
>     OS_FAMILY       STRING    'unix'
>     OS_NAME         STRING    'Mac OS X'
>     RELEASE         STRING    '8.2'
>     BUILD_DATE      STRING    'Apr 10 2012'
>     MEMORY_BITS     INT             64
>     FILE_OFFSET_BITS
>                     INT             64
> IDL>
> IDL> url =
> 'ftp://ftp.ngdc.noaa.gov/STP/GEOMAGNETIC_DATA/INDICES/KP_AP/2008'
> IDL> read_ftp, url, file = '2008'
> <220-                     ----- Notice -----
> <220-
> <220- This is a United States Department of Commerce computer system, which
> <220- may be accessed and used only for official Government business by
> <220- authorized personnel. Unauthorized access or use of this computer
> <220- system may subject violators to criminal, civil, and/or
> administrative
> <220- action.
> <220-
> <220- All information on this computer system may be intercepted, recorded,
> <220- read, copied, and disclosed by and to authorized personnel for
> <220- official purposes, including criminal investigations. Access or
> use of
> <220- this computer system by any person whether authorized or
> unauthorized,
> <220- constitutes consent to these terms.
> <220-
> <220-                     ----- Notice -----
> <220-
> <220- Questions/Problems should be directed to ngdc.linux-admin@noaa.gov
> <220
>  >USER anonymous
>  >PASS test@test.com
>  >TYPE I
>  >CWD /STP/GEOMAGNETIC_DATA/INDICES/KP_AP
>  >SIZE 2008
>  >PASV
>  >RETR 2008
>  >QUIT
> IDL>
>
> And as David noted, the IDLnetUrl method was almost instantaneous. The
> socket method took 45 sec.

Curious. Worked for me on Linux:

IDL> print, !version
{ x86_64 linux unix linux 8.0.1 Oct  7 2010      64      64}

But not on OS X (Lion):

IDL> print, !version
{ x86_64 darwin unix Mac OS X 8.2 Apr 10 2012      64      64}

Mike
-- 
Michael Galloy
www.michaelgalloy.com
Modern IDL: A Guide to IDL Programming (http://modernidl.idldev.com)
Research Mathematician
Tech-X Corporation

FTP have two mode: active and passive. 
Try to look at FTP_CONNECTION_MODE property and may be change it
Some FTP servers supports only passive mode
from IDL help FTP_CONNECTION_MODE - "Active (this is the default)"
 
 

On Sunday, July 29, 2012 12:24:25 PM UTC-7, skymaxwell wrote:
> FTP have two mode: active and passive. 
> 
> Try to look at FTP_CONNECTION_MODE property and may be change it
> 
> Some FTP servers supports only passive mode
> 
> from IDL help FTP_CONNECTION_MODE - "Active (this is the default)"



Thank you so much, Skymaxwell! After I specified FTP_CONNECTION_MODE = 0 inside obj_new( ) for oUrl, the code worked like a snap.

JBT