f



Using HTTPS with LWP::UserAgent = Bad Service

Hi all,

I'm trying to access https://www.nodeworks.com/ through a proxy which
requires a proxy username and password. The code I have is as follows:

#!/usr/bin/perl
use strict;
use LWP::UserAgent;
use Data::Dumper qw(Dumper);
# If they provide an argument, connect to https else http
my $url = "http://www.nodeworks.com";
$url = "https://www.nodeworks.com" if ($ARGV[0]);
print "Going to get $url\n";
my $ua = new LWP::UserAgent;
# According to Crypt::SSLeay:
#   "Use of the HTTPS_PROXY environment variable in this way is
#   similar to LWP::UserAgent->env_proxy() usage, but calling that
#   method will likely override or break the Crypt::SSLeay support,
#   so do not mix the two."
# so only define $ua->proxy when using http.
if ($url =~ /^https/i)
{
  print "Using HTTPS proxy $ENV{HTTPS_PROXY}\n";
}
else
{
  $ua->env_proxy();
  print "Using HTTP proxy $ENV{HTTP_PROXY}\n";
}
$ua->timeout(15);
my $req = new HTTP::Request('HEAD', $url);
my $res = $ua->request($req);
print "Result is:\n";
print Dumper($res);

My environment variable for http_proxy is set up like so:

http_proxy=http://richard:mypassword@10.10.200.44:8080/
HTTP_PROXY=http://richard:mypassword@10.10.200.44:8080/
export ...all of the above...

and running the code above with no arguments produces the desired
result.

However, if I try the code above with an argument (so it trys https)
with the following environment variables:

https_proxy=http://richard:mypassword@10.10.200.44:8080/
HTTPS_PROXY=http://richard:mypassword@10.10.200.44:8080/
export ...all of the above...

or even:

https_proxy=http://10.10.200.44:8080/
HTTPS_PROXY=http://10.10.200.44:8080/
https_proxy_username=richard
HTTPS_PROXY_USERNAME=richard
https_proxy_password=mypassword
HTTPS_PROXY_PASSWORD=mypassword
export ...all of the above...

then I always get the following error:

$VAR1 = bless( {
'_content' => '500 Can\'t connect to www.nodeworks.com:443 (Bad service
\'8080/\')',
'_rc' => 500,
'_headers' => bless( {
'client-warning' => 'Internal response',
'client-date' => 'Wed, 30 Mar 2005 13:41:12 GMT',
'content-type' => 'text/plain'
}, 'HTTP::Headers' ),
'_msg' => 'Can\'t connect to www.nodeworks.com:443 (Bad service
\'8080/\')',
'_request' => bless( {
'_content' => '',
'_uri' => bless( do{\(my $o = 'https://www.nodeworks.com')},
'URI::https' ),
'_headers' => bless( {
'user-agent' => 'libwww-perl/5.803'
}, 'HTTP::Headers' ),
'_method' => 'HEAD'
}, 'HTTP::Request' )
}, 'HTTP::Response' );

Can anyone offer any suggestions?

Many thanks,

Richard

0
3/30/2005 1:45:33 PM
comp.lang.perl.misc 33233 articles. 2 followers. brian (1246) is leader. Post Follow

5 Replies
30054 Views

Similar Articles

[PageSpeed] 14

["Followup-To:" set to comp.lang.perl.misc.]

Richard  Lawrence <richard.lawrence@gmail.com> kirjoitti 30.03.2005:
>
> I'm trying to access https://www.nodeworks.com/ through a proxy which
> requires a proxy username and password. The code I have is as follows:

I don't have a proxy that would require a username and a password, but
I've tested your code as far as I've can.  As far as I can tell, your
code works just as it should -- the problem is elsewhere.


> my $req = new HTTP::Request('HEAD', $url);
> my $res = $ua->request($req);
> print "Result is:\n";
> print Dumper($res);

Just so you know, you can get a much cleaner dump of the response by
using $res->as_string().  In fact, those four lines can be reduced to:

  print "Result is:\n", $ua->head($url)->as_string;


> However, if I try the code above with an argument (so it trys https)
> with the following environment variables:
>
> https_proxy=http://richard:mypassword@10.10.200.44:8080/

You don't need this; the uppercase version is the proper one.

> HTTPS_PROXY=http://richard:mypassword@10.10.200.44:8080/

It appears that Crypt::SSLeay doesn't support this syntax, and will
fail with an error similar to what you report.  So this is likely to
be at least a part of your problem.


> HTTPS_PROXY=http://10.10.200.44:8080/
> HTTPS_PROXY_USERNAME=richard
> HTTPS_PROXY_PASSWORD=mypassword

This, however, seems to work just fine.  Of course, I can't really
test if the authentication works or not, but at least there is no
error.  You're reporting that you get the same "Bad service" error
each time, but I'd really suggest you try it one more time, and make
sure $ENV{HTTPS_PROXY} really contains what you think it does.  You
could add something like this to your code to dump the relevant
environment variables:

  /^HTTPS?_PROXY/ and print "$_=$ENV{$_}\n" for sort keys %ENV;


Hope this helps.

-- 
Ilmari Karonen
To reply by e-mail, please replace ".invalid" with ".net" in address.
0
Ilmari
3/30/2005 9:42:24 PM
Ilmari Karonen wrote:
> > HTTPS_PROXY=http://10.10.200.44:8080/
> > HTTPS_PROXY_USERNAME=richard
> > HTTPS_PROXY_PASSWORD=mypassword
>
> This, however, seems to work just fine.  Of course, I can't really
> test if the authentication works or not, but at least there is no
> error.  You're reporting that you get the same "Bad service" error
> each time, but I'd really suggest you try it one more time, and make
> sure $ENV{HTTPS_PROXY} really contains what you think it does.  You
> could add something like this to your code to dump the relevant
> environment variables:
>
>   /^HTTPS?_PROXY/ and print "$_=$ENV{$_}\n" for sort keys %ENV;

Hi there,

Thanks for your advise. I've made the changes you recommended and get
the following output:

$ perl testproxy.pl 1
HTTPS_PROXY=http://10.10.200.44:8080/
HTTPS_PROXY_PASSWORD=mypassword
HTTPS_PROXY_USERNAME=richard
HTTP_PROXY=http://10.10.199.45:80/
Going to get https://www.nodeworks.com
Using HTTPS proxy http://10.10.200.44:8080/
Result is:
500 (Internal Server Error) Can't connect to www.nodeworks.com:443 (Bad
service '8080/')
Content-Type: text/plain
Client-Date: Thu, 31 Mar 2005 08:35:51 GMT
Client-Warning: Internal response

500 Can't connect to www.nodeworks.com:443 (Bad service '8080/')

Interestingly enough, using "wget" and "lynx" both give me a timeout
error so I'm wondering if it is actually something to do with my
environment settings.

In that case, I think I need to find another newsgroup!

Regards,

Richard.

0
Richard
3/31/2005 8:43:18 AM
Richard  Lawrence wrote:
> $ perl testproxy.pl 1
> HTTPS_PROXY=http://10.10.200.44:8080/
> HTTPS_PROXY_PASSWORD=mypassword
> HTTPS_PROXY_USERNAME=richard
> HTTP_PROXY=http://10.10.199.45:80/
> Going to get https://www.nodeworks.com
> Using HTTPS proxy http://10.10.200.44:8080/
> Result is:
> 500 (Internal Server Error) Can't connect to www.nodeworks.com:443
(Bad
> service '8080/')
> Content-Type: text/plain
> Client-Date: Thu, 31 Mar 2005 08:35:51 GMT
> Client-Warning: Internal response
>
> 500 Can't connect to www.nodeworks.com:443 (Bad service '8080/')
>
> Interestingly enough, using "wget" and "lynx" both give me a timeout
> error so I'm wondering if it is actually something to do with my
> environment settings.

As an update, I've set up a cascading proxy on my computer which runs
locally and then redirects everything through the main proxy with the
correct username and password.

Setting HTTP_PROXY and HTTPS_PROXY to http://127.0.0.1:8080/ with no
username and password (plus their lowercase versions for the benefit of
lynx which doesn't see the upper-case ones) works just fine.

However, my perl code still reports "500 (Internal Server Error) Can't
connect to www.nodeworks.com:443 (Bad service '8080/')" which brings me
back around full circle to thinking that my implementation is wrong.

If you have any suggestions, it would be appreciated!

Thanks,

Richard.

0
Richard
3/31/2005 8:57:34 AM
Richard  Lawrence <richard.lawrence@gmail.com> kirjoitti 31.03.2005:
>
> $ perl testproxy.pl 1
> HTTPS_PROXY=http://10.10.200.44:8080/
> HTTPS_PROXY_PASSWORD=mypassword
> HTTPS_PROXY_USERNAME=richard
> HTTP_PROXY=http://10.10.199.45:80/
> Going to get https://www.nodeworks.com
> Using HTTPS proxy http://10.10.200.44:8080/
> Result is:
> 500 (Internal Server Error) Can't connect to www.nodeworks.com:443 (Bad
> service '8080/')

Ah, I think I see what the problem is now.

Try getting rid of the trailing slash after ":8080".

-- 
Ilmari Karonen
To reply by e-mail, please replace ".invalid" with ".net" in address.
0
Ilmari
4/1/2005 5:18:00 PM
Ilmari Karonen wrote:
> Richard  Lawrence <richard.lawrence@gmail.com> kirjoitti 31.03.2005:
> >
> > $ perl testproxy.pl 1
> > HTTPS_PROXY=http://10.10.200.44:8080/
> > HTTPS_PROXY_PASSWORD=mypassword
> > HTTPS_PROXY_USERNAME=richard
> > HTTP_PROXY=http://10.10.199.45:80/
> > Going to get https://www.nodeworks.com
> > Using HTTPS proxy http://10.10.200.44:8080/
> > Result is:
> > 500 (Internal Server Error) Can't connect to www.nodeworks.com:443
(Bad
> > service '8080/')
>
> Ah, I think I see what the problem is now.
>
> Try getting rid of the trailing slash after ":8080".

Worked like a charm. Thank you very much for all your help.

Richard.

0
Richard
4/4/2005 4:46:43 PM
Reply: