How Do I Use Proxy in Net:HTTP?

  • Permalink
  • submit to reddit
  • Email
  • Follow


Can anyone tell me how the proxy is implemented in LWP:UserAgent? I have to
use a proxy in Net:HTTP, but proxy is not supported in it.


0
Reply ir (3) 10/15/2003 6:26:03 PM

See related articles to this posting


On Wed, 15 Oct 2003 18:26:03 GMT, "Public Interest" <ir@labranche.com>
wrote:

>Can anyone tell me how the proxy is implemented in LWP:UserAgent? 
>I have to use a proxy in Net:HTTP, but proxy is not supported in it.

$ua->proxy(['http'] => "http://$proxy:$port");
0
Reply Helgi 10/15/2003 7:07:43 PM

"Public Interest" <ir@labranche.com> writes:

> Can anyone tell me how the proxy is implemented in LWP:UserAgent? I have to
> use a proxy in Net:HTTP, but proxy is not supported in it.

Net::HTTP support proxies just fine.  RFC 2616 explains how to do it.
You basically just provide full absolute URLs with the requests you
send.

-- 
Gisle Aas
0
Reply gisle 10/15/2003 10:22:10 PM

Please give me a line or two showing how to use proxy..

In UserAgent it is:
$ua->proxy(['http', 'ftp'], 'http://proxy.sn.no:8001/');

Can do use the same line in Net:Http? I don' know how use use 2616.

<gisle@activestate.com> ????
news:m3ad82rvjh.fsf@eik.i-did-not-set--mail-host-address--so-shoot-me...
> "Public Interest" <ir@labranche.com> writes:
>
> > Can anyone tell me how the proxy is implemented in LWP:UserAgent? I have
to
> > use a proxy in Net:HTTP, but proxy is not supported in it.
>
> Net::HTTP support proxies just fine.  RFC 2616 explains how to do it.
> You basically just provide full absolute URLs with the requests you
> send.
>
> -- 
> Gisle Aas


0
Reply Public 10/19/2003 4:51:53 AM

Public Interest wrote:

>Please give me a line or two showing how to use proxy..
>
>In UserAgent it is:
>$ua->proxy(['http', 'ftp'], 'http://proxy.sn.no:8001/');
>
>Can do use the same line in Net:Http? I don' know how use use 2616.

No. As Gisle Aas wrote, you have to do a low level connection yourself.
Thank goodness, the difference between a direct connection and a
connection via proxy is minimal. The gist is:

For a normal connection, you 
 - connect to  a host/port, and 
 - request for a URL that looks like an absolute Unix path. (= relative
to the domain root)

For a proxy connection, you
 - connect to the proxy, and
 - request for an absolute URL, including protocol ("http"), domain and
optionally, port.

Here's an example modified from the Synopsis of Net::Http, which
requests a page on a port different from the default for http, 80:

	use Net::HTTP;
	my $s = Net::HTTP->new(Host => "modperl.com", PeerPort => 9000)
 	  or die $@;
	$s->write_request(GET => "/perl_networking/errata.html");
	my($code, $mess, %h) = $s->read_response_headers;

	print "Response code: $code ($mess)\n";
	use Data::Dumper;
	print Data::Dumper->Dump([\%h], ['*headers']);
	print "\n";

	{
	    my $n = $s->read_entity_body(my $buf, 1024);
	    die "read failed: $!" unless defined $n;
	    print $buf;
	    redo if $n;
	}

Here is it again, for another URL, and modified to connect via the proxy
of my ISP (<http://proxy.pandora.be:8080>):

	use Net::HTTP;
	my $s = Net::HTTP->new(Host => "httpd.apache.org",
	  PeerAddr => 'proxy.pandora.be', PeerPort => 8080) or die $@;
	$s->write_request(GET =>
	  "http://httpd.apache.org/docs/misc/FAQ.html");
	my($code, $mess, %h) = $s->read_response_headers;

	print "Response code: $code ($mess)\n";
	use Data::Dumper;
	print Data::Dumper->Dump([\%h], ['*headers']);
	print "\n";

	{
	    my $n = $s->read_entity_body(my $buf, 1024);
	    die "read failed: $!" unless defined $n;
	    print $buf;
	    redo if $n;
	}

IMO the code to retrieve data via a proxy is actually simpler than for a
normal connection, because (almost) everything but the URL for the GET
is constant.

n.b. I think the Host property is mainly important if you encounter name
based virtual hosts, otherwise you could get pages from the wrong
domain.

-- 
	Bart.
0
Reply Bart 10/19/2003 11:42:41 AM

Dear Master Bart,

Thank YOU.
My own code was:
 my $s = Net::HTTP->new(Host => '64.161.246.100:8080') || die $@;
 $s->write_request(GET => 'http://quote.yahoo.com', 'User-Agent' =>
'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)' );

So I did not know the syntax of using proxy, because it was not said in it.

My final question on this: Why do I have to put host here? Because I put
peeradd which is proxy server, peerport and the abs url. Your said the Host
is important for virtual domain, but you already has the full url to let
httpd to know? Can I do?

> my $s = Net::HTTP->new(
>   PeerAddr => 'proxy.pandora.be', PeerPort => 8080) or die $@;
> $s->write_request(GET =>
>   "http://httpd.apache.org/docs/misc/FAQ.html");

or

> my $s = Net::HTTP->new(Host => "httpd.apache.org",
>   PeerAddr => 'proxy.pandora.be', PeerPort => 8080) or die $@;
> $s->write_request(GET =>
>   "/docs/misc/FAQ.html");


..


0
Reply Public 10/20/2003 6:25:13 AM

One of things I don't undertand is you and I both read the same document on
Net::Http, there is no such info as what I need, but you can write a correct
code and I could not. Do you have a better documation file or a better cook
book? What you explained to me on how to use the proxy is no where found.
Don't know how you first guessed the syntax right.


0
Reply Public 10/20/2003 6:54:13 AM
comp.lang.perl.misc 32794 articles. 7 followers. Post

6 Replies
521 Views

Similar Articles

[PageSpeed] 11


  • Permalink
  • submit to reddit
  • Email
  • Follow


Reply:

Similar Artilces:

Net::HTTP::Proxy using one or more proxies
Hi to net/http/proxy developers, is possible to use multiple proxies in Net::HTTP::Proxy? Tips pls. Thanks and kind regards -botp Hi, In mail "Net::HTTP::Proxy using one or more proxies" "Pe=F1a, Botp" <botp@delmonte-phil.com> wrote: > is possible to use multiple proxies in Net::HTTP::Proxy? > Tips pls. In a word, No. You must use multiple Net::HTTP objects explicitly. Regards, Minero Aoki ...

Using Net::HTTP to proxy HTTPS
As far as I can tell from my experimenting, there is no way to proxy HTTPS requests using the Net::HTTP classes. My question is: can it be done? And if so, how? I've tried numerous variations, but the main gist of what I've tried is this code fragment: httpobj = Net::HTTP.new( host, port ) # this Proxy version below doesn't work to post to an HTTPS url through a proxy even though # it's supposed to have # the same methods as Net::HTTP, errors with a use_ssl is not a method # httpobj = Net::HTTP.Proxy( @proxy_host, @proxy_port, @proxy_user, @proxy_password ) httpobj...

How to use proxy in Net::HTTP, not in LWP::UserAgent?
I need some features that are in Net::HTTP but not in LWP::UserAgent. I used proxy in LWP::UserAgent, and I think LWP::UserAgent is using Net::HTTP, but I can not find how I can use proxy in Net::HTTP directly... deals@slip-12-64-108-121.mis.prserv.net (Great Deals) writes: > I need some features that are in Net::HTTP but not in LWP::UserAgent. Which feature is that? > I used proxy in LWP::UserAgent, and I think LWP::UserAgent is using > Net::HTTP, but I can not find how I can use proxy in Net::HTTP > directly... Read the HTTP spec. Or monitor how some application (like LW...

Nuby: using net/http*s* from behind a proxy server
Hey All, I've had reasonable luck using net/http from behind a proxy server here at work, but now I need to draw down SSL-encrypted pages via https. Here's my latest, unsuccessful attempt: # ------------------------------- require "uri" require "net/https" uri = URI.parse("https://appliedresearch.cancer.gov/crnportal/ directory") ghc_proxy = Net::HTTP.Proxy(PROXYSERVER, PROXYPORT) ghc_proxy.start(uri.host, uri.port) do |sess| sess.use_ssl = true puts("in block") end # ------------------------------- That gets me one of these here: c...

easy to use http proxy for windows that supports basic proxy authentication?
Will the suffering ever end?! I am searching for an easy to use http proxy for windows that supports basic proxy authentication. I've been trying quite a few, including winproxy, winroute, tinyproxy, et al, and nothing has fully worked yet... or some of them have worked TOO well (i.e. apparently assumed that I didn't need to authenticate because I was connecting from localhost, when basic proxy authentication is the thing I need to test!) All I'm trying to do is debug an app I'm writing that is using basic proxy authentication by checking that it is supplying some creden...

easy to use http proxy for windows that supports basic proxy authentication?
Will the suffering ever end?! I am searching for an easy to use http proxy for windows that supports basic proxy authentication. I've been trying quite a few, including winproxy, winroute, tinyproxy, et al, and nothing has fully worked yet... or some of them have worked TOO well (i.e. apparently assumed that I didn't need to authenticate because I was connecting from localhost, when basic proxy authentication is the thing I need to test!) All I'm trying to do is debug an app I'm writing that is using basic proxy authentication by checking that it is supplying some creden...

Using HTTP::Proxy
I am having trouble getting this to work. Can anyone tell me what I am doing wrong? This is simple test code. use HTTP::Proxy; use HTTP::Proxy::BodyFilter::simple; use HTTP::Proxy::BodyFilter::tags; use LWP::RobotUA; my $proxy = HTTP::Proxy->new; $proxy->port( 3128 ); # the classical accessors are here! $proxy->host(undef); print "post set\n"; $proxy->logmask(ALL); # you can also use your own UserAgent my $agent = LWP::RobotUA->new('my-robot/0.1', 'me@foo.com'); print "agent = $agent\n"; $proxy->agent( $agent ); $proxy->push_filter( ...

How to use max_siz in Net:Http
max_size is included in LWP:UserAgent? How do I use max_size or similar in Net:Http? ...

net/http: using cookies
Hey everyone, I'm trying to post a form to a site using net/http. I think it works, I'm not sure, but does net/http store cookies? Because I think that's the problem: loggin in isn't very hard, but the cookies aren't stored. Does anyone know of a way to do that? -- Best regards, Chris Eidhof >From a library I use at work ... def login(username, password, domain = nil) path = 'Login' data = %Q{<?xml version="1.0" ?> <authorization> <username><![CDATA[#{username}]]></username> ...

How to use a particular interface in net/http
Hi, Sorry if this has already been asked but I wasn't able to find the response via search. I'm doing some page scraping for a client and need to connect to their vpn server. So I have the vpn client running on my linux box, and this creates a new interface, say ppp0. Now, when I run Mechanize/open-uri, etc. I need to be able to bind specifically to this particular interface, because the other interfaces are to the internet in general. How do I do this? Thanks F -- Posted via http://www.ruby-forum.com/. ...

Net::HTTP, inheritance + proxy problem
Hi all, I am trying to create a child class from Net::HTTP and the following seems to work: class Connection < Net::HTTP def initialize(addr, port) super(addr, port) end end I can now write: conn = Connection.new(addr, port) instead of: conn = Net::HTTP.new(addr, port) But what I really would like is to be able to use the Proxy method of Net::HTTP in initialize. So that i could use: conn = Connection.new(addr, port, paddr, pport) instead of: conn = Net::HTTP::Proxy(paddr, pprort).new(addr, port) But I have as of yet no idea how to incorporate the proxy in the Connection class. ...

Net::HTTP using SSL certificate
Hi, I'm attempting to login to a URL over SSL using Net:HTTP. CERT_FILE = 'cert.pem' http = Net::HTTP.new('dev.company.com, 443) http.use_ssl = true http.cert = OpenSSL::X509::Certificate.new( File.read(CERT_FILE) ) http.key = OpenSSL::PKey::RSA.new( File.read(CERT_FILE) ) http.verify_mode = OpenSSL::SSL::VERIFY_NONE path = '/login.form' response, body = http.post(path, data, headers) puts "Code = #{response.code}" puts "Message = #{response.message}" puts "Body= #{body}" but I get the following error message: C:/rubydev/olb/olb.rb:...

How to get CPAN to use Http Proxy?
How can I force CPAN to use an http proxy for all it's downloads? When configuring CPAN, I specified only an http proxy (thus leaving ftp proxy blank). However, it is still trying to use ftp to grab it's files. log: Trying with "/usr/bin/wget -O -" to get ftp://ftp.perl.org/pub/CPAN/MIRRORED.BY --11:45:00-- ftp://ftp.perl.org/pub/CPAN/MIRRORED.BY => `-' Resolving ftp.perl.org... done. Connecting to ftp.perl.org[207.45.221.24]:21... failed: Connection timed out. Connecting to ftp.perl.org[153.106.5.70]:21... failed: Connection timed out. Retrying. --11...

use net:smtp behind the proxy
I would like to use net:smtp to send email through smtp.gmail.com. The problem is that I am behind a proxy, so I need to tell the smtp module that it should use the proxy (otherwise it won’t be able to resolve smtp.gmail.com). How can this be done? There is no obvious way for me to specify the proxy. I am using version 1.8.6 of ruby. -- Posted via http://www.ruby-forum.com/. Peter (Guo) Pei wrote: > I would like to use net:smtp to send email through smtp.gmail.com. The > problem is that I am behind a proxy, so I need to tell the smtp module > that it should use the proxy (otherwise...

too slow in using Net::HTTP.post_form
Hi I am using Net::HTTP.post_form to post cgi request to a website and get answers. Below is my code and I found that it always take about 30s to return the result. I think my Internt connection is not so slow as I paste the same cgi request url and run in my firefox, it will get result in less then 10 second. puts Time.now post_result= Net::HTTP.post_form(URI.parse(url),params) puts Time.now So, how can I make the post_form returns quickly? And, I also found many tags like <img src=xxx.gif> in the returned html file. Did the post_form download all those gif files? If so, how can I di...

Which HTTP proxy am I using and how is the traffic being routed?
Hi. In my school, all HTTP traffic has to be filtered through a proxy server, say proxy.school.edu:8080. However, it appears that my school network is linked to the Internet via a leased line from a local ISP which has its own proxy server, say proxy.isp.com:8080. I experimented with using both servers as my HTTP proxy setting in my web browser and found that both can work. I hope to know exactly which HTTP proxy am I using anyway? I'm surprised that I can use a proxy server outside of my school network over at the ISP side and I wonder how my HTTP traffic managed to get...

DELETE HTTP request using Net library?
I want to issue a DELETE HTTP request. Ruby-doc actually mentions Net::HTTP::Delete class at http://www.ruby-doc.org/stdlib/libdoc/net/http/rdoc/classes/Net/HTTP/Delete.html but my Ruby distribution (1.8.3) doesn't seem to contain HTTP Delete class or method it in the Net library. I can only use GET, PUT, POST and HEAD. ...

Sending a cookie in the request using Net::HTTP
I can't find this answered in the RDoc, so I'll ask here: How do I send the cookie string in an HTTP request using Net::HTTP? I tried doing something like: Net::HTTP.start( 'server.host.here' ) { |http| response = http.get( '/local/file.rb', 'Cookie: cookie_name=cookie_val' ) } hoping that the "initheader" in HTTP#get is where I should drop the cookie value, but that just gave me a traceback. Francis Hwang http://fhwang.net/ On Sun, Dec 05, 2004 at 02:07:17AM +0900, Francis Hwang wrote: > I can't find this answered in the RDoc, s...

Using an HTTP proxy to record web traffic
As part of debugging WWW::Mechanize, I realized that the best way to compare what my browser is doing to what mechanize is doing would be to compare their HTTP transactions incoming and outgoing. What I want is an HTTP proxy that can record the transactions and also supports SSL. The Ruby project doesn't have any code http://rubyforge.org/projects/rpp/ This free program doesn't support SSL, and costs $600. http://www.proxy-sniffer.com/download.html This doesn't support SSL but is supposed to soon. http://kevinlangdon.com/serviceCapture/ Any other suggestions? Dan Kohn wrote:...

How to use open uri or net/http class
Hi I am not able to access any http via Ruby program.I access internet using proxy server. On runniong following program , it generates error : =20 c:/ruby/lib/ruby/1.8/net/protocol.rb:83:in `initialize': getaddrinfo: no address associated with hostname. (SocketError) from c:/ruby/lib/ruby/1.8/net/protocol.rb:83:in `new' from c:/ruby/lib/ruby/1.8/net/protocol.rb:83:in `connect' from c:/ruby/lib/ruby/1.8/net/protocol.rb:82:in `timeout' from c:/ruby/lib/ruby/1.8/timeout.rb:55:in `timeout' from c:/ruby/lib...

Net::HTTP.get2 failure when using debugger
Hi, Can someone tell me if I have found a bug or am doing something wrong using Net::HTTP.get ? If I run the code without the -rdebug flag, then it works perfectly. If I use the flag and then simply do a continue, then I get the errors shown after the code below. I have stepped through the code and shown that it is the get2 call that causes the error. Also happens with the get method. Jeff #!/usr/bin/ruby require 'net/http' SITE = "http://campus.acm.org" PATH = '/Public/fmr/' USERAGENT = ' Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/418.9 (KHTML,...

Timeout errors using Net::HTTP on Windows
Hi all, I'm having a problem here running some file upload code on Windows. I've installed Ruby 1.8.6 from the one-click installer on Windows 2000 Server in VMware and on real hardware with Windows XP. The client, when running on Windows, always times/errors out no matter what. Here's the error: c:/ruby/lib/ruby/1.8/net/http.rb:2094:in `error!': 408 "Request Timeout " (Net::HTTPServerException) from c:/ruby/lib/ruby/1.8/net/http.rb:2103:in `value' from put_client.rb:25:in `upload_file' from c:/ruby/lib/ruby/1.8/net/http.rb:543:in `sta...

Net::HTTP::Proxy Access Denied (authentication_failed)
Hi, two pc's show different behavior, although they are under the same domain in a company network: require 'net/http' Net::HTTP::Proxy('<proxy-address-here>', 8080).start('www.google.com') { |http| puts http.get('/search?q=ruby') } This code displays google's page content as expected on my computer, while on a colleague's pc "Access Denied (authentication_failed) Your credentials could not be authenticated: "Credentials are missing.". You will not be permitted access until your credentials can be verified. This is typic...

vb.net sending files using http
I need to send a file from a vb.net application to a server which will accept this file and send a response back. The server requires http headers to be sent with the file. Is it possible to send a file but also send the http headers. I have attempted to do this using the MSXML2.XMLHTTP40 method but i can't send the file. If anyone could help. Cheers Davy ...