|
|
IOException for URL
When I try the code below with "http://www.google.com/search?source=ig&hl=en&rlz=&q=something" in the URLs.txt file, I get anIOException error. The server is returning an HTTP response code403. However, when I open the same URL in my browser, it worksfine. A different but similarly formated URL to another server workedfine. Maybe this server is expection something additional? I am new at this, and I wonder if it is obvious what theproblems is.Thanks, Alan// download text content of URLimport java.net.*;import java.io.*;public class URLget{ public static void main ( String[] args ) throws IOException { try { BufferedReader infile = new BufferedReader(newFileReader("URLs.txt")); PrintWriter out = new PrintWriter(newFileOutputStream("downloaded.html")); String aURL, str; while ((aURL = infile.readLine()) != null) { URL url = new URL(aURL); System.out.println(aURL); BufferedReader in = new BufferedReader(newInputStreamReader(url.openStream())); while ((str = in.readLine()) != null) { out.println(str); } in.close(); } infile.close(); out.close(); } catch (MalformedURLException e) {e.printStackTrace();} catch (IOException e) {e.printStackTrace();} }}
|
|
0
|
|
|
|
Reply
|
jalanthomas (125)
|
10/22/2007 3:01:09 AM |
|
Alan wrote:> When I try the code below with "http://www.google.com/search?> source=ig&hl=en&rlz=&q=something" in the URLs.txt file, I get an> IOException error. The server is returning an HTTP response code> 403. However, when I open the same URL in my browser, it works> fine.> > A different but similarly formated URL to another server worked> fine. Maybe this server is expection something additional?I think Google tests on browser type.In 2004 the following worked: URL url = new URL("http://www.google.dk/search?q=hej"); HttpURLConnection con = (HttpURLConnection)url.openConnection(); con.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"); con.setRequestProperty("Referrer", "http://www.google.dk/"); con.connect(); if(con.getResponseCode() == HttpURLConnection.HTTP_OK) { InputStream is = con.getInputStream(); byte[] b = new byte[1000]; int n; while((n = is.read(b)) >= 0) { System.out.println(new String(b,0,n)); } is.close(); } else { System.out.println(con.getResponseCode() + " " + con.getResponseMessage()); } con.disconnect();Arne
|
|
0
|
|
|
|
Reply
|
ISO
|
10/22/2007 3:07:53 AM
|
|
|
1 Replies
311 Views
(page loaded in 0.056 seconds)
|
|
|
|
|
|
|
|
|