There are certain file types (.html, .htm, .txt) that will not generate a FileNotFoundException if the server returns a >400 response code. This sometimes leads to an error stream being read as if it were the requested document. If another type of file is requested a FileNotFoundException is thrown and the error stream is not accessible without requesting the document again.

The RFE 4087140 which was approved by the CCC describes the method getErrorStream() which would provide an error stream in the case of a FileNotFoundException. Probably the best course would be to always throw the FileNotFoundException on >400 responses for any type of document. Applications could then catch this exception and call getErrorStream() if they wanted access to the error stream (as in the case of HotJava, where it displays the server error message)




The implementation of java.net.HttpURLConnection.getErrorStream() is "return null".  Furthermore, the implementation of sun.net.www.protocol.http.HttpURLConnection.getErrorStream() is also "return null".

The fact that these methods are unimplemented indicates that they will never work as defined.

This is a major problem, becuase of the behaivor of the getInputStream() methods on the same classes.  These methods throw a FileNotFoundException if the HTTP response code is >= 400.  Nearly every web server in existence returns a document along with an error code -- for example, if the error code is 404, the document usually says "Document not found", and generally includes the e-mail address of the webmaster, and possibly other useful information.  Sometimes this document even indicates that a search returned no matches.  In any case, it is often important that this "error" document is made available to the user -- but there is NO WAY to get this document using the URLConnection classes.

I'm trying to write an HTTP proxy in Java.  I CANNOT do so, however, becuase I can't handle error conditions -- which, of course, happen all the time on the Internet.  The end user's browser gets a "document contained no data" or similar error instead of the real error document that the remote web server created.

import java.io.*;
import java.net.*;

public class Connect {
        static  byte[]                  buf;
        static  InputStream             is, es;
        static  URLConnection   conn;
    
    public static void main(String[] args) {
                buf = new byte[80];

        try{
            URL url = new URL("http://java.sun.com/products/jdk/1.3/docs/badLastSeg");
                        conn = url.openConnection();
                        is = conn.getInputStream();
                        es = ((HttpURLConnection)conn).getErrorStream();
                        System.out.println("input stream IS:"  +  is);
                        System.out.println("error stream IS:"  +  es);
                        is.read(buf);
                        System.out.println("buf is:" + buf);
                        for(int i=0; i < buf.length; i++) {
                                System.out.print(buf[i]);
                        }
                        System.out.println("-----");
                }
                catch(Exception e){
                        System.err.println("YIKES!");
                        System.out.println(e);
                        try {
                                System.out.println("err stream is:" + es);
                                // commence workaround... (which doesn't work yet)
                                es = ((HttpURLConnection)conn).getErrorStream();
                                System.out.println(
                                        "err stream (after 2nd getErrorStream()) is (still) :  " + es);
                                if(es != null) {
                                        es.read(buf);
                                        for(int i=0; i < buf.length; i++) {
                                                System.out.print(buf[i]);
                                        }
                                }
                        }
                        catch(Exception e2) {
                                System.exit(1);
                        }
                        System.exit(1);
                }
                
        }
}

(Review ID: 95930)
======================================================================




C:\Ian\demo>c:\jdk1.2.2\bin\java -version
java version "1.2.2"
Classic VM (build JDK-1.2.2-001, native threads, symcjit)

C:\Ian\demo>

Two attachments, as it were, here.  First the stack trace and second the test
program.  Use this test program to surf to a cite that gives an unauthorized
response (401).  I'm sure you can find a site like that (ours is from an
internal servlet, you don't really want that too).  Try the test program with a
site that returns 200, 301... anything
less that 400 and everything is okay.  Try anything 400 and over and you get
a file not found exception and 404 is NOT the only over 400 code. 401 should
be returned!

LoginTest.java is a complete program just run it from the command line.

**********************Stack Trace
C:\Ian\demo>rl

C:\Ian\demo>c:\jdk1.2.2\bin\java LoginTest
java.io.FileNotFoundException: http://192.168.0.129/servlet/Demo1.LoginHandler.L
oginHandler
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLCon
nection.java:530)
        at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:145
)
        at LoginTest.main(LoginTest.java:14)

C:\Ian\demo>rl

C:\Ian\demo>c:\jdk1.2.2\bin\java LoginTest
204
No Content
************* end stack trace

*******************************file LoginTest.java
class LoginTest
{
    public static final void main(String args[])
    {
	try
	{
	    java.net.URL url = new
java.net.URL("http://192.168.0.129/servlet/Demo1.LoginHandler.LoginHandler");
	    java.net.HttpURLConnection hurlc =
(java.net.HttpURLConnection)url.openConnection();
	    hurlc.setDoOutput(true);
	    hurlc.setDoInput(true);
	    java.io.OutputStream out = hurlc.getOutputStream();
	    out.write("name=micky&passwd=mouse".getBytes());
	    System.out.println(hurlc.getResponseCode());
	    System.out.println(hurlc.getResponseMessage());
	    java.io.InputStream in = hurlc.getInputStream();
	    byte[] b = new byte[2000];
	    in.read(b);
	    System.out.println(new String(b));
	}
	catch(Exception e)
	{
	    e.printStackTrace();
	}
      
    }
}
*********** end file LoginTest.java
(Review ID: 98576)
======================================================================

If the requested file does not exist, and ends in .html, .htm, .txt or /, you will get the error stream with no exception thrown. If the file does not end like any of these you can catch the exception and immediately request it again
to get the error stream. The response code can be obtained with getResponseCode().




There is NO WORKAROUND for this bug!  Short of using raw sockets to retrieve the error document.  But this may be difficult or impossible, depending on what information must be passed to the server in order to generate the error document.
(Review ID: 95930)
======================================================================




You can't use a buffered-anything, because that would possibly trigger the
Exception without returning data.  Include a check for a SocketException around
your read() loop, and handle it like an EOF.
(Review ID: 103500)
======================================================================

Related to bug 4222009. We should get rid of the error handling inside HttpURLConnection.getInputStream, and let applications rely on response code and message for further action. As it should have been done in the first place.

Will fix in Merlin.

 xxxxx@xxxxx  1999-10-11

When the file ends with those extension, what is the difference in terms of
output returned by getErrorStream() and getInputStream() ?
Both of them will return the error page returned by the server, won't they ?

FileNotFoundException should never be thrown.
It is worse than meaning less in most cases and there is no benefit to be
gained; infact it just adds a level of complexity/processing.
Each app should be able to access the data that was returned from the server
and make its OWN decision on what to do with it.

First of all, the implementation of getErrorStream() in Solaris_JDK_1.2.1_02 is
"return null", so don't even call that a workaround.  Secondly,
getInputStream should RETURN THE INPUT STREAM, no matter what the HTTP response
is, and no matter what the file extension is.  Why is HttpURLConnection
deciding whether or not we should get to see the document content?  If there is
no stream, return null.  That's it.  There is no reason for an exception here.

Has anyone found that on Linux 1.2.2, when there's been a
404 error, the input stream crashes midway or at EOF with a
SocketException?  I´ve got a workaround underway...

JDK 1.3 GA still have this bug.

2 YEARS.  Hey Sun you gonna fix this one or what?  Throw 
away useful info, throw an unrelated exception: That 
approaching a Microsoft level of incompetence.

> Will fix in Merlin.

i sure do hope so - this BUG (denied by some) has been 
around so long that its has at least 10 bug reports against 
it!  my particular complaint is that it obscures errors from 
servlet managers (which are typically in the 5xx range) even 
though there is valid error text available on the 
inputStream.

this error is not in "Top 25 Bugs" but certainly in "Top 25 
oldest Bugs" .....

argggh

Trying to retrieve XML documents from a servlet with a .xml 
extension. So far a partial work-around I have found (JDK 
1.3) is that I can get all the header information using 
interations of the getHeaderField method. The error content 
cannot be retrieved, even though it is there. Immediately 
retrying the request in the exception handling code does 
not provide anything new.

If the error is a result of a basic authentication error 
(401), I can get the response by appending a ".txt" onto 
the end of the URL and retrying a request for the non-
existent document. Ugly.

Pretty much along the same lines as ghamann1, except it works more broadly. I also came across the fact 
that if you use the HttpURLConnection getHeaderFieldKey or getHeaderField methods, then the 
FileNotFoundException is not thrown and you can successfully read the response code using 
getResponseCode. This fact in itself is weird and can be explained by a catch in getHeaderFieldKey and 
getHeaderField that eats the FileNotFoundException

You can then look at the response code and decide what to do. In my case I append a new parm to the 
URL to avoid the jdk bug and try the connection again. In my case ?foo=/ or &foo=/ worked. You could also 
end it with .html, .htm or .txt (as in ?foo=.html etc.). This time you will get the body

Here's the test code, an application Test500.java and a jsp 500error.jsp. Don't forget to change TESTJSP 
for your environment.

import javax.servlet.*;
import javax.servlet.http.*;
import java.io.*;
import java.util.*;
import java.lang.*;
import java.net.*;
import java.security.*;

public class Test500
{
	public static final String TESTJSP = "http://moggy.mcom.com:7779/500error.jsp";

	public static void main(String args[]) 
		throws IOException 
	{
		System.out.println("Accessing Test Page at -  "+TESTJSP);
		URL url=new URL(TESTJSP);
		System.out.println(url.getFile());

		try {

			// Open the URL Connection
			HttpURLConnection conn=(HttpURLConnection)(url.openConnection());
			
			int i=1;
			String key, field;
			while(true) {

				// Make sure we get a key and it's corresponding field
				key = conn.getHeaderFieldKey(i);
				field = conn.getHeaderField(i);
				if((key == null) && (field == null))
					break;

				System.out.println("Set header back to client -- > Key: "+key+", 
Field: "+field);
				i++;
			}
			
			int code=conn.getResponseCode();
			System.out.println("Response Code: "+code+" "+conn.getResponseMessage());

			// Did we tweak the jdk bug (should be more checks for .htm, .html, .txt and /
			if(code>=400) {

				// Tweak the URL and try again
				String urlStr=url.toString();
				if(urlStr.indexOf('?')==-1)
					urlStr+="?foo=/";
				else
					urlStr+="&foo=/";

				System.out.println("Need to retry the url: "+urlStr);
				url=new URL(urlStr);
				conn=(HttpURLConnection)(url.openConnection());
				
			}

			BufferedReader reader = new BufferedReader(new 
InputStreamReader(conn.getInputStream()));
			
			String c;
			while((c=reader.readLine())!=null) {
				System.out.println(c);
			}

		}
		catch(Exception e) {

			System.out.println("Exception: "+e);
			e.printStackTrace();
		}
	}
}

And the jsp (500error.jsp)

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<META HTTP-EQUIV="expires" CONTENT="0">

<HTML>
<BODY bgcolor="#FFFFFF">
<%
	response.setHeader("Cache-Control", "no-cache");
	response.setHeader("pragma", "no-cache");
	response.setStatus(500);
%>

<H1> This page returns a 500 http error </H1>

</BODY></HTML>

And finally, here is the output showing that it worked

moggy% java Test500
Accessing Test Page at -  http://moggy.mcom.com:7779/500error.jsp
/500error.jsp
Set header back to client -- > Key: Server, Field: Netscape-Enterprise/4.1
Set header back to client -- > Key: Date, Field: Fri, 16 Mar 2001 02:28:22 GMT
Set header back to client -- > Key: Content-type, Field: text/html; charset=ISO8859_1
Set header back to client -- > Key: Set-cookie, Field: 
NSES40Session=d%253A3ab17a46%253A12bc58936d5ddec8;path=/;expires=Fri, 16-Mar-2001 02:58:23 
GMT
Set header back to client -- > Key: Cache-control, Field: no-cache
Set header back to client -- > Key: Pragma, Field: no-cache
Set header back to client -- > Key: Connection, Field: close
Response Code: 500 Server Error
Need to retry the u

3+ years and still no fix!

The HTTP specification clearly states that a server should 
return an entity describing the reason for the error.

The bug was reported on Jul 24, 1998. The status is "fixed".
I am using JDK1.3 and HttpURLConnection is throwing 
IOException for status code > 400. The bug is still not 
fixed. 
What is the value of HttpURLConnection if it works properly 
only for status codes < 400????

Its quite unbelievable that a company like Sun, so involved 
in the Internet, as been carrying that bug for so long...
I am currently quite stuck now, as we went to production 
with a fix for that bug, and now, the fix is causing me 
quite a headache...

Argh!  Please fix this!

I tried the workaround given by roryward, but the technique 
of calling getHeaderFieldKey didn't prevent an exception 
being thrown when I called getResponseCode().  

I'm on JDK 1.3 on Windows 2000.

As reported, the bug still seems to be in JDK 1.3.1.  Based on 
the roryward's workaround, I (accidentally) found a way to 
always get the response code.

String field = connection.getHeaderField(null) 

for an HttpURLConnection seems to return the first row of the 
HTTP response. The row contains the status code, but 
no "key". From there you can parse the code easily.

I installed 1.4 and I'm still getting this error/bug. Is this fixed or not?

As far as I am concerned, it still NOT work with 1.5 !!!

How is that possible ??

For my problem, I had to replace the code from getURLInputStream()  in URLTag.java (io taglib) as follow :

    public InputStream getURLInputStream() throws IOException {
        URLConnection connection = getConnection();
        connection.connect();
        try {
        	return connection.getInputStream();
        }
        catch (IOException e) {
        	return((HttpURLConnection)connection).getErrorStream();
        }
    }

with this code, getURLInputStream()  sends the content of the error message instead of throwing the exception.

1.5.0_06 Still no fix?