Java Solaris Communities Sun Store Join SDN My Profile Why Join?
 
Bug Database
Bug Detail
Quick Lists
Top 25 Bugs
Top 25 RFE's
Recently Closed Bugs
Printable Page Printable Page


Bug Database
Bug ID: 4304013
Votes 103
Synopsis RFE: Add UTF-7 to encoding converters
Category java:char_encodings
Reported Against 1.4 , merlin-beta
Release Fixed
State 6-Fix Understood, request for enhancement
Priority: 4-Low
Related Bugs 4679523
Submit Date 11-JAN-2000
Description





Add UTF-7 to the standard set of supported encoding converters.  This is useful 
when trying to pass Unicode data through transports that are only ASCII safe. 

======================================================================

Here is some additional evidence of why supporting this encoding within
the J2SE core set of converters might need to be considered:
Taken from RFE #4679523






FULL PRODUCT VERSION :
java version "1.3.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0)
Classic VM (build 1.3.0, J2RE 1.3.0 IBM build co130-20020124 (JIT enabled: jitc)
)


FULL OPERATING SYSTEM VERSION :
The Operating System/2 Version is 4.50

A DESCRIPTION OF THE PROBLEM :
JavaMail should support all mail-specific charsets, such as
"unicode-1-1-utf-7" and perhaps  xxxxx s.

Hotmail.com, among  xxxxx s, frequently uses this encoding and
when JavaMail encounters it it results in the specified
error message (see below).

I have attached a testcase mail for you to parse below.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. See testcase
2.
3.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.io.UnsupportedEncodingException: unicode-1-1-utf-7
	at sun.io.Converters.getConverterClass(Converters.java:97)
	at sun.io.Converters.newConverter(Converters.java:128)
	at sun.io.ByteToCharConverter.getConverter(ByteToCharConverter.java:71)
	at java.io.InputStreamReader.<init>(InputStreamReader.java:85)
	at com.sun.mail.handlers.text_plain.getContent(text_plain.java:65)
	at javax.activation.DataSourceDataContentHandler.getContent(DataHandler.java:755)
	at javax.activation.DataHandler.getContent(DataHandler.java:511)
	at javax.mail.internet.MimeBodyPart.getContent(MimeBodyPart.java:531)

This bug can be reproduced always.

---------- BEGIN SOURCE ----------
Return-Path: <>
Received: from hotmail.com (mc1-s11.law16.hotmail.com [65.54.236.60]) by
 HSE-QuebecCity-ppp3503426.sympatico.ca (Weasel v1.40) for
 < xxxxx@xxxxx >; 16 Apr 2002 22:17:09
Received: from hotmail.com ([65.54.236.21]) by hotmail.com with Microsoft
SMTPSVC(5.0.2195.4905);
	 Tue, 16 Apr 2002 19:16:08 -0700
  From:  xxxxx@xxxxx 
  To:  xxxxx@xxxxx 
Date: Tue, 16 Apr 2002 19:15:49 -0700
MIME-Version: 1.0
Content-Type: multipart/report; report-type=delivery-status;
	boundary="9B095B5ADSN=_01C1E58710E1D33C0000BFAChotmail.com"
Message-ID: < xxxxx@xxxxx >
  Subject: Delivery Status Notification (Failure)
Return-Path: <>
X-OriginalArrivalTime: 17 Apr 2002 02:16:09.0046 (UTC) FILETIME=[D64F7F60:01C1E5B5]

This is a MIME-formatted message.
Portions of this message may be unreadable without a MIME-capable mail program.

--9B095B5ADSN=_01C1E58710E1D33C0000BFAChotmail.com
Content-Type: text/plain; charset=unicode-1-1-utf-7

This is an automatically generated Delivery Status Notification.

Delivery to the following recipients failed.

       zarina_95@hotmail.com

--9B095B5ADSN=_01C1E58710E1D33C0000BFAChotmail.com--
---------- END SOURCE ----------
(Review ID: 145766) 
======================================================================


 xxxxx@xxxxx  2002-05-03
JavaMail depends on the charset support in the JDK, so support for this
charset needs to be added to the JDK.
Work Around
N/A
Evaluation
Once we have the character converter SPI (RFE 4287465), anybody could implement this converter, but it's probably worthwhile having it as part of the J2RE.
 xxxxx@xxxxx  2000-01-26




Committing to merlin release


======================================================================

UTF-7 charset support (as a bundled charset component of J2SE) will not
happen for 1.5. Apart from lateness in time to implement compared to
1.5 release cutoffs this is a good example where frameworks, eg
Ref implementations of JavaMail,etc should be able to use the public
API/SPI to write and provide a UTF-7 implementation rather than have
it bundled as a core J2SE supported encoding.
 xxxxx@xxxxx  2003-11-07
Comments
  
  Include a link with my name & email   

Submitted On 23-AUG-2001
kilyas
I have come across various emails in Java Forums facing 
similar porblems with UTF-7.  But unfortunately there are 
no answers.  Nor has the java documentation provided any 
information as to why this charset(UTF-7) was excluded and 
can we handle UTF-7 using any other char set already in 
place.  Its pretty sad.


Submitted On 19-NOV-2001
danedwards1
Is there anyway to work around this?  Perhaps masquerade 
utf-7 as ASCII?


Submitted On 02-AUG-2002
peterwlynch
It's been two and a half years, why not stick it in 1.4.1? 
Continues to be a problem...


Submitted On 20-SEP-2002
pmtyson
Not A Workaround, just another vote to try and get the 
charset included, this strikes me as a really big problem.  It is 
now affecting Coldfusion MX which sits on top of Java


Submitted On 24-SEP-2002
iainmac
get it sorted!  I've got enough to do without having to 
reimplement wheels!


Submitted On 27-SEP-2002
dnewman-datixinc
Our company is having this same problem of encoding 7 not 
working. This is not good!  We need this included please!


Submitted On 31-DEC-2002
gvelli
I could not find why this was ommitted in the first place. Can 
we have it in near future or shall we have to wait till JDK99...?


Submitted On 29-JAN-2003
tholly
Try this for hotmail bounces. It works for me in 1.4.1_01...

<code>
public String getMessageText(Message message) throws 
java.io.IOException,javax.mail.MessagingException
  {
    String body = null;

    Object o = message.getContent();

    if (o instanceof MimeMultipart)
    {
      MimeMultipart mm = (MimeMultipart)o;
      mm = (MimeMultipart)o;
      BodyPart bp = null;

        for(int j = 0; j < mm.getCount(); j++)
        {
          bp = mm.getBodyPart(j);

          try
          {
            o = bp.getContent();
          }
          catch (IOException e)
          {
            if (bp.getContentType().equals("text/plain; 
charset=unicode-1-1-utf-7"))
            {
              ByteArrayOutputStream bao = new 
ByteArrayOutputStream();
              bp.writeTo(bao);
              body = bao.toString();
            }

            break;
          }

          if (o instanceof MimeMessage)
          {
           body = ((MimeMessage)o).getContent().toString();
           break;
          }
          else if (o instanceof String)
          {
            body = (String)o;
            break;
          }
        }
    }
    else
    {
      body = message.getContent().toString();
    }


    return body;
  }
</code>


Submitted On 22-MAY-2003
clementi
Good heavens!

We run a e-mail newsletter subscription service at our site 
(plug: http://nn.byu.edu/), running on ColdFusion MX, but the 
unsubscription script pukes all over me now whenever it deals 
with UTF-7 charsets! So, the script, which is supposed to run 
every day, was failing without my knowledge--that is, until 
we started getting unsubscribe requests from angry users. 
And to think that processing Hotmail messages won't work! 
That is critical! We need a fix *yesterday!*


Submitted On 22-MAY-2003
clementi
BTW, I'm referring to J2EE (1.3)


Submitted On 09-JUN-2003
AndriPopa
yes, this needs to be fixed ASAP.
we are having major problems with emails


Submitted On 22-AUG-2003
sacocats
fix it 


Submitted On 01-OCT-2003
jchen06
There are two issues at here: UTF-7 and unicode-1-1-utf-7.

UTF-7 can be easily implemented via JDK 1.4's 
java.nio.charset.Charset. In fact, ICU has a C++ version. It is 
not hard to implement UTF-7 because RFC and C++ codes are 
there. At least, I've implemented it, and tested it againt 
Micrisoft Outlook. It tooks couple weekends.

Secondly, unicode-1-1-utf-7 is obsolete by UTF-7 in RFC 
2152. There is no point to implement an obsolete data. In 
fact, if you would like, you can make unicode-1-1-utf-7 as a 
alia of UTF-7 because there are almost the same.

I encourage that developers can implement UTF-7 by their 
own, instead of waiting for Sun. 
(I might publish my implementation, don't wait for me.)


Submitted On 23-OCT-2003
vladnc
This bug must be fixed asap. Our mail client doesn't support 
the unicode-1-1-utf-7 and US-ASCII because of this bug.


Submitted On 04-FEB-2004
oswedor
I need this fixed too...


Submitted On 13-APR-2004
ostapz
This, need to be fixed.


Submitted On 22-JUN-2004
allancliff
I need this fixed to work with coldfusion MX cfpop tag.
We cannot pop emails that come from Hotmail or Microsoft Exchange.


Submitted On 13-SEP-2004
mightyIan
this has been going on for years, it means you can't process non delivery reports from Exchange Server Internet Mail Connector. A total pain.


Submitted On 16-DEC-2004
maartenbrug
please fix it


Submitted On 26-JAN-2005
x.frisaye
When our application receive mail from Exchange 2000 System administrator, it use utf-7 charset so please fix it.


Submitted On 13-JUN-2005
Amichai
I've developed an open source UTF-7 charset implementation, you can get it at
http://www.freeutils.net/source/jcharset

I hope this helps!

(Should Sun decide to incorporate it into the standard JRE, I'd be happy to be of assistance.)


Submitted On 30-JUN-2005
volkenborn
Hey, one more vote for this bug, and it will make the current top 25 list...

We use a workaround for this missing Converter in our email response management software since 2001.
It's hard to understand why this charset still isn't supported when it's so widely used apparently!?


Submitted On 10-JAN-2006
LinuxLars
Please, please, please fix this. It's making it extremely difficult to do any serious mail processing with Java.  


Submitted On 20-APR-2006
You know, it really breaks my heart that this bug has been running for six years now. The JavaMail people seem to point fingers at the JRE for not having an implementation and seem unwilling to do anything about it.

Is the political situation in Sun Microsystems really so dominated by buck passing and petty bullshit?

UTF7 encoding support is essential for mail services, and that's a 'bottom line' statement. I'm more than a little disgusted that the JavaMail development team is incapable of, unwilling to or just plain disinterested in rolling their own implementation of UTF-7 support, apart from the runtime environment, just to offer their supporters the level of tool that was promised in the first place.


Submitted On 21-SEP-2006
jeff@redcondor.com
This "in progress" bug is nearing 7 years old.  It has a large number of votes, and the only excuse for not getting it done is laziness.  JCharset has had this for years, but its GPL license is not suitable for many applications.  

This bug is just another reason why Java should be open source.  If Sun has abandoned custodianship of the product, then the community should be allowed to serve its needs.  


Submitted On 24-DEC-2006
JaapBeetstra
I have just released a JAR on sourceforge which implements UTF-7 encoding. It uses an MIT license, so you should be able to use it in any  project you like.
http://sf.net/projects/jutf7


Submitted On 25-DEC-2006
amgad
I got the same error when dealing with exchangeserver 2003 

I really shocked when i saw the date & the bug status 

hope this bug solved b4 the new century 


Submitted On 26-JAN-2007
I’m an avowed javanut that believed that the sun shone only because Sun Microsystems told it to. It’s been 7 years!!! Please get it fixed. 


Submitted On 26-MAR-2007
verdyp
 jeff@redcondor.com said:
"JCharset has had this for years, but its GPL license is not suitable for many applications."

Did you ask to the JCharset author if he could provide you a LGPL licence? If JCharset is now a collective work, it could be a legal problem, but this shuold be solved according to the copyright assignment notice (that any GPL project requires... so you don't need to ask to all possible authors, given that they already have reassigned the copyright in their modifications, so the original author can still provide such secondary licence, without having to void the existing GPL licence granted to others).


Submitted On 28-MAR-2007
Stumbled across this problem while dealing with java-mail and IMAP in order to build an internal mail archival solution, and by now I will have a look at jutf7 (thanks jaap). 

Anyhow, what is the state in having a solution for that implemented in a recent Java version? Having this bug open for 7 years and across two minor(?) releases severely shakes my trust in Java. :(


Submitted On 30-JUL-2007
camelwoo
In progress? really?


Submitted On 07-NOV-2007
Borys
The following link provides a UTF7 implementation in Java:

http://sourceforge.net/projects/jutf7



Submitted On 20-FEB-2008
ralfhauser
see also http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4619777


Submitted On 03-JUN-2008
mr_weasel
Can someone at Sun raise the priority of this bug?  I think it should be higher than "Low" because it prevents stuff from working.  (i.e. you can't process UTF-7 or unicode-1-1-utf-7 content, which seems to be quite common in email)



PLEASE NOTE: JDK6 is formerly known as Project Mustang