[PATCH] US-ASCII messages containing non-ASCII characters get truncated

[r@…]

If a message claiming "Content-Type: TEXT/PLAIN; charset=US-ASCII" somehow contains a non-ASCII character like a “, the message will be truncated at the first non-ASCII character. I'm sure this applies to other character sets as well, not just US-ASCII.

For example if the body contains "This program deals with listening, the “forgotten skill,” and how it impacts our lives. It will include a number of exercises, role plays and games. It addresses topics such as comprehension, body language and tune-out.", it will be truncated to "This program deals with listening, the ". No doubt the inclusion of the non-ASCII character is a violation of RFCs 822 and 2045, however RoundCube should be more forgiving.

The problem occurs in the rcube_charset_convert function in program/include/main.inc. iconv just stops when it encounters a character in a character set that shouldn't be there. There is a parameter to iconv to ignore such problems instead of stopping when they're encountered.

I propose the following patch to avoid truncated messages due to invalid characters unexpectedly appearing in a given character set.

--- main.inc        Thu May 24 00:00:15 2007
+++ main.inc.new    Mon Jun  4 00:16:40 2007
@@ -927,7 +927,7 @@

   // convert charset using iconv module
   if (function_exists('iconv') && $from!='UTF-7' && $to!='UTF-7')
-    return iconv($from, $to, $str);
+    return iconv($from, $to . "//IGNORE", $str);

   $conv = new utf8();

[thomasb]

Added to trunk in [b8e65ce3]