VCards from OSX address book with UTF-16 not importing

[dynedain]

When importing VCards, Roundcube chokes on VCards generated by the OSX address book. This causes Roundcube to think the VCard is invalid.

I suspect that this has something to do with Mac end-of-line characters being present in the file. I opened the VCard with a text-editor and resaved using standard UNIX end of line characters and the file then imported properly. There isn't a problem with actual contents as I didn't modify the data.

[dynedain]

From a bit of research comparing the file originally exported by the OSX address book app, it appears that the file is encoded as UTF-16BE.

Some more testing... I tried modifying line 174 of rcube_vcard.php to force roundcube to be converting from UTF-16BE and the import worked! So my next guess is that roundcube is not properly detecting that the VCard is UTF-16BE encoded.

[dynedain]

A little more info as I try to track this down... the detect_encoding function in rcube_vcard is not detecting UTF-16BE by xFE\xFF on this VCard (not sure why) and so is reverting to rc_detect_encoding from rcube_shared.inc.

Apparently mb_detect_encoding() can't detect UTF-16BE ​http://bugs.php.net/bug.php?id=45993 so the rc_detect_encoding() wrapper is returning the wrong result.

I don't know if there's a performance hit to replacing it with mb_check_encoding($string, 'UTF-16BE') but it might be workaround.

[dynedain]

Some more info, UTF-16BE isn't being noticed because the first 2 characters of the filestream from file_get_contents() do not match \xFE\xFF. Why this is, I don't know. I'm at my wits end as file encoding is all new to me.

[dynedain]

Patch

[dynedain]

I've created a patch for rcube_vcard that adds md_check_encoding() functions to test for UTF-16 encoding. I don't know if it is sufficient as conceivably other encoding types could produce false positives.

[alec]

Please, attach sample vcard file in utf-16 for testing.

[dynedain]

I'm having trouble generating a VCard that exhibits the problem. The VCard I do have that generates the issue I cannot share because of private data and I'm worried that anonimizing it may cause re-encoding.

I'll see what I can do.

[thomasb]

mb_check_encoding($string,'UTF-16BE') wrongly detects ISO-8859-1 files as UTF-16 :-(

[chadm]

This will probably be difficult to figure out with mb_* functions. mb_detect_encoding's return is nothing more than an educated guess using heuristics. and mb_check_encoding just checks if the string can pass as the specified encoding.

It really really blows that Vcard specification doesn't require some sort of data as what the encoding is.

[questionman]

Replying to dynedain:

When importing VCards, Roundcube chokes on VCards generated by the OSX address book. This causes Roundcube to think the VCard is invalid.

I suspect that this has something to do with Mac end-of-line characters being present in the file. I opened the VCard with a text-editor and resaved using standard UNIX end of line characters and the file then imported properly. There isn't a problem with actual contents as I didn't modify the data.

---

I have Mac OSX and i can import VCards generated using Address Book. No errors.

[dynedain]

It's been a while now, but if I remember correctly, I was having the issue with vcards that contain multiple contacts.

[questionman]

Replying to dynedain:

It's been a while now, but if I remember correctly, I was having the issue with vcards that contain multiple contacts.

Yes. Has the error been fixed (for you)?

[netbit]

We need a multiple contacts sample vCard file in UTF-16 for testing. As we had no response, I'm closing this ticket. Reopen if needed.