Press "Enter" to skip to content

Tag: unicode

Code Snippets: UTF-8 to UTF-16 revisited and expanded

June 28, 2026
Leave a Comment

Yes, I know I posted yesterday about converting from UTF-8 to UTF-16BE/UTF-16LE – but I wasn’t happy with the code to convert to UTF16. It relied on mb_convert_encoding which whilst clear, it did mean that sequences sent could be silently “fixed”, lone high/low surrogate code points would be refused to be output (which I actually needed as they were one thing I was trying to test against!), and I just wanted more “insight” into the whole UTF8 to UTF16 system.

Unicode is NOT UTF-8

One thing to remember (and which caused me a timesink) was that Unicode is NOT UTF-8! Unicode is a collection of characters which are usually represented in UTF-8 byte sequences (but can be represented in UTF-16BE/UTF-16LE and UTF32 and others).

Continue ReadingCode Snippets: UTF-8 to UTF-16 revisited and expanded

Code Snippets: PHP: Converting to/from UTF-8 to UTF16-BE

June 27, 2026
Leave a Comment

I needed to convert some Unicode UTF16-BE strings (as used in Java) to UTF-8 (which is “byte-orientated” and so doesn’t need to worry about endianness) – however, there didn’t seem to many examples online. I’m not going to say these methods are robust, 100% accurate for every use case or even the best way to do it – but just ways to do it.

It’s worth noting that Java uses the escape sequence \uXXXX (where X is a hexadecimal code), and PHP uses the nearly similar escape sequence \u{XXXX} . Both seem to use \xXX for a single character.

ADDED: I’ve improved the UTF-8 to UTF-16 code conversion in a newer post.

Continue ReadingCode Snippets: PHP: Converting to/from UTF-8 to UTF16-BE