A string of unexpected lengths - Blog - Recurse Center

Abstraction: Three ways string byte-length diverges from terminal display width

Key points:

• Multi-byte encodings: a single visible character like Ä may occupy 2+ bytes; Python 2 byte strings report len() as byte count, not character count; switching to Unicode strings fixes this for most cases
• ANSI escape codes: color sequences like \x1b[31m inflate len() beyond visible width — e.g., a 4-char colored string reports length 13; Python's clint library tracks visual width but breaks on concatenation
• Fullwidth Unicode characters (e.g., CJK ideographs 扶桑) display as double-width but count as 1 code point each; combining characters (e.g., Fusō) add no width but increase len()
• POSIX wcswidth function correctly computes display width for Unicode strings; Python binding available via wcwidth package
• Python < 3.3 "narrow" builds may report len() == 2 for code points above U+FFFF (check via sys.maxunicode)

Connections: Recurse Center · Python · Unicode · Character Encoding · String Handling

Source: https://www.hackerschool.com/blog/74-a-string-of-unexpected-lengths