Unicode & Emoji Encoding Guide: Code Points to…

Unicode: One Encoding Standard for All Characters

Unicode assigns a unique number (code point) to every character in virtually every writing system in the world.

Concept	Description	Example
Code Point	A character's unique number	U+4E2D (中)
Plane	65,536 code points per plane	BMP (Plane 0)
BMP	Basic Multilingual Plane, U+0000~U+FFFF	Most common characters
Supplementary planes	U+10000~U+10FFFF	Emoji, ancient scripts, rare characters

1. The UTF Encoding Family

Encoding	Byte Length	Encoding of "中"	Characteristics
UTF-8	1-4 bytes	E4 B8 AD	Variable length, ASCII-compatible, the Web standard
UTF-16	2 or 4 bytes	4E2D	Variable length, used internally by JS
UTF-32	Fixed 4 bytes	00004E2D	Fixed length, high memory usage

UTF-8 Encoding Rules

Code Point Range	Bytes	Encoding Template
U+0000 ~ U+007F	1	`0xxxxxxx`
U+0080 ~ U+07FF	2	`110xxxxx 10xxxxxx`
U+0800 ~ U+FFFF	3	`1110xxxx 10xxxxxx 10xxxxxx`
U+10000 ~ U+10FFFF	4	`11110xxx 10xxxxxx 10xxxxxx 10xxxxxx`

Encoding Example

Character: 中
Code point: U+4E2D = 0100 1110 0010 1101

UTF-8 (3 bytes):
  1110xxxx 10xxxxxx 10xxxxxx
  1110 0100 10 111000 10 101101
  = E4 B8 AD

UTF-16 (2 bytes):
  4E2D

UTF-32 (4 bytes):
  00004E2D

2. JavaScript's Unicode Pitfalls

String Length Is Unreliable

'中'.length              // 1 ✅
'𠮷'.length             // 2 ❌ (supplementary plane character, surrogate pair)
'👨‍👩‍👧‍👦'.length           // 11 ❌ (Emoji ZWJ sequence)
'🏳️‍🌈'.length             // 6 ❌ (flag Emoji)

Why: JS Uses UTF-16

Characters in supplementary planes (code points > U+FFFF) require a surrogate pair (2 UTF-16 code units):

Character: 𠮷 (U+20BB7)
UTF-16 surrogate pair: D842 DF87
JS: length = 2 (two code units)

Correct Approaches

// Method 1: spread operator
[...'𠮷'].length           // 1 ✅
[...'👨‍👩‍👧‍👦'].length         // 1 ✅ (but ZWJ sequences count as one)

// Method 2: Array.from
Array.from('𠮷').length    // 1 ✅

// Method 3: for...of
for (const char of '𠮷👨‍👩‍👧‍👦') {
  console.log(char);  // iterates character by character
}

Working with Code Points

// Get a code point
'中'.codePointAt(0)        // 20013 (0x4E2D)
'𠮷'.codePointAt(0)        // 134047 (0x20BB7)

// Create a character from a code point
String.fromCodePoint(0x4E2D)  // '中'
String.fromCodePoint(0x20BB7) // '𠮷'

// Note: charCodeAt cannot handle supplementary planes
'𠮷'.charCodeAt(0)         // 55362 (only the first surrogate)

3. Emoji Encoding Deep Dive

The Four Types of Emoji

Type	Example	Code Points	Length
Basic Emoji	😀	U+1F600	1 (code point)
Modifier Emoji	👩🏽	U+1F469 + U+1F3FD	2 (code points)
ZWJ sequence	👨‍👩‍👧‍👦	Multiple code points + ZWJ	7 (code points)
Flag Emoji	🇨🇳	U+1F1E8 + U+1F1F3	2 (code points)

Zero Width Joiner (ZWJ)

ZWJ (U+200D) combines multiple emoji into a single composite emoji:

👨 (U+1F468) + ZWJ (U+200D) + 👩 (U+1F469) + ZWJ (U+200D) + 👧 (U+1F467) + ZWJ (U+200D) + 👦 (U+1F466)
= 👨‍👩‍👧‍👦 (family of four)

Skin Tone Modifiers

Skin tone modifiers (U+1F3FB ~ U+1F3FF) change an emoji's skin tone:

👩 (U+1F469) + 🏽 (U+1F3FD) = 👩🏽 (woman with medium skin tone)

Modifier	Code Point	Skin Tone
🏻	U+1F3FB	Light
🏼	U+1F3FC	Medium-light
🏽	U+1F3FD	Medium
🏾	U+1F3FE	Medium-dark
🏿	U+1F3FF	Dark

Flag Emoji

Flags are composed of two regional indicator symbols (ISO 3166-1 alpha-2):

🇨 (U+1F1E8) + 🇳 (U+1F1F3) = 🇨🇳 (China)
🇺 (U+1F1FA) + 🇸 (U+1F1F8) = 🇺🇸 (United States)
🇯 (U+1F1EF) + 🇵 (U+1F1F5) = 🇯🇵 (Japan)

4. Variation Selectors

VS16 (U+FE0F) switches a text-presentation character to emoji presentation:

❤ (U+2764)           → text presentation: ❤ (black heart)
❤️ (U+2764 + U+FE0F) → emoji presentation: ❤️ (red heart)

Character	Without VS16	With VS16
☕	☕ (text)	☕️ (emoji)
⚡	⚡ (text)	⚡️ (emoji)
❤	❤ (text)	❤️ (emoji)

5. Working with Unicode Using the Toolbox

Unicode Encode/Decode

Open the Unicode Encode/Decode Tool
Enter text or code points
View the conversion results:
- Text → Unicode escape sequences (\u4E2D)
- Code points → characters
- UTF-8/16/32 byte sequences

Emoji Encoding

Open the Emoji Encoding Tool
Enter an emoji
View complete encoding details:
- Code point sequence
- UTF-8 bytes
- ZWJ/modifier breakdown
- CLDR short name

6. Common Pitfalls and Solutions

Pitfall 1: String Truncation

// Wrong: slicing may cut a surrogate pair in half
text.substring(0, 10)  // may produce invalid characters

// Correct: slice by code point
function sliceByCodePoint(str, start, end) {
  const chars = [...str];
  return chars.slice(start, end).join('');
}

Pitfall 2: String Reversal

// Wrong: reversal breaks composite characters
'👨‍👩‍👧‍👦'.split('').reverse().join('')  // garbled output

// Correct: reverse by code point
[...'👨‍👩‍👧‍👦'].reverse().join('')  // ✅

Pitfall 3: Regex Matching

// Wrong: . does not match supplementary plane characters
/^.$/.test('𠮷')  // false

// Correct: use the u flag
/^.$/u.test('𠮷')  // true

Pitfall 4: Sorting

// Default sorting compares UTF-16 code units
['中', '𠮷', 'a'].sort()  // may not order as expected

// Correct: sort by code point
['中', '𠮷', 'a'].sort((a, b) =>
  a.codePointAt(0) - b.codePointAt(0)
)

7. Useful APIs

// Normalization (handling canonical equivalence of composed characters)
'café'.normalize('NFC')   // café (composed)
'café'.normalize('NFD')   // café (decomposed)

// Detecting emoji
function isEmoji(char) {
  const code = char.codePointAt(0);
  return (code >= 0x1F600 && code <= 0x1F64F) ||  // Emoticons
         (code >= 0x1F300 && code <= 0x1F5FF) ||  // Misc Symbols
         (code >= 0x1F680 && code <= 0x1F6FF) ||  // Transport
         (code >= 0x1F900 && code <= 0x1F9FF);    // Supplemental
}

// Getting the true grapheme length of a string
function graphemeLength(str) {
  const segmenter = new Intl.Segmenter('zh', { granularity: 'grapheme' });
  return [...segmenter.segment(str)].length;
}

graphemeLength('👨‍👩‍👧‍👦')  // 1 ✅
graphemeLength('🏳️‍🌈')    // 1 ✅