Why Your Emojis and Multilingual SMS Messages Aren’t Displaying Correctly—And How to Fix It

Does your SMS look like this?

???????? Get 50% off ???? today only! ???? Use code: ??????
Ends tonight: 23:59. Don’t miss out!

When this is the SMS you wanted to send?

🌟 ¡Hola! Get 50% off 🛍️ today only! 🎉 Use code: AHORA 🌍
Ends tonight: 23:59. Don’t miss out!

If you’ve sent out a message only to have it come back looking like a strange collection of question marks, you're not alone. This happens more often than you'd think. Whether you're sending a simple emoji-laden "Thanks 😊" or trying to wow your global audience with messages in different languages, you might be hit with these frustrating "???" placeholders. But why? And, more importantly, how do we fix it?

The Root Cause: SMS Encoding Issues

The problem boils down to encoding. There are two main encoding schemes that determine how SMS messages are processed and displayed: GSM-7 and UCS-2.

GSM-7: The Old Standard

GSM-7 is the most common encoding for SMS, packing each character into 7 bits. It’s great for basic Latin characters (A-Z, 0-9), but that’s about where its usefulness ends. If you try to send something outside this character set—like emojis, accented characters, or non-Latin scripts—things start to fall apart. You’ll see question marks or weird symbols because GSM-7 simply doesn’t understand them.

UCS-2: The Unicode Hero

Enter UCS-2, a 16-bit encoding system that can handle almost any character you throw at it: emojis, Chinese characters, Arabic script, you name it. It’s the go-to encoding when you want to send messages with diverse characters or a little bit of emoji flair.

The Fix: How to Send Fun and Multilingual Messages Without Breaking

Here’s what you need to know to make sure your SMS messages appear exactly the way you want:

1. Know Your SMS Platform’s Encoding Settings

Every platform handles encoding a bit differently, so here’s how some of the top players do it:

• Twilio: Automatically switches to UCS-2 when your message contains non-GSM-7 characters (like emojis or non-English scripts).
• Vonage: Lets you explicitly set the encoding to unicode in the API to avoid encoding issues.
• Plivo: Similar to Vonage, Plivo allows you to set the encoding type to unicode in your API request to ensure proper UCS-2 encoding.
• MessageBird: Also switches to UCS-2 automatically but uses UTF-8 by default. No need to set the encoding unless you’re going for something specific.

2. Use the Right Encoding in Your JSON Payload

Depending on the platform, you might need to specify the encoding manually.

Example (Vonage/Nexmo):

{
    "api_key": "your_api_key",
    "api_secret": "your_api_secret",
    "to": "447700900000",
    "from": "YourApp",
    "text": "¡Hola! 🌍 Get 50% off today!",
    "type": "unicode"
}

By setting the type to unicode, you ensure that your message is sent in UCS-2 and looks exactly as intended, with emojis and non-Latin text displayed correctly.

3. Test Your Message Across Devices

Before launching that fun emoji-laden campaign, test the message on different devices and carriers to ensure it shows up as expected. Different carriers and devices may handle encoding slightly differently, so testing is your best bet to avoid the dreaded "???" issue.

Why Your SMS Messages Are Broken:

1. Emojis or Special Characters: GSM-7 can’t handle them. It’s like trying to run a modern video game on a 1990s computer—things just don’t compute.
2. Accented Letters or Non-Latin Scripts: Languages like Spanish, Arabic, or Chinese require UCS-2 encoding because they have characters that fall outside the GSM-7 set.

How to Ensure Correct Display of Multilingual Messages:

1. Use the Right Encoding: If you’re using special characters or non-Latin scripts, make sure your message is being sent with UCS-2 encoding.
2. Watch Your Character Count: With UCS-2, the maximum number of characters per SMS drops from 160 to 70. Keep your messages short and sweet to avoid splitting them into multiple parts.
3. Test, Test, Test: Make sure your message looks good on multiple devices and carriers.

Stop letting your multilingual and emoji-filled messages get lost in translation. By understanding SMS encoding and choosing the right settings for your platform, you can ensure your texts are sent exactly as intended—whether they contain Chinese characters, Arabic script, or just a lot of fun emojis. No more question marks!