Looks like the diffing doesn't consider whether a character is non-ASCII and breaks unicode emojis in pieces if they are different but occupy the same space. That would result in some unknown character � after all is said and done. Assuming it's conversion related.

Looking into it out of curiosity.

Looks like @yetanotherape has solved it for their PHP fork:

• Remove UCS-2LE encoding/decoding? yetanotherape/diff-match-patch#9
• yetanotherape/diff-match-patch@3e7b024

The two current attempts at solving it in this repo have both had complications:

• Use UTF-32 Aware String in JavaScript #13
• JS: Handle surrogate pairs correctly #69

Any updates here?

@ndvbd I ended up "solving" it by just escaping all the special unicode stuff with text = encodeURI(text) before saving the text (and using decodeURI(...) to undo it, of course). Bit of a hack but it works for my use case.

Check out #80, as referenced above @ndvbd's comment. It should handle all the surrogate pairs properly.