UTF-8 characters are stripped out of basket item name

[judgej]

This is done in

omnipay-sagepay/src/Message/AbstractRequest.php

Line 177 in 2474c68

protected function filterDiscountName($name)

It treats the item description as an ASCII string and strips out any non-ASCII character. This results in Western European international characters being removed, which are commonly used.

Instead, an attempt should be made to map the UTF-8 string (testing that it is valid UTF-8 first) to extended ASCII (ISO-8859-1) which Sage Pay uses, so we get to keep as much of the item's meaning as possible.

It should also be noted that recently some of the Sage Pay APIs started accepting UTF-8 characters and makes its own attempt at converting to ISO8859-1. This has been a long time coming (I've been complaining about this for five years) but some end-to-end tests may tell us more about exactly what character sets can be sent through the basket XML.