Unicode issues

[dnalor]

• It seems the catalog number can be non-ascii, so there was a encoding error in program.py.
  Fix: common/program.py line 370:

                if metadata.catalogNumber:
                    self._stdout.write("Cat no  : %s\n" %
                                       metadata.catalogNumber.encode('utf-8'))

• Track names can contain non-latin-1 characters (maybe Windows cp1252):
  Album: Front 242 - Tyranny >For You<
  CDDB Disc ID: 860cd90a
  MusicBrainz Disc ID: im8gRiFeyibqd0fTZdGkgUrx9.Q-
  MusicBrainz lookup url: https://musicbrainz.org/cdtoc/attach?toc=1+10+246857+150+20575+39182+58992+85427+106325+126762+147380+173607+188130&tracks=10&id=im8gRiFeyibqd0fTZdGkgUrx9.Q-

Ripping track 5 of 10: 05 - Gripped by Fear.flac
CRCs match for track 5                    
Peak level: 99.52% 
Rip quality: 100.00%
Traceback (most recent call last):
  File "/home/user/appdata/venv-whipper/bin/whipper", line 11, in <module>
    load_entry_point('whipper==0.5.1', 'console_scripts', 'whipper')()
  File "/home/user/appdata/venv-whipper/lib/python2.7/site-packages/whipper-0.5.1-py2.7.egg/whipper/command/main.py", line 40, in main
    ret = cmd.do()
  File "/home/user/appdata/venv-whipper/lib/python2.7/site-packages/whipper-0.5.1-py2.7.egg/whipper/command/basecommand.py", line 124, in do
    return self.cmd.do()
  File "/home/user/appdata/venv-whipper/lib/python2.7/site-packages/whipper-0.5.1-py2.7.egg/whipper/command/basecommand.py", line 124, in do
    return self.cmd.do()
  File "/home/user/appdata/venv-whipper/lib/python2.7/site-packages/whipper-0.5.1-py2.7.egg/whipper/command/cd.py", line 190, in do
    self.doCommand()
  File "/home/user/appdata/venv-whipper/lib/python2.7/site-packages/whipper-0.5.1-py2.7.egg/whipper/command/cd.py", line 479, in doCommand
    _ripIfNotRipped(i + 1)
  File "/home/user/appdata/venv-whipper/lib/python2.7/site-packages/whipper-0.5.1-py2.7.egg/whipper/command/cd.py", line 375, in _ripIfNotRipped
    if os.path.exists(path):
  File "/home/user/appdata/venv-whipper/lib64/python2.7/genericpath.py", line 26, in exists
    os.stat(path)
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u25b7' in position 69: ordinal not in range(256)

Other examples:
Jean-Michel Jarre - Revolutions / Track 3 / https://musicbrainz.org/release/e2d4719c-34d2-445f-a1f1-c6877b614411
R.E.M. - Green / Track 8 (Character u'\u2010')

Fix: command/cd.py line 362:

            logger.debug('ripIfNotRipped: path %r' % path)
            trackResult.number = number
# convert
            path = unicode( path.encode('latin1','ignore'), 'latin1', 'ignore')

[JoeLametta]

@Freso You're the MusicBrainz expert here: could you please comment about the non ASCII catalog number issue?

• Fix catalog number issue
• Check if the track names issue still exists in the latest whipper version

[Freso]

What kind of comment do you want? I don't think MusicBrainz puts arbitrary restrictions on what characters can go into a cat. no., and AFAIK all text is stored in the MB db as UTF-8 (and served as such too).

[JoeLametta]

@Freso Thanks for your confirmation: didn't know if it was "normal" to have a non ASCII catalog number.

[Freso]

It isn't. I haven't encountered it before. :) I'm not saying it's normal, I'm just saying it isn't "out of spec".

[JoeLametta]

@Freso It seems that the track names issue is no longer present in latest whipper version... (otherwise I suppose #319 would have failed before).

Fix included in: #323.

[Freso]

Yeah, I've ripped tons of discs with non-ASCII release names, artist names, track names, etc. They all work fine. I don't think I've ever encountered one with a non-ASCII cat. no. though. :)