Skip to content
This repository has been archived by the owner on Apr 16, 2023. It is now read-only.

GIS exports broken due to translit from giggls/mapnik-german-l10n master #870

Open
das-g opened this issue May 23, 2017 · 0 comments
Open
Assignees
Labels

Comments

@das-g
Copy link
Member

das-g commented May 23, 2017

The master branch of giggls/mapnik-german-l10n (currently giggls/mapnik-german-l10n@40e107f a.k.a. v2.2.5) has changed in ways that break our current use of it: Exporting GIS formats with a freshly built worker docker image fails with

exception caught while processing /home/py/osmaxx/conversion/converters/converter_gis/bootstrap/sql/functions/0030_transliterate.sql
11:18:28 sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) required extension "unaccent" is not installed
 [SQL: "---------------------------------------------------------------------------------------\n-- write or print (a letter or word) using the closest corresponding letters of\n-- a different alphabet or language.\n-- EG. select osml10n_translit('Москва́');\n--     Moskvá\n---------------------------------------------------------------------------------------\nCREATE EXTENSION IF NOT EXISTS postgis;\nCREATE EXTENSION IF NOT EXISTS osml10n;\n"]
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
psycopg2.ProgrammingError: required extension "unaccent" is not installed


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/rq/worker.py", line 700, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.4/site-packages/rq/job.py", line 500, in perform
    self._result = self.func(*self.args, **self.kwargs)
  File "/home/py/osmaxx/conversion/converters/converter.py", line 43, in convert
    converter.perform_export(**params)
  File "/home/py/osmaxx/conversion/converters/converter_gis/gis.py", line 32, in perform_export
    gis.create_gis_export()
  File "/home/py/osmaxx/conversion/converters/converter_gis/gis.py", line 75, in create_gis_export
    _bootstrapper.bootstrap()
  File "/home/py/osmaxx/conversion/converters/converter_gis/bootstrap/bootstrap.py", line 29, in bootstrap
    self._setup_db_functions()
  File "/home/py/osmaxx/conversion/converters/converter_gis/bootstrap/bootstrap.py", line 54, in _setup_db_functions
    self._execute_sql_scripts_in_folder(os.path.join(self._script_base_dir, 'sql', 'functions'))
  File "/home/py/osmaxx/conversion/converters/converter_gis/bootstrap/bootstrap.py", line 111, in _execute_sql_scripts_in_folder
    self._postgres.execute_sql_file(script_path)
  File "/home/py/osmaxx/conversion/converters/converter_gis/helper/postgres_wrapper.py", line 27, in execute_sql_file
    return self.execute_sql_command(psql_command_file.read())
  File "/home/py/osmaxx/conversion/converters/converter_gis/helper/postgres_wrapper.py", line 35, in execute_sql_command
    result = connection.execute(sqlalchemy.text(sql))
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) required extension "unaccent" is not installed
 [SQL: "---------------------------------------------------------------------------------------\n-- write or print (a letter or word) using the closest corresponding letters of\n-- a different alphabet or language.\n-- EG. select osml10n_translit('Москва́');\n--     Moskvá\n---------------------------------------------------------------------------------------\nCREATE EXTENSION IF NOT EXISTS postgis;\nCREATE EXTENSION IF NOT EXISTS osml10n;\n"]
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
psycopg2.ProgrammingError: required extension "unaccent" is not installed


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/rq/worker.py", line 700, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.4/site-packages/rq/job.py", line 500, in perform
    self._result = self.func(*self.args, **self.kwargs)
  File "/home/py/osmaxx/conversion/converters/converter.py", line 43, in convert
    converter.perform_export(**params)
  File "/home/py/osmaxx/conversion/converters/converter_gis/gis.py", line 32, in perform_export
    gis.create_gis_export()
  File "/home/py/osmaxx/conversion/converters/converter_gis/gis.py", line 75, in create_gis_export
    _bootstrapper.bootstrap()
  File "/home/py/osmaxx/conversion/converters/converter_gis/bootstrap/bootstrap.py", line 29, in bootstrap
    self._setup_db_functions()
  File "/home/py/osmaxx/conversion/converters/converter_gis/bootstrap/bootstrap.py", line 54, in _setup_db_functions
    self._execute_sql_scripts_in_folder(os.path.join(self._script_base_dir, 'sql', 'functions'))
  File "/home/py/osmaxx/conversion/converters/converter_gis/bootstrap/bootstrap.py", line 111, in _execute_sql_scripts_in_folder
    self._postgres.execute_sql_file(script_path)
  File "/home/py/osmaxx/conversion/converters/converter_gis/helper/postgres_wrapper.py", line 27, in execute_sql_file
    return self.execute_sql_command(psql_command_file.read())
  File "/home/py/osmaxx/conversion/converters/converter_gis/helper/postgres_wrapper.py", line 35, in execute_sql_command
    result = connection.execute(sqlalchemy.text(sql))
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) required extension "unaccent" is not installed
 [SQL: "---------------------------------------------------------------------------------------\n-- write or print (a letter or word) using the closest corresponding letters of\n-- a different alphabet or language.\n-- EG. select osml10n_translit('Москва́');\n--     Moskvá\n---------------------------------------------------------------------------------------\nCREATE EXTENSION IF NOT EXISTS postgis;\nCREATE EXTENSION IF NOT EXISTS osml10n;\n"]
11:18:28 Invoking exception handler <bound method Worker.move_to_failed_queue of <rq.worker.Worker object at 0x7fa9c2acaf98>>
11:18:28 Moving job to 'failed' queue

The last published build of the container uses giggls/mapnik-german-l10n@704ad2a a.k.a. v2.2.0

$> docker run -t --rm geometalab/osmaxx-worker:v3.10.0 bash -c 'cd $CODE/mapnik-german-l10n && git describe --tags && git rev-parse HEAD'
[ ok ] Starting PostgreSQL 9.4 database server: main.
v2.2.0
704ad2af30cae74891da5931f4abbcceb7a8a75d
@das-g das-g added the bug label May 23, 2017
@das-g das-g self-assigned this May 23, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

1 participant