.pkgr.yml to enable packaging on packager.io

[crohr]

This change enables packaging on Packager.io for the major linux distributions (see https://packager.io/gh/pkgr/kinto-heroku/build_runs/3 for an example build).

I tried to test the resulting package, but not sure what's going on after I set a proper DATABASE_URL:

Install the package:

$ wget -qO - https://deb.packager.io/key | sudo apt-key add -
$ echo "deb https://deb.packager.io/gh/pkgr/kinto-heroku trusty pkgr" | sudo tee /etc/apt/sources.list.d/kinto-heroku.list
$ sudo apt-get update
$ sudo apt-get install kinto                                                                                  
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  kinto
0 upgraded, 1 newly installed, 0 to remove and 180 not upgraded.
Need to get 39.9 MB of archives.
After this operation, 138 MB of additional disk space will be used.
Fetched 39.9 MB in 6s (6,614 kB/s)
Selecting previously unselected package kinto.
(Reading database ... 92411 files and directories currently installed.)
Preparing to unpack .../kinto_0.0.0-1456475295.f903350.trusty_amd64.deb ...
Unpacking kinto (0.0.0-1456475295.f903350.trusty) ...
Processing triggers for ureadahead (0.100.0-16) ...
Setting up kinto (0.0.0-1456475295.f903350.trusty) ...

Trying to run the web process:

vagrant@vagrant-ubuntu-trusty-64:/vagrant/pkgr$ sudo kinto run web
Traceback (most recent call last):
  File "./runapp.py", line 14, in <module>
    database)
ValueError: DATABASE_URL is not correctly defined: None

Setting a proper DATABASE_URL:

vagrant@vagrant-ubuntu-trusty-64:/vagrant/pkgr$ sudo kinto config:set DATABASE_URL=postgres://dbuser:[email protected]:5432/dbname

Then re-running web process:

vagrant@vagrant-ubuntu-trusty-64:/vagrant/pkgr$ sudo kinto run web
Traceback (most recent call last):
  File "./runapp.py", line 16, in <module>
    app = loadapp('config:kinto.ini', relative_to='.')
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
    return context.create()
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
    return self.object_type.invoke(self)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
    return fix_call(context.object, context.global_conf, **context.local_conf)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
    val = callable(*args, **kw)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/kinto/__init__.py", line 43, in main
    default_settings=DEFAULT_SETTINGS)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/cliquet/initialization.py", line 549, in initialize
    config.include("cliquet", route_prefix=api_version)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/pyramid/config/__init__.py", line 798, in include
    c(configurator)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/cliquet/__init__.py", line 145, in includeme
    step_func(config)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/cliquet/initialization.py", line 227, in setup_storage
    backend = storage_mod.load_from_config(config)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/cliquet/storage/postgresql/__init__.py", line 741, in load_from_config
    client = create_from_config(config, prefix='storage_')
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/cliquet/storage/postgresql/client.py", line 94, in create_from_config
    engine = sqlalchemy.engine_from_config(settings, prefix=prefix, url=url)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/__init__.py", line 427, in engine_from_config
    return create_engine(url, **options)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/__init__.py", line 386, in create_engine
    return strategy.create(*args, **kwargs)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 49, in create
    u = url.make_url(name_or_url)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/url.py", line 186, in make_url
    return _parse_rfc1738_args(name_or_url)
  File "/opt/kinto/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/url.py", line 235, in _parse_rfc1738_args
    "Could not parse rfc1738 URL from string '%s'" % name)
sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string ''

Can you tell what's wrong?

[Natim]

For the database URL you actually need to define multiple variables, see: https:/Kinto/kinto/blob/master/docker-compose.yml#L12-L18

[Natim]

Ideally we wanted to create a /etc/kinto.ini file with the default configuration.

[crohr]

@Natim ok, what would be the default config? I can put that in a postinstall file

[Natim]

Ideally we should propose a production ready conf.

Having systemd managing a uwsgi daemon. You can have a first version of this configuration file (with most of the production settings as comments) by running kinto --ini ./kinto.ini --backend postgres init

[almet]

Oh, I missed this! Thanks for taking the time to do this @crohr.

You can generate a default configuration file by running kinto init. The resulting default file looks like that:

[server:main]
use = egg:waitress#main
host = 0.0.0.0
port = 8888


[app:main]
use = egg:kinto

# Backends.
# http://kinto.readthedocs.org/en/latest/configuration/settings.html#storage
kinto.storage_backend = cliquet.storage.postgresql
kinto.storage_url = postgres://postgres:postgres@localhost/postgres
kinto.cache_backend = cliquet.cache.postgresql
kinto.cache_url = postgres://postgres:postgres@localhost/postgres
kinto.permission_backend = cliquet.permission.postgresql
kinto.permission_url = postgres://postgres:postgres@localhost/postgres

kinto.userid_hmac_secret = 28460c66a3fc0e2644e7bdbb8c4ecfdc41455ed253ed63c4c8309b784aa7996a
multiauth.policies = basicauth

kinto.experimental_collection_schema_validation = true

kinto.includes = kinto.plugins.default_bucket
[loggers]
keys = root, cliquet

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = INFO
handlers = console

[logger_cliquet]
level = DEBUG
handlers =
qualname = cliquet

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s

Let me know if you need any more help on this.

[Natim]

Any status update on this? I'd really like to have this feature.

[crohr]

@Natim I can have another look. However, do you know if the kinto.ini file can take environment variables instead of hard-coded values?

[Natim]

do you know if the kinto.ini file can take environment variables instead of hard-coded values?

Yes it can 👍

[Natim]

Should we update this PR or close it?

[crohr]

@Natim if you think having native packages still has value, then I'm available to help push this, otherwise I suppose you can close it. Let me know either way :)

[Natim]

I still think it can be interesting yes.