Configure reading .env files

[sloria]

#230 introduced support for reading .env files and using envvards in docker-sync configuration.

I use autoenv in order to do more than just set environment variables in my .env file (setting my Python virtual environment, for example). Therefore, I get an error when I run docker-sync start:

/usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv/parser.rb:63:in `parse_line': Line "source activate osf" doesn't match format (Dotenv::FormatError)
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv/parser.rb:47:in `block in call'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv/parser.rb:46:in `each'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv/parser.rb:46:in `call'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv/parser.rb:36:in `call'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv/environment.rb:13:in `load'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv/environment.rb:9:in `initialize'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv.rb:15:in `new'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv.rb:15:in `block (2 levels) in load'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv.rb:59:in `ignoring_nonexistent_files'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv.rb:14:in `block in load'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv.rb:46:in `block in with'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv.rb:45:in `each'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv.rb:45:in `reduce'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv.rb:45:in `with'
        from /usr/local/lib/ruby/gems/2.4.0/gems/dotenv-2.2.0/lib/dotenv.rb:13:in `load'
        from /usr/local/lib/ruby/gems/2.4.0/gems/docker-sync-0.2.0/lib/docker-sync/sync_manager.rb:19:in `initialize'
        from /usr/local/lib/ruby/gems/2.4.0/gems/docker-sync-0.2.0/tasks/sync/sync.thor:30:in `new'
        from /usr/local/lib/ruby/gems/2.4.0/gems/docker-sync-0.2.0/tasks/sync/sync.thor:30:in `start'
        from /usr/local/lib/ruby/gems/2.4.0/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
        from /usr/local/lib/ruby/gems/2.4.0/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
        from /usr/local/lib/ruby/gems/2.4.0/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
        from /usr/local/lib/ruby/gems/2.4.0/gems/thor-0.19.4/lib/thor/base.rb:444:in `start'
        from /usr/local/lib/ruby/gems/2.4.0/gems/docker-sync-0.2.0/bin/docker-sync:14:in `<top (required)>'
        from /usr/local/bin/docker-sync:22:in `load'
        from /usr/local/bin/docker-sync:22:in `<main>'

One solution to this would be to add a configuration option that allows a user to specify which file to read envvars from. This would look something like

# docker-sync.yml
version: '2'
options:
  env_file: 'docker-sync.env'
# ...

If set to null or false, no env file will be read.

Thoughts?

[EugenMayer]

Teach me wrong, but would this setting more be a global setting, not a project specific setting?
Telling you that, we have ~/docker-sync-global.yml - i like your point, the questions is, if people really define this per-project ever

[sloria]

I don't personally need it set per-project at the moment (I'm only using docker-sync for one project), but I could see a use case for it. Different projects may have different expectations for how .env is used.

[sloria]

P.S. I would be fine configuring this in ~/docker-sync-global.yml. I think it makes sense as global config--you can always add support for per-project config in the future.

[EugenMayer]

i see, i personally have no usage for .env that why i ask for the use cases.

What do you think, are you able to provide a PR?
We would need to modify https:/EugenMayer/docker-sync/blob/master/lib/docker-sync/config.rb#L11 that said, we do not have any config when loading this, so conceptually things get pretty complicated.

I am currently not even quiet sure when initialize is run, since DockerSyncConfig seems to be used statically only, never DockerSyncConfig.new, probably including require 'dotenv' does it right away.

Well you have to dig into it yourself right now, i cannot dedicate time to you right now, sorry

[sloria]

I understand that you can't work on this immediately.

I won't be able to look into a PR just yet, but I might take a look at the code when I have some time off of work.

Thank you for your prompt responses to my issue.

[EugenMayer]

Fair enough i guess, lets see how we can wrap this up :)

[sloria]

One possible intermediate solution that would at least get rid of the errors would be to remove usage of dotenv in favor of function that skips over non-envvar lines (e.g. source activate myenv). Foreman uses the following: https:/ddollar/foreman/blob/5f5502efed74c68f5cf381d38b2ad497676837ec/lib/foreman/env.rb#L8-L18 .

I've suggested that dotenv implement this behavior: bkeepers/dotenv#285 .

[matleh]

You could go the other way around and specify a different file to use for autoenv instead of .env using AUTOENV_ENV_FILENAME.

[sloria]

@matleh Good idea. I'll do that in the meantime.

[dsdobrzynski]

We use .env heavily in multiple projects and cannot modify them just to make docker-sync work. Is there a plan and timeline to fix this?

[EugenMayer]

There is no timeline here, specificly i do not plan to work on this feature myself yet. Sure iam happy for any contribution or pull request

[matleh]

Wouldn't it be the easiest thing to add an env-var DOCKER_SYNC_ENV_FILE which can be used to overwrite the location of the docker-sync .env file? Then people are free to either set that variable in their shell init script or people using autoenv can use that to overwrite DOCKER_SYNC_ENV_FILE on a per-project basis. But that would introduce another "place" for configuration. @EugenMayer what do you think of that?

[EugenMayer]

Why not, sounds reasonable. We just should default to .env, but making in configureable is something simple and effective

[EugenMayer]

Thank you @matleh for joining forces here an implementing DOCKER_SYNC_ENV_FILE, i adjusted the docs https:/EugenMayer/docker-sync/wiki/Variable-Replace---Dynamic-Configuration - thank you!