Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-209] [Bug] Error parsing env vars in on-run-start or on-run-end hooks #4703

Closed
1 task done
fbertsch opened this issue Feb 9, 2022 · 1 comment
Closed
1 task done
Labels
bug Something isn't working duplicate This issue or pull request already exists

Comments

@fbertsch
Copy link

fbertsch commented Feb 9, 2022

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

If the on-run-start or on-run-end hook contains jinja that calls the env_var function, the compilation will fail with an error like the following:

 Encountered an error:
'$PROJECT://./dbt_project.yml'

Expected Behavior

I would expect the compilation to succeed when using the env_var function.

Steps To Reproduce

  1. Add an environment variable: export TEST_ENV_VAR="true"
  2. Add that env var to the on-run-start as a jinja function call:
on-run-start:
  - "{{ env_var('TEST_ENV_VAR') }}"
  1. Run dbt compile

Relevant log output

19:42:41.432145 [debug] [MainThread]: Flushing usage events
19:42:41.432955 [error] [MainThread]: Encountered an error:
'bug_test://./dbt_project.yml'
19:42:41.439341 [debug] [MainThread]: Traceback (most recent call last):
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/main.py", line 127, in main
    results, succeeded = handle_and_check(args)
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/main.py", line 192, in handle_and_check
    task, res = run_from_args(parsed)
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/main.py", line 246, in run_from_args
    results = task.run()
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/task/runnable.py", line 454, in run
    self._runtime_initialize()
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/task/runnable.py", line 156, in _runtime_initialize
    super()._runtime_initialize()
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/task/runnable.py", line 94, in _runtime_initialize
    self.load_manifest()
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/task/runnable.py", line 81, in load_manifest
    self.manifest = ManifestLoader.get_full_manifest(self.config)
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/parser/manifest.py", line 200, in get_full_manifest
    manifest = loader.load()
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/parser/manifest.py", line 326, in load
    parser_types
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/parser/manifest.py", line 482, in parse_project
    hook_parser.parse_file(file_block)
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/parser/hooks.py", line 119, in parse_file
    self.parse_node(hook)
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/parser/base.py", line 402, in parse_node
    self.render_update(node, config)
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/parser/base.py", line 377, in render_update
    context = self.render_with_context(node, config)
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/parser/base.py", line 268, in render_with_context
    parsed_node.raw_sql, context, parsed_node, capture_macros=True
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/clients/jinja.py", line 600, in get_rendered
    return render_template(template, ctx, node)
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/clients/jinja.py", line 549, in render_template
    return template.render(ctx)
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/jinja2/sandbox.py", line 462, in call
    return __context.call(__obj, *args, **kwargs)
  File "/Users/frankbertsch/repos/all-the-things/deployable/dbt/src/.venv37/lib/python3.7/site-packages/dbt/context/providers.py", line 1192, in env_var
    source_file = self.manifest.files[self.model.file_id]
KeyError: 'bug_test://./dbt_project.yml'


### Environment

```markdown
- OS: MacOS 11.5.2
- Python: Tested on 3.7.5 and 3.7.8
- dbt: 1.0.0

What database are you using dbt with?

snowflake

Additional Context

No response

@fbertsch fbertsch added bug Something isn't working triage labels Feb 9, 2022
@github-actions github-actions bot changed the title [Bug] Error parsing env vars in on-run-start or on-run-end hooks [CT-209] [Bug] Error parsing env vars in on-run-start or on-run-end hooks Feb 9, 2022
@fbertsch
Copy link
Author

fbertsch commented Feb 9, 2022

Fixed with #4524 - sorry for the noise! Upgrading to 1.0.1

@fbertsch fbertsch closed this as completed Feb 9, 2022
@jtcohen6 jtcohen6 added duplicate This issue or pull request already exists and removed triage labels Feb 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

2 participants