Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"dbt seed" with dependencies -> error in path resolution #1723

Closed
beckjake opened this issue Sep 4, 2019 · 0 comments · Fixed by #1783
Closed

"dbt seed" with dependencies -> error in path resolution #1723

beckjake opened this issue Sep 4, 2019 · 0 comments · Fixed by #1783
Labels
bug Something isn't working

Comments

@beckjake
Copy link
Contributor

beckjake commented Sep 4, 2019

Describe the bug

I have a project with a dependency that has seeds. When I run dbt seed in the parent project, I get an error about the dependency seeds:
[Errno 2] No such file or directory: 'data/seedtable.csv'

Steps To Reproduce

  1. Create a project with data/seedtable.csv named a
  2. Create another project named b and with a packages.yml like:
packages:
  - local: /path/to/project/a
  1. run dbt deps && dbt seed from the project b directory
  2. feel sad that it gives you an error

Expected behavior

I expect my dependency's seed to run successfully.

Screenshots and log output

2019-09-04 16:36:25,854704 (MainThread): Acquiring new postgres connection "seedtable".
2019-09-04 16:36:25,854894 (MainThread): Re-using an available connection from the pool (formerly model_2).
2019-09-04 16:36:25,963811 (MainThread): Unhandled error while executing seed.minimal.seedtable
[Errno 2] No such file or directory: 'data/seedtable.csv'
2019-09-04 16:36:25,964249 (MainThread):
Traceback (most recent call last):
  File "/Users/jake/src/fishtown/dbt/core/dbt/node_runners.py", line 203, in safe_run
    result = self.compile_and_execute(manifest, ctx)
  File "/Users/jake/src/fishtown/dbt/core/dbt/node_runners.py", line 146, in compile_and_execute
    result = self.run(ctx.node, manifest)
  File "/Users/jake/src/fishtown/dbt/core/dbt/node_runners.py", line 245, in run
    return self.execute(compiled_node, manifest)
  File "/Users/jake/src/fishtown/dbt/core/dbt/node_runners.py", line 357, in execute
    materialization_macro.generator(context)()
  File "/Users/jake/src/fishtown/dbt/core/dbt/clients/jinja.py", line 127, in call
    return macro(*args, **kwargs)
  File "/Users/jake/.pyenv/versions/3.6.5/envs/dbt36/lib/python3.6/site-packages/jinja2/runtime.py", line 575, in __call__
    return self._invoke(arguments, autoescape)
  File "/Users/jake/.pyenv/versions/3.6.5/envs/dbt36/lib/python3.6/site-packages/jinja2/asyncsupport.py", line 110, in _invoke
    return original_invoke(self, arguments, autoescape)
  File "/Users/jake/.pyenv/versions/3.6.5/envs/dbt36/lib/python3.6/site-packages/jinja2/runtime.py", line 579, in _invoke
    rv = self._func(*arguments)
  File "<template>", line 346, in macro
  File "/Users/jake/.pyenv/versions/3.6.5/envs/dbt36/lib/python3.6/site-packages/jinja2/sandbox.py", line 438, in call
    return __context.call(__obj, *args, **kwargs)
  File "/Users/jake/.pyenv/versions/3.6.5/envs/dbt36/lib/python3.6/site-packages/jinja2/runtime.py", line 262, in call
    return __obj(*args, **kwargs)
  File "/Users/jake/src/fishtown/dbt/core/dbt/context/common.py", line 364, in load_agate_table
    table = dbt.clients.agate_helper.from_csv(path)
  File "/Users/jake/src/fishtown/dbt/core/dbt/clients/agate_helper.py", line 65, in from_csv
    with open(abspath, encoding='utf-8') as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'data/seedtable.csv'

System information

Which database are you using dbt with?
Any

The output of dbt --version:
0.15.0-a1

The operating system you're using:
Any

The output of python --version:
Any

Additional context

I'm pretty sure this is related to the new FilePath construct I introduced. It doesn't know about the project root of the project it was parsed from, so original_file_path (which we use when loading seeds at runtime) is a relative path. I think the proper fix is for FilePath to know about the project_root of the project it came from and construct the original_file_path property in a way that includes the project_root.

This is only a problem for seeds, as all other files are read in during parse time.

@beckjake beckjake added bug Something isn't working triage labels Sep 4, 2019
@drewbanin drewbanin added this to the Louisa May Alcott milestone Sep 17, 2019
@drewbanin drewbanin removed the triage label Sep 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants