Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dbt-core 1.3 upgrade: Incremental mats: more standard and more error-proof #44

Closed
jlarue26 opened this issue Oct 24, 2022 · 1 comment · Fixed by #109
Closed

dbt-core 1.3 upgrade: Incremental mats: more standard and more error-proof #44

jlarue26 opened this issue Oct 24, 2022 · 1 comment · Fixed by #109
Assignees
@ArgusLi
Labels
upgrade

Comments

@jlarue26
Copy link
Contributor

jlarue26 commented Oct 24, 2022 •
edited
Loading

From dbt-labs/dbt-core#6011:

user-facing changes
new in 1.3:
breaking change. the default strategy of incremental models used to be delete+insert. it is now append. See point below to override this.
a new macro, adapter_get_incremental_default_sql(), in which adapter maintainers can specify a 'default' strategy for incremental models.
a new Adapter method, valid_incremental_strategies() in which adapter maintainers can define all supported strategies. Previously, if i user defined an unsupported incremental strategy, they would not learn of their error until compile time or later. Now dbt will throw an error before attempting to compile.
non-user facing changes
there was significant work (dbt-labs/dbt-core#5245 dbt-labs/dbt-core#5359) put into refactoring the default incremental materialization. If your adapter overrides the default, we recommend that you either:
2. refactor your materialization to match that of the default
even better, determine exactly why you currently need to override the default, and perhaps we can find a way to bring that flexibility into the core adapter so that you can stay up to date with the default mat
@ArgusLi ArgusLi self-assigned this Nov 1, 2022
@jlarue26 jlarue26 mentioned this issue Nov 8, 2022
Open
3 tasks
@jlarue26
Copy link
Contributor Author

jlarue26 commented Nov 9, 2022

dbt-dremio only supported append and thus there will be no breaking changes.
Tests still remain to be run and issues addressed.

@ArgusLi ArgusLi mentioned this issue Jan 25, 2023
Merged
1 task
@ArgusLi ArgusLi linked a pull request Jan 25, 2023 that will close this issue
44 dbt core 13 upgrade incremental mats more standard and more error proof #109
Merged
1 task
ArgusLi added a commit that referenced this issue Feb 1, 2023
@ArgusLi
44 dbt core 13 upgrade incremental mats more standard and more error …
fad73d9 
…proof (#109)

### Summary

Upgrade incremental materialization to new dbt-core 1.3 logic while
retaining Dremio specific changes.

### Description

- Add valid_incremental_strategies() with append being the only valid
strategy.
- Refactor materialization to match new default incremental
materialization.
- Add macro dremio__rename_relation() that uses CTAS and DROP instead of
default ALTER TABLE and RENAME to.

### Test Results

30/30 tests passed.

### Changelog

-   [x] Added a summary of what this PR accomplishes to CHANGELOG.md

### Related Issue
[44](#44)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ArgusLi ArgusLi

Labels
upgrade
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

44 dbt core 13 upgrade incremental mats more standard and more error proof dremio/dbt-dremio
2 participants
@ArgusLi @jlarue26