Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Add cross db ceiling macro #332

Closed
wants to merge 8 commits into from
Closed

WIP: Add cross db ceiling macro #332

wants to merge 8 commits into from

Conversation

emilieschario
Copy link
Contributor

@emilieschario emilieschario commented Feb 8, 2021 •
edited
Loading

This is a:

  • bug fix PR with no breaking changes — please ensure the base branch is master
  • new functionality — please ensure the base branch is the latest dev/ branch
  • a breaking change — please ensure the base branch is the latest dev/ branch

Description & motivation

This adds a default call and an alternate call for ceil or ceiling so that it can be used across more systems. Notably, this makes things Spark compatible. dbt-labs/spark-utils#5

Checklist

  • I have verified that these changes work locally on the following warehouses (Note: it's okay if you do not have access to all warehouses, this helps us understand what has been covered)
    • BigQuery
    • Postgres
    • Redshift
    • Snowflake
  • I have updated the README.md (if applicable)
  • I have added tests & descriptions to my models (and macros if applicable)
  • I have added an entry to CHANGELOG.md

@emilieschario emilieschario marked this pull request as draft February 8, 2021 21:49
@emilieschario
Copy link
Contributor Author

@clrcrl can you share what's failing? I can't see the pipeline details.

clrcrl
clrcrl suggested changes Apr 21, 2021
View reviewed changes
Copy link
Contributor

@clrcrl clrcrl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😅
sorry about the delay here!

macros/cross_db_utils/ceil.sql
ceil({{ field }})
{%- endmacro %}

{% macro alternative_ceil(field) %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which adapters need to use ceiling over ceil? We'll want to adjust this to something like:

Suggested change
{% macro alternative_ceil(field) %}
{% macro redshift__ceil(field) %}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's spark specifically (so dbx, but I thought it took the default then the alternative then database specific ones? since there was only on alternative (redshift, bq, and snowflake are all ceil), I thought one catchall might make more sense? If you don't think so, I can move this PR to spark_utils.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess in both cases we'd need the default in dbt-utils, and the override in spark utils. Since none of our core adapters use ceiling, I think we'd want the spark-specific version over in spark-utils

integration_tests/models/cross_db_utils/test_ceil.sql Outdated Show resolved Hide resolved
clrcrl
clrcrl suggested changes May 4, 2021
View reviewed changes
macros/cross_db_utils/ceil.sql Outdated Show resolved Hide resolved
@clrcrl clrcrl force-pushed the dev/0.7.0 branch 2 times, most recently from 445dbfd to 4843069 Compare May 19, 2021 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@clrcrl clrcrl clrcrl requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@emilieschario @clrcrl