Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add cross db ceiling macro #332

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,14 @@ While these macros are cross database, they do not support all databases.
These macros are provided to make date calculations easier and are not a core part of dbt.
Most date macros are not supported on postgres.

#### ceil ([source](macros/cross_db_utils/ceil.sql))
This macro returns the smallest integer.

Usage:
```
{{ dbt_utils.ceil(field) }}
```

#### current_timestamp ([source](macros/cross_db_utils/current_timestamp.sql))
This macro returns the current timestamp.

Expand Down Expand Up @@ -95,7 +103,7 @@ Usage:
---
### Date/Time
#### date_spine ([source](macros/datetime/date_spine.sql))
This macro returns the sql required to build a date spine. The spine will include the `start_date` (if it is aligned to the `datepart`), but it will not include the `end_date`.
This macro returns the sql required to build a date spine. The spine will include the `start_date` (if it is aligned to the `datepart`), but it will not include the `end_date`.

Usage:
```
Expand Down
8 changes: 8 additions & 0 deletions integration_tests/data/cross_db/data_ceil.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
input,output
2.0,2.0
2.3,3.0
2.8,3.0
2.5,3.0
-2.3,-2.0
-2.8,-2.0
-2.5,-2.0
32 changes: 19 additions & 13 deletions integration_tests/models/cross_db_utils/schema.yml
Original file line number Diff line number Diff line change
@@ -1,78 +1,84 @@
version: 2

models:
- name: test_ceil
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_concat
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_current_timestamp
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_date_trunc
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_dateadd
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_datediff
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_hash
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_last_day
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_length
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_safe_cast
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_split_part
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_replace
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_right
tests:
- assert_equal:
actual: actual
expected: expected

- name: test_position
tests:
- assert_equal:
Expand All @@ -83,4 +89,4 @@ models:
tests:
- assert_equal:
actual: actual
expected: expected
expected: expected
12 changes: 12 additions & 0 deletions integration_tests/models/cross_db_utils/test_ceil.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@

with data as (

select * from {{ ref('data_ceil') }}

)

select
{{ dbt_utils.ceil('input') }} as actual,
output as expected

from data
11 changes: 11 additions & 0 deletions macros/cross_db_utils/ceil.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{% macro ceil(field) -%}
{{ return(adapter.dispatch('ceil', packages = dbt_utils._get_utils_namespaces())(fields)) }}
emilieschario marked this conversation as resolved.
Show resolved Hide resolved
{%- endmacro %}

{% macro default__ceil(field) -%}
ceil({{ field }})
{%- endmacro %}

{% macro alternative_ceil(field) %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which adapters need to use ceiling over ceil? We'll want to adjust this to something like:

Suggested change
{% macro alternative_ceil(field) %}
{% macro redshift__ceil(field) %}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's spark specifically (so dbx, but I thought it took the default then the alternative then database specific ones? since there was only on alternative (redshift, bq, and snowflake are all ceil), I thought one catchall might make more sense? If you don't think so, I can move this PR to spark_utils.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess in both cases we'd need the default in dbt-utils, and the override in spark utils. Since none of our core adapters use ceiling, I think we'd want the spark-specific version over in spark-utils

ceiling({{ field }})
{% endmacro %}