Skip to content

Commit

Permalink
Update parameter
Browse files Browse the repository at this point in the history
  • Loading branch information
clrcrl committed Dec 2, 2019
1 parent ee0c4d7 commit b2ee6cf
Show file tree
Hide file tree
Showing 3 changed files with 55 additions and 9 deletions.
37 changes: 32 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@ models:
- dbt_utils.mutually_exclusive_ranges:
lower_bound_column: min_age
upper_bound_column: max_age
allow_gaps: false
gaps: not_allowed

# test that each customer can only have one subscription at a time
- name: subscriptions
Expand All @@ -256,7 +256,7 @@ models:
lower_bound_column: started_at
upper_bound_column: ended_at
partition_by: customer_id
allow_gaps: true
gaps: required
```
**Args:**
* `lower_bound_column` (required): The name of the column that represents the
Expand All @@ -266,7 +266,8 @@ upper value of the range. Must be not null.
* `partition_by` (optional): If a subset of records should be mutually exclusive
(e.g. all periods for a single subscription_id are mutually exclusive), use this
argument to indicate which column to partition by. `default=none`
* `allow_gaps` (optional): Whether there can be gaps between each range. `default=true`
* `gaps` (optional): Whether there can be gaps are allowed between ranges.
`default='allowed', one_of=['not_allowed', 'allowed', 'required']`

**Note:** Both `lower_bound_column` and `upper_bound_column` should be not null.
If this is not the case in your data source, consider passing a coalesce function
Expand All @@ -281,8 +282,34 @@ models:
lower_bound_column: coalesce(started_at, '1900-01-01')
upper_bound_column: coalesce(ended_at, '2099-12-31')
partition_by: customer_id
allow_gaps: true
```
gaps: allowed
```

**Understanding the `gaps` parameter:**
Here are a number of examples for each allowed `gaps` parameter.
* `gaps:not_allowed`: The upper bound of one record should be the lower bound of
the next record.
| lower_bound | upper_bound |
|-------------|-------------|
| 0 | 1 |
| 1 | 2 |
| 2 | 3 |

* `gaps:allowed` (default): There may be a gap between the upper bound of one
record and the lower bound of the next record.
| lower_bound | upper_bound |
|-------------|-------------|
| 0 | 1 |
| 2 | 3 |
| 3 | 4 |

* `gaps:required`: There must be a gap between the upper bound of one record and
the lower bound of the next record (common for date ranges).
| lower_bound | upper_bound |
|-------------|-------------|
| 0 | 1 |
| 2 | 3 |
| 4 | 5 |
---
### SQL helpers
#### get_column_values ([source](macros/sql/get_column_values.sql))
Expand Down
10 changes: 8 additions & 2 deletions integration_tests/models/schema_tests/schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,12 +56,18 @@ models:
- dbt_utils.mutually_exclusive_ranges:
lower_bound_column: lower_bound
upper_bound_column: upper_bound
allow_gaps: false
gaps: not_allowed

- name: data_test_mutually_exclusive_ranges_with_gaps
tests:
- dbt_utils.mutually_exclusive_ranges:
lower_bound_column: valid_from
upper_bound_column: coalesce(valid_to, '2099-01-01')
partition_by: subscription_id
allow_gaps: true
gaps: allowed

- dbt_utils.mutually_exclusive_ranges:
lower_bound_column: valid_from
upper_bound_column: coalesce(valid_to, '2099-01-01')
partition_by: subscription_id
gaps: required
17 changes: 15 additions & 2 deletions macros/schema_tests/mutually_exclusive_ranges.sql
Original file line number Diff line number Diff line change
@@ -1,7 +1,20 @@
{% macro test_mutually_exclusive_ranges(model, lower_bound_column, upper_bound_column, partition_by=None, allow_gaps=True) %}
{% macro test_mutually_exclusive_ranges(model, lower_bound_column, upper_bound_column, partition_by=None, gaps='allowed') %}

{% if gaps == 'not_allowed' %}
{% set allow_gaps_operator='=' %}
{% elif gaps == 'allowed' %}
{% set allow_gaps_operator='<=' %}
{% elif gaps == 'required' %}
{% set allow_gaps_operator='<' %}
{% else %}
{{ exceptions.raise_compiler_error(
"`gaps` argument for mutually_exclusive_ranges test must be one of ['not_allowed', 'allowed', 'required'] Got: '" ~ gaps ~"'.'"
) }}

{% endif %}

{% set partition_clause="partition by " ~ partition_by if partition_by else '' %}
{% set allow_gaps_operator='<=' if allow_gaps else '=' %}

with calc as (

select
Expand Down

0 comments on commit b2ee6cf

Please sign in to comment.