-
Notifications
You must be signed in to change notification settings - Fork 45
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Initialize * in progress * Fully running * Merge 0.7.3 * Use is_adapter macro w default adapters * Ready for 0.9.0: all-around upgrade (#79) * Rebase + rework * Use better adapter macros. Full pg support * More dbt-onic convert_timezone. Last catches * Prettify code. Add docs for macros, seed * Prettify + tidy up * Update README * Enable override of set_default_adapters * Update DAG img * Add docs for set_default_adapters
- Loading branch information
Showing
46 changed files
with
488 additions
and
357 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Validating CODEOWNERS rules …
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
* @jtcohen6 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
--- | ||
name: Bug report | ||
about: Report a bug or an issue you've found with this package | ||
title: '' | ||
labels: bug, triage | ||
assignees: '' | ||
|
||
--- | ||
|
||
### Describe the bug | ||
<!--- | ||
A clear and concise description of what the bug is. What command did you run? What happened? | ||
---> | ||
|
||
### Steps To Reproduce | ||
<!--- | ||
In as much detail as possible, please provide steps to reproduce the issue. Sample data that triggers the issue, example model code, etc is all very helpful here. | ||
---> | ||
|
||
### Expected results | ||
<!--- | ||
A clear and concise description of what you expected to happen. | ||
---> | ||
|
||
### Actual results | ||
<!--- | ||
A clear and concise description of what you expected to happen. | ||
---> | ||
|
||
### Screenshots and log output | ||
<!--- | ||
If applicable, add screenshots or log output to help explain your problem. | ||
---> | ||
|
||
### System information | ||
**Which database are you using dbt with?** | ||
- [ ] postgres | ||
- [ ] redshift | ||
- [ ] bigquery | ||
- [ ] snowflake | ||
- [ ] other (specify: ____________) | ||
|
||
|
||
**The output of `dbt --version`:** | ||
``` | ||
<output goes here> | ||
``` | ||
|
||
**The operating system you're using:** | ||
|
||
**The output of `python --version`:** | ||
|
||
### Additional context | ||
<!--- | ||
Add any other context about the problem here. | ||
---> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
## Description & motivation | ||
<!--- | ||
Describe your changes, and why you're making them. | ||
--> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
version: 2 | ||
|
||
seeds: | ||
- name: country_codes | ||
description: > | ||
English names for countries based on their two-letter ISO code, which is | ||
stored in the `geo_country` column of `snowplow_page_views` and | ||
`snowplow_sessions`. Not directly used in any of the snowplow package's | ||
sessionization logic. | ||
columns: | ||
- name: name | ||
- name: two_letter_iso_code | ||
- name: three_letter_iso_code |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
6 changes: 0 additions & 6 deletions
6
macros/adapters/bigquery/pageviews/snowplow_web_events_scroll_depth.sql
This file was deleted.
Oops, something went wrong.
6 changes: 0 additions & 6 deletions
6
macros/adapters/bigquery/pageviews/snowplow_web_events_time.sql
This file was deleted.
Oops, something went wrong.
2 changes: 0 additions & 2 deletions
2
macros/adapters/bigquery/pageviews/snowplow_web_page_context.sql
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
{%- macro convert_timezone(in_tz, out_tz, in_timestamp) -%} | ||
{{ adapter_macro('convert_timezone', in_tz, out_tz, in_timestamp) }} | ||
{%- endmacro -%} | ||
|
||
{% macro default__convert_timezone(in_tz, out_tz, in_timestamp) %} | ||
convert_timezone({{in_tz}}, {{out_tz}}, {{in_timestamp}}) | ||
{% endmacro %} | ||
|
||
{% macro postgres__convert_timezone(in_tz, out_tz, in_timestamp) %} | ||
({{in_timestamp}} at time zone {{in_tz}} at time zone {{out_tz}})::timestamptz | ||
{% endmacro %} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
|
||
{%- macro get_max_sql(relation, field = 'collector_tstamp') -%} | ||
|
||
select | ||
|
||
coalesce( | ||
max({{field}}), | ||
'0001-01-01' -- a long, long time ago | ||
) as start_ts | ||
|
||
from {{ relation }} | ||
|
||
{%- endmacro -%} | ||
|
||
|
||
{%- macro get_most_recent_record(relation, field = 'collector_tstamp') -%} | ||
|
||
{%- set result = run_query(get_max_sql(relation, field)) -%} | ||
|
||
{% if execute %} | ||
{% set start_ts = result.columns['start_ts'].values()[0] %} | ||
{% else %} | ||
{% set start_ts = '' %} | ||
{% endif %} | ||
|
||
{{ return(start_ts) }} | ||
|
||
{%- endmacro -%} | ||
|
||
|
||
{%- macro get_start_ts(relation, field = 'collector_tstamp') -%} | ||
{{ adapter_macro('get_start_ts', relation, field) }} | ||
{%- endmacro -%} | ||
|
||
|
||
{%- macro default__get_start_ts(relation, field = 'collector_tstamp') -%} | ||
({{get_max_sql(relation, field)}}) | ||
{%- endmacro -%} | ||
|
||
|
||
{%- macro bigquery__get_start_ts(relation, field = 'collector_tstamp') -%} | ||
|
||
{%- set partition_by = config.get('partition_by', none) -%} | ||
{%- set partitions = config.get('partitions', none) -%} | ||
|
||
{%- set start_ts -%} | ||
{%- if config.incremental_strategy == 'insert_overwrite' -%} | ||
|
||
{%- if partitions -%} least({{partitions|join(',')}}) | ||
{%- elif partition_by.data_type == 'date' -%} _dbt_max_partition | ||
{%- else -%} date(_dbt_max_partition) | ||
{%- endif -%} | ||
|
||
{%- else -%} | ||
|
||
{%- set rendered -%} | ||
{%- if partition_by.data_type == 'date' -%} {{partition_by.field}} | ||
{%- else -%} date({{partition_by.field}}) {%- endif -%} | ||
{%- endset -%} | ||
{%- set record = get_most_recent_record(relation, rendered) -%} | ||
'{{record}}' | ||
|
||
{%- endif -%} | ||
{%- endset -%} | ||
|
||
{%- do return(start_ts) -%} | ||
|
||
{%- endmacro -%} |
Oops, something went wrong.