Skip to content

Latest commit

 

History

History
165 lines (124 loc) · 10.6 KB

CHANGELOG.md

File metadata and controls

165 lines (124 loc) · 10.6 KB
Raw

dbt-utils v0.7.1

Under the hood

  • Declare compatibility with dbt v0.21.0, which has no breaking changes for this package (#398)

Features

  • Allow user to provide any case type when defining the exclude argument in dbt_utils.star() (#403)

dbt-utils v0.7.0

Breaking changes

🚨 New dbt version

dbt v0.20.0 or greater is required for this release. If you are not ready to upgrade, consider using a previous release of this package.

In accordance with the version upgrade, this package release includes breaking changes to:

  • Generic (schema) tests
  • dispatch functionality

🚨 get_column_values

The order of (optional) arguments has changed in the get_column_values macro.

Before:

{% macro get_column_values(table, column, order_by='count(*) desc', max_records=none, default=none) -%}
...
{% endmacro %}

After:

{% macro get_column_values(table, column, max_records=none, default=none) -%}
...
{% endmacro %}

If you were relying on the position to match up your optional arguments, this may be a breaking change — in general, we recommend that you explicitly declare any optional arguments (if not all of your arguments!)

-- before: This works on previous version of dbt-utils, but on 0.7.0, the `50` would be passed through as the `order_by` argument
{% set payment_methods = dbt_utils.get_column_values(
        ref('stg_payments'),
        'payment_method',
        50
) %}

-- after
{% set payment_methods = dbt_utils.get_column_values(
        ref('stg_payments'),
        'payment_method',
        max_records=50
) %}

Features

  • Add new argument, order_by, to get_column_values (code originally in #289 from @clausherther, merged via #349)
  • Add slugify macro, and use it in the pivot macro. 🚨 This macro uses the re module, which is only available in dbt v0.19.0+. As a result, this feature introduces a breaking change. (#314)
  • Add not_null_proportion schema test that allows the user to specify the minimum (at_least) tolerated proportion (e.g., 0.95) of non-null values

Under the hood

  • Update the default implementation of concat macro to use || operator (#373 from @ChristopheDuong). Note this may be a breaking change for adapters that support concat() but not ||, such as Apache Spark.
  • Use power() instead of pow() in generate_series() and haversine_distance() as they are synonyms in most SQL dialects, but some dialects only have power() (#354 from @swanderz)

dbt-utils v0.6.6

Fixes

  • make sequential_values schema test use dbt_utils.type_timestamp() to allow for compatibility with db's without timestamp data type. #376 from @swanderz

dbt-utils v0.6.5

Features

Fixes

  • Handle booleans gracefully in the unpivot macro (#305 @avishalom)
  • Fix a bug in get_relation_by_prefix that happens with Snowflake external tables. Now the macro will retrieve tables that match the prefix which are external tables (#351)
  • Fix cardinality_equality test when the two tables' column names differed (#334 @joellabes)

Under the hood

  • Fix Markdown formatting for hub rendering (#336 @coapacetic)
  • Reorder readme and improve docs

dbt-utils v0.6.4

Fixes

  • Fix insert_by_period to support dbt v0.19.0, with backwards compatibility for earlier versions (#319, #320)

Under the hood

  • Speed up CI via threads, workflows (#315, #316)
  • Fix equality test when used with ephemeral models + explicit column set (#321)
  • Fix get_query_results_as_dict integration test with consistent ordering (#322)
  • All macros are now properly dispatched, making it possible for non-core adapters to implement a shim package for dbt-utils (#312) Thanks @chaerinlee1 and @swanderz
  • Small, non-breaking changes to accomodate TSQL (can't group by column number references, no real TRUE/FALSE values, aggregation CTEs need named columns) (#310) Thanks @swanderz
  • Make get_relations_by_pattern and get_relations_by_prefix more powerful by returning relation.type (#323)

dbt-utils v0.6.3

  • Bump require-dbt-version to [">=0.18.0", "<0.20.0"] to support dbt v0.19.0 (#308, #309)

dbt-utils v0.6.2

Fixes

  • Fix the logic in get_tables_by_pattern_sql to ensure non-default arguments are respected (#279)

dbt-utils v0.6.1

Fixes

  • Fix the logic in get_tables_by_pattern_sql for matching a schema pattern on BigQuery (#275)

dbt-utils v0.6.0

Breaking changes

  • 🚨 dbt v0.18.0 or greater is required for this release. If you are not ready to upgrade, consider using a previous release of this package
  • 🚨 The get_tables_by_prefix, union_tables and get_tables_by_pattern macros have been removed

Migration instructions

  • Upgrade your dbt project to v0.18.0 using these instructions.
  • Upgrade your packages.yml file to use version 0.6.0 of this package. Run dbt clean and dbt deps.
  • If your project uses the get_tables_by_prefix macro, replace it with get_relations_by_prefix. All arguments have retained the same name.
  • If your project uses the union_tables macro, replace it with union_relations. While the order of arguments has stayed consistent, the tables argument has been renamed to relations. Further, the default value for the source_column_name argument has changed from '_dbt_source_table' to '_dbt_source_relation' — you may want to explicitly define this argument to avoid breaking changes.
-- before:
{{ dbt_utils.union_tables(
    tables=[ref('my_model'), source('my_source', 'my_table')],
    exclude=["_loaded_at"]
) }}

-- after:
{{ dbt_utils.union_relations(
    relations=[ref('my_model'), source('my_source', 'my_table')],
    exclude=["_loaded_at"],
    source_column_name='_dbt_source_table'
) }}
  • If your project uses the get_tables_by_pattern macro, replace it with get_tables_by_pattern_sql — all arguments are consistent.

Features

  • Switch usage of adapter_macro to adapter.dispatch, and define dbt_utils_dispatch_list, enabling users of community-supported database plugins to add or override macro implementations specific to their database (#267)
  • Use add_ephemeral_prefix instead of hard-coding a string literal, to support database adapters that use different prefixes (#267)
  • Implement a quote_columns argument in the unique_combination_of_columns schema test (#270 @JoshuaHuntley)

Quality of life

  • Remove deprecated macros get_tables_by_prefix and union_tables (#268)
  • Remove get_tables_by_pattern macro, which is equivalent to the get_tables_by_pattern_sql macro (the latter has a more logical name) (#268)

dbt-utils v0.5.1

Quality of life