Add slugify macro to utils, use in pivot macro (#314)

CHANGELOG.md

@@ -70,6 +70,7 @@ If you were relying on the position to match up your optional arguments, this ma
 - Bump `require-dbt-version` to `[">=0.18.0", "<0.20.0"]` to support dbt v0.19.0 ([#308](https:/fishtown-analytics/dbt-utils/pull/308), [#309](https:/fishtown-analytics/dbt-utils/pull/309))
 
 ## Fixes
+* Add `slugify` macro, and use it in the pivot macro. :rotating_light: This macro uses the `re` module, which is only available in dbt v0.19.0+. As a result, this feature introduces a breaking change. ([#314](https:/fishtown-analytics/dbt-utils/pull/314))
 
 # dbt-utils v0.6.2
 

README.md

@@ -960,8 +960,8 @@ Usage:
 ```
 
 ---
-### Logger
-#### pretty_time ([source](macros/logger/pretty_time.sql))
+### Jinja Helpers
+#### pretty_time ([source](macros/jinja_helpers/pretty_time.sql))
 This macro returns a string of the current timestamp, optionally taking a datestring format.
 ```sql
 {#- This will return a string like '14:50:34' -#}
@@ -971,7 +971,7 @@ This macro returns a string of the current timestamp, optionally taking a datest
 {{ dbt_utils.pretty_time(format='%Y-%m-%d %H:%M:%S') }}
 ```
 
-#### pretty_log_format ([source](macros/logger/pretty_log_format.sql))
+#### pretty_log_format ([source](macros/jinja_helpers/pretty_log_format.sql))
 This macro formats the input in a way that will print nicely to the command line when you `log` it.
 ```sql
 {#- This will return a string like:
@@ -980,7 +980,7 @@ This macro formats the input in a way that will print nicely to the command line
 
 {{ dbt_utils.pretty_log_format("my pretty message") }}
 ```
-#### log_info ([source](macros/logger/log_info.sql))
+#### log_info ([source](macros/jinja_helpers/log_info.sql))
 This macro logs a formatted message (with a timestamp) to the command line.
 ```sql
 {{ dbt_utils.log_info("my pretty message") }}
@@ -991,6 +991,40 @@ This macro logs a formatted message (with a timestamp) to the command line.
 11:07:31 + my pretty message
 ```
 
+#### slugify ([source](macros/jinja_helpers/slugify.sql))
+This macro is useful for transforming Jinja strings into "slugs", and can be useful when using a Jinja object as a column name, especially when that Jinja object is not hardcoded.
+
+For this example, let's pretend that we have payment methods in our payments table like `['venmo App', 'ca$h-money']`, which we can't use as a column name due to the spaces and special characters. This macro does its best to strip those out in a sensible way: `['venmo_app',
+'cah_money']`.
+
+```sql
+{%- set payment_methods = dbt_utils.get_column_values(
+ table=ref('raw_payments'),
+ column='payment_method'
+) -%}
+
+select
+order_id,
+{%- for payment_method in payment_methods %}
+sum(case when payment_method = '{{ payment_method }}' then amount end)
+ as {{ slugify(payment_method) }}_amount,
+
+{% endfor %}
+...
+```
+
+```sql
+select
+order_id,
+
+sum(case when payment_method = 'Venmo App' then amount end)
+ as venmo_app_amount,
+
+sum(case when payment_method = 'ca$h money' then amount end)
+ as cah_money_amount,
+...
+```
+
 ### Materializations
 #### insert_by_period ([source](macros/materializations/insert_by_period_materialization.sql))
 `insert_by_period` allows dbt to insert records into a table one period (i.e. day, week) at a time.

dbt_project.yml

@@ -1,7 +1,8 @@
 name: 'dbt_utils'
 version: '0.1.0'
 
-require-dbt-version: [">=0.18.0", "<0.20.0"]
+require-dbt-version: [">=0.19.0", "<0.20.0"]
+
 config-version: 2
 
 target-path: "target"

integration_tests/tests/jinja_helpers/test_slugify.sql

@@ -0,0 +1,7 @@
+{% if dbt_utils.slugify('!Hell0 world-hi') == 'hell0_world_hi' %}
+ {# Return 0 rows for the test to pass #}
+ select 1 limit 0
+{% else %}
+ {# Return >0 rows for the test to fail #}
+ select 1
+{% endif %}

macros/jinja_helpers/slugify.sql

@@ -0,0 +1,12 @@
+{% macro slugify(string) %}
+
+{#- Lower case the string -#}
+{% set string = string | lower %}
+{#- Replace spaces and dashes with underscores -#}
+{% set string = modules.re.sub('[ -]+', '_', string) %}
+{#- Only take letters, numbers, and underscores -#}
+{% set string = modules.re.sub('[^a-z0-9_]+', '', string) %}
+
+{{ return(string) }}
+
+{% endmacro %}

macros/sql/pivot.sql

@@ -78,7 +78,7 @@ Arguments:
  {% if quote_identifiers %}
  as {{ adapter.quote(prefix ~ v ~ suffix) }}
  {% else %}
- as {{prefix ~ v ~ suffix }}
+ as ({{ dbt_utils.slugify(prefix ~ v ~ suffix) }})
  {% endif %}
  {% endif %}
  {% if not loop.last %},{% endif %}