Merge pull request #6931 from dbt-labs/merge_feature_click_cli_main

Merge feature/click-cli into main

Co-authored-by: Ian Knox <[email protected] >
Co-authored-by: Stu Kilgore <[email protected]>
Co-authored-by: Michelle Ark <[email protected] >
Co-authored-by: Kshitij Aranke <[email protected] >
Co-authored-by: Jeremy Cohen <[email protected] >

.changes/1.4.0-b1.md

@@ -0,0 +1,89 @@
+## dbt-core 1.4.0-b1 - December 15, 2022
+
+### Features
+
+- Added favor-state flag to optionally favor state nodes even if unselected node exists ([#2968](https:/dbt-labs/dbt-core/issues/2968))
+- Update structured logging. Convert to using protobuf messages. Ensure events are enriched with node_info. ([#5610](https:/dbt-labs/dbt-core/issues/5610))
+- Friendlier error messages when packages.yml is malformed ([#5486](https:/dbt-labs/dbt-core/issues/5486))
+- Migrate dbt-utils current_timestamp macros into core + adapters ([#5521](https:/dbt-labs/dbt-core/issues/5521))
+- Allow partitions in external tables to be supplied as a list ([#5929](https:/dbt-labs/dbt-core/issues/5929))
+- extend -f flag shorthand for seed command ([#5990](https:/dbt-labs/dbt-core/issues/5990))
+- This pulls the profile name from args when constructing a RuntimeConfig in lib.py, enabling the dbt-server to override the value that's in the dbt_project.yml ([#6201](https:/dbt-labs/dbt-core/issues/6201))
+- Adding tarball install method for packages. Allowing package tarball to be specified via url in the packages.yaml. ([#4205](https:/dbt-labs/dbt-core/issues/4205))
+- Added an md5 function to the base context ([#6246](https:/dbt-labs/dbt-core/issues/6246))
+- Exposures support metrics in lineage ([#6057](https:/dbt-labs/dbt-core/issues/6057))
+- Add support for Python 3.11 ([#6147](https:/dbt-labs/dbt-core/issues/6147))
+- incremental predicates ([#5680](https:/dbt-labs/dbt-core/issues/5680))
+
+### Fixes
+
+- Account for disabled flags on models in schema files more completely ([#3992](https:/dbt-labs/dbt-core/issues/3992))
+- Add validation of enabled config for metrics, exposures and sources ([#6030](https:/dbt-labs/dbt-core/issues/6030))
+- check length of args of python model function before accessing it ([#6041](https:/dbt-labs/dbt-core/issues/6041))
+- Add functors to ensure event types with str-type attributes are initialized to spec, even when provided non-str type params. ([#5436](https:/dbt-labs/dbt-core/issues/5436))
+- Allow hooks to fail without halting execution flow ([#5625](https:/dbt-labs/dbt-core/issues/5625))
+- Clarify Error Message for how many models are allowed in a Python file ([#6245](https:/dbt-labs/dbt-core/issues/6245))
+- After this, will be possible to use default values for dbt.config.get ([#6309](https:/dbt-labs/dbt-core/issues/6309))
+- Use full path for writing manifest ([#6055](https:/dbt-labs/dbt-core/issues/6055))
+- [CT-1284] Change Python model default materialization to table ([#6345](https:/dbt-labs/dbt-core/issues/6345))
+- Repair a regression which prevented basic logging before the logging subsystem is completely configured. ([#6434](https:/dbt-labs/dbt-core/issues/6434))
+
+### Docs
+
+- minor doc correction ([dbt-docs/#5791](https:/dbt-labs/dbt-docs/issues/5791))
+- Generate API docs for new CLI interface ([dbt-docs/#5528](https:/dbt-labs/dbt-docs/issues/5528))
+- ([dbt-docs/#5880](https:/dbt-labs/dbt-docs/issues/5880))
+- Fix rendering of sample code for metrics ([dbt-docs/#323](https:/dbt-labs/dbt-docs/issues/323))
+- Alphabetize `core/dbt/README.md` ([dbt-docs/#6368](https:/dbt-labs/dbt-docs/issues/6368))
+
+### Under the Hood
+
+- Put black config in explicit config ([#5946](https:/dbt-labs/dbt-core/issues/5946))
+- Added flat_graph attribute the Manifest class's deepcopy() coverage ([#5809](https:/dbt-labs/dbt-core/issues/5809))
+- Add mypy configs so `mypy` passes from CLI ([#5983](https:/dbt-labs/dbt-core/issues/5983))
+- Exception message cleanup. ([#6023](https:/dbt-labs/dbt-core/issues/6023))
+- Add dmypy cache to gitignore ([#6028](https:/dbt-labs/dbt-core/issues/6028))
+- Provide useful errors when the value of 'materialized' is invalid ([#5229](https:/dbt-labs/dbt-core/issues/5229))
+- Clean up string formatting ([#6068](https:/dbt-labs/dbt-core/issues/6068))
+- Fixed extra whitespace in strings introduced by black. ([#1350](https:/dbt-labs/dbt-core/issues/1350))
+- Remove the 'root_path' field from most nodes ([#6171](https:/dbt-labs/dbt-core/issues/6171))
+- Combine certain logging events with different levels ([#6173](https:/dbt-labs/dbt-core/issues/6173))
+- Convert threading tests to pytest ([#5942](https:/dbt-labs/dbt-core/issues/5942))
+- Convert postgres index tests to pytest ([#5770](https:/dbt-labs/dbt-core/issues/5770))
+- Convert use color tests to pytest ([#5771](https:/dbt-labs/dbt-core/issues/5771))
+- Add github actions workflow to generate high level CLI API docs ([#5942](https:/dbt-labs/dbt-core/issues/5942))
+- Functionality-neutral refactor of event logging system to improve encapsulation and modularity. ([#6139](https:/dbt-labs/dbt-core/issues/6139))
+- Consolidate ParsedNode and CompiledNode classes ([#6383](https:/dbt-labs/dbt-core/issues/6383))
+- Prevent doc gen workflow from running on forks ([#6386](https:/dbt-labs/dbt-core/issues/6386))
+- Fix intermittent database connection failure in Windows CI test ([#6394](https:/dbt-labs/dbt-core/issues/6394))
+- Refactor and clean up manifest nodes ([#6426](https:/dbt-labs/dbt-core/issues/6426))
+- Restore important legacy logging behaviors, following refactor which removed them ([#6437](https:/dbt-labs/dbt-core/issues/6437))
+
+### Dependencies
+
+- Update pathspec requirement from ~=0.9.0 to >=0.9,<0.11 in /core ([#5917](https:/dbt-labs/dbt-core/pull/5917))
+- Bump black from 22.8.0 to 22.10.0 ([#6019](https:/dbt-labs/dbt-core/pull/6019))
+- Bump mashumaro[msgpack] from 3.0.4 to 3.1.1 in /core ([#6108](https:/dbt-labs/dbt-core/pull/6108))
+- Update colorama requirement from <0.4.6,>=0.3.9 to >=0.3.9,<0.4.7 in /core ([#6144](https:/dbt-labs/dbt-core/pull/6144))
+- Bump mashumaro[msgpack] from 3.1.1 to 3.2 in /core ([#4904](https:/dbt-labs/dbt-core/issues/4904))
+
+### Contributors
+- [@andy-clapson](https:/andy-clapson) ([dbt-docs/#5791](https:/dbt-labs/dbt-docs/issues/5791))
+- [@chamini2](https:/chamini2) ([#6041](https:/dbt-labs/dbt-core/issues/6041))
+- [@daniel-murray](https:/daniel-murray) ([#2968](https:/dbt-labs/dbt-core/issues/2968))
+- [@dave-connors-3](https:/dave-connors-3) ([#5990](https:/dbt-labs/dbt-core/issues/5990))
+- [@dbeatty10](https:/dbeatty10) ([dbt-docs/#6368](https:/dbt-labs/dbt-docs/issues/6368), [#6394](https:/dbt-labs/dbt-core/issues/6394))
+- [@devmessias](https:/devmessias) ([#6309](https:/dbt-labs/dbt-core/issues/6309))
+- [@eve-johns](https:/eve-johns) ([#6068](https:/dbt-labs/dbt-core/issues/6068))
+- [@haritamar](https:/haritamar) ([#6246](https:/dbt-labs/dbt-core/issues/6246))
+- [@jared-rimmer](https:/jared-rimmer) ([#5486](https:/dbt-labs/dbt-core/issues/5486))
+- [@josephberni](https:/josephberni) ([#2968](https:/dbt-labs/dbt-core/issues/2968))
+- [@joshuataylor](https:/joshuataylor) ([#6147](https:/dbt-labs/dbt-core/issues/6147))
+- [@justbldwn](https:/justbldwn) ([#6245](https:/dbt-labs/dbt-core/issues/6245))
+- [@luke-bassett](https:/luke-bassett) ([#1350](https:/dbt-labs/dbt-core/issues/1350))
+- [@max-sixty](https:/max-sixty) ([#5946](https:/dbt-labs/dbt-core/issues/5946), [#5983](https:/dbt-labs/dbt-core/issues/5983), [#6028](https:/dbt-labs/dbt-core/issues/6028))
+- [@paulbenschmidt](https:/paulbenschmidt) ([dbt-docs/#5880](https:/dbt-labs/dbt-docs/issues/5880))
+- [@pgoslatara](https:/pgoslatara) ([#5929](https:/dbt-labs/dbt-core/issues/5929))
+- [@racheldaniel](https:/racheldaniel) ([#6201](https:/dbt-labs/dbt-core/issues/6201))
+- [@timle2](https:/timle2) ([#4205](https:/dbt-labs/dbt-core/issues/4205))
+- [@dave-connors-3](https:/dave-connors-3) ([#5680](https:/dbt-labs/dbt-core/issues/5680))

.changes/1.4.0/Dependency-20220923-000646.yaml

@@ -0,0 +1,6 @@
+kind: "Dependencies"
+body: "Update pathspec requirement from ~=0.9.0 to >=0.9,<0.11 in /core"
+time: 2022-09-23T00:06:46.00000Z
+custom:
+ Author: dependabot[bot]
+ PR: "5917"

.changes/1.4.0/Dependency-20221007-000848.yaml

@@ -0,0 +1,6 @@
+kind: "Dependencies"
+body: "Bump black from 22.8.0 to 22.10.0"
+time: 2022-10-07T00:08:48.00000Z
+custom:
+ Author: dependabot[bot]
+ PR: "6019"

.changes/1.4.0/Dependency-20221020-000753.yaml

@@ -0,0 +1,6 @@
+kind: "Dependencies"
+body: "Bump mashumaro[msgpack] from 3.0.4 to 3.1.1 in /core"
+time: 2022-10-20T00:07:53.00000Z
+custom:
+ Author: dependabot[bot]
+ PR: "6108"

.changes/1.4.0/Dependency-20221026-000910.yaml

@@ -0,0 +1,6 @@
+kind: "Dependencies"
+body: "Update colorama requirement from <0.4.6,>=0.3.9 to >=0.3.9,<0.4.7 in /core"
+time: 2022-10-26T00:09:10.00000Z
+custom:
+ Author: dependabot[bot]
+ PR: "6144"

.changes/1.4.0/Dependency-20221205-002118.yaml

@@ -0,0 +1,7 @@
+kind: "Dependencies"
+body: "Bump mashumaro[msgpack] from 3.1.1 to 3.2 in /core"
+time: 2022-12-05T00:21:18.00000Z
+custom:
+ Author: dependabot[bot]
+ Issue: 4904
+ PR: 6375

.changes/1.4.0/Docs-20220908-154157.yaml

@@ -0,0 +1,6 @@
+kind: Docs
+body: minor doc correction
+time: 2022-09-08T15:41:57.689162-04:00
+custom:
+ Author: andy-clapson
+ Issue: "5791"

.changes/1.4.0/Docs-20221007-090656.yaml

@@ -0,0 +1,6 @@
+kind: Docs
+body: Generate API docs for new CLI interface
+time: 2022-10-07T09:06:56.446078-05:00
+custom:
+ Author: stu-k
+ Issue: "5528"

.changes/1.4.0/Docs-20221017-171411.yaml

@@ -0,0 +1,5 @@
+kind: Docs
+time: 2022-10-17T17:14:11.715348-05:00
+custom:
+ Author: paulbenschmidt
+ Issue: "5880"

.changes/1.4.0/Docs-20221116-155743.yaml

@@ -0,0 +1,6 @@
+kind: Docs
+body: Fix rendering of sample code for metrics
+time: 2022-11-16T15:57:43.204201+01:00
+custom:
+ Author: jtcohen6
+ Issue: "323"

.changes/1.4.0/Docs-20221202-150523.yaml

@@ -0,0 +1,6 @@
+kind: Docs
+body: Alphabetize `core/dbt/README.md`
+time: 2022-12-02T15:05:23.695333-07:00
+custom:
+ Author: dbeatty10
+ Issue: "6368"

.changes/1.4.0/Features-20220408-165459.yaml

@@ -0,0 +1,7 @@
+kind: Features
+body: Added favor-state flag to optionally favor state nodes even if unselected node
+ exists
+time: 2022-04-08T16:54:59.696564+01:00
+custom:
+ Author: daniel-murray josephberni
+ Issue: "2968"

.changes/1.4.0/Features-20220817-154857.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Update structured logging. Convert to using protobuf messages. Ensure events are enriched with node_info.
+time: 2022-08-17T15:48:57.225267-04:00
+custom:
+ Author: gshank
+ Issue: "5610"

.changes/1.4.0/Features-20220823-085727.yaml

@@ -0,0 +1,7 @@
+kind: Features
+body: incremental predicates
+time: 2022-08-23T08:57:27.640804-05:00
+custom:
+ Author: dave-connors-3
+ Issue: "5680"
+ PR: "5702"

.changes/1.4.0/Features-20220912-125935.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Friendlier error messages when packages.yml is malformed
+time: 2022-09-12T12:59:35.121188+01:00
+custom:
+ Author: jared-rimmer
+ Issue: "5486"

.changes/1.4.0/Features-20220914-095625.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Migrate dbt-utils current_timestamp macros into core + adapters
+time: 2022-09-14T09:56:25.97818-07:00
+custom:
+ Author: colin-rogers-dbt
+ Issue: "5521"

.changes/1.4.0/Features-20220925-211651.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Allow partitions in external tables to be supplied as a list
+time: 2022-09-25T21:16:51.051239654+02:00
+custom:
+ Author: pgoslatara
+ Issue: "5929"

.changes/1.4.0/Features-20221003-110705.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: extend -f flag shorthand for seed command
+time: 2022-10-03T11:07:05.381632-05:00
+custom:
+ Author: dave-connors-3
+ Issue: "5990"

.changes/1.4.0/Features-20221102-150003.yaml

@@ -0,0 +1,7 @@
+kind: Features
+body: This pulls the profile name from args when constructing a RuntimeConfig in lib.py,
+ enabling the dbt-server to override the value that's in the dbt_project.yml
+time: 2022-11-02T15:00:03.000805-05:00
+custom:
+ Author: racheldaniel
+ Issue: "6201"

.changes/1.4.0/Features-20221107-105018.yaml

@@ -0,0 +1,8 @@
+kind: Features
+body: Adding tarball install method for packages. Allowing package tarball to be specified
+ via url in the packages.yaml.
+time: 2022-11-07T10:50:18.464545-05:00
+custom:
+ Author: timle2
+ Issue: "4205"
+ PR: "4689"

.changes/1.4.0/Features-20221114-185207.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Added an md5 function to the base context
+time: 2022-11-14T18:52:07.788593+02:00
+custom:
+ Author: haritamar
+ Issue: "6246"

.changes/1.4.0/Features-20221130-112913.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Exposures support metrics in lineage
+time: 2022-11-30T11:29:13.256034-05:00
+custom:
+ Author: michelleark
+ Issue: "6057"

.changes/1.4.0/Features-20221206-150704.yaml

@@ -0,0 +1,7 @@
+kind: Features
+body: Add support for Python 3.11
+time: 2022-12-06T15:07:04.753127+01:00
+custom:
+ Author: joshuataylor MichelleArk jtcohen6
+ Issue: "6147"
+ PR: "6326"

.changes/1.4.0/Fixes-20220916-104854.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Account for disabled flags on models in schema files more completely
+time: 2022-09-16T10:48:54.162273-05:00
+custom:
+ Author: emmyoop
+ Issue: "3992"

.changes/1.4.0/Fixes-20221010-113218.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Add validation of enabled config for metrics, exposures and sources
+time: 2022-10-10T11:32:18.752322-05:00
+custom:
+ Author: emmyoop
+ Issue: "6030"

.changes/1.4.0/Fixes-20221011-160715.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: check length of args of python model function before accessing it
+time: 2022-10-11T16:07:15.464093-04:00
+custom:
+ Author: chamini2
+ Issue: "6041"

.changes/1.4.0/Fixes-20221016-173742.yaml

@@ -0,0 +1,7 @@
+kind: Fixes
+body: Add functors to ensure event types with str-type attributes are initialized
+ to spec, even when provided non-str type params.
+time: 2022-10-16T17:37:42.846683-07:00
+custom:
+ Author: versusfacit
+ Issue: "5436"

.changes/1.4.0/Fixes-20221107-095314.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Allow hooks to fail without halting execution flow
+time: 2022-11-07T09:53:14.340257-06:00
+custom:
+ Author: ChenyuLInx
+ Issue: "5625"

.changes/1.4.0/Fixes-20221115-081021.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Clarify Error Message for how many models are allowed in a Python file
+time: 2022-11-15T08:10:21.527884-05:00
+custom:
+ Author: justbldwn
+ Issue: "6245"

.changes/1.4.0/Fixes-20221124-163419.yaml

@@ -0,0 +1,7 @@
+kind: Fixes
+body: After this, will be possible to use default values for dbt.config.get
+time: 2022-11-24T16:34:19.039512764-03:00
+custom:
+ Author: devmessias
+ Issue: "6309"
+ PR: "6317"

.changes/1.4.0/Fixes-20221202-164859.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Use full path for writing manifest
+time: 2022-12-02T16:48:59.029519-05:00
+custom:
+ Author: gshank
+ Issue: "6055"

.changes/1.4.0/Fixes-20221213-112620.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: '[CT-1284] Change Python model default materialization to table'
+time: 2022-12-13T11:26:20.550017-08:00
+custom:
+ Author: aranke
+ Issue: "6345"

.changes/1.4.0/Fixes-20221214-155307.yaml

@@ -0,0 +1,7 @@
+kind: Fixes
+body: Repair a regression which prevented basic logging before the logging subsystem
+ is completely configured.
+time: 2022-12-14T15:53:07.396512-05:00
+custom:
+ Author: peterallenwebb
+ Issue: "6434"

.changes/1.4.0/Under the Hood-20220927-194259.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Put black config in explicit config
+time: 2022-09-27T19:42:59.241433-07:00
+custom:
+ Author: max-sixty
+ Issue: "5946"

.changes/1.4.0/Under the Hood-20220929-134406.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Added flat_graph attribute the Manifest class's deepcopy() coverage
+time: 2022-09-29T13:44:06.275941-04:00
+custom:
+ Author: peterallenwebb
+ Issue: "5809"

.changes/1.4.0/Under the Hood-20221005-120310.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Add mypy configs so `mypy` passes from CLI
+time: 2022-10-05T12:03:10.061263-07:00
+custom:
+ Author: max-sixty
+ Issue: "5983"

.changes/1.4.0/Under the Hood-20221007-094627.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Exception message cleanup.
+time: 2022-10-07T09:46:27.682872-05:00
+custom:
+ Author: emmyoop
+ Issue: "6023"

.changes/1.4.0/Under the Hood-20221007-140044.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Add dmypy cache to gitignore
+time: 2022-10-07T14:00:44.227644-07:00
+custom:
+ Author: max-sixty
+ Issue: "6028"

.changes/1.4.0/Under the Hood-20221013-181912.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Provide useful errors when the value of 'materialized' is invalid
+time: 2022-10-13T18:19:12.167548-04:00
+custom:
+ Author: peterallenwebb
+ Issue: "5229"

.changes/1.4.0/Under the Hood-20221017-151511.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Fixed extra whitespace in strings introduced by black.
+time: 2022-10-17T15:15:11.499246-05:00
+custom:
+ Author: luke-bassett
+ Issue: "1350"

.changes/1.4.0/Under the Hood-20221017-155844.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Clean up string formatting
+time: 2022-10-17T15:58:44.676549-04:00
+custom:
+ Author: eve-johns
+ Issue: "6068"

.changes/1.4.0/Under the Hood-20221028-104837.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Remove the 'root_path' field from most nodes
+time: 2022-10-28T10:48:37.687886-04:00
+custom:
+ Author: gshank
+ Issue: "6171"

.changes/1.4.0/Under the Hood-20221028-110344.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Combine certain logging events with different levels
+time: 2022-10-28T11:03:44.887836-04:00
+custom:
+ Author: gshank
+ Issue: "6173"

.changes/1.4.0/Under the Hood-20221108-074550.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Convert threading tests to pytest
+time: 2022-11-08T07:45:50.589147-06:00
+custom:
+ Author: stu-k
+ Issue: "5942"