Feature: reports

[beckjake]

resolves #2730

Description

Add report nodes to dbt.

They participate in the graph (as a "sink": not ref-able, but can use ref/source). That means they also honor selector behavior, which means things like dbt ls --select +my_report behave, dbt run --models '+state:modified' runs all ancestors of a modified report, etc.

I made two changes unrelated to this PR:

• bumped the dbt version to 0.18.1a1, as it was getting a bit confusing to run locally/switch branches around
• I moved the RPC tests to a folder starting with 100 - this is a quality of life change to make it easier to run all non-RPC integration tests locally. I removed some unused folders there, too.

Checklist

• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have updated the CHANGELOG.md and added information about my change to the "dbt next" section.

[beckjake]

I'm not sure why it's not showing up here, but this is the azure pipelines build for e96f4a5

[jtcohen6]

This looks great! I messed around with some reports in our internal analytics project, there's a draft PR open there if you're curious.

Neither of these is a blocker for this PR, just things to discuss/modify between now and 0.18.0-rc1:

node selection syntax

Because reports are like inverse sources, I had been thinking about a report: node selector. This would get us a few small things:

• Explicitness. A report can have the same name as a model (which feels ok), so we'd disambiguate between dbt run -m +report:customers and dbt run -m +customers.
• List all resources upstream of all reports: dbt ls -s +report:*. By the same token, list all resources that are not exposed to any reports: dbt ls --exclude +report:*.

dbt-docs

Open issue: dbt-labs/dbt-docs#135

We just need to fix before we can release. They still work fine for projects that don't have reports defined.

[beckjake]

@jtcohen6 does that mean that reports shouldn't act like nodes in FQN selection?

For example, sources can be selected with source: but they don't show up in the fqn/default selector. I think that would be good for reports, it'd remove some ambiguity that currently exists where a report with the same name as a model.

[jtcohen6]

@beckjake That feels right to me. dbt run -m +ambiguous_name should default to the model, and excluding the same-named report has no functional difference since it isn't run. Is that a big change?

[beckjake]

I think it just involves tweaking the FQN selector to not look at reports and adding a selector that's a lot like source:, but for reports.

[beckjake]

That branch I just pushed is not quite right! Still need to handle package.report_name format, and need to fix report/source to output the correct selectors. Currently the dbt ls selector output is sort of wrong!