Allow references in documentation

[jdreaver]

Feature

I've only been using dbt for a day, and it is fantastic!

I searched the documentation and existing issues to see if this feature already exists, but I couldn't find it. If something like this already exists, my apologies, and please close this issue!

Feature description

When documenting a model, I'd like to be able to reference other dbt models. For example:

version: 2

models:
  - name: events
    description: A table containing clickstream events from the marketing website

    columns:
        - name: user_id
          description: The ID of the user who recorded the event. Points to the {{ ref('users') }} table.

Note the sentence Points to the {{ ref('users') }} table. Ideally, this would provide two benefits:

1. This should produce a hyperlink in the documentation from dbt docs generate.
2. This acts as a check against referencing a model that has been renamed or removed. If a model is renamed/removed, then ideally these references fail so you know where to update your documentation.

I'm not sure calling this function ref() is the best idea. We might want doc_ref() that allows an argument for the model name like ref, but also maybe another argument like a column name:

Points to the {{ doc_ref('users', 'id') }} column in the {{ doc_ref('users') }} table.

Relation to tests

Clearly you can express this relationship via a test:

tests:
    - relationships:
        to: ref('users')
        field: id

A test isn't quite sufficient for two reasons:

1. A hyperlink isn't produced in the documentation for tests. (This one could be fixed, probably.)
2. Sometimes a table points to another table, but there might be e.g. missing users, so the test would fail. That is, there might be some events with a user_id but no corresponding users.id. This is not uncommon in the data warehousing world when pulling/combining data from different sources.

Who will this benefit?

Anyone who uses dbt to document their models would benefit from this. It would make dbt's generated documentation more useful by adding hyperlinks to other models inside documentation and by making sure modifying a model doesn't break any upstream hyperlinks to it.

[drewbanin]

Hey @jdreaver - thanks for the feature request! We're already tracking ref hyperlinks in the docs context here: #1334

I think @beckjake or @cmcarthur suggested using href, which I really like :)

You've also mentioned the relationships test here -- can you say more about that? We received a report that Relationships tests weren't rendering as hyperlinks recently, but I wasn't able to reproduce that error. When a relationships test is defined, the column should link back to the parent model in the documentation website:

Are you seeing something different?

[jdreaver]

Thanks for the response @drewbanin!

Ah I must have totally messed up my original test for the foreign key hyperlink feature. Womp womp.

I'll go ahead and follow #1334. Thanks for the heads up!