Add a Function to Allow Easy Hyperlinking Between Model Documentation

[jrandrews]

Function to Allow Easy Hyperlinking Between Model Documentation

Feature description

Right now, if you are writing a doc block for a model and want to embed a hyperlink to a different doc block, it is necessary to do syntax like this: [link text]link title. This is a bit clunky and difficult to follow. It would be nice to have a function like "link(model_name)" to make it easier to embed hyperlinks to other models in documentation.

Who will this benefit?

All users who want to be able to construct links between various documentation pages for models when using the "dbt docs" functionality.

[drewbanin]

Really great idea! I think this should be relatively straightforward for someone to add to dbt -- going to add a Good first issue label here. Happy to help out anyone who wants to take a stab at it!

[jrandrews]

In addition, it would be nice to have the ability to have SQL queries executed and have their output actually appear in dbt docs. One use case could be including a line of documentation that says "The earliest record in X table has a timestamp of XXXXXXX" and have the value of the timestamp populated at runtime. Or have an aggregate query execute to count the number of transaction values by a certain lookup value, e.g. cars by color, and then have a table get populated in the documentation output to actually display what this looks like in the database.

[drewbanin]

Tentatively pulling this issue in the wilt-chamberlain release, pending questions about what sort of dbt context we can provide in the docs blocks. See also: #1255

[drewbanin]

Kicking this one out of Wilt Chamberlain, but excited to tackle it for a subsequent release! Just prioritized for Louisa May Alcott, which should be the first feature release after 0.14.0 (so, either 0.14.x, or 0.15.0, depending)

[cmcarthur]

related to #1503

[eugene-nikolaev]

Hi! Keen to get model references in docs!
What is the current status of this request? Will it be implemented?

[jtcohen6]

Hey @eugene-nikolaev, this isn't prioritized right now, but it's good to hear this is something you're interested in!

All resources in the dbt-docs site today have URLs like:

[base url]/#!/[resource_type]/[resource_type].[package_name].[resource_name]

So if you want to link to a specific model in a Markdown description, it can be as simple as:

- name: model
   description: >
     Check out this [other_model](/#!/model/model.my_project.other_model)
     Or this [source](/#!/source/model.my_project.my_source.src_table)
     Or this [macro from a package](/#!/macro/macro.dbt_utils.date_spine)

It is possible to render Jinja templates in the description fields, but dbt doesn't have access to its full custom Jinja context today: it's just the base context plus doc(). The proposal in this issue is to add another special function that provides shorthand for the link construction above—perhaps a link() function, in lieu of trying to incorporate the ref() or source() macros themselves. I think that's feasible as a community contribution, with two small challenges:

• It will require following some dbt-Jinja code paths around (e.g.) DocumentationExtension and get_docs_macro_name
• The link function would need to introspect the parsed manifest to figure resource type and package name

[eugene-nikolaev]

@jtcohen6 thanks for reply!

[superfinn23]

Would love to have this function in the future! Glad to have the href though!

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

[ConstantinoSchillebeeckx]

What's the status on this? As someone who makes heavy use of docs, I'd love to see this feature. How can I help?

[keunsoopark]

Waiting for this being released!!

[jtcohen6]

Our team had a chance to discuss this in more detail yesterday. The idea of writing something like:

# resources.yml
models:
  - name: one_model
    description: "{{ link_to_ref('another_model') }}"

And having it render as a Markdown relative link ([another_model](/#!/model/model.my_project.another_model)), for use in the dbt-docs site, is definitely compelling.

I'm not sure if that relative link makes much sense in other contexts where descriptions are used, though:

• database comments, if persist_docs is enabled — what does "hyperlinking" to another model look like in the context of the database? Returning its relation name (database.schema.identifier) instead?
• dbt Cloud's metadata API, which can provide each model's description to external tools. It might be weird to see relative links to the docs site, but I suppose it could be combined with a dbt Cloud base URL for the account / job to link to that model's specific page of documentation!

If we put those questions aside for the moment, and just solve for the dbt-docs site as it exists currently: I think the actual implementation of this could be quite straightforward. It differs slightly from the approach outlined I above (over a year ago). The relevant changes would actually be needed in DocsRuntimeContext

    @contextmember
    # should this just be called ref()?
    def link_to_ref(self, *args: str) -> str:
        # return Markdown for a relative URL to the other resource's page
        if len(args) == 1:
            target_model_package = None
            target_model_name = args[0]
        elif len(args) == 2:
            target_model_package, target_model_name = args
        else:
            raise dbt.exceptions.InternalException(
                f"Refs should always be 1 or 2 arguments - got {len(ref)}"
            )

        target_model = self.manifest.resolve_ref(
            target_model_name,
            target_model_package,
            self._project_name,
            self.node.package_name,
        )
        
        name = target_model.name
        unique_id = target_model.unique_id
        resource_type = target_model.resource_type
        
        return f"[{name}](/#!/{resource_type}/{unique_id})"
        
    @contextmember
    def link_to_source(self, *args: str) -> str:
        ... similar idea ...

Then:

# resources.yml
version: 2
models:
  - name: one_model
    description: "{{ link_to_ref('another_model') }}"

@gshank Could you advise on the performance implications of an approach like that? I know we've discussed in the past that providing macros to the context for description field rendering would be quite slow (#3277, #3827). The whole manifest is already loaded and available, as DocsRuntimeContext.manifest, but I guess because the full MacroManifest isn't available for the rendering context...?

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest; add a comment to notify the maintainers.

[olsavmic]

I'm sad to see this closed, any possibility to revive this idea? Even the official docs says current solution with markdown href is quite hacky

https://docs.getdbt.com/reference/resource-properties/description#link-to-another-model-in-a-description