Add a get-manifest API method (#2168)

[beckjake]

resolves #2168

Description

When a user calls 'get-manifest', compile all the nodes in the project and return the compiled manifest from memory.

get-manifest takes no arguments.
'get-manifest' returns an async ID, just like the existing compile/run/etc. The final result will be an object with logs and manifest attributes.

See #2169 for the PR about slimming this down.

Checklist

• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have updated the CHANGELOG.md and added information about my change to the "dbt next" section.

[drewbanin]

@beckjake nice, this looks great!

One question: can we make this method block instead of running async? Here's what I'm thinking:

We want to use this method to render the dbt docs site. If this method is async, then we need to:

1. call the method
2. poll for a response
3. write the response out to a file
4. serve the index.html file for the docs site

If the method was blocking, then we could just proxy a request to get manifest.json to actually hit this endpoint on the dbt server.

I recognize that this method might be really slow for projects that contain a bunch of introspective queries. The thing I'm trying to figure out is: should the compilation be an async background process, or should the compilation happen directly when someone wants to view the docs represented in the state of a dbt server.

Curious to hear what you think! cc @cmcarthur

[beckjake]

One question: can we make this method block instead of running async? Here's what I'm thinking:

We can, but I really don't recommend it. The process takes several seconds, probably minutes on huge projects. Web requests that block for minutes tend to get cut off by various timeout mechanisms along the network path and result in inscrutable network-y failures. If we want to return the manifest directly, we probably have to compile projects in the background on sighup, or something.

[drewbanin]

Ok - I buy that - this is a good starting point and we can iterate on the exact interface in the future if need be

[drewbanin]

LGTM - let's ship it :)