Workspace support plan stage 1

Proposed first stage support for workspaces

Expectations for the first stage design

• PuppetDB should only expect to need handle up to about 3000 nodes (total) and up to 25 workspaces (total) when workspaces are in use. It might well be able to handle more, but that’s not a requirement.
• Once a workspace has been mentioned to PuppetDB, it will never be forgotten, even if all of its members are removed.
• Workspace lifetimes and workspace memberships are completely independent of node lifetimes, i.e. workspaces may refer to certnames that PuppetDB knows nothing about.
• PuppetDB 6 does not need to support workspaces (so we’re planning to only add it to 7 (current main branch)).
• There is no current need to be able to sync workspaces, so we may or may not include that support in this initial work.
• There is no current need for a separate /pdb/query/v4/workspaces endpoint.
• There is no initial requirement to add workspaces to PuppetDB exports and imports.

Declaring workspace memberships

We propose to add a new configure workspaces command that only allows the specification of workspaces by providing complete lists of their members:

{"producer_timestamp": "2021-10-13T17:56:24-05:00",
 "workspace":
  {"3a573c12-22f6-11ec-8477-eb2e86ffd2e5": ["cert-1", "cert-2", ...],
   "5490e984-22f6-11ec-81b7-f75939532fff": ["cert-1", "cert-3", ...]}}

Future additional commands, or adjustments to this command may provide additional flexibility.

Queries

Restricting queries to a set of workspaces

Incoming queries will be allowed to restrict restrict the results of the query to information associated with a specific set of workspaces. They will do so via a new query key (name subject to change):

{"query": ["from", "nodes"],
 "in-workspaces": ["5490e984-22f6-11ec-81b7-f75939532fff" ...]}

If it ends up being a feasible and preferable approach, we may add workspace support to AST, in which case, the in-workspaces specification may just be used to augmenting the query and subquery filters in the incoming AST.

Querying for workspace information (not currently required)

If sync or export become requirements, we will add support for queries for workspace-related information. At a minimum, we would expect to add a /pdb/query/v4/workspaces endpoint providing the following fields in its response maps:

workspace_id (string)

UUID naming the workspace

last_update (timestamp)

last time the information was updated

certnames (vector of strings)

workspace members

For the initial implementation, filtering support might be limited to what is needed by the query restrictions described above.

Archiving (not currently required)

If this becomes a requirement we will adjust the archive endpoint to include workspaces in exports and imports. This will also require a /pdb/query/v4/workspaces endpoint (see above).

Sync support (not currently required)

If this becomes a requirement we will adjust PuppetDB sync to sync workspace information. This will depend on the configure workspaces command, and a /pdb/query/v4/workspaces endpoint (see above).

Data storage

This section is just advisory, and the internal details described are subject to change.

We may investigate a few possibilities during development, but since the initial capacity has been limited to around 3000 nodes, finding an acceptable arrangement should be less difficult.

We need to record enough information to maintain the constraints required by sync, which will likely mean per-workspace or per-certname-workspace-membership timestamps.

Requiring the incoming workspace membership data to be comprehensive for now (as the current configure workspaces command does) may allow for more efficient storage with respect to both size and ingestion/query performance and sync.

Currently, we’re leaning in favor of a “membership table” approach like this:

workspaces

workspace-id	updated
355ac85a-…	2021-…

workspace_members

workspace-id	certname
355ac85a-…	host-1
355ac85a-…	host-2
00a1dd64	host-2