Add FileSet Resource

[bobcatfish]

Expected Behavior

Tasks that need to operate on files should be able to depend on a generic FileSet resource so that they can operate on Files without needing to know if the files came from git, from a configmap, from a hardcoded string, whatever.

Tasks that do need extra information, e.g. a git commit, should be able to still rely on more specifically typed PipelienResources

Proposal

To create one

You can supply one of:

• rawData (string representing one file)
• configMapRef (just use all the files in a configmap)
• ResourceRef
  • pointing to a Git resource or a Storage resource (or maybe other types later)
  • And a set of glob strings (to select which files)

Other ideas for "sources":

• Container Image + file globs (we would extract them from an image in a registry)
• URL (download the contents at the URL)
• maybe more!

When passed as an Input to a Task

The files to operate on will be under:
/workspace/$resourcename/
in an arbitrary directory structure.
Only the files the task needs to operate on are present in this structure.
The task should operate on everything in this directory recursively.

The task will also receieve an environment variable named: $SOMETHING (also available for interpolation)
containing a list of filenames, separated by spaces.

The goal is for users to be able to write tasks with a step like:

kubectl apply -f ${something} and have that work correctly on a list of filenames.

When produced as an Output of a Task

The task is responsible for filling a directory at:
/workspace/outputs/$resourcename
with the files.

Actual Behavior

Today in Tekton, tasks that consume files must operate on a typed Input resource, usually of Type Git or Storage.
This leads to a few problems:

• Tasks must choose which to support, when both should be equivalent
• Pipelines of Tasks that operate on specific files must invent out-of-band mechanisms to specify which files to operate on

Ideally, Tasks that operate on Files would have a more generic type of resource that can be populated from various sources, and Tasks should not need to communicate about file paths or names out-of-band.

Steps to Reproduce the Problem

For example, the golang build Task in the catalog depends on a git input:

https:/tektoncd/catalog/blob/a849fc273516b166a8c3193eab7a21f6cbd247c5/golang/build.yaml#L28-L31

    resources:
    - name: source
      type: git
      targetPath: src/${inputs.params.package}

This means that if a user wanted to use this Task on files that came from somewhere else, they'd need to build a new Task.

Additional Info

Written by @dlorenc in https://gist.github.com/dlorenc/58f6dae545ad4a7d0b3e8f7f508ee490

Next steps:

• Create a design doc with examples

[vtereso]

A PipelineResource referencing another PipelineResource is somewhat concerning, but otherwise I think this is great.

[dibyom]

One use case: Pass entire event bodies from Triggers to a Pipeline

[bobcatfish]

Now that we have workspaces I think this is no longer relevant! + @sbwsg