Feature request: Support packaging resource files with workflow

[jasonterando]

Describe your idea/feature/enhancement

Maybe there's a way to already do this, but I can't figure out it out...

When building and packaging a Lambda function, provide the capability to package resource files with the package to deploy. My need is for esbuild, but this may be useful for other workflows as well.

For example, in Metadata, add an optional property to Include files, which would be a list of globs to include in the output. Directories/files would be copied in with relative paths (i.e. copying "./my-resources/*.res" would create "{output dir}/my-resources/foo1.res", "{output dir}/my-resources/foo2.res", etc.

  MyLambda:
    Type: AWS::Serverless::Function
    Metadata:
      Include:
         - ./my-resources/*.res
      BuildMethod: esbuild
      BuildProperties:
        Minify: true
        Target: "es2017"
        Sourcemap: false
        EntryPoints:
          - ./src/index.ts
        UseNpmCi: true     

Proposal

1. Add new Purpose - COPY_RESOURCES
2. In NodejsNpmEsbuildWorkflow.Init, check for kwargs.options.include
3. If kwargs.options.include is defined, add an action to list of self.actions, after subprocess_esbuild, copy specified directories/files to the artifacts director

Additional Details

May require creation of OSUtils.glob wrapper. If that's problematic, we could stick with specific directory/file names

I'm happy to do the initial lift on this if the concept sounds viable.

[torresxb1]

Hi @jasonterando , thanks for bringing this to our attention. Is your feature request asking for the same/similar thing to this other one: #185? Asking so that we can track all in one place (we could close this one, and you could add a comment on the other one)

[jasonterando]

I'm not sure... In my case, there are files that I explicitly want to include in the bundled output that aren't part of the build. With issue 185, I think they are more interested in filtering what's included or not. With something like esbuild, where you're creating a "built" (minified) file, the source directory and its contents don't get copied anyways.

[sriram-mv]

Consistency across INCLUDE_FILES and EXCLUDED_FILES is something we need to have. Especially since aws-lambda-builders could be used standalone as well as library via aws-sam-cli.

Should this start as a design PR on the aws-lambda-builders repo with a specific emphasis on the interface?

[jasonterando]

Hard to say for sure. I know when I am doing .NET Core, for example, I can include files for publish with entries in the .csproj project file. No such mechanism exists for npm or npm-esbuild (other than custom scripts and 3rd party packages).

I suspect you can include files during publish with Maven/Gradle, but don't know for sure. I have no experience with Go. This might be something useful for Python though.

For what it's worth, to unblock my project, I've had to move on from using SAM's integrated esbuild, and resort to manually triggering esbuild using NPM scripts. Thus, if you need to reject this idea in its entirety, it won't be the end of the world. If we can make this work, I can look into using the integrated solution again.

[jasonterando]

One other thing randomly came to mind this morning. When it comes to Include/Exclude functionality, we may want to differentiate between file inclusion/exclusion during build versus packaging (for deployment).

[pkit]

Especially since aws-lambda-builders could be used standalone as well as library via aws-sam-cli.

I'm not sure I follow, is there any documentation how lambda-builders can be run verbatim from aws-sam-cli ?

[mildaniel]

Created a separate issue to consolidate the discussion for include/exclude functionality #516.