.net core - sam build command fails when private feeds in Nuget.Config

[fvilches17]

Description

Unable to use commands like sam build when working with .net core templates.

The sam cli uses dotnet global tools(s).

E.g. from debug logs (i.e. when running sam build --debug):
executing dotnet: ['dotnet.exe', 'tool', 'install', '-g', 'Amazon.Lambda.Tools']

When executing these dotnet commands, the dotnet cli will perform a dotnet restore.

dotnet restore scans the user's machine global Nuget.Config file in order to get feed sources. *On Windows this config file is located in C:\Users<user>\AppData\Roaming\NuGet

If the Nuget.Config file contains private feed sources (e.g. Azure Artifacts) this will generate an Http 401 Unauthorized error, which in turn generates an unrecoverable exception.

As a result, when running the command sam build , we get an error which includes this message

executing dotnet: ['dotnet.exe', 'tool', 'update', '-g', 'Amazon.Lambda.Tools']
C:\Program Files\dotnet\sdk\3.1.201\NuGet.targets(124,5): error : Unable to load the service index for source

To solve this, Microsoft introduced the flag --ignore-failed-sources. E.g. dotnet tool install "Amazon.Lambda.Tools" -g --ignore-failed-sources

This bug is pretty much a request to add this flag when the sam cli uses the dotnet cli under the hood for the sam build method (and any others applicable ones)

Steps to reproduce

1. Add a private feed. Easiest way to do is through Visual Studio (not vscode). Tools > Nuget Package Manager > Package Manager Settings > Package Sources > Add

2. Make sure you have the Sam cli installed. https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install-windows.html

3. In command prompt or powershell, run the following command: sam init

4. Choose option 1 (AWS Quick Start Templates)

5. Choose option 6 (dotnetcore3.1)

6. Press enter for default project name

7. step over to the sam-app directory. e.g. cd sam-app

8. run the command sam build --debug

Observed result

PS C:\Users\<user>\Desktop\sam-training\sam-app-netcore> sam build --debug
Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
No Parameters detected in the template
2 resources found in the template
Found Serverless function with name='HelloWorldFunction' and CodeUri='./src/HelloWorld/'
Building resource 'HelloWorldFunction'
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Found workflow 'DotnetCliPackageBuilder' to support capabilities 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Running workflow 'DotnetCliPackageBuilder'
Running DotnetCliPackageBuilder:GlobalToolInstall
Installing Amazon.Lambda.Tools Global Tool
executing dotnet: ['dotnet.exe', 'tool', 'install', '-g', 'Amazon.Lambda.Tools']

Error installing probably due to already installed. Attempt to update to latest version.
executing dotnet: ['dotnet.exe', 'tool', 'update', '-g', 'Amazon.Lambda.Tools']
C:\Program Files\dotnet\sdk\3.1.201\NuGet.targets(124,5): error : Unable to load the service index for source https://pkgs.dev.azure.com/vmob/_packaging/Plexure%40Local/nuget/v3/index.json. [C:\Users\<user>\AppData\Local\Temp\bnf4whzu.d3g\restore.csproj]
C:\Program Files\dotnet\sdk\3.1.201\NuGet.targets(124,5): error :   Response status code does not indicate success: 401 (Unauthorized). [C:\Users\<user>\AppData\Local\Temp\bnf4whzu.d3g\restore.csproj]
DotnetCliPackageBuilder:GlobalToolInstall failed
Traceback (most recent call last):
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\dotnet_clipackage\actions.py", line 33, in execute
    self.subprocess_dotnet.run(["tool", "install", "-g", "Amazon.Lambda.Tools"])
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\dotnet_clipackage\dotnetcli.py", line 62, in run
    raise DotnetCLIExecutionError(message=err.decode("utf8").strip())
aws_lambda_builders.workflows.dotnet_clipackage.dotnetcli.DotnetCLIExecutionError: Dotnet CLI Failed: Tool 'amazon.lambda.tools' is already installed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\dotnet_clipackage\actions.py", line 37, in execute
    self.subprocess_dotnet.run(["tool", "update", "-g", "Amazon.Lambda.Tools"])
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\dotnet_clipackage\dotnetcli.py", line 62, in run
    raise DotnetCLIExecutionError(message=err.decode("utf8").strip())
aws_lambda_builders.workflows.dotnet_clipackage.dotnetcli.DotnetCLIExecutionError: Dotnet CLI Failed: Tool 'amazon.lambda.tools' failed to update due to the following:
The tool package could not be restored.
Tool 'amazon.lambda.tools' failed to install. This failure may have been caused by:

* You are attempting to install a preview release and did not use the --version option to specify the version.
* A package by this name was found, but it was not a .NET Core tool.
* The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
* You mistyped the name of the tool.

For more reasons, including package naming enforcement, visit https://aka.ms/failure-installing-tool

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflow.py", line 269, in run
    action.execute()
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\dotnet_clipackage\actions.py", line 39, in execute
    raise ActionFailedError("Error configuring the Amazon.Lambda.Tools .NET Core Global Tool: " + str(ex))
aws_lambda_builders.actions.ActionFailedError: Error configuring the Amazon.Lambda.Tools .NET Core Global Tool: Dotnet CLI Failed: Tool 'amazon.lambda.tools' failed to update due to the following:
The tool package could not be restored.
Tool 'amazon.lambda.tools' failed to install. This failure may have been caused by:

* You are attempting to install a preview release and did not use the --version option to specify the version.
* A package by this name was found, but it was not a .NET Core tool.
* The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
* You mistyped the name of the tool.

For more reasons, including package naming enforcement, visit https://aka.ms/failure-installing-tool

Build Failed
Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 3561, 'exitReason': 'WorkflowFailedError', 'exitCode': 1, 'requestId': '53cfbe8e-f1ae-474d-8da0-559087c288a8', 'installationId': '63c43b0f-6ab0-4772-ba8b-9940b250a669', 'sessionId': 'c9bf8eb8-82eb-4f42-8919-df6cf1d66b55', 'executionEnvironment': 'CLI', 'pyversion': '3.7.6', 'samcliVersion': '0.48.0'}}]}
HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: DotnetCliPackageBuilder:GlobalToolInstall - Error configuring the Amazon.Lambda.Tools .NET Core Global Tool: Dotnet CLI Failed: Tool 'amazon.lambda.tools' failed to update due to the following:
The tool package could not be restored.
Tool 'amazon.lambda.tools' failed to install. This failure may have been caused by:

* You are attempting to install a preview release and did not use the --version option to specify the version.
* A package by this name was found, but it was not a .NET Core tool.
* The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
* You mistyped the name of the tool.

For more reasons, including package naming enforcement, visit https://aka.ms/failure-installing-tool
PS C:\Users\<user>\Desktop\sam-training\sam-app-netcore>

Expected result

As a workaround, if I uncomment any private feeds in the Nuget.Config file, then it works.

E.g.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
    <add key="Microsoft Visual Studio Offline Packages" value="C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\" />
    <!-- <add key="myfeed" value="https://pkgs.dev.azure.com/vmob/_packaging/myfeed/nuget/v3/index.json" /> -->
  </packageSources>
  <packageRestore>
    <add key="enabled" value="True" />
    <add key="automatic" value="True" />
  </packageRestore>
  <bindingRedirects>
    <add key="skip" value="False" />
  </bindingRedirects>
  <packageManagement>
    <add key="format" value="0" />
    <add key="disabled" value="False" />
  </packageManagement>
</configuration>

Result
sam build

PS C:\Users\<user>\Desktop\gh\sam-app> sam build
Building resource 'HelloWorldFunction'
Running DotnetCliPackageBuilder:GlobalToolInstall

Tool 'amazon.lambda.tools' was reinstalled with the latest stable version (version '4.0.0').
Running DotnetCliPackageBuilder:RunPackageAction
Amazon Lambda Tools for .NET Core applications (4.0.0)
Project Home: https:/aws/aws-extensions-for-dotnet-cli, https:/aws/aws-lambda-dotnet

Executing publish command
... invoking 'dotnet publish', working folder 'C:\Users\<user>\Desktop\gh\sam-app\src\HelloWorld\bin\Release\netcoreapp3.1\publish'
... publish: Microsoft (R) Build Engine version 16.5.0+d4cbfca49 for .NET Core
... publish: Copyright (C) Microsoft Corporation. All rights reserved.
... publish:   Restore completed in 201.84 ms for C:\Users\<user>\Desktop\gh\sam-app\src\HelloWorld\HelloWorld.csproj.
... publish:   HelloWorld -> C:\Users\<user>\Desktop\gh\sam-app\src\HelloWorld\bin\Release\netcoreapp3.1\linux-x64\HelloWorld.dll
... publish:   HelloWorld -> C:\Users\<user>\Desktop\gh\sam-app\src\HelloWorld\bin\Release\netcoreapp3.1\publish\
Zipping publish folder C:\Users\<user>\Desktop\gh\sam-app\src\HelloWorld\bin\Release\netcoreapp3.1\publish to C:\Users\<user>\Desktop\gh\sam-app\.aws-sam\build\HelloWorldFunction\HelloWorld.zip
Creating directory C:\Users\<user>\Desktop\gh\sam-app\.aws-sam\build\HelloWorldFunction
... zipping: Amazon.Lambda.APIGatewayEvents.dll
... zipping: Amazon.Lambda.Core.dll
... zipping: Amazon.Lambda.Serialization.Json.dll
... zipping: HelloWorld.deps.json
... zipping: HelloWorld.dll
... zipping: HelloWorld.pdb
... zipping: HelloWorld.runtimeconfig.json
... zipping: Newtonsoft.Json.dll
Created publish archive (C:\Users\<user>\Desktop\gh\sam-app\.aws-sam\build\HelloWorldFunction\HelloWorld.zip).
Lambda project successfully packaged: C:\Users\<user>\Desktop\gh\sam-app\.aws-sam\build\HelloWorldFunction\HelloWorld.zip

Build Succeeded

Built Artifacts  : .aws-sam\build
Built Template   : .aws-sam\build\template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Windows 10 1909

2. sam --version: 0.48.0

3. .net core sdk version installed on machine: 3.1.201

[fvilches17]

I've created a Pull Request which should fix this issue

aws/aws-lambda-builders#169

[fvilches17]

Update, PR merged to develop

aws/aws-lambda-builders#169

[fvilches17]

This issue has now been resolved. This is the commit to the master branch: aws/aws-lambda-builders@4493a84

You can get the fix if you download the latest version of SAM CLI

[nsivaji61]

Hello , I have updated the latest version of SAM and trying to build .netcore lambda and getting the same exception. Can you please suggest what needs to be done .
Running DotnetCliPackageBuilder:GlobalToolInstall

error NU3034: Package 'Amazon.Lambda.Tools 5.0.2' from source 'https://api.nuget.org/v3/index.json': This package is signed but not by a trusted signer.

Build Failed
Error: DotnetCliPackageBuilder:GlobalToolInstall - Error configuring the Amazon.Lambda.Tools .NET Core Global Tool: Dotnet CLI Failed: Tool 'amazon.lambda.tools' failed to update due to the following:
The tool package could not be restored.
Tool 'amazon.lambda.tools' failed to install. This failure may have been caused by:

• You are attempting to install a preview release and did not use the --version option to specify the version.
• A package by this name was found, but it was not a .NET tool.
• The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
• You mistyped the name of the tool.

For more reasons, including package naming enforcement, visit https://aka.ms/failure-installing-tool

[fvilches17]

Hi @nsivaji61 I wasn't able to replicate the issue. I have 2 private NUGET feeds configured in my machine and sam build ran just fine.

Environment
Windows 10 Enterprise Version 20H2 OS Build 19042.870.

Steps

1. Downloaded the latest version of sam cli (x64).
2. Ran sam init and chose the dotnetcore3.1 template.
3. Ran sam build. Build succeeded.

Note: I only got a nuget warning, pretty much saying that my 2 private feeds couldn't be used during dotnet restore (because they need authentication), so they were ignored. This is expected.

This looks like an unrelated issue. I suggest checking out this SO question.
https://stackoverflow.com/questions/56334526/error-nu3034-this-package-is-signed-but-not-by-a-trusted-signer