The coalesce macro should work with built-in dotnet new parameters

[ardove]

I'm trying to use the value provided via the "output" parameter (-o) to replace a path that's referenced in one of my source files. That file happens to be a Jenkinsfile that needs to know where a sln file is located in order to build it. The jenkinsfile will live in the same directory as my sln.

When I don't specify a value, I'm fine with falling back to my default (since the developer knows to scaffold the solution in a directory which matches the default).

My ultimate goal is to enable a developer to run the template engine, check in the source, and build the application through jenkins via a jenkinsfile.

Using the code below, you'll notice that you ALWAYS get the default value when you invoke the template to generated the transformed Jenkinsfile. I'm unaware of another way to do this (tried using an evaluate and also tried using "target" instead of "output").

Am I missing something or is this just a gap?

Here's the sample template.json

{  
  "$schema": "http://json.schemastore.org/template",
  "author": "Test",
  "classifications": [
    "Web",
    "WebAPI"
  ],
  "name": "TEST ASP.NET Core Web API Solution",
  "identity": "TEST.WebApi.SolutionTemplate",
  "shortName": "test-webapi-sln",
  "sourceName": "WebApiTemplate",
  "preferNameDirectory": true,
  "tags": {  
    "language": "C#"
  },
  "primaryOutputs": [
    {  
      "path": "WebApiTemplate.sln"
    }
  ],
  "symbols": {
    "DefaultSourceDirectory": {
      "type": "generated",
      "generator": "constant",
      "parameters": {
        "value": "src"
      }
    },
    "SourceDirectoryReplacer": {
      "type": "generated",
      "generator": "coalesce",
      "parameters": {
        "sourceVariableName": "output",
        "fallbackVariableName": "DefaultSourceDirectory"
      },
      "replaces": "templates/test-webapi-sln"
    }
  }
}

And the Jenkinsfile which it works on

pipeline {
  agent none
  options {
    skipDefaultCheckout true
  }
  stages {
    stage('Compile and test') {
      agent {
        label 'dotnet-2.2'
      }
      steps { 
        checkout scm
        stash 'source'
        dir("$WORKSPACE/templates/test-webapi-sln"){
          echo 'Building WebApiTemplate.sln...'
          sh 'dotnet build WebApiTemplate.sln'
          echo 'Running unit tests...'
          sh 'dotnet test UnitTests/test.WebApiTemplate.UnitTests.csproj --no-build'
        }        
      }
    }
    stage('Build docker image') {
      agent {        
          label 'docker'
      }
      steps {
        unstash 'source'
        dir("$WORKSPACE/templates/test-webapi-sln"){
          sh 'DOCKER_REGISTRY=072100331905.dkr.ecr.us-west-2.amazonaws.com/ IMAGE_VERSION=0.0.0.$BUILD_NUMBER docker-compose build'
        }
      }
    }
    stage('Login to ECR') {
      agent {
        label 'aws-cli'
      }
      steps {
        echo 'Generate script to enable docker-compose push to ECR'
        sh 'aws ecr get-login --no-include-email --region us-west-2 > docker-login.sh'
        sh 'chmod +x docker-login.sh'
        stash includes: 'docker-login.sh', name: 'docker-login'
      }
    }
    stage('Publish docker image') {
      agent {        
          label 'docker'
      }
      steps {
        unstash 'docker-login'
        sh './docker-login.sh'
        unstash 'source'
        dir("$WORKSPACE/templates/test-webapi-sln"){
          sh 'DOCKER_REGISTRY=072100331905.dkr.ecr.us-west-2.amazonaws.com/ IMAGE_VERSION=0.0.0.$BUILD_NUMBER docker-compose push'
        }        
      }
    }
  }
}

When I run dotnet new -o output, I'd expect that templates/test-webapi-sln would be replaced with output, but instead, it is replaced with src (the default value in the coalesce macro).

[mlorbetske]

In this particular case, -o|--output isn't actually passed to the template, it's a configuration parameter to the engine itself that sets the directory in which to work. I think this ask really boils down to promoting the output engine parameter to a pseudo-parameter that cannot be overridden in the template. @seancpeters what would you think of that?

cc @vijayrkn

[mlorbetske]

Thinking a bit more about this - are you certain that you'd want the physical path (relative the the location they ran the command - or possibly the absolute path on their disk) the user specifies as the root of content creation to be emitted into the template? If the project is cloned or moved elsewhere, this would leave the original pathing information in the project and potentially break it, which doesn't seem like the desired behavior.

[ardove]

Well, I think I would really only want to include the output paremeter if the user had generated the template providing it (maybe by using the derived macro instead of coalescing).

You're right, they could move the files around, but if they make manual modifications to the files, we assume they are an experienced user with the know how to troubleshoot problems they cause by doing so. We think a dumb user will just run the scaffold, check it in, and fire up jenkins. Rather than expecting them to know how that all works, we want to cater to a "dumb" user of our cli tooling rather than an experienced user. The dumb user should to be able to run the dotnet new command, run a git init and push, and then fire up jenkins.

A more experienced user will be the one wanting to use customized configuration, and we expect the experienced user to be able to troubleshoot on their own.

[mlorbetske]

But wouldn't the action of pushing to git be enough to cause the unexpected results (in the case that -o is specified)?

Consider the command dotnet new templateNameHere -o c:\users\MyUserName\projects\x. If I run that, the contents of the template are generated into the x folder in c:\users\MyUserName\projects. If the content that gets generated then takes a dependency on that path, running the project on a machine without that path present (or building the project if the path is depended on for a build step) would be in a state that requires troubleshooting by default.

Beyond this, if I'm already in the c:\users\MyUserName\projects directory, I could run dotnet new templateNameHere -o x to put the content in the same place. If I were in the c:\users\MyUserName\projects\y directory, I could run dotnet new templateNameHere -o ..\x. Having those equivalent commands produce different outputs seems odd as well.

[ardove]

That's true. I think my use case is specific to a solution directory relative to the place that I'm scaffolding the solution. I don't think that using an absolute path applies for our use case at all (I don't expect anyone would ever want to scaffold a directory based on an absolute path to a specific location somewhere on disk and then check it in). Generally speaking, I'm of the opinion that scaffolding a solution in location y from location x just seems unnatural (unless y is a child directory of x -- hence the relativity).

To workaround the issue, we landed on a conventional solution repository structure that relies on the scaffolded solution ending up in src/ (at the root of the repository). We just don't support a repository that contains files in non-standard places. So, if you guys end up wanting to reject this, that's fine by me.

[mlorbetske]

I'm not strictly against it, but I think there may things beyond just making -o available for coalesce to get directly to the commonly desired information more directly. Maybe something along the lines of a built-in "directoryName" symbol that gives just the name of the directory the content was generated into would do the trick?

@seancpeters

[ardove]

Yeah, I mean I don't have strong feelings about how it's implemented, just thought it might be a useful feature for the community (and you're right, there are probably other macros that could use it too).

It could be handy if there were some built in parameters that contained contextual information about the invocation (absolute path to command invocation location, command invoked, version of the dotnet cli, target dotnet framework, etc.)

[donJoseLuis]

Hello @ardove , we're prioritizing active items & produce a delivery plan against a prioritized backlog. Is this something you are still interested in?

[ardove]

@donJoseLuis - yes, I think this feature would still be useful.

[bekir-ozturk]

We have discussed this internally and found the concerns valid. We want to extend this to other useful built-in parameters such as fullOutputPath, hostIdentifier, hostVersion etc.
For the time being, we are aiming for .NET 7.

[heaths]

I'm trying to create a template for the Azure SDK for .NET as well and running into this. We have a particular directory structure and naming convention, where we want the users to pass -o sdk/{service-directory} -n Azure.{ServiceName} but then have to have them repeat some of those same tokens instead of just using regex parameters or forms to get those same values since they need to be replaced in various forms throughout files.

[vlada-shubina]

We have discussed this internally and found the concerns valid. We want to extend this to other useful built-in parameters such as fullOutputPath, hostIdentifier, hostVersion etc. For the time being, we are aiming for .NET 7.

Update on this: it is already possible to use HostIdentifier and HostVersion (for dotnet new) via bind symbols, however output path is still not accessible. Adding outputPath as implicit variable (as name for --name) has to be considered.
At the moment there is no timeline for that unfortunately.