Templates Snapshot Testing Tooling v2

[JanKrivanek]

Background

This is continuation of #3868
Though we already have some ideas on next iteration, it should be mainly based on feedback from the initial adoption

Ideas for next iteration

• Support for switching sdk versions
  Simplified approach to achieve this:
  1. switch sdk for new processes

  ren global.json global.json.bak
  dotnet new globaljson --sdk-version <version>
  ren global.json.bak global.json
  

  2. run the instantiation
  3. run Verify while injecting the proper naming suffixes for the sdk (we cannot rely on Verify as it'll run in current process without altered sdk)
  4. cleanup the global.json from step 1
• Simplify and unify working with paths (include/exclude patterns, paths given in custom scruber and custom verifier etc.) - tooling should be permissive and be able to accept mix-n-matches of dir separator chars, plus it should have settings for enforcing separator char used in path it's passing out (custom scrubber and verifier). Paths passed out should probably be passed as custom type - allowing to fetch relative path (to template root or test root), absolute path, paths with custom spearators
• Rewrite more snapshots-based integration test for templating instantiation in sdk to use the tooling
• Add telemetry
• Implement context detection and extraction for nondeterministic generators handling (so e.g. for Port generator, the logic should be able to detect the resulting value in the generated output and then process the output by replacing all instances of the generator being used).
• Add Template Validator as another tool in the authoring toolset. Implement just a sample of most importatnt validations (more comprehensive list: Authoring tools: templates & template packages validation #2623)
  • Create MSBuild Task version of the Template Validator
  • Design and use continuable errors during validation - so that as much errors as possible cna be reported during single run (while not reporting nonsense issues caused by inconsistent data detected in previous steps).
• Build in property based testing
• Extend API verifier to support custom filter: see Cover TemplateSamples with tests #5794 (comment)
• Surface addtextextension setting in TemplateVerifierOptions #5785