Enable FDD + (implicit) RID-specific apps with RuntimeSpecific

[richlander]

Enable FDD + (implicit) RID-specific apps with RuntimeSpecific

We've been on a slow path to make it easier and more intuitive to produce RID-specific apps, for both framework-dependent (FDD) and self-contained (SCD) deployment models. Framework dependent + (implicit) RID-specific apps remain a significant gap.

Runtime-specific or Runtime Identifier (RID) specific apps target a specific operating system + architecture combination, like Linux x64. RIDs have a special syntax, like linux-x64. This makes the apps more similar to C++ or Go, which are always compiled specific to a given OS+Arch combination. There is a performance advantage to producing apps as RID-specific.

The addition of a new RuntimeSpecific boolean property (that doesn't assume SCD) can significantly deliver on that need.

Alternatively, the most obvious approach is to make a breaking change to what -r means (w/o any other arguments being specified). We may make that change in .NET 8, but not now. This proposed change either removes the need for that break or is a bridge towards it. We'll likely find out based on feedback and further exploration of this domain.

Note: Property naming is TBD. Another name is ArchitectureSpecific.

Context

We started .NET Core with one key flaw (in this topic area), which was that specifying a RID resulted in a self-contained app. It's now a breaking change to resolve that.

In particular, the following command (alone) results in a RID-specific app (now with a warning):

dotnet build -r win-x64

We should have recognized:

• Framework-dependent apps are the default, and
• Specifying a RID is a valid option for framework-dependent apps, and
• Changing the deployment type (FDD -> SCD) for an otherwise valid option was bad.

We have to recover for that in some way. In particular, we want to make it easier to produce RID-specific FDD apps. RID-specific FDD apps are smaller, simpler, and potentially faster to load. This is particularly valuable for containers.

We've made multiple proposals and changes along this path:

• Update self contained/ runtime command line options for build and publish #18837
• Adding OS and arch command line options #18889
• Enable implicit RID with --sc #21405
• .NET 8: [breaking] dotnet build/publish produces RID-specific apps by default #23540
• Use implicit RID for Publish Properties #26028

Solution

Perhaps there is another way, particularly now that we've adopted an implicit RID scheme.

We can offer a RuntimeSpecific boolean property that does the following:

• Makes the app RID-specific.
• Uses the implicit RID by default.
• Does't assume a deployment type (which means FDD by default).

This is almost identical to what the -a and --os arguments do, except without needing to be specify either of them.

Field test of the solution

We maintain several (many!) sample Dockerfiles. There are a lot of them, in part due to wanting to make and promote container apps as FDD + RID-specific as the best option. Unfortunately, that forces us to create a Dockerfile per architecture, which is a very bad default requirement for a platform with multi-arch container images. It's really bad.

Note: It pained me when I created these samples. I didn't see another way at the time.

Let's take a good look at these samples. It's going to be ugly.

Let's start with the good Dockerfile. It is the default (that's why it is "good"; Docker users will know this). Unfortunately, we're unable to follow our preferred pattern since that Dockerfiles needs to work everywhere (from a Rasbperry PI to GitHub Actions). That means that people using this Dockerfile as a guide don't follow the preferred FDD+RID-specifc pattern. Ughh! More pain.

Let's take a look at the good/processor-agnostic Dockerfile:

RUN dotnet publish -c release -o /app --no-restore

Looks decent.

Let's look at the x64 Dockerfile:

RUN dotnet restore -r linux-x64

# copy and publish app and libraries
COPY . .
RUN dotnet publish -c release -o /app -r linux-x64 --self-contained false --no-restore

Oh my. That's bad. Look at the careful RID management of restore and publish and the requirement of --self-contained false. All ugly.

Same with the Arm64 Dockerfile:

RUN dotnet restore -r linux-arm64

# copy and publish app and libraries
COPY . .
RUN dotnet publish -c release -o /app -r linux-arm64 --self-contained false --no-restore

Same with the Arm32 Dockerfile:

RUN dotnet restore -r linux-arm

# copy and publish app and libraries
COPY . .
RUN dotnet publish -c release -o /app -r linux-arm --self-contained false --no-restore

These are repeated to make a point. Let's see if this property would improve this situation.

Moment of truth

If the RuntimeSpecific property was set, the good Dockerfile would do the same thing as the platform-specific Dockerfiles, with zero code changes and a much simpler pattern. Nice! We'd be able to delete those platform-specific Dockerfiles, with no loss in customer value. That's a strong indication of progress.

Note: The final FROM statements would differ between the platform-agnostic and the current platform-specific Dockerfiles, but that's all mechanics and further confusing users.

The good Dockerfile also includes -c Release.

RUN dotnet publish -c release -o /app --no-restore

With the plan to enable -c Release for dotnet publish, we could further simplify that line to be:

RUN dotnet publish -o /app --no-restore

This assumes a project or Directory.Build.props file with the following:

<PropertyGroup>
    <RuntimeSpecific>true</RuntimeSpecific>
    <PublishRelease>true</PublishRelease>
</PropertyGroup>

We would update our container samples to do just that.

We need to have a follow-on discussion on whether we should add either of these properties to templates. Probably not for .NET 7, but something to consider. They are probably best left as guidance for now and to get more feedback on the direction. It's best to decide what we're going to do with the breaking change proposal for .NET 8.

Visual Studio Publishing

Visual Studio has a number of publishing scenarios that rely on apps NOT being runtime-specific. This feature is opt-in so doesn't break Visual Studio publishing, per-se. Runtime-specific apps already exist, so nothing is changing in principle. The intent of this feature is that it will be much easier to produce FDD + RID-specific apps. As a result, we should think through how Visual Studio tools (and any other like it) should react.

One common pattern in Visual Studio:

• Build on Windows.
• Volume mount into a Linux container.

Another is:

• Build on Windows.
• Push to an Azure app service (which could be Windows or Linux).

There are other similar scenarios. They rely on the app being a portable app. In particular, the app should contain native assets for all the NuGet packages it depends on. Runtime-specific apps, by design, don't do that.

Visual Studio publishing tools can defensively enable this scenario. They should set the following properties to false:

• RuntimeSpecific
• SelfContained

That will ensure that they always build portable apps by default. Visual Studio should include a switch to disable that default to enable users to get a different behavior when they want an advanced experience.

@baronfel @MichaelSimons @mthalman @marcpopMSFT @nagilson @jander-msft @DamianEdwards @mhutch

[DamianEdwards]

@bradygaster what are your thoughts on this from a "publishing to Azure" (etc.) viewpoint?

[richlander]

I assume our general guidance is "don't use RID-specific apps with 'public to Azure' unless you know what you are doing." This feature would align with that. The only difference is that it would make RID-specific apps easier and more compelling to use than today.

[bradygaster]

Agreed with @richlander atm, but would be keen to show this to our partners in Azure to see what their concerns would be. Presumably, anything K8S would "almost always be guessable," but App Service runs Linux and Windows, and both 32 and 64 bitness on Windows, so I'd want to make sure we validate this with those partners. I'd also presume we not make this something that a "tool can fix" - meaning, when one right-click publishes in VS, VS does the right thing, but other tools force the customer to know what they're doing.

[richlander]

The end game on this that VS tools build with extra properties set to ensure that they the app is built the right way for the target. Relying on the defaults or the developer not having "messed stuff up" is a losing proposition, IMO. The whole point of tools is that they are value-added.

[btardif]

• @JasonFreeberg
  This passes my sniff test, I think it would be mostly a no-op for App Service hosted apps being deployed through VS / VS Code etc...

We might want to take a look at our GitHub Actions templates to make sure we are following the latest greatest conventions.

[richlander]

I added a section about Visual Studio publishing at the end.

[tannergooding]

An interesting scenario is potentially the difference between "developer inner loop", where you often have a "full sdk" and therefore want to minimize the work to build and test. For example, in VS, copying the framework and runtime to the output each "iteration" is potentially expensive, especially on say a laptop where resources are often more limited or constrained.

On the other hand, this is often the desired configuration for actual deployments or CI setups where you are building "stable" bits for shipping out to customers.

It would be good to hear more about these potentially competing scenarios and if the defaults will be the same or different between the two.

[richlander]

I'm not sure I'm seeing the same tension. Isn't your split the same as build vs publish?

[teo-tsirpanis]

What's the difference between UseCurrentRuntimeIdentifier and RuntimeSpecific?

[nagilson]

@teo-tsirpanis

That's a good question. The fact that it took a second to come up with a clear answer implies we need to be really clear communicating this.

UseCurrentRuntimeIdentifier is a flag that tells the SDK to use a RID based on the one of your current machine. It will generally be OS-tied but less specific: like win-x64 as opposed to win-7-x64. You can set it to false if you want to disable getting a RID automatically on properties that require one, like SelfContained.

RuntimeSpecific (which: warning, is not fully implemented yet) means the app is made for a certain RuntimeIdentifier, but not necessarily FDD (framework dependent), AKA not self contained, where previously setting a RuntimeIdentifier always meant you had SelfContained unless you set it to false. If we make the planned change in .NET 8 to make all apps have a RID by default, it will also be the opt out mechanism for that. It also uses the more specific RID.

[nagilson]

@richlander I think we can close this since the related components to #29950 are done? RuntimeSpecific wasn't added but I think we collectively determined we could go down a different path and didn't need it. It was actually implemented (or mostly so) in an older pr that we closed. Since it's been quite a while, I forget where we landed and why we decided it was not necessary in the new PR. I vaguely remember being that this was designed to provide runway to the breaking change but we ended up putting it in 8 so we didn't need that.

[baronfel]

Going to close this one in favor of continued discussion on #29950.