introduce operating system version ranges as part of the target

[andrewrk]

On Linux we have kernel versions: 3.14, 4.14, 4.18, etc.

On Windows we have versions, e.g.

• XP
• Vista
• 7
• 8
• 10

FreeBSD has the concept of versions: most recent e.g. 12.0

MacOS / iOS has the concept of versions too - e.g. 10.14.3

I propose for OS minimum version and maximum version to be part of the target. When you specify an OS minimum version as part of the target, it is telling userland code what ABI it can rely on. Each OS will have a default minimum version. Some standard library functions (or functions from third party packages) may give a @compileError("this API supported only on OS version xyz and later"). The OS minimum version will be available for userland code to observe in @import("builtin").

Maximum version is also useful, because it can avoid code bloat, when it is compile-time known that the code will not be able to take advantage of newer OS features.

When compiling for the native target, the minimum and maximum OS versions will be the same.

[bfloch]

I believe the story with windows is a bit more complicated than just using the "marketing" version:

https://docs.microsoft.com/en-us/windows/desktop/SysInfo/operating-system-version

It all started by genius marketing to call API version 6.1 "Windows 7" and went downhill from there. I guess after some negative press they just decided to re-version their 6.4 API to 10 [1].
To make things better the Version API in Windows 10 will return Windows 8 if the app was not manifested for 8.1 or 10, which I assume is done to solve backwards compatibility (?).

What does that mean in terms of target version? Not sure but I have the feeling that the API version is more useful then the marketing version.

Is Windows XP really more like a distro label, while 5.1/5.2 is what matters as platform/API?
Maybe a Windows dev can clarify this further.

[1] https://www.digitaltrends.com/computing/windows-10-kernel-to-leapfrog-from-6-4-directly-to-10-0/

[daurnimator]

Is Windows XP really more like a distro label, while 5.1/5.2 is what matters as platform/API?

Usually _WIN32_WINNT is what really matters

[shawnl]

It all started by genius marketing to call API version 6.1 "Windows 7"

We use to call it "Vista 7".

[andrewrk]

For macOS, there's an important linker flag -mmacos_version_min. 5784631 bumped the default from 10.10 to 10.14, but the better solution will be to have this as part of the target. And then the native target should use the current OS version for this value.

[andrewrk]

Regarding the Windows stuff, people in this thread who have a better understanding than me, can you help me come up with the enum that represents Windows API versions? Here's what I would do on first guess:

const WindowsVersion = enum {
    Legacy, /// Represents a version that is unsupported due to being too old.
    v7,
    v8,
    v10,
    v10_2015LTSB,
    v10_2016LTSB,
    v10_2019LTSC,
    Unreleased, /// Represents a version of windows not yet released.
};

The compiler will also need a way to detect the version of the native target.

[shawnl]

Linux has a special ELF section for this (and file reads it[1]), however it is not enforced for static binaries, so we would have to enforce it ourselves (which I think is a good idea). https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/noteabitag.html

Example: qt/qtbase@bb8a618#diff-e4c3c13b1c7ed0faa458312ad4d5c1da

[1] a.out: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=64dbf5608aa84b8eff7ced72abd13868652890e5, for GNU/Linux 5.3.28, not stripped
a.out: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=5b9ea55987638c03288db9433abad05bd0621111, for GNU/Linux 5.3.28, not stripped

[daurnimator]

can you help me come up with the enum that represents Windows API versions?

From window's sdkddkver.h: (see also https://docs.microsoft.com/en-us/cpp/porting/modifying-winver-and-win32-winnt?view=vs-2019 )

#define _WIN32_WINNT_NT4                    0x0400
#define _WIN32_WINNT_WIN2K                  0x0500
#define _WIN32_WINNT_WINXP                  0x0501
#define _WIN32_WINNT_WS03                   0x0502
#define _WIN32_WINNT_WIN6                   0x0600
#define _WIN32_WINNT_VISTA                  0x0600
#define _WIN32_WINNT_WS08                   0x0600
#define _WIN32_WINNT_LONGHORN               0x0600
#define _WIN32_WINNT_WIN7                   0x0601
#define _WIN32_WINNT_WIN8                   0x0602
#define _WIN32_WINNT_WINBLUE                0x0603
#define _WIN32_WINNT_WINTHRESHOLD           0x0A00 /* ABRACADABRA_THRESHOLD*/
#define _WIN32_WINNT_WIN10 0x0A00 /* ABRACADABRA_THRESHOLD*/

I think that should translate to the following zig definition:

const WindowsVersion = enum(u16) {
	WindowsNT4 = 0x0400,
	Windows2000 = 0x0500,
	WindowsXP = 0x0501,
	WindowsServer2003 = 0x0502,
	WindowsVista = 0x0600,
        WindowsServer2008 = 0x0600, // are duplicate enums allowed?
	Windows7 = 0x0601,
	Windows8 = 0x0602,
	Windows8_1 = 0x0603,
	Windows10 = 0x0A00,
}

[emekoi]

@daurnimator see #2115

[andrewrk]

#2847 did a proof of concept of adding more things to the target, and I think it is a good idea. This is part of zig's "optimality" claim mixed with "communicate intent precisely". When the programmer specifies the minimum target OS version, zig code can take advantage of that specification to generate better code. See also fc9e28e which solved #397.

[andrewrk]

If anyone wants to contribute to this issue, most of the work can be in Zig code, with the following sub-task:

Detect the native operating system version, for all the OSes in the support table.

[pixelherodev]

How is this meant to work for cross-compiling? e.g. is there going to be a specific option to target the Windows Seven API, or Linux 5.x, etc?

[andrewrk]

Updated syntax for -target to take into account OS version ranges:

# still valid. default version range
-target x86_64-windows-msvc

# minimum windows version: XP
# maximum windows version: 10
-target x86_64-windows.xp...win10-msvc

# minimum windows version: 7
# maximum windows version: latest
-target x86_64-windows.win7-msvc

# linux example
-target aarch64-linux.3.16...5.3.1-musl

# specifying glibc version
-target mipsel-linux.4.10-gnu.2.1

Here's what it will look like to populate a std.Target:

@@ -167,7 +167,7 @@ const test_targets = blk: {
             .target = Target{
                 .Cross = CrossTarget{
                     .cpu = Target.Cpu.baseline(.mipsel),
-                    .os = .linux,
+                    .os = Target.Os.defaultVersionRange(.linux),
                     .abi = .musl,
                 },
             },
@@ -178,7 +178,7 @@ const test_targets = blk: {
             .target = Target{
                 .Cross = CrossTarget{
                     .cpu = Target.Cpu.baseline(.x86_64),
-                    .os = .macosx,
+                    .os = Target.Os.defaultVersionRange(.macosx),
                     .abi = .gnu,
                 },
             },
@@ -190,7 +190,7 @@ const test_targets = blk: {
             .target = Target{
                 .Cross = CrossTarget{
                     .cpu = Target.Cpu.baseline(.i386),
-                    .os = .windows,
+                    .os = Target.Os.defaultVersionRange(.windows),
                     .abi = .msvc,
                 },
             },

Code that used Target.parse need not be updated.

[daurnimator]

How does this interact with libc version ranges?
-target aarch64-linux.3.16...5.3.1-gnu.2.20...2.31?
How do you specify open ended or "newest known"?

[andrewrk]

The only libc which has version options is glibc and it is a specific version to target, not a range.

[adontz]

@andrewrk may you give more details on this?

Maximum version is also useful, because it can avoid code bloat, when it is compile-time known that the code will not be able to take advantage of newer OS features.

A real life example or something?

[andrewrk]

Here are some examples:

zig/lib/std/fs.zig

Lines 155 to 156 in 907c558

	const amt = try in_stream.readFull(buf[0..]);
	try atomic_file.file.write(buf[0..amt]);

This code is to be updated to call copy_file_range if it is in the Linux version range. If the version range includes both a version without it, and a version with it, then it will try it and use ENOSYS at runtime to fall back to manual copying. If the target minimum version is >= the Linux version that introduced this syscall, then no fallback code will be generated, and ENOSYS => unreachable or at least ENOSYS => error.Unexpected. If the target maximum version is < the Linux version that introduced this syscall, then no attempted syscall is made; the "fallback" code is all there is. This avoids 1 pointless syscall at runtime for this codepath.

Another example, more consequential, is using io_uring rather than epoll in lib/std/event/loop.zig. Same principles as the previous paragraph. io_uring is quite new, so when compiling for the native OS target (which sets both min and max versions to the host OS) io_uring code would be compiled out and take up no space. This helps Zig with its "optimal" claim, because it means we can code for all the different abstractions that have been available over time, and Zig code at compile-time can determine what particular set to target.

Finally, another use case for this is http://libsound.io/. Sound APIs on Windows have changed with nearly every major release of Windows, and a Zig package that wants to emit sound on Windows has a lot to gain from knowing the version range. If you happen to know that your target platform is stuck on an older Windows, then you can note that in the target version and there will be no API calls to newer features.

Some people target old versions of operating systems as a hobby, such as Windows XP, and this also caters to this use case.

[adontz]

@andrewrk

This code is to be updated to call copy_file_range if it is in the Linux version range. If the version range includes both a version without it, and a version with it, then it will try it and use ENOSYS at runtime to fall back to manual copying.

I don't think it should work like this. If there are two code paths and we prefer one over another during entire lifetime of the application, then I'd expect the check result to be cached. Your method really "avoids 1 pointless syscall at runtime for this codepath." but that should be one call per entire application lifetime, not per call.

So, there are three cases:

1. Application needs any version higher than N to use the only API, the modern one. Specifying minimum version is enough. Application will crash in some ugly manner if API is not available, probably not even load completely, but that is fine, since that is exactly what developer wanted.
2. Application supports versions between N and M and as a result several different APIs. APIs have strict priority, which API will be used is predetermined by external conditions, like OS version or dynamic library availability and will not change during application lifetime. I can't recall any, even crazy, case when anyone would like to switch between APIs, because API became available at runtime. I don't even think it's a thing. If there are several APIs available for version higher than N, application should use run-time dispatch with caching. So, API availability check result should be cached. It's a few syscalls per entire application runtime, quite negligible price. We can read OS version with uname on Linux or directly request for specific version with VerifyVersionInfoW on Windows. Maybe in a lazy way, on demand. Maybe in a not so lazy way, on startup. I'm not so sure about Linux, because kconfig allows fine grained cherrypicking, so ENOSYS may be the only reliable way to detect features, but again result should be cached for sure.
   Also, application should not link statically against newer APIs, because corresponding system libraries may not be available.
3. Application intends to use only old APIs because reasons. So, if we cache API availability check, and if we already support case parseh unable to detect noreturn function attribute #2 and do everything to support environments where newer APIs are not available, why is this a problem? Larger binary size? Even if so, it's better to use explicit flags for build system, rather simple OS version comparison, because in case of Linux kernel can come with quite various set of kernel features as well as userland libraries, so just saying "I'll need kernels older than 5.2" is not enough to guarantee that Ubuntu did not backport some feature, as well as saying "I'll use kernels newer than 4.9" does not mean that some buildroot compiled kernel will not lack modern feature, to say nothing about OpenGL availability. In case of Windows it's not that simple either. Some components, may not be installed despite Windows version supports them in general. Some components are server edition only. So I see build flags per API as the only reasonable way to go.

What I would really like to see for supporting multiple APIs is

• Dynamic imports, becase cannot link statically to optional APIs. So "pub extern" may become "pub defer extern". It will use stub which loads library, resolves export and replaces itself with jump to actual function address. Something like .Net's DllImportAttribute. Or any other way to make LoadLibrary/GetProcAddress/dlopen/dlsym less painful.
• Maybe new builtin to allow resolve function address with non-default library name. Like @resolve(function_name, library_path) builtin. I needed it a few times, but not sure if such special cases need to be supported.
• What to do if import fails is an open question. I think it's OK to just panic. Also @resolve(function_name) may return some result to check.
• Exporting pointers to functions as functions. Will allow more or less what C CRT does for memcpy/memmove based on CPU arch. Such pointers will be initialized with address of stub function, which updates pointer based on checks.

[andrewrk]

Follow-up issues:

• implement the native operating system version detection for Windows #4581
• implement the native operating system version detection for FreeBSD #4582
• implement the native operating system version detection for macOS #4583