Proposal: Fixed Sized Buffers should support formatted value types that contain only the already supported types.

[tannergooding]

Ported from dotnet/roslyn#11018

At present, fixed size buffers only support bool, byte, char, short, int, long, sbyte, ushort, uint, ulong, float, and double.

This should be expanded to also support formatted value types (structs) that contain only the above supported types.

This would allow interop code, such as the following, to be written:

public struct DXGI_RGB
{
    public float Red;
    public float Green;
    public float Blue;
}

unsafe public struct DXGI_GAMMA_CONTROL
{
    public DXGI_RGB Scale;
    public DXGI_RGB Offset;
    public fixed DXGI_RGB GammaCurve[1025];
}

Other Thoughts:
The existing backing structure that is emitted by the compiler to support fixed sized buffers would likely need to be modified to support this.

With the current 'backing structure' a modification to the private fields in a struct would cause a break in the generated code. For example:

Assembly A:

public struct S1
{
    private float _x;
}

Assembly B:

unsafe public struct S2
{
    public fixed S1 Buffer[10];
}

The above code will generate something akin to the following:

Assembly A:

public struct S1
{
    private float _x;
}

Assembly B:

unsafe public struct S2
{
    [CompilerGenerated, StructLayout(LayoutKind.Sequential, Pack = 0, Size = 40), UnsafeValueType]
    public struct <Buffer>e__FixedBuffer
    {
        public S1 FixedElementField;
    }

    [FixedBufferAttribute(typeof(S1), 10)]
    public <Buffer>e__FixedBuffer Buffer;
}

Then, modifying Assembly A in the following manner will cause a break (the fixed buffer has a size of 40, when it should have a size of 80, due to the additional field added).

public struct S1
{
    private float _x;
    private float _y;
}

This could be worked around, by modifying the generated structure in the following manner:

unsafe public struct S2
{
    [CompilerGenerated, StructLayout(LayoutKind.Sequential, Pack = 1), UnsafeValueType]
    public struct <Buffer>e__FixedBuffer
    {
        public S1 _0;
        public S1 _1;
        public S1 _2;
        public S1 _3;
        public S1 _4;
        public S1 _5;
        public S1 _6;
        public S1 _7;
        public S1 _8;
        public S1 _9;
    }

    [FixedBufferAttribute(typeof(S1), 10)]
    public <Buffer>e__FixedBuffer Buffer;
}

Under the newly generated code, the modification to Assembly A would allow Assembly B to still work. This is because the size is computed dynamically, based on the number of elements.

Another interesting scenario is that, with the new system, the emitted structure could be modified even further, so-as to provide a type of 'safe' fixed buffer.

unsafe public struct S2
{
    [CompilerGenerated, StructLayout(LayoutKind.Sequential, Pack = 1), UnsafeValueType]
    public struct <Buffer>e__FixedBuffer
    {
        private S1 _0;
        private S1 _1;
        private S1 _2;
        private S1 _3;
        private S1 _4;
        private S1 _5;
        private S1 _6;
        private S1 _7;
        private S1 _8;
        private S1 _9;

        public S1 this[int index]
        {
            get
            {
                if ((index < 0) || (index > 10)) // This would ideally be replaced with an actual array range check so that IndexOutOfRangeException is thrown.
                {
                    throw new ArgumentOutOfRangeException(nameof(index));
                }

                fixed (S1* pFixedBuffer = &_0)
                {
                    return pFixedBuffer[index];
                }
            }

            set
            {
                if ((index < 0) || (index > 10)) // This would ideally be replaced with an actual array range check so that IndexOutOfRangeException is thrown.
                {
                    throw new ArgumentOutOfRangeException(nameof(index));
                }

                fixed (S1* pFixedBuffer = &_0)
                {
                    pFixedBuffer[index] = value;
                }
            }
        }
    }

    [FixedBufferAttribute(typeof(S1), 10)]
    public <Buffer>e__FixedBuffer Buffer;
}

Doing this allows validating the index is within the bounds of the fixed array (which is known).

Having this support for fixed buffer structs (especially in a safe manner) would greatly improve the ability to create/work with SOA (structure of arrays) and AOSOA (array of structure of arrays).

It is probably also worth noting, that if the fixed buffers where modified to emit a backing structure in the last manner, they could be extended to support all blittable types (including those that do not have a fixed size, such as IntPtr and UIntPtr).

[Korporal]

This is a very good suggestion.

[tannergooding]

I had a very rough implementation of this proposal here: dotnet/roslyn#13064 (I should probably clean it up).

[tannergooding]

FYI. @jaredpar, we had talked about this briefly over lunch one time (I think it fits into the bigger picture of supporting blittable at a language level).

[tannergooding]

@MadsTorgersen or @gafter, for finding a champion for a proposal...

Are we able to champion them ourselves (and what would that entail) or do we have to find someone on the actual compiler team to champion it?

Does championing only involve taking it through LDM or does it also involve finding someone to implement it (even if that means implementing it yourself)?

[gafter]

@tannergooding A "champion" is a member of the LDM who believes we should do the proposal and is willing to drive the process through all of the steps to get it designed, specified, agreed, implemented, and scheduled for inclusion in a particular version of the compiler. For this particular proposal I suggest @jaredpar is likely to be interested.

[Thaina]

I was making this proposal

primitive struct as primitive dotnet/roslyn#16095

And I think it's more generalized

[tannergooding]

@Thaina, dotnet/roslyn#16095 would not cover this scenario if implemented, and likewise, this proposal would not cover your scenario if implemented.

This proposal is suggesting that fixed-sized arrays be extended to support the full-set of blittable types and types composed of blittable types. While your proposal is suggesting that literals be extended to additionally support types composed of other literal types.

[Thaina]

@tannergooding I think in general fixed size buffer just work on any primitive type?

[tannergooding]

@Thaina, not exactly...

Fixed-sized buffers, today, work on the following blittable types: bool, byte, short, int, long, sbyte, ushort, uint, ulong, float, and double. They also works on char (which is sometimes blittable) and pointers (which are always blittable). It does not work on structures composed of these types. It does not work on System.IntPtr or System.UIntPtr (which are always blittable).

Constants today work on the following blittable types: bool, byte, short, int, long, sbyte, ushort, uint, ulong, float, and double. It also works on char (sometimes blittable), decimal (a blittable struct), string (sometimes blittable), and any reference type (provided it is null). It does not work on structures composed of these types. It does not work on System.IntPtr or System.UIntPtr (the size of these types varies based on the target platform, so constants up to 32-bits could be supported, but probably not anything over). It does not work with pointers (which could only be null).

Enums today work on a subset of the constant types: byte, short, int, long, sbyte, ushort, uint, ulong.

Additionally, anything that is not a bool, byte, short, int, long, sbyte, ushort, uint, ulong, float, double, char, string, or a reference type (set to null) is not actually a constant (from the list above, this is decimal and any other structs composed of constant types), as there is no runtime support for it.

Instead, it is a static readonly field (this is how other .NET languages would see it) with a special attribute that indicates how the constructor should be called. The compiler then creates a new instance of the type (by calling the constructor) anytime the 'constant' is used (this is how decimal works today). This is contrast to the runtime constant types which are just have their literal value pushed onto the stack before calling the method.

[Thaina]

@tannergooding Thanks for your clarification

But then what I proposed is, in general, whatever the category it is, if any struct contains only things of that category, it should automatically be that category if possible

So in your case, if any struct contains only blittable type it should also be blittable type. Then if fixed size buffer look for blittable type it should just accept that struct. I just take this logic to apply with anything include const, enum, and integer

[tannergooding]

But then what I proposed is, in general, whatever the category it is, if any struct contains only things of that category, it should automatically be that category if possible

I'm not sure your proposal makes that very clear. My understanding from reading it, and the examples provided, was that it was directly related to extending constant support.

In any case, I don't think overly generic proposal are good or likely to be picked up. I would think the language team needs something fairly concrete in order to properly weigh its value. That is, proposing something like: "any type composed of types that are useable in a scenario should also be useable in that scenario" is generic and covers a ton of stuff, but it isn't likely something that can be taken to a design meeting and properly discussed (and even if it were, a ton of stuff would likely get missed).

I think this proposal covers a very specific area, with very specific examples, a fairly explicit use-case, and with clear benefits/drawbacks. I think your proposal (as I understood it) does the same.

[Thaina]

@tannergooding I was in process of trying to port that proposal into this repo. And your case is another one I look for to added for making that proposal be more generalized

But you have a good point about being specific. I might rethink about it

[tannergooding]

With the existence of ref returns and ref locals this could actually be expanded to be safe and support fixed sized buffers of any type.

@jaredpar, If I update the proposal, is this something you would be willing to champion?

[jaredpar]

@tannergooding yes. Not sure what release but agree it should, can be done.

[tannergooding]

Proposal is here: #685

[4creators]

Related but different take on the issue #749 jointly with #653 with some modifications. Implemented together with 'fixed-size-buffers' takes C# expressiveness to new level.

[Xyncgas]

I thought stack alloc T[] already works with structs with other types

[jaredpar]

Yes stackalloc works with any type that is unmanaged but fixed buffers operate on a different set of rules. They are specifically limited to a few set of primitive types. Expanding that out requires a bit of work. The latest proposal on how to make that work is captured here

https:/dotnet/csharplang/blob/main/proposals/low-level-struct-improvements.md#safe-fixed-size-buffers

[HaloFour]

IIRC that's because the size of the buffer to be allocated by stackalloc can be calculated at runtime based on the actual size of the type, whereas the total size of the fixed buffer needs to be calculated at compile time as the buffer needs to be part of the layout of the type.