Proposal: Ranges with stepping

[hez2010]

In C# 8.0, the Ranges feature (#185 ) allows us to write arr[1..5], arr[1..], arr[..3], arr[2..^2] and etc.

I propose to add a step filed which allow us to control the stepping while slicing:

Proposed syntax:

array[..] // new syntax
array[start..]
array[..end]
array[start..end]
array[start..step..end] // new syntax

step is optional, and both start and end are optional if step exists.

For example:

var arr = new [] { 1, 2, 3, 4, 5, 6 };
arr[1..] // 2, 3, 4, 5, 6
arr[1..3] // 2, 3, 4
arr[..3] // 1, 2, 3, 4
arr[1..2..5] // 2, 4, 6 (new syntax)
arr[3..-1..1] // 4, 3, 2 (new syntax)
arr[1..^1] // 2, 3, 4, 5, 6
arr[1..2..^1] // 2, 4, 6 (new syntax)
arr[^1..-2..1] // 6, 4, 2 (new syntax)
arr[1..2..] // 2, 4, 6 (new syntax)
arr[..2..] // 1, 3, 5 (new syntax)
arr[..2..^3] // 1, 3 (new syntax)
arr[..] // 1, 2, 3, 4, 5, 6 (new syntax)

This is especially useful in machine learning, such as operations as below:

[Unknown6656]

I quite like the idea of an optional stepping parameter, however, I am wondering how this could be implemented. IIRC, a range is currently only a span slice - meaning that the underlying data will not be copied. This requires the underlying data to be located in continuous block. Steps, however, will introduce "holes" in this continuous data alignment - requiring the data to be copied when using the step-parameter.
This will lead to inconsistent slicing APIs as some use a copying mechanism and others do not. I would therefore propose a slicing/stepping API to be added to LINQ.

[yaakov-h]

IIRC, a range is currently only a span slice - meaning that the underlying data will not be copied.

A range is its own datatype.

[svick]

@Unknown6656

IIRC, a range is currently only a span slice - meaning that the underlying data will not be copied.

Range is a range, it does not dictate how applying it on a collection works. Applying it on a Span<T> will create a slice, so a range with steps could not work well in that case. But applying it on an array creates a new sub-array, which does copy the data, so a range with steps could work in that case.

[msedi]

See my comments here #2510, although the whole issue is a bit bloated with emotions it has a lot in common

[fawdlstty]

The python grammar is more concise than this one

[DavidArno]

Seems a sensible addition to ranges to me. At the risk of bike shedding, the only criticism I have of it is the odd syntax of [start .. step .. end]. Is that a convention in any other language? [start .. end, step] seems more "normal" to me, but maybe that's because of experience with a different set of languages.

[hez2010]

@DavidArno this may conflict with 2d-array in C#

[DavidArno]

@hez2010,

Ah, I think you are right. Oh well, in that case, [start .. step .. end] gets my full support 😃

[miloush]

@DavidArno this seems to be similar syntax to what MATLAB has, but yes, I find it confusing too. It's not intuitive if you have not seen it before, unlike most of the language. Moreover, it contradicts the for syntax of "start; end; step". I can imagine newcomers stressing about which is where and getting it wrong.

Could we not do better?

[spydacarnage]

You may not like 2D arrays, but I'm sure a lot of other people do. A language feature isn't going to get removed just because you don't like it...

…

On Wed, 29 May 2019 at 13:39, fawdlstty ***@***.***> wrote: maybe remove 2d-array and use pythonic grammar is better.i dont't like 2d-array because it cannot be used for linq — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https:/dotnet/csharplang/issues/2562?email_source=notifications&email_token=ADIEDQISJN7MDFPWKDZ7ANDPXZ2QBA5CNFSM4HP2DQG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWPGB6A#issuecomment-496918776>, or mute the thread <https:/notifications/unsubscribe-auth/ADIEDQPI4QAUQS7VIHIATMTPXZ2QBANCNFSM4HP2DQGQ> .

[HaloFour]

Whatever the syntax would be it would have to be independent of indexer syntax, so anything that depends on being surrounded with [ and ] is out and because of this I don't think , makes any sense. You can assign a range to a variable, after all.

[MillKaDe]

Just an idea ...

[ start ; end ; step ] // similar to `for ( start ; end ; step ) ..`

.. or maybe ..

[ start .. end ; step ]

[vladd]

@MillKaDe In this case, the visual difference between rectangular array access and step is as tiny as the difference between , and ;. So the people will confuse one with another.

If we want an alternate syntax, I'd go for [1..^1 step 2].

[msedi]

I don't want to sound repetitive, but the MATLAB syntax has the most generic indexing, considering the suggested syntaxes here. I think it really would make sense to take a deeper look here:

https://www.mathworks.com/help/matlab/math/array-indexing.html

For the fast readers:

double [] A = new double[10];
double [,]B = new double[10];

Range indexing with continous index: A[0:9]
Range indexing with step: A[0:2:9]
Range indexing with unknown end: A[0:end]
Range indexing with arbitrary indexes: [0;1;2;3;5;9]
Range indexing with all element: [:]
Range indexing with arbitrary indexes with two dimensions: B[0;1;2;3;5;9, 2;2]

Can be extended to any dimension.

and many more...

As stated above that this is useful for machine learning, it's also useful for image processing for a lot of cases (resampling, feature selection, etc) or data analysis in general. In image processing two dimensions are essential, although we always put the images in one dimensional arrays. When it comes to 3D it gets even harder, you need a 3D indexer/range. For time dependent 3D volumes you even need more.

The MATLAB syntax might look complex and it is, because it allows a lot. But for simple cases it is as easy as Range but without the odd ^ syntax. C# was always more descriptive than other languages. For example instead of introducing the ref keyword you could have also used the C++ reference symbol & but you didn't. The same holds true for the indexing, but here it's the other way around. You introduced ^ where you could have used end instead ;-)

I could think of leaving the range as it is and add another "indexer" like structure that allows more cases.
The question here is about performance of the indexing (can I do it better if I do it myself).
Another question is how can the compiler distinguish between easy Range or Index syntax and between a new one. Here's some example, how we did it years ago, without the possibility that the compiler can create the structures, not fully functional and there's more to be done, but just to get the clue:

using System;

namespace ConsoleApp15
{
    class Program
    {
        static void Main(string[] args)
        {
            // Take every second element.
            // We need to do this with string parsing since we have no other way to resolve these things at compile time.
            var range = IndexedRange.Parse("0:2:end, 0:2:end");

            Matrix m1 = new Matrix(512, 512);
            Matrix m2 = m1[range];
            Matrix m3 = m1["0:2:end, 0:2:end"];
        }
    }

    public readonly struct IndexedRange
    {
        public readonly int Dimension;

        public readonly int[][] Index;

        public IndexedRange(int dimensions, int[][] index)
        {
            Dimension = index.GetLength(0);
            Index = index;
        }

        public static implicit operator IndexedRange (string indexer)=> IndexedRange.Parse(indexer);

        public int GetLength(int dim) => Index[dim].Length;
        public int[] GetRange(int dim) => Index[dim];

        public static IndexedRange Parse(string indexer)
        {
            // For brevity just return the IndexedRange.
            return new IndexedRange();
        }
    }

    public class Matrix
    {
        public int N;
        public int M;
        public float[,] Data;

        public Matrix(int n, int m)
        {
            M = m;
            N = n;
            Data = new float[M, N];
        }

        public ref float this[int m, int n] => ref Data[m, n];

        public Matrix this[in IndexedRange r]
        {
            get
            {
                if (r.Dimension != 2)
                    throw new RankException();

                int newM = r.GetLength(0);
                int newN = r.GetLength(1);

                int[] idxM = r.GetRange(0);
                int[] idxN = r.GetRange(1);

                Matrix mtx = new Matrix(newM, newN);

                for (int m = 0; m < newM; m++)
                    for (int n = 0; n < newN; n++)
                        mtx[m, n] = this[idxM[m], idxN[n]];

                return mtx;
            }
        }
    }
}

[miloush]

I agree the ^ is the most confusing part on the current range syntax, however, I don't have any better concise alternative and also I thought it's already a decided thing. I share the sentiment with @msedi though that it's not a very precedented C# syntax.

I like the @MillKaDe idea that matches for syntax with [start; end; step] and does not collide with indexers or dimensions.

The possibility of arbitrary list of indices does not really sound like a "range".

[msedi]

@miloush

The possibility of arbitrary list of indices does not really sound like a "range".

True. Compared to MATLAB everything is a vector or matrix (an array of elements) so that this is a natural feature of MATLAB. In the end, I don't really care so much about how the structures are named, I'm sure Microsoft will find a suitable name for it ;-) For me just the behavior counts ;-)

[HaloFour]

@miloush

I like the @MillKaDe idea that matches for syntax with [start; end; step] and does not collide with indexers or dimensions.

The [ and ] aren't parts of the range syntax, though. They are part of the invocation of the indexer. How would you assign such a range to a variable or pass it as a parameter?

[miloush]

@HaloFour oh that's what you meant, I see. Yeah, that's not going to work, unless [ ] would be part of the syntax, like ( ) is for tuples. I guess the brackets could be optional in indexers (in Foo((4)) sense), but this is probably wrong time and place for such discussions.

If there is no syntactical start or end for the range, then sadly I don't see much ways how step could be anywhere but in the middle. Perhaps something like 1..4 by 2 but obviously if keywords did not make it to the core syntax than I don't expect it to be justifiable for steps either.

[ghord]

F# Is an example of language with this syntax, and IMO it is intuitive, https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/sequences

[adrientetar]

Step could rather be a method, similar to (1..10).step_by(2) in Rust. A very useful part of that would be the -1 step to iterate in reverse.

[solvingproblemswithtechnology]

I'm sticking with Python syntax. We wanted this feature to be an expression and compact so: start..end..step would be nice.
In the original silicing/range issue this was already discussed.

[AmrAlSayed0]

I have 2 points to make

1. I don't think the name Range fits any more. It should be a new type called Sequence or Set (already taken) or something expressive. Indexers could be overloaded to receive this new type.
2. I think, when used, it should either return a copy of the required element or my personal preference is an enumerable that only returns the required elements when enumerated. That way no copies are made (thought their is still the overhead of the enumerator).

[CyrusNajmabadi]

I don't think the name Range fits any more. It should be a new type called Sequence or Set (already taken) or something expressive

Range seems to be a perfectly reasonable name.

I think, when used, it should either return a copy of the required element or my personal preference is an enumerable that only returns the required elements when enumerated. That way no copies are made (thought their is still the overhead of the enumerator).

this is unrelated to the language (and hence dotnet/csharplang). That is a decision for implementations that have a Range-accepting member.