Proposal: Active Patterns

[alrz]

Background

Active patterns enable you to define custom patterns to be used either on record types or non-record types. Currently, according to #206 custom patterns will be supported through is operator:

public class Cartesian(double X, double Y);

public static class Polar
{
    public static bool operator is(Cartesian c, out double R, out double Theta)
    {
        R = Math.Sqrt(c.X*c.X + c.Y*c.Y);
        Theta = Math.Atan2(c.Y, c.X);
        return c.X != 0 || c.Y != 0;
    }
}

var c = Cartesian(3, 4);
if (c is Polar(R, *)) Console.WriteLine(R);

Which is very confusing already, because:

• Invoking a static class (described in this #206 comment though)
• At first glance you'd say this is an extension operator (Defining operators via Extension Methods #5165, Extension operators #4945) but it's not.
• All those out parameters which eventually came out as Polar parameters. Ouch.

Meanwhile, active patterns in F# are pretty powerful and interestingly implemented:

Case 1: complete, with one possibility, implemented as 't -> 'r:

let (|A|) t = ... 

Case 2: partial, with one possibility, implemented as 't -> 'r option (this is the only case possible via is operator):

let (|A|_|) t =  ... 

Case 3: complete, with up to 7 possibilities, implemented as 't -> Choice<'r1, 'r2, ...>. Note that no new type will be created, pattern possibilities actually map to the Choice members.

let (|A|B|...) t = ... 

where 't is the target type and 'r is the return type.

There is no case 4: partial, with multiple possibilities. But you can define multiple partial active patterns and combine them:

let (|A|_|) t =  ... 
let (|B|_|) t =  ... 

match t with A & B -> ... // AND pattern
match t with A | B -> ... // OR pattern

According to aforementioned issue, there is no support for pattern combination. (see #5154 comment)

Proposal

This proposal supports all four cases above. Active patterns will be defined as user-defined overloads of the operator is.

operator-declarator:
 unary-operator-declarator
 binary-operator-declarator
 conversion-operator-declarator
 is-operator-declarator

is-operator-declarator:
 partial operator is ( type identifier ) returns-clause
 complete operator is ( type identifier ) returns-clause

returns-clause:
 returns inline-record-declarations

inline-record-declarations:
 inline-record-declaration
 inline-record-declarations , inline-record-declaration 

inline-record-declaration:
 identifier record-parameters _opt

Remarks

• This can use a special kind of record-parameters with an optional identifier. Because these record types will essentially be used directly in patterns, one may simply do not care about identifiers. This also keeps the operator signature concise.
• Inline records are implicitly generated as class because if more than one record were defined then all of them implicitly inherit from a common compiler-generated abstract class.
• Inline records in complete patterns will be generated as sealed class with a sealed abstract base class, as proposed in Proposal: Add completeness checking to pattern matching draft specification #188.
• null is used for when partial patterns fail, so compiler would generate an error if a null is returned throughout a complete pattern.
• To be able to define active patterns on existing types, like String they should be allowed to be defined as an extension operator.
• With active patterns, it would make sense to combine patterns in switch statements or match or is expressions, just like F# AND and OR patterns.
• With this design, parametrized active patterns are an open issue.

Examples

// case 1
public static complete operator is(Color col) returns RGB(double R, double G, double B) => 
    RGB(col.R, col.G, col.B);

// case 2
public static partial operator is(string str) returns Integer(int value) =>
    int.TryParse(str, ref int value) ? Integer(value) : null;

public static partial operator is(Cartesian c) returns Polar(double R, double Theta) => 
    c.X != 0 || c.Y != 0 ? Polar(Math.Sqrt(c.X*c.X + c.Y*c.Y), Math.Atan2(c.Y, c.X)) : null;

// case 3
public static complete operator is(int num) returns Even, Odd =>
    num % 2 == 0 ? Even() : Odd();

// case 4
public static partial operator is(int num) returns One, Two =>
    num match(case 1: One() case 2: Two() case *: null);

An example from here:

// case 3
public static complete operator is(string input)
    returns Paragraph(int, string[]), Whitespace, Word(string), Sentence(string[])
{
    var input = input.Trim();
    if(input == "") return Whitespace();
    else if (input.IndexOf(".") !=  -1) {
        var sentences = input.Split(new[] { "." }, StringSplitOptions.None);
        return Pharagraph(sentences.Length, sentences);
    } else if (input.IndexOf(" ") != -1) {
        return Sentence(input.Split(new[] { " " }, StringSplitOptions.None));
    } else return Word(input);
}

static int CountLetters(string str) => str match(
    case WhiteSpace(): 0
    case Word(x): x.Length
    case Sentence(words): words.Select(CountLetters).Sum()
    case Paragraph(*, sentences): sentences.Select(CountLetters).Sum()
);

[gafter]

This suggestion uses a number of hypothetical C# language features that it doesn't describe in enough detail to evaluate. We're going to continue working on specifying and prototyping the elements of the language one-by-one and evaluate those elements against the use cases.

[alrz]

@gafter I just revamped the original proposal, so all of the previous assumptions went away.

[gafter]

I don't know where these declared operators are proposed to be allowed. You appear to be declaring them without any enclosing type. I don't know what the scope of the names appearing in the returns clause is. You say they are classes, but then you return them as if they are expressions. As you say, "ouch".

You're defining operator is as a kind of conversion to an algebraic data type (ADT). We're working on ADTs that are not "inline" like this (see "Draft approach for algebraic data types" in #5757). Once you have that, and conversions applied implicitly in switch (third bullet under "Making the extension of switch backward-compatible" in #5757), you don't get any additional benefit from what you're calling active patterns.

[alrz]

@gafter

ADTs that are not "inline" like this

Inline records here are not a special kind, they will be translate to what #5757 suggests (depending on complete or partial declarations) eventually.

Once you have that, and conversions applied implicitly in switch

You're saying that what have proposed in #206 can be written like this?

public class Polar(double R, double Theta);
public class Cartesian(double X, double Y) {
    public static implicit operator Polar(Cartesian c) =>
        Polar(Math.Sqrt(c.X*c.X + c.Y*c.Y), Math.Atan2(c.Y, c.X));
}

var c = Cartesian(3, 4);
if (c is Polar(R, *)) Console.WriteLine(R);

So what is the point of declaring is operator anyway? (and what about closed types like int?) Maybe you don't want a specific type for Polar case, and just want to use it as a pattern. Besides, I think this is very verbose compared to what I'm calling active patterns; and will be more cumbersome when it comes to other cases that I've mentioned.

Moreover, this is exactly a port of F# active patterns, so you don't have to explicitly declare a new type when you just want to use them as pattern on other types.

[gafter]

operator is can fail, while a conversion is considered to (statically) always succeed. You've introduced the modifiers complete and partial for that distinction. I would say that operator is is closer to Scala's unapply than to F# active patterns. Since we don't have Option well integrated into the language and platform, we use a bool return type to indicate failure, while you propose to use null. Since we have a single return value in C#, we use out parameters for the data, which is the standard convention in .net (e.g. int.Parse).

There are many contexts in which types are implicitly declared in F#. We're not planning to do that. At least not right now.

[alrz]

@gafter Using null in such ways is totally acceptable in the platform and language (as unspecified/empty parameters: methodInfo.Invoke(instance, null); default value for optional parameters: void F(object obj = null); or as return value: Type.GetMethod). Well, in F# it's not. Even regular classes have to be marked with a [<AllowNullLiteralAttribute>] to be allowed to be null.

One advantage of using null over bool and out convention is that there is no out parameters! e.g. in Polar example, you have to assign every out argument even if it's a failed match. Best you can do is assign default(T) value to every single out argument. Which, over time, makes the code ugly and the developer AAAARGH.

Yes, for methods like int.TryParse or Dictionary.TryGetValue this pattern totally fits. But we're talking about an operator. Besides, we're actually addressing variables in Polar(r, θ) pattern, isn't it better to use the same syntax Polar( ... , ... ) to address them?

[alrz]

There are many contexts in which types are implicitly declared in F#

In C# as well; iterator and async methods, closures, anonymous types, to name a few. Although they are all internal but I had even suggested a way to make the latter more useful in this #3304 comment. It's inevitable if you want a robust language.

I don't know where these declared operators are proposed to be allowed. You appear to be declaring them without any enclosing type.

Actually I did describe it. In the 5th bullet. They are declared in the target type (like Cartesian) but for string or int they must be declared in another static class as an extension operator.

I don't know what the scope of the names appearing in the returns clause is. You say they are classes, but then you return them as if they are expressions.

Yes, they are record types and from examples in #206 you can create them without new keyword, right?

[gafter]

Yes, they are record types and from examples in #206 you can create them without new keyword, right?

If we do that, they would be invocable without new. But that's not what you are doing.

[alrz]

@gafter you mean I missed the parentheses for records without members like Odd or Whitespace? Since they're optional in record declarations I assumed they are optional in patterns and invocations too.

[gafter]

I don't know what you mean by "optional in record declarations". If you don't have parens in the declaration, then you don't have a record. You have an ordinary class declaration.

[alrz]

That means a lot of parens should be added to the examples above? This can be still optional in returns clause, according to the proposed syntax:

inline-record-declaration:
 identifier record-parameters _opt

you won't be able to declare ordinary classes in there anyway.

[gafter]

@alrz As I said, there are a lot of undescribed language proposals hidden in your examples.

[alrz]

@gafter I'm just saying that out parameters are not a good fit for this scenario. Since patterns are null-checked, even implicit operators that you pointed out, with #4945, are rather more readable, and you can have all the four cases above (in a more verbose way though):

// case 1
public sealed class RGB(double R, double G, double B);

public static implicit operator RGB(Color col) =>
    RGB(col.R, col.G, col.B);

// case 2
public sealed class Integer(int value);

public static implicit operator Integer(string str) =>
    int.TryParse(str, ref int value) ? Integer(value) : null;

public sealed class Polar(double R, double Theta);

public static implicit operator Polar(Cartesian c) =>
    c.X != 0 || c.Y != 0 ? Polar(Math.Sqrt(c.X*c.X + c.Y*c.Y), Math.Atan2(c.Y, c.X)) : null;

// case 3
public abstract sealed class P {
    public sealed class Even() : P;
    public sealed class Odd() : P;
}

public static implicit operator P(int num) =>
    num % 2 == 0 ? Even() : Odd();

// case 4
public abstract class N;
public sealed class One() : N;
public sealed class Two() : N;

public static implicit operator N(int num) =>
    num match(case 1: One() case 2: Two() case *: null);

So yes, they are pretty much like conversion operators. The advantages of using partial and complete operators are that when you explicitly declare partial or complete patterns, not only appropriate record types will be created for you, but also according to 4th bullet above, the completeness will be checked at compile-time. and of course, all the out parameters go away. Besides, defining classes like Even or Odd wouldn't make sense, while as a pattern, it would.

operator is can fail, while a conversion is considered to (statically) always succeed. You've introduced the modifiers complete and partial for that distinction.

Yes it can, but what if it doesn't at all (in case of a complete pattern)? Its possibility of failure (which must be determined at compile-time) is not based on the return value, it's based on the target record type completeness, in this case, Polar class, which is not even a record type in #206 example.

[alrz]

Closing in favor of extension is operators (#6136).