Add Enumerable.WhereNotNull()

[terrajobst]

The compiler has limitations what it can infer and express with respect to nullability, especially in the context of generics. When using LINQ, it's common to deal with sequences that should not contain null values. Unfortunately, there is no easy way to make the type checker aware when they got removed.

Consider this:

private static IReadOnlyList<ToolCommand> GetCommands()
{
    return typeof(Program).Assembly
                          .GetTypes()
                          .Where(t => typeof(ToolCommand).IsAssignableFrom(t) &&
                                      !t.IsAbstract && t.GetConstructor(Array.Empty<Type>()) != null)
                          .Select(t => (ToolCommand?)Activator.CreateInstance(t))
                          .OrderBy(t => t.Name)
                          .ToArray();
}

Since the Activator.CreateInstance may return a null value, the compiler correctly warns me that .OrderBy(t => t.Name) may null-ref. Sadly, inserting .Where(t => t != null) doesn't fix this because it doesn't change the return type of the Where() method. One also has to insert .Select(t => t!) to tell the type system that the values are now not-null.

private static IReadOnlyList<ToolCommand> GetCommands()
{
    return typeof(Program).Assembly
                          .GetTypes()
                          .Where(t => typeof(ToolCommand).IsAssignableFrom(t) &&
                                      !t.IsAbstract && t.GetConstructor(Array.Empty<Type>()) != null)
                          .Select(t => (ToolCommand?)Activator.CreateInstance(t))
                          .Where(t => t != null)
                          .Select(t => t !)
                          .OrderBy(t => t.Name)
                          .ToArray();
}

Proposed API

We can address this problem by introducing a method that explicitly changes the return type:

namespace System.Linq
{
    public partial class Enumerable
    {
        public static IEnumerable<T> WhereNotNull<T>(this IEnumerable<T?> source)
            where T: class;
    }
}

It also makes the common tasks of excluding null values easier.

Usage Examples

private static IReadOnlyList<ToolCommand> GetCommands()
{
    return typeof(Program).Assembly
                          .GetTypes()
                          .Where(t => typeof(ToolCommand).IsAssignableFrom(t) &&
                                      !t.IsAbstract && t.GetConstructor(Array.Empty<Type>()) != null)
                          .Select(t => (ToolCommand?)Activator.CreateInstance(t))
                          .WhereNotNull()
                          .OrderBy(t => t.Name)
                          .ToArray();
}

[ghost]

Tagging subscribers to this area: @eiriktsarpalis
See info in area-owners.md if you want to be subscribed.

[GrabYourPitchforks]

Is there any way to remove the where T : class restriction? I can imagine scenarios where I want to use this within a helper function, but my helper function isn't constrained to where T : class. In cases where T is a non-nullable value type the method's implementation would essentially be "return this".

public static class Extensions
{
    public static IEnumerable<T> WhereNotNull<T>(this IEnumerable<T?> source) => throw null;
}

Behavior:

• If T is a reference type, filter out items which are null.
• If T is a non-nullable value type, return source.
• If T is a nullable value type, filter out items which are null, but since you have to preserve the return type you'd still technically be returning IEnumerable<Nullable<T>>. It's not the end of the world. If you want to make this a little better then you can add another overload, as shown below.

public static class Extensions
{
    public static IEnumerable<T> WhereNotNull<T>(this IEnumerable<T?> source) => throw null;
    public static IEnumerable<T> WhereNotNull<T>(this IEnumerable<T?> source) where T : struct => throw null;
}

I confirmed in a sample app that this syntax is legal and that the nullability feature behaves as expected when it sees calls to these APIs.

[mburbea]

What about just using OfType<T>()? It removes null entries and casts a IEnumerable<T?> into IEnumerable<T>.

https://sharplab.io/#v2:C4LgTgrgdgNAJiA1AHwAICYCMBYAUKgBgAJVMA6AGQEsoBHAbjwwKfQGYiBvPI3k9dFx58RANwCGYIgGciAXiJQApgHcSmAgH4A2gF1OAInEGYUCABtzAX0a4RI0gE4AFNLIB5AGYAVAJ4AHJQAeUgIAPmcASk0yAGUlcyUAY2BXOTC3CiUoAHNgAAtIsgBhAHtoVMjI2xErPCsgA===

[FiniteReality]

@mburbea That's still an extra method call:

var values = mySequence
    .Where(x => x != null)
    .OfType<T>();
// or
var values = mySequence
    .Where(x => x != null)
    .Select(x => x!);

The goal is:

var values = mySequence
    .WhereNotNull();

EDIT: So it turns out I shouldn't be reading issues shortly after I've woken up 🙃
I checked the implementation and only saw if (value is TResult result) and completely forgot that also adds an implicit null check...

[stephentoub]

Dup of #30381