Law testing + Traits update + RWST

Laws testing

Functors, applicatives, and monads all have laws that make them what they are. Some think that Map just means the same as Map in mathematics, when in fact functors are more constrained and are strucutre-preserving.

The full set of laws are:

Functors

• Identity law
• Composition law
• Structure preservation law

Applicatives

• Identity law
• Composition law
• Homomorphism law
• Interchange law
• Applicative-functor law

Monads

• Left-identity law
• Right-identity law
• Associativity law

When you write your own functors/applicatives/monads you are expected to honour these laws. In reality it's pretty hard to go wrong if you just follow the type-signatures and implement the traits in the most obvious way, but still, it is possible to make a mistake and some of the guarantees of the traits start to fail.

assert

The type-system isn't able to enforce many of the laws above, so we need to do it ourselves. I have now made that process much easier. If you implement a monadic type (using the new traits system) then can simply call:

MonadLaw<M>.assert();

Where M is your monad trait implementation type.

For example, this tests that Option complies with all of the laws listed above.

MonadLaw<Option>.assert();

If your type isn't a monad, but is an applicative, then you can call:

ApplicativeLaw<M>.assert();

And if your type isn't an applicative, but is a functor, then you can call:

var mx = M.Pure(123);
FunctorLaw<M>.assert(mx);

Functors don't know how to instantiate new functors (unlike applicatives and monads), so you must provide an instance to the assert function.

Note that, if your type is a monad and you call MonadLaw<M>.assert, you do not need to call ApplicativeLaw<M>.assert or FunctorLaw<M>.assert. Those will be tested automatically.

validate

The assert functions listed above are perfect for unit-tests, but you can call validate instead. It will return a Validation<Error, Unit> which will collect a set of failures for any failing laws.

var result = MonadLaw<Option>.validate();

Equality

The functions that test that the laws hold need to be able to test equality of functor/monad/applicative values. Unfortunately, not all functors/applicatives/monads support equality. Types like Reader, for example, are computations (not values), and so must be evaluated to extract a concrete value. The generic traits don't know how to evaluate them to extract the values.

And so, for types that have no valid Equals implementation, you must provide an equality function to assert and validate.

Here's an example for Eff<int>:

  bool eq(K<Eff, int> vx, K<Eff, int> vy) => 
      vx.Run().Equals(vy.Run());
  
  MonadLaw<Eff>.assert(eq);

It's pretty simple, it just runs the effect and compares the result.

Examples

You can look at the unit-tests for all of the functor/applicative/monad types in language-ext:

• Functors
• Applicatives
• Monads

Future

• More laws tested for more traits!
• Potentially add these assertions to a Roslyn analyzer (if anyone wants to try, please do!)

Removal of Alternative and SemiAlternative

I have removed Alternative and SemiAlternative traits. I really disliked the name SemiAlternative (which was a combination of SemigroupK and Applicative. I was OK with Alternative (MonoidK and Applicative) but it doesn't make sense without its semigroup partner. So, for now, we will only have SemigroupK and MonoidK (semigroup and monoid that work for K<F, A> rather than A).

I'm still refining the types and am not 100% happy with this, but am short of ideas for better names or approaches. Feel free to let me know what you think.

Pure to pure

The computation types: Reader, ReaderT, State, StateT, Writer, and WriterT have all had their module Pure function renamed to pure -- as it's not strictly a constructor, it simply lifts a pure value into those computations.

RWST

Reader/Write/State monad-transformer. This is still WIP but it should be usable. It just doesn't have all the bells and whistles yet.

Map and Apply standardisation

Now that language-ext has Functor and Applicative traits, there are a number of extension methods[1][2] and module functions[3][4] that work with those traits.

Those all return the abstract K<F, A> type rather than the more specialised types that derive from K<F, A>. So, to get over that, I had previously added bespoke extensions for types like Either, Option, etc. that would call the generic behaviours and then cast back to the specialised type.

Unfortunately, over the years there's been an inconsistent application of these extension methods to the various functor/applicative types. So, I have now made all functors/applicatives support the exact same set of Map, Apply, Action extensions as well as the exact same set of map, apply, action functions in the Prelude.

That means for some types you may have lost some extensions/functions and for some they have gained. But, they are now all consistent, so going forward at least there's no second guessing.

One big change is that the multiple operand Apply has gone, so you can't do this now:

var mf = Some((int x, int y) => x + y);
var mx = Some(100);
var my = Some(100);

var mr = mf.Apply(mx, my);

You must fluently chain calls to Apply (which is just what it did behind the scenes anyway):

var mr = mf.Apply(mx).Apply(my);

The variants of Map and Apply that took multi-argument Func delegates as their first argument are now all only available as generic extensions and only accept a single operand, e.g:

 public static K<AF, Func<B,Func<C, D>>> Apply<AF, A, B, C, D>(
    this K<AF, Func<A, B, C, D>> mf, 
    K<AF, A> ma)
    where AF : Applicative<AF> =>
    AF.Apply(AF.Map(curry, mf), ma);

public static K<AF, Func<B,Func<C, Func<D, E>>>> Apply<AF, A, B, C, D, E>(
    this K<AF, Func<A, B, C, D, E>> mf, 
    K<AF, A> ma)
    where AF : Applicative<AF> =>
    AF.Apply(AF.Map(curry, mf), ma);

// ... etc.  up to 10 parameter Func delegates

Note, how the multi-parameter Func delegates turn into single parameter curried Func results.

What that means is each bespoke extension method (say, for Option, like below) just needs to handle Func<A, B> and not all variants of n-argument function. All chained Apply calls will eventually bake down to a concrete type being returned, removing the need to call .As() afterwards.

public static Option<B> Apply<A, B>(this Option<Func<A, B>> mf, K<Option, A> ma) =>
    Applicative.apply(mf, ma).As();

public static Option<B> Apply<A, B>(this K<Option, Func<A, B>> mf, K<Option, A> ma) =>
    Applicative.apply(mf, ma).As();

The Prelude now has map, apply, and action functions for all applicatives. I think it's worth pointing out that map is particularly useful in making the use of applicatives a bit easier. Previously, if you needed to lift up an anonymous lambda, you'd need to call fun(x => ...) to make the delegate available:

var mr = fun((int x, int y) => x + y)
             .Map(mx)
             .Apply(my);

Now, with the map overrides, you can avoid the initial lifting of the function and perform the lift and map all in one:

var mr = map((int x, int y) => x + y, mx)
             .Apply(my);

Of course, the tuple-based approach is also available for all applicatives:

var mr = (mx, my).Apply((x, y) => x + y);

This however returns the generic K<F, A> and needs .As() to make it concrete.

Error and catching updates

Based on this discussion the Error type has had a few changes:

• The bespoke Equals operators have been removed. Meaning that all Error types use the built-in record structural equality.
• Is<E>() where E : Exception, the test for an exceptional error contained within the Error, has been renamed to HasException<E>().
• IsType<E>() where E : Error has been added to test if this contains an E. It's like this is E, but because this might contain many-errors, it checks for the existence of any Error of type E.

The Catch extensions and the @catch combinators have been updated:

• To fix some obvious bugs!
• To flip the operands when matching, so the predicate argument is on the left-hand-side where appropriate.
• Added support for error-codes in Catch (previously missing)

Previous releases:

I did some minor releases that didn't get release notes, so here's a quick summary:

• notFollowedBy in Parsec reports the correct position
• Explicit (useAsync) and implicit (via use) support for IAsyncDisposable resources
• SomeRun overrides for Eff weren't disposing of the EnvIO properly
• Fin.Apply matching fix
• LiftM support for StreamT:
  • Previously we could Lift(IAsyncEnumerable<A>) or Lift(IEnumerable<A>) to return a StreamT<M, A>
  • Now we can LiftM(IAsyncEnumerable<K<M, A>>) and LiftM(IEnumerable<K<M, A>>) to also return a StreamT<M, A>

IO eagerness fix

The recent IO refactor allowed for eager evaluation of pure lifted values, this causes stack-overflows and other side-issues with StreamT and other infinite recursion techniques. This minor release fixes that.

Fin, Try, and IO applicative behaviours + minor fixes

A question was asked about why Fin doesn't collect errors like Validation when using applicative Apply, seeing as Error (the alternative value for Fin) is a monoid. This seems reasonable and has now been added for the following types:

• Fin<A>
• FinT<M, A>
• Try<A>
• TryT<M, A>
• IO<A>
• Eff<A>
• Eff<RT, A>

I extended this for Try, IO, and Eff because their alternative value is also Error, so it makes sense in applicative scenarios.

The IO monad has also had its Apply internals updated to work with the new underlying IOAsync, IOSync, ... types. It now uses regular Task.WhenAll instead of forking to achieve concurrent execution. To achieve genuine parallel execution you can still call Fork on the operands.

IO has also had its Zip functions updated to use Apply instead of forking for the same reasons. That means forking of an IO operation is a choice by the programmer rather than something that is imposed in certain functions.

Because Eff<RT, A> and Eff<A> are both based on the IO monad they're also updated to this new behaviour.

Domain Type traits

Minor fixes to the Domain-Type interfaces:

In Locus<SELF, SCALAR, DISTANCE>, I have reordered the SCALAR and DISTANCE types and renamed SCALAR to SCALAR_DISTANCE; that means the new type is: Locus<SELF, DISTANCE, DISTANCE_SCALAR> -- so it's obvious that it's a scalar value for the distance rather than SELF. Also, removed Origin and now rely on the AdditiveIdentity from IAdditiveIdentity.

Credit card validation sample

Added a new Credit Card Validation sample, this is the example built in my Higher Kinds in C# series with all of the data-types converted to use the Domain Type traits.

IO performance improvements

In one of the proposals leading up to the big v5 refactor, I discussed the idea of using SpinWait as a lightweight waiting technique to avoid the use of the async/await machinery everywhere. I also mentioned that the idea might be too primitive. Well, it was.

So, I have modified the internals of the IO monad (which is where all async code lives now) to have four possible states: IOSync, IOAsync, IOPure, and IOFail. These are just types derived from IO (you never see them).

The idea is that any actual asynchronous IO will just use the regular async/await machinery (internally in IOAsync), any synchronous IO will be free of async/await (in IOSync), and any pure or failure values will have a super simplified implementation that has no laziness at all and just can pre-compute.

The TestBed.Web sample with the TestBed.Web.Runner NBomber test now runs both the sync and async versions with exactly the same performance and with no thread starvation; and without any special need to fork the IO operation on the sync version.

I consider that a big win which will allow users to avoid async/await entirely (if they so wish), one of the goals of 'Drop all Async variants' proposal.

app.MapGet("/sync", 
    () => {
        var effect = liftIO(async () =>
                            {
                                await Task.Delay(1000);
                                return "Hello, World";
                            });

        return effect.Run();
    });

app.MapGet("/async", 
    async () => {
        var effect = liftIO(async () =>
                            {
                                await Task.Delay(1000);
                                return "Hello, World";
                            });
        
        return await effect.RunAsync();
    });

Issue fix

• AsyncEnumerable extensions introduced in 5.0.0-beta-08 conflict with System.Linq.Async #1360 - thanks to @guythetechie

Domain-types update

Domain-types are still a relatively nascent idea for v5 that I am playing around with. I wouldn't use them in anger unless you're ok with updating your code when I change them. Because I will!

Anyway, the updates are:

• Improved documentation
• Changed their inheritance hierarchy so don't see so many where constraints
• DomainType<SELF, REPR> has a base DomainType<SELF>. The derived domain types (Identifier, Locus, VectorSpace, and Amount) inherit from DomainType<SELF>.
  • So, they don't need to specify a REPR type, simplifying the traits.
  • It does however mean that you will need to specify the DomainType<SELF, REPR> type as well as whatever derived domain type to gain a constructable value (see the Length example later)
• Changed From in DomainType<SELF, REPR> to return a Fin<SELF. This allows for validation when constructing the domain-type.
  • Because this isn't always desired, you can use an explicitly implemented interface method to override it.
    • See the Length example below
• Dropped the Quantity domain-type for now
  • I need to find a better approach with C#'s type system

public readonly record struct Length(double Value) :
    DomainType<Length, double>, //< note this is now needed, because Amount only impl DomainType<Length>
    Amount<Length, double> 
{
    public static Length From(double repr) => 
        new (repr);
    
    public double To() =>
        Value;

    // explicitly implemented `From`, so it's not part of the Length public interface
    static Fin<Length> DomainType<Length, double>.From(double repr) =>
        new Length(repr);

    public static Length operator -(Length value) => 
        new (-value.Value);

    public static Length operator +(Length left, Length right) => 
        new (left.Value + right.Value);

    public static Length operator -(Length left, Length right) => 
        new (left.Value - right.Value);

    public static Length operator *(Length left, double right) => 
        new (left.Value * right);

    public static Length operator /(Length left, double right) => 
        new (left.Value / right);

    public int CompareTo(Length other) => 
        Value.CompareTo(other.Value);

    public static bool operator >(Length left, Length right) =>
        left.CompareTo(right) > 0;

    public static bool operator >=(Length left, Length right) => 
        left.CompareTo(right) >= 0;

    public static bool operator <(Length left, Length right) => 
        left.CompareTo(right) < 0;

    public static bool operator <=(Length left, Length right) => 
        left.CompareTo(right) <= 0;
}

StreamT merging and zipping + parsing updates

This release follows on from the last release (which featured the new StreamT type): we can now merge and zip multiple streams. There's also an update to the Prelude.parse* functions (like the Option<int> returning parseInt).

Merging

Merging multiple StreamT streams has the following behaviours:

• async & async stream: the items merge and yield as they happen
• async & sync stream: as each async item is yielded, a sync item is immediately yielded after
• sync & async stream: each sync item is yielded immediately before each async item is yielded
• sync & sync stream: each stream is perfectly interleaved

If either stream finishes first, the rest of the stream that still has items keeps yielding its own items.

There is an example of merging on in the Streams sample:

public static class Merging
{
    public static IO<Unit> run =>
        example(20).Iter().As() >>
        emptyLine;

    static StreamT<IO, Unit> example(int n) =>
        from v in evens(n) & odds(n)
        where false
        select unit;
    
    static StreamT<IO, int> evens(int n) =>
        from x in Range(0, n).AsStream<IO>()
        where isEven(x)
        from _ in magenta >> write($"{x} ")
        select x;

    static StreamT<IO, int> odds(int n) =>
        from x in Range(0, n).AsStream<IO>()
        where isOdd(x)
        from _ in yellow >> write($"{x} ")
        select x;
    
    static bool isOdd(int n) =>
        (n & 1) == 1;

    static bool isEven(int n) =>
        !isOdd(n);
}

This creates two streams: odds and evens and them merges them into a single stream using:

evens(n) & odds(n)

The output looks like this:

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

With differing colours depending on whether odd or even.

You can merge any number of streams with the & operator, or concatenate streams with the + operator.

Other ways to merge:

var s = stream1.Merge(stream2, ...);
var s = StreamT.merge(stream1, stream2, ...);
var s = merge(stream1, stream2, ...);  // in the Prelude

Zipping

You can zip up to four streams and the result is a stream of tuples.

Obviously, to create a tuple all of the streams need to have yielded a value and so must wait for them on each stream. But, be sure that the async streams are running independently and not blocking before being tupled.

That also means the length of the tuple stream is clamped to the shortest stream length.

Useful aspects of zipping sync and async is that you can pair async events with identifiers:

For example, imagine you have a stream of messages coming from an external source (async):

static StreamT<IO, Message> messages =>
    // create an async message stream

And a stream of natural numbers, playing the role of an identifier (sync):

static StreamT<IO, long> ids =>
    Range(0, long.MaxValue).AsStream<IO>();

Then you can tag each message with a unique identifier like so:

static StreamT<IO, (long Id, Message)> incoming =>
    ids.Zip(messages);

There's also an example in the Streams sample. It's similar to the merging example, except, instead of interleaving the odd and even streams, it tuples them:

public static class Zipping
{
    public static IO<Unit> run =>
        from x in example(10).Iter().As()
        select unit;

    static StreamT<IO, Unit> example(int n) =>
        from v in evens(n).Zip(odds(n))
        from _ in writeLine(v)
        where false
        select unit;

    static StreamT<IO, int> evens(int n) =>
        from x in Range(0, n).AsStream<IO>()
        where isEven(x)
        select x;

    static StreamT<IO, int> odds(int n) =>
        from x in Range(0, n).AsStream<IO>()
        where isOdd(x)
        select x;
    
    static bool isOdd(int n) =>
        (n & 1) == 1;

    static bool isEven(int n) =>
        !isOdd(n);
}

The output looks like this:

(0, 1)
(2, 3)
(4, 5)
(6, 7)
(8, 9)

There are no operators for zipping (because operators don't support generics), these are the options:

var s = stream1.Zip(stream2, .., stream4);
var s = StreamT.zip(stream1, .., stream4);
var s = zip(stream1, .., stream4);  // in the Prelude

Parsing

parseInt and its variants (parseLong, parseGuid, etc.) all return Option<A> where A is the type being generated from the parse. With the advent of the trait-types - in particular the Alternative<M> trait - we can now parse to any type that implements the Alternative<M> trait.

Alternative<M> is like a monoid for higher-kinds and it has an Empty<A>() function that allows us to construct a 'zero' version of higher-kind (think None in Option, but also Errors.None in types with an alternative value of Error).

The original parse* functions (that return Option), remain unchanged, but there is now an extra overload for each variant that takes the trait-implementation type as a generic parameter:

Here's the original parseInt with the new parseInt<M>:

public static Option<int> parseInt(string value) =>
    Parse<int>(int.TryParse, value);

public static K<M, int> parseInt<M>(string value)
    where M : Alternative<M> =>
    Parse<M, int>(int.TryParse, value);

To see how this helps, take a look at the run function from the SumOfSquares example:

Before:

public static class SumOfSquares
{
    public static IO<Unit> run =>
        from _ in writeLine("Enter a number to find the sum of squares")
        from s in readLine
        from n in parseInt(s).Match(Some: IO.pure, None: IO.fail<int>("expected a number!"))
        from x in example(n).Iter().As()
        select unit;

    ..
}

After

public static class SumOfSquares
{
    public static IO<Unit> run =>
        from _ in writeLine("Enter a number to find the sum of squares")
        from s in readLine
        from n in parseInt<IO>(s)
        from x in example(n).Iter().As()
        select unit;
   
    ..
}

We lift directly into the IO monad instead of into Option first (only to have to match on it straight away).

Obviously, the default alternative value might not be right, and so you can then use the | operator to catch the failure:

public static class SumOfSquares
{
    public static IO<Unit> run =>
        from _ in writeLine("Enter a number to find the sum of squares")
        from s in readLine
        from n in parseInt<IO>(s) | IO.fail<int>("expected a number!")
        from x in example(n).Iter().As()
        select unit;

    ..
}

Instead of raising an error, you could also provide a default if the parse fails:

parseInt<IO>(s) | IO.pure(0)

This is nice and elegant and, I think, shows the usefulness of the traits. I wouldn't mind removing the Option bearing parse* functions, but I don't think it hurts to keep them in.

As always, any questions or comments, please reply below.

New features: Monadic action operators, StreamT, and Iterable

Features:

• Monadic action operators
• New Iterable monad
• New StreamT monad-transformer
  • Support for recursive IO with zero space leaks
• Typed operators for |
• Atom rationalisation
• FoldOption
• Async helper
• IAsyncEnumerable LINQ extensions

Monadic action operators

The monadic action operator >> allow the chaining of two monadic actions together (like a regular bind operation), but we discard the result of the first.

A good example of why we want this is the LINQ discards that end up looking like BASIC:

public static Game<Unit> play =>
    from _0 in Display.askPlayerNames
    from _1 in enterPlayerNames
    from _2 in Display.introduction
    from _3 in Deck.shuffle
    from _4 in playHands
    select unit;

We are always discarding the result because each operation is a side-effecting IO and/or state operation.

Instead, we can now use the monadic action operator:

public static Game<Unit> play =>
    Display.askPlayerNames >>
    enterPlayerNames       >>
    Display.introduction   >>
    Deck.shuffle           >>
    playHands;

Here's another example:

static Game<Unit> playHands =>
    from _   in initPlayers >>
                playHand >>
                Display.askPlayAgain
    from key in Console.readKey
    from __  in when(key.Key == ConsoleKey.Y, playHands)
    select unit;

In the above example you could just write:

static Game<Unit> playHands =>
    initPlayers >>
    playHand >>
    Display.askPlayAgain >>
    from key in Console.readKey
    from __  in when(key.Key == ConsoleKey.Y, playHands)
    select unit;

It's really down to taste. I like things to line up!

Because operators can't have generics, we can only combine operands where the types are all available. For example:

public static IO<A> operator >> (IO<A> lhs, IO<A> rhs) =>
    lhs.Bind(_ => rhs);

But, we can de-abstract the K versions:

public static IO<A> operator >> (IO<A> lhs, K<IO, A> rhs) =>
    lhs.Bind(_ => rhs);

And, also do quite a neat trick with Unit:

public static IO<A> operator >> (IO<A> lhs, IO<Unit> rhs) =>
    lhs.Bind(x => rhs.Map(_ => x));

That propagates the result from the first operation, runs the second (unit returning) operation, and then returns the first-result. This is actually incredibly useful, I find.

Because, it's not completely general case, there will be times when your types don't line up, but it's definitely useful enough, and can drastically reduce the amount of numbered-discards! I also realise some might not like the repurposing of the shift-operator, but I chose that because it's the same operator used for the same purpose in Haskell. Another option may have been to use &, which would be more flexible, but in my mind, less elegant. I'm happy to take soundings on this.

The CardGame sample has more examples.

New Iterable monad

The EnumerableM type that was a wrapper for IEnumerable (that enabled traits like foldable, traversable, etc.) is now Iterable. It's now more advanced than the simple wrapper that existed before. You can Add an item to an Iterable, or prepend an item with Cons and it won't force a re-evaluation of the lazy sequence, which I think is pretty cool. The same is true for concatenation.

Lots of the AsEnumerable have been renamed to AsIterable (I'll probably add AsEnumerable() back later (to return IEnumerable again). Just haven't gotten around to it yet, so watch out for compilation failures due to missing AsEnumerable.

The type is relatively young, but is already has lots of features that IEnumerble doesn't.

New StreamT monad-transformer

If lists are monads (Seq<A>, Lst<A>, Iterable<A>, etc.) then why can't we have list monad-transformers? Well, we can, and that's StreamT. For those that know ListT from Haskell, it's considered to be done wrong. It is formulated like this:

   K<M, Seq<A>>

So, the lifted monad wraps the collection. This has problems because it's not associative, which is one of the rules of monads. It also feels instinctively the wrong way around. Do we want a single effect that evaluates to a collection, or do we want a collection of effects? I'd argue a collection of effects is much more useful, if each entry in a collection can run an IO operation then we have streams.

So, we want something like this:

Seq<K<M, A>>

In reality, it's quite a bit more complicated than this (for boring reasons I won't go into here), but a Seq of effects is a good way to picture it.

It's easy to see how that leads to reactive event systems and the like.

Anyway, that's what StreamT is, it's ListT done right.

Here's a simple example of IO being lifted into StreamT:

StreamT<IO, long> naturals =>
    Range(0, long.MaxValue).AsStream<IO>();

static StreamT<IO, Unit> example =>
    from v in naturals
    where v % 10000 == 0
    from _ in writeLine($"{v:N0}")
    where false
    select unit;

So, naturals is an infinite lazy stream (well, up to long.MaxValue). The example computation iterates every item in naturals, but it uses the where clause to decide what to let through to the rest of the expression. So, where v % 10000 means we only let through every 10,000th value. We then call Console.writeLine to put that number to the screen and finally, we do where false which forces the continuation of the stream.

The output looks like this:

10,000
20,000
30,000
40,000
50,000
60,000
70,000
80,000
90,000
100,000
110,000
120,000
130,000
140,000
150,000
...

That where false might seem weird at first, but if it wasn't there, then we would exit the computation after the first item. false is essentially saying "don't let anything thorugh" and select is saying "we're done". So, if we never get to the select then we'll keep streaming the values (and running the writeLine side effect).

We can also lift IAsyncEnumerable collections into a StreamT (although you must have an IO monad at the base of the transformer stack -- it needs this to get the cancellation token).

static StreamT<IO, long> naturals =>
    naturalsEnum().AsStream<IO, long>();

static StreamT<IO, Unit> example =>
    from v in naturals
    from _ in writeLine($"{v:N0}")
    where false
    select unit;

static async IAsyncEnumerable<long> naturalsEnum()
{
    for (var i = 0L; i < long.MaxValue; i++)
    {
        yield return i;
        await Task.Delay(1000);
    }
}

We can also fold and yield the folded states as its own stream:

static StreamT<IO, int> naturals(int n) =>
    Range(0, n).AsStream<IO>();

static StreamT<IO, Unit> example(int n) =>
    from v in naturals(n).FoldUntil(0, (s, x) => s + x, (_, x) => x % 10 == 0)
    from _ in writeLine(v.ToString())
    where false
    select unit;

Here, FoldUntil will take each number in the stream and sum it. In its predicate it returns true every 10th item. We then write the state to the console. The output looks like so:

0
55
210
465
820
1275
1830
2485
3240
4095
..

Support for recursive IO with zero space leaks

I have run the first StreamT example (that printed every 10,00th entry forever) to the point that this has counted over 4 billion. The internal implementation is recursive, so normally we'd expect a stack-overflow, but for lifted IO there's a special trampoline in there that allows it to recurse forever (without space leaks either). What this means is we can use it for long lived event streams without worrying about memory leaks or stack-overflows.

To an extent I see StreamT as a much simpler pipes system. It doesn't have all of the features of pipes, but it is much, much easier to use.

To see more examples, there's a 'Streams' project in the Samples folder.

Typed operators for |

I've added lots of operators for | that keeps the .As() away when doing failure coalescing with the core types.

Atom rationalisation

I've simplified the Atom type:

• No more effects inside the Swap functions (so, no SwapEff, or the like).
• Swap doesn't return an Option any more. This was only needed for atoms with validators. Instead, if a validator fails then we just return the original unchanged item. You can still use the Changed event to see if an actual change has happened. This makes working with atoms a bit more elegant.
• New Prelude functions for using atoms with IO:
  • atomIO to construct an atom
  • swapIO to swap an item in an atom while in an IO monad
  • valueIO to access a snapshot of the Atom
  • writeIO to overwrite the value in the Atom (should be used with care as the update is not based on the previous value)

FoldOption

New FoldOption and FoldBackOption functions for the Foldable trait. These are like FoldUntil, but instead of a predicate function to test for the end of the fold, the folder function itself can return an Option. If None the fold ends with the latest state.

Async helper

• Async.await(Task<A>) - turns a Task into a synchronous process. This is a little bit like Task.Result but without the baggage. The idea here is that you'd use it where you're already in an IO operation, or something that is within its own asynchronous state, to pass a value to a method that doesn't accept Task.
• Async.fork(Func<A>, TimeSpan) and `Async.fork(Func<Task>, TimeS...

IO and effects refactoring

In the last release I wrote this:

"Because the traits are all interfaces we can't use operator | for error handling"

I realised that because @catch creates a temporary struct (the various Catch* record structs) that I could attach operator | to those types and make @catch work for K<F, A> types that are Fallible<F> or Fallible<E, F>.

This took me down a massive rabbit hole! So this release has quite a few changes. If you're using v5 then you'll need to pay attention. And, if you're using runtimes, you'll really need to pay attention!

@catch

As, many of you know, in v4 we can @catch errors raised in the Eff<RT, A> and Eff<A> types by using the | operator like a coalescing operator. For example:

   public static Eff<RT, Unit> main =>
        from _1 in timeout(60 * seconds, longRunning)
                 | @catch(Errors.TimedOut, unit)
        from _2 in Console<Eff<RT>, RT>.writeLine("done")
        select unit;

This imposes a time-limit on the longRunning operation, which throws a TimedOut error if it doesn't finish in time. It then catches the timeout and continues safely by returning a default value of unit.

There were a number of types that @catch (depending on the overload) could create:

• CatchValue<A> - for returning default values (as above)
• CatchValue<E, A> - for returning default values (with generic error type)
• CatchError - for returning an alternative error
• CatchError<E> - for returning an alternative error (with generic error type)
• CatchIO<A> - for returning and lifting an IO<A> as the result
• CatchIO<E, A> - for returning and lifting an IO<A> as the result (with generic error type)
• CatchM<M, A> - for returning and lifting an K<M, A> as the result
• CatchM<E, M, A> - for returning and lifting an K<M, A> as the result (with generic error type)

Each one carries a predicate function and an action function. If the predicate returns true for the error raised then the action is run, otherwise the result it left alone. This means a chain of | catch(...) operations can effectively pattern match the errors raised.

Most importantly: the arguments to @catch can make the inference of the generic parameters automatic, so we don't have to manually write @catch<Error, Eff, int>(...) -- this makes catch usable.

Back to the idea that we have a Fallible<E, F> (and Fallible<F> which is equivalent to Fallible<Error, F>). Because, operator declarations can't have generic parameters, all generic parameters must come from the type.

To be able to leverage the Fallible<E, F> trait then we need F (the trait type), E (the error type), and A (the bound value type):

public interface Fallible<E, F>
{
    public static abstract K<F, A> Catch<A>(
        K<F, A> fa,
        Func<E, bool> Predicate, 
        Func<E, K<F, A>> Fail);

   ...
}

Only one of the Catch* record structs has all of those generics:

• CatchM<E, M, A> - for returning and lifting an K<M, A> as the result (with generic error type)

So, that's the only type that can support an operator | that can work with Fallible<E, M>:

public readonly record struct CatchM<E, M, A>(Func<E, bool> Match, Func<E, K<M, A>> Value)
    where M : Fallible<E, M>
{
    public static K<M, A> operator |(K<M, A> lhs, CatchM<E, M, A> rhs) =>
        lhs.Catch(rhs.Match, rhs.Value);
}

So, I had a couple of options:

1. Add support only to CatchM and leave the other Catch* types as non-Fallible supporting
2. Remove all of the other Catch* types that can't support Fallible

Option 1 would mean that some usages of @catch would work with Eff<A> but not K<Eff, A>. This felt unsatisfactory.
Option 2 would mean that some of the convenience @catch overrides would have to be removed. So, you couldn't write this anymore:

   @catch(Errors.TimedOut, unit)

You'd have to write (one of):

   @catch(Errors.TimedOut, SuccessEff(unit))
   @catch(Errors.TimedOut, pure<Eff, Unit>(unit))
   @catch(Errors.TimedOut, unitEff)  // unitEff is a static readonly of SuccessEff

Option 2 is the option I've gone with. The reasons for this are primarily for consistency between the concrete types (Eff<A>) and their abstract pairs (K<Eff, A>), but also...

Every single Fallible type gets to use @catch!

So, previously, @catch only worked for Eff<RT, A>, Eff<A>, and IO<A>. It now works for:

• IO<A>
• Eff<RT, A>
• Eff<A>
• Either<L, R>
• EitherT<L, M, R>
• Fin<A>
• FinT<M, A> - more on this later
• Option<A>
• OptionT<M, A>
• Try<A>
• TryT<A>
• Validation<F, A>
• ValidationT<F, M, A>

So now all Fallible types get to use @catch and they all get to use the same set (well, some are specifically for the Error type, like @expected and @exceptional, but other than that they're all the same).

Things to note about this change:

• Because @catch is now entirely generic and based around Fallible types, the | operator can only return K<M, A>, so you may need to use .As() if you need to get back to the concrete type.
• For catch-all situations, it's better to not use @catch at all, unless you need access to the error value.

MonadIO refactor

The generalisation of catching any errors from Fallible led to me doing some refactoring of the Eff<RT, A> and Eff<A> types. I realised not all errors were being caught. It appeared to be to do with how the IO monad was lifted into the Eff types. In the Monad<M> trait was a function: WithRunInIO which is directly taken from the equivalent function in Haskell's IO.Unlift package.

I decided that was too complicated to use. Every time I used it, it was turning my head inside out, and if it's like that for me then it's probably unusable for others who are not fully aware of unlifting and what it's about. So, I removed it, and UnliftIO (which depended on it).

I have now moved all lifting and unlifting functions to MonadIO:

public interface MonadIO<M>
    where M : MonadIO<M>, Monad<M>
{
    public static virtual K<M, A> LiftIO<A>(IO<A> ma) =>
        throw new ExceptionalException(Errors.LiftIONotSupported);

    public static virtual K<M, A> LiftIO<A>(K<IO, A> ma) =>
        M.LiftIO(ma.As());

    public static virtual K<M, IO<A>> ToIO<A>(K<M, A> ma) =>
        throw new ExceptionalException(Errors.UnliftIONotSupported);

    public static virtual K<M, B> MapIO<A, B>(K<M, A> ma, Func<IO<A>, IO<B>> f) =>
        M.ToIO(ma).Bind(io => M.LiftIO(f(io)));
}

Monad<M> inherits MonadIO<M>, which isn't how it should be, but because of the limitations of C#'s type system we have all monads expose the MonadIO functionality (otherwise monad-transformers won't work). I'm still thinking through alternative approaches, but I'm a little stumped at the moment. So, for now, there are default implementations for LiftIO and ToIO that throw exceptions. You only implement them if your type supports IO.

• LiftIO as most will know, will lift an IO<A> into your monad-transformer.
• ToIO is the opposite and will unpack the monad-transformer until it gets to the IO monad and will then return that as the bound value.

For example, this is the implementation for ReaderT:

    public static ToIO<A>(K<ReaderT<Env, M>, A> ma) =>
        new ReaderT<Env, M, IO<A>>(env => ma.As().runReader(env).ToIO());

So, we run the reader function with the env environment-value, it will return a K<M, A> which we then call ToIO() on to pass it down the transformer stack. Eventually it reaches the IO monad that just returns itself. This means we run the outer shell of the stack and not the inner IO.

That allows methods like MapIO to operate on the IO<A> monad, rather than the <A> within it:

M.ToIO(ma).Bind(io => M.LiftIO(f(io)));

What does this mean?

• It means you can call .MapIO(...) on any monad that has an IO monad within it (as long as ToIO has been implemented for the whole stack)
• Once we can map the IO we can generalise all of the IO behaviours...

Generalised IO behaviours

The IO<A> monad has many behaviours attached to it:

• Local - for creating a local cancellation environment
• Post - to make the IO computation run on the SynchronizationContext that was captured at the start of the IO operation
• Fork - to make an IO computation run on its own thread
• Await - for awaiting a forked IO operation's completion
• Timeout - to timeout an IO operation if it takes too long
• Bracket - to automatically track resource usage and clean it up when done
• Repeat, RepeatWhile, RepeatUntil - to repeat an IO operation until conditions cause the loop to end
• Retry, RetryWhile, RetryUntil - to retry an IO operation until successful or conditions cause the loop to end
• Fold, FoldWhile, FoldUntil - to repeatedly run an IO operation and aggregating a result until conditions cause the loop to end
• Zip - the ability to run multiple IO effects in parallel and join them in a tuppled result.

Many of the above had multiple overrides, meaning a few thousand lines of code. But, then we put our IO monad inside monad-transformers, or encapsulate them inside types like Eff<A> and suddenly those functions above are not available to us at all. We can't get at the IO<A> monad within to pass as arguments to the IO behaviours.

That's where MapIO comes in. Any monadic type or transformer type that has implemented ToIO (and has an IO<A> monad encapsulated within) can now directly invoke ...