New trait: Fallible

In Haskell there's a trait called MonadFail for monadic types to raise errors. It's not particularly effective as most tend to avoid it. I wanted to create a trait (for types that can fail) that's effective and could help standardise error handling.

That's the new Fallible<E, F> and Fallible<F> trait type...

Fallible

Traits that are fallible can fail (I'm quite smug about the name, I think it's pretty cool, haha)!

• Fallible<E, F> - can have a parameterised failure value E for structure F (usually a functor, applicative, or monad)
• Fallible<F> is equivalent to Fallible<Error, F> - which simplifies usage for the commonly used Error type

Anything that is fallible must implement:

public static abstract K<F, A> Fail<A>(E error);

public static abstract K<F, A> Catch<A>(
    K<F, A> fa,
    Func<E, bool> predicate, 
    Func<E, K<F, A>> fail);

• Fail is for the raising of errors
• Catch can be used to catch an error if it matches a predicate; and if so, it runs the fail function to produce a new structure (which may also be failing, but could be used to rescue the operation and provide a sensible succeeding default).

Fallible module

In the Fallible module there are functions to raise failures:

public static class Fallible
{
    public static K<F, A> fail<E, F, A>(E error)
        where F : Fallible<E, F> =>
        F.Fail<A>(error);
    
    public static K<F, Unit> fail<E, F>(E error)
        where F : Fallible<E, F> =>
        F.Fail<Unit>(error);    
    
    public static K<F, A> error<F, A>(Error error)
        where F : Fallible<Error, F> =>
        F.Fail<A>(error);
    
    public static K<F, Unit> error<F>(Error error)
        where F : Fallible<Error, F> =>
        F.Fail<Unit>(error);    
}

• fail raises the parameterised error types
• error raises the Error type

Because the traits are all interfaces we can't use operator | for error handling (the operators can still be used for concrete types, like Eff<A>, IO<A>, etc.) -- and so there are now lots of Catch extension methods for catching errors in Fallible structures. You can view them here.

Prelude

The Prelude now has:

public static K<F, A> pure<F, A>(A value)
    where F : Applicative<F>;

public static K<F, A> fail<E, F, A>(E error)
    where F : Fallible<E, F>;

public static K<F, Unit> fail<E, F>(E error)
    where F : Fallible<E, F>;

public static K<F, A> error<F, A>(Error error)
    where F : Fallible<F>;

public static K<F, Unit> error<F>(Error error)
    where F : Fallible<F>;

So, for example, you can now construct any type (as long as it implements the Applicative trait) using pure:

var effect = pure<Eff, int>(100);
var option = pure<Option, string>("Hello");
var either = pure<Either<Error>, bool>(true);

And you can construct any type (as long as it implements the Fallible<E, F> trait) using fail or with error (when Fallible<F>):

var effect = error<Eff, int>(Errors.SequenceEmpty);
var trying = error<Try, int>(Errors.EndOfStream);
var option = fail<Unit, Option, string>(unit);
var either = fail<string, Either<string>, bool>("failed!");

Types that have been made Fallible

• IO<A>
• Eff<RT, A>
• Eff<A>
• Either<L, R>
• EitherT<L, M, R>
• Fin<A>
• Option<A>
• OptionT<M, A>
• Try<A>
• TryT<A>
• Validation<F, A>
• ValidationT<F, M, A>

Which means you can use .Catch(...) on all of those types now. For example:

var res = error<Eff, int>(Errors.Cancelled)
            .Catch(Errors.EndOfStream, _ => 0)             // Catch a specific error and ignore with a default 
            .Catch(Errors.Closed, _ => pure<Eff, int>(-1)) // Run an alternative effect
            .Catch(Errors.Cancelled, _ => IO.pure(-2))     // For monads that support IO, launch an IO operation
            .Catch(e => Errors.ParseError(e.ToString()));  // Catch-all mapping to another error

IO changes

• IO.Pure has been renamed IO.pure
  • Capital letters are used for constructor cases (like Some and None in a discriminated union)
  • IO.Pure doesn't create an IO<A> type with a Pure case, it constructs a lambda that returns an A
  • So, I'm going to change the construction functions where they're not really doing what they claim
• IO.Fail has been renamed IO.fail (see above)

Eff running extensions

The various .Run* methods for Eff<A> and Eff<RT, A> have been made into extension methods that work with K<Eff, A> and K<Eff<RT>, A>. That means if you end up with the more abstract representation of Eff you can run it without calling .As() first.

I'll be doing this for other types that are 'run'.

I have also tidied up some of the artefacts around the MinRT runtime used by Eff<A>. Because Eff<A> is now backed by a transformer stack with IO<A> as its inner monad, the MinRT doesn't need to carry any IO environment any more, so I've removed it from MinRT, making MinRT into a completely empty struct. This removes some constraints from the Run* extensions.

Prelude.ignoreF and Functor.IgnoreF

The prelude function ignoreF and equivalent extension method to Functor<F>, IgnoreF are the equivalent of calling .Map(_ => unit) to ignore the bound-value of a structure and instead return unit.

Transducers removed

I have removed the Transducers completely from v5. They were originally going to be the building blocks of higher-kinds, but with the new trait-system I don't think they add enough value, and frankly I do not have the time to bring them through this v5 release process (which is already a mammoth one)! As much as I like transducers, I think we can do better with the traits system now.

Not needed traits removed

The following traits have been removed:

• HasCancel - This was used in the Aff monad and now isn't needed because the IO monad has its own environment which carries the cancellation token
• HasFromError - was used by the transducers, so not needed
• HasIO - was used by the MinRT runtime, which isn't needed anymore
• HasSyncContextIO - as above

New Sample

Those of you who are subscribed to my blog at paullouth.com will have seen the first newsletter this week. It took a while to get off the ground because I refused to use the terrible Mailgun integration in GhostCMS.

Instead I rolled my own, which I've been working on the past few days. So it was an opportunity to test out the effect system and trait system. I took it as far as it can go and the entire application is trait driven. Only when you invoke the application do you specify what monad and runtime to use.

This is the main operation for generating the newsletter and emailing it out to all of the members:

public static class Send<M, RT>
    where RT : 
        Has<M, WebIO>,
        Has<M, JsonIO>,
        Has<M, FileIO>,
        Has<M, EmailIO>,
        Has<M, ConsoleIO>,
        Has<M, EncodingIO>,
        Has<M, DirectoryIO>,
        Reads<M, RT, Config>,
        Reads<M, RT, HttpClient>
    where M :
        Monad<M>,
        Fallible<M>,
        Stateful<M, RT>
{
    public static K<M, Unit> newsletter =>
        from posts     in Posts<M, RT>.readLastFromApi(4)
        from members   in Members<M, RT>.readAll
        from templates in Templates<M, RT>.loadDefault
        from letter    in Newsletter<M, RT>.make(posts, templates)
        from _1        in Newsletter<M, RT>.save(letter)
        from _2        in Display<M, RT>.showWhatsAboutToHappen(members)
        from _3        in askUserToConfirmSend
        from _4        in Email<M, RT>.sendToAll(members, letter)
        from _5        in Display<M, RT>.confirmSent
        select unit;
  
    ..
}

Note how the computation being run is entirely generic: M. Which is constrained to be a Monad, Fallible, and Stateful. The state is RT, also generic, which is constrained to have various IO traits as well as a Config and HttpClient state. This can be run with any type that supports those traits. Completely generic and abstract from the underlying implementation.

Only when we we pass the generic argument to Send<> do we get a concrete implementation:

var result = Send<Eff<Runtime>, Runtime>.newsletter.Run(runtime);

Here, we run the newsletter operation with an Eff<Runtime> monad. But, it could be with any monad we build.

Importantly, it works, so that's good :)

Source code is here . Any questions, ask in the comments below...

StateT bug fix + monadic conditionals

• Fixed: a bug in the StateT monad-transformer. One of the SelectMany overloads wasn't propagating the state correctly.
• Changed: Prelude.local that creates a local IO and resource environment renamed to localIO to avoid conflicts with ReaderT.local and Reader.local
• Added: general purpose liftIO in Prelude
• Added: variants of when and unless that take a K<M, bool> as the source of the flag. Means any monad that binds a bool can be used directly in when and unless, rather than having to lower it first.
• Added: new monadic conditional: iff - works like when and unless, but has an else case. K<M, bool> can be used directly also, meaning that if/then/else monadic expressions can be built without lowering.
• Added: applicative actions to the Prelude. Allows for chaining n applicative actions, discarding their results, apart from the last one, which is returned

Fix for: use of custom sub-type errors in IO monads

This is a minor release to fix: issue 1340.

Thanks to @HernanFAR for raising the issue with concise repro steps 👍

LanguageExt v5 first beta

I'm now moving the v5 release from alpha to beta. Not because I'm feature complete, but because from my real-world testing of v5 (with my new startup project) it is much more stable than I expected. In fact I haven't hit any issues at all outside of missing functionality.

So, this is more of a 'soft launch beta', primarily for those who were perhaps not ready to use language-ext in alpha form but are more likely to in beta form.

Removed ResourceT transformer / integrated into IO

This release removes the ResourceT<M, A> monad-transformer from language-ext and instead moves the functionality into the IO<A> monad. ResourceT required IO to be in the transformer stack and so it really was adding complexity to a feature that's closely linked. This adds a tiny overhead to the IO monad -- the IO monad already carried an environment through its computations, so this doesn't change much -- in the big scheme of things it's likely to bring performance benefits.

Some big improvements because of this:

• use and release are now available in the Prelude, which makes them easier to work with (no need for any manual generic arguments), everything is inferable from usage.
• Forking an IO computation (launching it on a new thread) automatically creates a local resource environment for the fork and cleans it up when the forked operation is complete.
• Repeating an IO computation (repeat(computation)) - will automatically clean up any resources acquired by use in the computation (on each iteration).
• Retrying an IO computation (retry(computation)) - will automatically clean up any resources (acquired with use) when the computation fails, that mean retries don't accumulate resources unnecessarily.
• New function local(computation) works as a 'super using' -- in that it will automatically clean up any resources acquired with use in the computation. This allows you to create local scopes where you just freely acquire resources and then have a clean-up happen automatically.
  • By the way, I am open to different names for this, as we already have IO.local for local cancellation contexts and Reader.local for local environments. I'm also open to changing the names of the others. Ideally any name would be a single word so it's easy on the eye. So, nothing like localResource or cleanUp.
• New functions bracket(Acq, Use, Err, Fin) and bracket(Acq, Use, Fin) - these are like try \ catch \ finally blocks for more explicit resource usage:
  • Acq - acquires the resource
  • Use - uses the resource
  • Err - is the catch block
  • Fin - is the finally block

All the usual caveats apply: this is an alpha, this isn't fully tested, use at your own risk.

New Try monad and updated TryT transformer

I've re-added a Try<A> monad (I always intended to re-add it, just hadn't got around to it). And I've and reimplemented the TryT<M, A> monad-transformer in terms of Try (K<M, Try<A>>), previously it was implemented in terms of Fin (Func<K<M, Fin<A>>>).

I have also:

• Added Match and IfFail methods for pattern matching on the success/fail state.
• Added | operator @catch overrides to allow for easy error handling.

The IO monad also has a .Try() method that will run the IO monad in a try/catch block returning IO<Fin<A>> for more manual handling of IO errors.

Still needs some in-depth testing to make sure all exceptions are captured, but it's effectively feature complete.

Language-Ext 5.0 alpha-3

WARNING: THIS IS AN ALPHA RELEASE AND SHOULD BE CONSUMED WITH CARE! NOT FOR PRODUCTION.

Bug fixing and TODO resolving release, with some minor featurettes!

For those that don't know yet (and there's no reason to think you should, because I haven't announced it yet) -- the Pipes Effect system now has the ability to lift any monad into its stack (previously it only allowed Aff to be lifted). It is now a general monad transformer like ReaderT, OptionT, EitherT, etc.

As, with all monad-transfomers, when you 'run' the transformer, it generates the lifted monad. You can think of this being like a mini-compiler that takes the monad stack and compiles down to the inner-most monad, which can then just be run as normal.

The problem for Pipes is that there's usually lots of recursion, repetition (using repeat, retry), or iteration (using yieldAll, etc.). This is problematic when you don't know anything about the inner monad. The transformer can't run the inner monad, because it only has access to the Monad interface (Bind) and the inherited interfaces of Applicative and Functor (Apply, Action, Map, and Pure). So, doing iteration requires recursion, and recursion blows the stack in C#.

Previously Pipes were able to directly Run the Aff because the Pipe system knew it was working only with Aff. This allowed it to flatten the recursion.

Anyway, now Pipes has internal support for any Foldable. That means yieldAll(...) can take a sequence from any foldable (Range, EnumerableM, HashMap, HashSet, Lst, Map, Seq, Either, Option, Validation, Identity, ... and any you write) and yield the values within the structure through the pipe. Functions like repeat(ma) - which continually repeat an operation until it fails - have also been implemented internally as something that iterates over an infinite foldable.

This functionality has been enabled by adding a new method to the Applicative trait: Actions. You might know the existing Action(K<M, A> ma, K<M, B> mb) method that runs the first applicative (ma), ignores its result, and then runs the second applicative mb, returning its result.

Actions instead takes an IEnumerable<K<M, A>>:

K<F, A> Actions<A>(IEnumerable<K<F, A>> fas)

It runs each applicative action and ignores its result, returning the result of the last item. That means a sequence of Proxy values (Proxy is the monad-transformer for pipes) can be mapped - the map will just run (using RunEffect) the Proxy - producing a sequence of whatever the lifted inner-monad is for the Proxy. This lazy sequence of monads can then be invoked by calling Actions on it, which will lazily walk the sequence, evaluating the inner-monad one-by-one.

There is a default implementation, but it has the same lack of knowledge that Pipes had, so it should be overridden for computation based applicatives (that usually need invoking with without an argument). Here's the override for Eff<RT, A>:

static K<Eff<RT>, A> Applicative<Eff<RT>>.Actions<A>(IEnumerable<K<Eff<RT>, A>> fas) =>
    from s in getState<A>()
    from r in Eff<RT, A>.Lift(
        rt =>
        {
            Fin<A> rs = Errors.SequenceEmpty;
            foreach (var kfa in fas)
            {
                var fa = kfa.As();
                rs = fa.Run(rt, s.Resources, s.EnvIO);
                if (rs.IsFail) return rs;
            }
            return rs;
        })
    select r;

You can see how:

• It's able to gather information, like the runtime, resources, and IO environment.
• It knows how to run itself, whereas the generic transformer can't.
• It can shortcut the operation when any effect fails.

And so, if you want to use your own monads with Pipes then you should implement Actions.

There's still more to do with Pipes, but all of the examples in EffectsExamples now work, which is a good sign!

WARNING: THIS IS AN ALPHA RELEASE AND SHOULD BE CONSUMED WITH CARE! NOT FOR PRODUCTION.

Language-Ext 5.0 alpha-2

WARNING: THIS IS AN ALPHA RELEASE AND SHOULD BE CONSUMED WITH CARE! NOT FOR PRODUCTION.

General updates

• Free monad doesn't need Alternative trait: removed
• All semigroup and monoid-like types have their Append operator renamed to Combine. 'Combine' works semantically for more of the monoidal associative operations than Append (which really only makes sense with collections).
• Added new SemigroupK and MonoidK -- these are like the Semigroup and Monoid traits except they work on K<M, A> instead of A. These are almost identical to SemiAlternative and Alternative execept they don't require the underlying value to an an Applicative. The idea here is that SemigroupK and MonoidK would be used on types like collections that 'sum' when the Combine operator is applied, whereas SemiAlternative and Alternative provide an alternative value when the Combine operator is applied (coalescing).
• Added missing repeat variants, retry variants, and timeout for the IO monad
• Added IO.yieldFor(TimeSpan). This is like Task.Delay but for the IO monad. The war against async means that this does the thread-yielding internally, no need to call await. I figured yieldFor is more meaningful than Delay, it indicates that the thread is yielding, not simply blocking.
• Added support for guards in the IO monad
• Initial pass at a continuation-monad transformer: ContT<R, M, A> -- just the raw type for now.
• HeadOrNone, HeadOrInvalid, HeadOrLeft, LastOrNone, etc. have been removed.
• Head and Last are now Option returning. This is a breaking change. Can be mitigated by either matching, casting, or invocation of .ValueUnsafe() extension.
• Improved Range type -- previously there were several types (IntegerRange, CharRange, etc.) -- now there's just one: Range<A>. It leverages the new traits built into .NET (IComparisonOperators, IAddtionOperators, etc.)
• Started adding support for more of the .NET traits in the collection types and various other places like Foldable.Sum, Foldable.Max, etc.: (IComparisonOperators, IAddtionOperators, IAdditiveIdentity etc.) -- these are Microsoft's ugly named versions of monoids etc.

War on Extension Methods

I'm rapidly coming to the conclusion that extension-methods are a terrible idea. Especially in a library like language-ext where I am trying to present a consistent set of interfaces to types that share common traits. It's just impossible to enforce consistency and relies on the human eye -- and that errs regularly!

The latest move toward using traits is really starting to help reduce the extension methods, or at least mean the extension methods are hanging off traits rather than individual instance-types.

One change that I have made recently is to change Foldable to require implementation of FoldWhile and FoldWhileBack instead of Fold and FoldBack. This means that so many more default behaviours can hang off of Foldable -- and most of them are optimal. For example, Exists -- which can stop processing as soon as its predicate returns true -- couldn't early-out before.

And so, the foldable trait is now growing to have a ton of functionality. Also nested foldables!

However, quite a lot of those methods, like Sum, Count, etc. also exist on IEnumerable. And so, for a type like Seq which derives from both IEnumerable and K<Seq, A>, there will be extension method resolution issues.

So, the choice is to provide extension methods for IEnumerable (an ill defined type) or for Foldable - a super featureful type with the opportunity for implementers to provide bespoke optimised overrides.

Really, the choice should be easy: extensions for Foldable are just better than extensions for IEnumerable. So, I have done that. The downside is that this will be another breaking change (because the IEnumerable extensions have been removed). The fix is to convert from IEnumerable<A> to EnumerableM<A> using .AsEnumerableM(). EnumerableM<A> supports Foldable (and other traits).

Conclusion

So, I've been working to remove as many non-trait extension methods as I can -- and I will continue to do so leading up to the beta. This will bring consistency to the code-base, reduce the amount of code, and provide ample opportunities for bespoke optimisations. Just be aware that this is another fix-up job.

WARNING: THIS IS AN ALPHA RELEASE AND SHOULD BE CONSUMED WITH CARE! NOT FOR PRODUCTION.

Language-Ext 5.0 alpha-1

This release should only be consumed by those who are interested in the new features coming in the monster v5 release.

Just to give you an idea of the scale of this change:

• 193 commits
• 1,836 files changed
• 135,000 lines of code added (!)
• 415,000 lines of code deleted (!!)

It is a monster and should be treated with caution...

• It is not ready for production
• It is not feature complete
• The new features don't have unit tests yet and so are probably full of bugs
• I haven't yet dogfooded all the new functionality, so it may not seem as awesome as it will eventually become!

If you add it to a production project, you should only do so to see (potentially) how many breaking changes there are. I would not advise migrating a production code-base until I get close to the final release.

I am also not going to go into huge detail about the changes here, I will simply list them as headings. I will do a full set of release notes for the beta release. You can however follow the series of articles I am writing to help you all prep for v5 -- it goes (and will go) into much more detail about the features.

New Features

• Higher-kinded traits
  • K<F, A> - higher-kinds enabling interface
  • Includes:
    • Defintions (interfaces listed below)
    • Static modules (Functor.map, Alternative.or, StateM.get, ...)
    • Extension methods (.Map, .Or, Bind, etc.),
    • Extension methods that replace LanguageExt.Transformers (BindT, MapT, etc. ), now fully generic.
    • Trait implementations for all Language-Ext types (Option, Either<L>, etc.)
  • Functor<F>
  • Applicative<F>
  • Monad<M>
  • Foldable<F>
  • Traversable<T>
  • Alternative<F>
  • SemiAlternative<F>
  • Has<M, TRAIT>
  • Reads<M, OUTER_STATE, INNER_STATE>
  • Mutates<M, OUTER_STATE, INNER_STATE>
  • ReaderM<M, Env>
  • StateM<M, S>
  • WriterM<M, OUT>
  • MonadT<M, N> - Monad transformers
    • ReaderT<Env, M, A>
    • WriterT<Out, M, A>
    • StateT<S, M, A>
    • IdentityT<M, A>
    • EitherT<L, M, R>
    • ValidationT<F, M, S>
    • OptionT<M, A>
    • TryT<M, A>
    • IdentityT<M, A>
    • ResourceT<M, A>
• Free<F, A> - Free monads
• IO<A> - new IO monad that is the base for all IO
• Eff<RT, A> monad rewritten to use monad-transformers (StateT<RT, ResourceT<IO>, A>)
• Eff<RT, A> doesn't need HasCancel trait (or any trait)
• Transducers
• Pure / Fail monads
• Lifting
• Improved guards, when, unless
• Nullable annotations - still WIP, mostly complete on Core)
• Collection initialisers

Breaking changes

• netstandard2.0 no longer supported (.NET 8.0+ only)
• Seq1 made [Obsolete]
• 'Trait' types now use static interface methods
• The 'higher-kind' trait types have all been refactored
• The Semigroup<A> and Monoid<A> types have been refactored
• The static TypeClass class has been renamed Trait
• Apply extensions that use raw Func removed
• Manually written Sequence extension methods have been removed
• Manually written Traverse extension methods have been removed
• ToComparer doesn't exist on the Ord<A> trait any more
• Renamed LanguageExt.ClassInstances.Sum
• Guard<E> has become Guard<E, A>
• UnitsOfMeasaure namespace converted to a static class
• Either doesn't support IEnumerable<EitherData> any more
• Either 'bi' functions have their arguments flipped
• Nullable (struct) extensions removed
• Support for Tuple and KeyValuePair removed
• Types removed outright
  • Some<A>
  • OptionNone
  • EitherUnsafe<L, R>
  • EitherLeft<L>
  • EitherRight<L>
  • Validation<MFail, Fail, A>
  • Try<A>
  • TryOption<A>
  • TryAsync<A>
  • TryOptionAsync<A>
  • Result<A>
  • OptionalResult<A>
  • Async extensions for Option<A>
  • ExceptionMatch, ExceptionMatchAsync, ExceptionMatchOptionalAsync
• Libraries removed outright
  • LanguageExt.SysX
  • LanguageExt.CodeGen
  • LanguageExt.Transformers

Big fixes and minor improvements release

New:

• Support for IfFail in Try, TryOption, TryAsync, and TryOptionAsync - Thanks @mark-pro 👍
• Update prelude for Arithmetic typeclass - Thanks @benjstephenson 👍
• Improvements to the Effects samples

Bug fixes:

• Fix for EitherUnsafe MapLeft throws ResultIsNullException
• Fix for Stack overflow with MapT on T
• Fix for CancellationTokenSource not disposed