Makes Eff Harder, Better, Faster, Stronger

[safareli]

Cont of safareli/purescript-ef#1

fixes #29

Same do optimization is still done, here is fragment from output/Test.Main/index.js:

var main = (function () {
    var test = function (msg) {
        return function (eff) {
            return function __do() {
                $foreign.time(msg)();
                eff();
                return $foreign.timeEnd(msg)();
            };
        };
    };
    return function __do() {
        test("testLift2")(testLift2)();
        test("testBindRight")(testBindRight(1000000))();
        test("testBindLeft")(testMap(1000000))();
        test("testMap")(testMap(5000000))();
        return test("testApply")(testApply(1000000))();
    };
})();

TODO

• add comments to new implementation
• add note about stack issue because of compiler optimization
• use published version of ps-minibench
• push instance of some "class" for APPLY_FUNC operation
• check if it's better to run operations on pure values directly add more optimization for pure node operations purescript-contrib/purescript-aff#133

[natefaubion]

I think this is at least worth considering:

• Eff is still semantically just a thunk as it was before.
• All existing FFI code should continue to work.
• The compiler optimizations still apply.
• Stack safety.
• Appears to be faster.

I think having both Eff and Aff (the core IO types) stack safe is huge.

[natefaubion]

Stack safety.

One problem though is that compiler optimized code can blow the stack, while combinator code would work fine.

-- Blows the stack?
loop x = do
  next <- inc x
  loop next

-- Doesn't?
loop x = inc x >>= loop

[paf31]

I think this is interesting and very cool, but it seems to change a basic library in a way which is incompatible with the original project goals. Namely, it's not minimal, and it's not simple (I don't fully understand it yet, and I've been reading it off and on since you originally posted it in the other repo).

I think it needs more documentation for me to understand it properly, for one.

It seems like this is purely an optimization, so I think making this fork available in an alternative package set for extensive testing in real world apps would be helpful. That way, we can test other things like how this impacts debugging.

Worst case, it's a drop-in replacement for eff until we can do more of this sort of testing, right?

[safareli]

Will totally add comments on how things are working .

We can hold merging it for some time, try to make this change in some real world apps and see if there are any issues:

- "purescript-eff": "^3.0.0",
+ "purescript-eff": "safareli/purescript-eff#fast"

In terms of debugging we can even implement custom toString function method for eff nodes we create so we have more options to improve debugging this way.

Yes it's drop in replacment of Eff.

[safareli]

This is output of npm run bench

bench	type	n	mean	stddev	min	max
>>=R build	Eff	100	130.67 μs	166.84 μs	12.70 μs	248.64 μs
>>=R build	Aff	100	69.36 μs	77.55 μs	14.53 μs	124.20 μs
>>=R run	Eff	100	934.69 μs	21.74 μs	919.31 μs	950.06 μs
>>=R run	Aff	100	1.34 ms	965.71 μs	661.46 μs	2.03 ms
>>=R build	Eff	1000	45.49 μs	62.18 μs	1.52 μs	89.45 μs
>>=R build	Aff	1000	29.73 μs	40.75 μs	914.00 ns	58.55 μs
>>=R run	Eff	1000	1.33 ms	1.20 ms	486.63 μs	2.18 ms
>>=R run	Aff	1000	2.00 ms	1.28 ms	1.10 ms	2.90 ms
>>=R build	Eff	5000	22.30 μs	30.37 μs	820.00 ns	43.77 μs
>>=R build	Aff	5000	26.66 μs	35.77 μs	1.37 μs	51.95 μs
>>=R run	Eff	5000	4.93 ms	164.93 μs	4.81 ms	5.04 ms
>>=R run	Aff	5000	9.08 ms	621.88 μs	8.64 ms	9.52 ms
>>=L build	Eff	100	93.03 μs	123.84 μs	5.46 μs	180.60 μs
>>=L build	Aff	100	27.74 μs	37.98 μs	883.00 ns	54.60 μs
>>=L run	Eff	100	364.22 μs	162.70 μs	249.17 μs	479.27 μs
>>=L run	Aff	100	806.67 μs	865.82 μs	194.44 μs	1.42 ms
>>=L build	Eff	1000	21.20 μs	28.68 μs	922.00 ns	41.48 μs
>>=L build	Aff	1000	22.45 μs	30.63 μs	786.00 ns	44.11 μs
>>=L run	Eff	1000	446.72 μs	75.71 μs	393.18 μs	500.26 μs
>>=L run	Aff	1000	1.89 ms	1.17 ms	1.06 ms	2.72 ms
>>=L build	Eff	5000	24.88 μs	33.94 μs	883.00 ns	48.88 μs
>>=L build	Aff	5000	22.39 μs	30.40 μs	887.00 ns	43.88 μs
>>=L run	Eff	5000	3.28 ms	1.67 ms	2.10 ms	4.47 ms
>>=L run	Aff	5000	8.45 ms	763.14 μs	7.91 ms	8.99 ms
map build	Eff	100	82.48 μs	109.40 μs	5.12 μs	159.84 μs
map build	Aff	100	28.01 μs	37.71 μs	1.35 μs	54.67 μs
map run	Eff	100	332.08 μs	411.04 μs	41.43 μs	622.73 μs
map run	Aff	100	1.14 ms	1.14 ms	335.09 μs	1.94 ms
map build	Eff	1000	22.00 μs	29.81 μs	922.00 ns	43.08 μs
map build	Aff	1000	40.76 μs	43.69 μs	9.87 μs	71.66 μs
map run	Eff	1000	807.00 μs	438.30 μs	497.07 μs	1.12 ms
map run	Aff	1000	1.41 ms	1.49 ms	356.32 μs	2.46 ms
map build	Eff	5000	27.71 μs	35.36 μs	2.71 μs	52.72 μs
map build	Aff	5000	45.81 μs	46.76 μs	12.74 μs	78.87 μs
map run	Eff	5000	1.12 ms	76.03 μs	1.06 ms	1.17 ms
map run	Aff	5000	7.73 ms	4.15 ms	4.80 ms	10.67 ms
apply build	Eff	100	96.37 μs	128.97 μs	5.17 μs	187.56 μs
apply build	Aff	100	61.38 μs	64.40 μs	15.84 μs	106.91 μs
apply run	Eff	100	619.51 μs	59.81 μs	577.21 μs	661.80 μs
apply run	Aff	100	1.19 ms	1.09 ms	422.74 μs	1.96 ms
apply build	Eff	1000	22.47 μs	27.18 μs	3.25 μs	41.69 μs
apply build	Aff	1000	44.37 μs	48.39 μs	10.15 μs	78.58 μs
apply run	Eff	1000	651.48 μs	80.40 μs	594.63 μs	708.33 μs
apply run	Aff	1000	6.73 ms	1.20 ms	5.87 ms	7.58 ms
apply build	Eff	5000	22.48 μs	30.54 μs	880.00 ns	44.07 μs
apply build	Aff	5000	40.63 μs	43.91 μs	9.58 μs	71.68 μs
apply run	Eff	5000	4.14 ms	925.88 μs	3.48 ms	4.79 ms
apply run	Aff	5000	28.01 ms	4.56 ms	24.78 ms	31.23 ms
-	-	-	-	-	-	-
>>=R build	Eff	10000	87.19 μs	115.05 μs	5.84 μs	168.55 μs
>>=R build	Aff	10000	51.24 μs	53.11 μs	13.69 μs	88.80 μs
>>=R run	Eff	10000	8.59 ms	5.80 ms	4.49 ms	12.69 ms
>>=R run	Aff	10000	13.46 ms	1.63 ms	12.31 ms	14.61 ms
>>=R build	Eff	50000	23.19 μs	31.36 μs	1.02 μs	45.36 μs
>>=R build	Aff	50000	45.53 μs	51.25 μs	9.29 μs	81.77 μs
>>=R run	Eff	50000	95.01 ms	30.11 ms	73.72 ms	116.30 ms
>>=R run	Aff	50000	141.67 ms	17.75 ms	129.12 ms	154.22 ms
>>=R build	Eff	100000	24.96 μs	34.02 μs	903.00 ns	49.01 μs
>>=R build	Aff	100000	50.97 μs	57.69 μs	10.17 μs	91.76 μs
>>=R run	Eff	100000	191.86 ms	864.94 μs	191.25 ms	192.47 ms
>>=R run	Aff	100000	260.19 ms	66.63 ms	213.08 ms	307.30 ms
>>=R build	Eff	1000000	38.57 μs	52.56 μs	1.41 μs	75.73 μs
>>=R build	Aff	1000000	57.62 μs	66.94 μs	10.29 μs	104.95 μs
>>=R run	Eff	1000000	2.28 s	289.05 ms	2.07 s	2.48 s
>>=R run	Aff	1000000	3.28 s	255.28 ms	3.10 s	3.46 s
>>=L build	Eff	10000	77.37 μs	100.37 μs	6.40 μs	148.34 μs
>>=L build	Aff	10000	80.40 μs	91.23 μs	15.89 μs	144.90 μs
>>=L run	Eff	10000	9.57 ms	7.01 ms	4.61 ms	14.53 ms
>>=L run	Aff	10000	17.03 ms	4.19 ms	14.07 ms	20.00 ms
>>=L build	Eff	50000	22.56 μs	30.60 μs	917.00 ns	44.20 μs
>>=L build	Aff	50000	42.55 μs	46.44 μs	9.71 μs	75.38 μs
>>=L run	Eff	50000	109.56 ms	12.99 ms	100.37 ms	118.75 ms
>>=L run	Aff	50000	205.05 ms	8.22 ms	199.25 ms	210.86 ms
>>=L build	Eff	100000	31.92 μs	43.78 μs	959.00 ns	62.88 μs
>>=L build	Aff	100000	74.85 μs	80.57 μs	17.88 μs	131.82 μs
>>=L run	Eff	100000	214.58 ms	49.42 ms	179.64 ms	249.53 ms
>>=L run	Aff	100000	308.45 ms	63.56 ms	263.50 ms	353.39 ms
>>=L build	Eff	1000000	29.98 μs	41.13 μs	895.00 ns	59.07 μs
>>=L build	Aff	1000000	68.67 μs	82.42 μs	10.39 μs	126.95 μs
>>=L run	Eff	1000000	3.06 s	277.36 ms	2.86 s	3.25 s
>>=L run	Aff	1000000	3.40 s	166.00 ms	3.28 s	3.52 s
map build	Eff	10000	77.41 μs	101.20 μs	5.85 μs	148.97 μs
map build	Aff	10000	117.85 μs	139.34 μs	19.33 μs	216.38 μs
map run	Eff	10000	2.85 ms	1.17 ms	2.02 ms	3.67 ms
map run	Aff	10000	20.04 ms	6.83 ms	15.21 ms	24.87 ms
map build	Eff	50000	41.92 μs	56.10 μs	2.25 μs	81.59 μs
map build	Aff	50000	109.44 μs	131.27 μs	16.62 μs	202.26 μs
map run	Eff	50000	19.60 ms	1.67 ms	18.42 ms	20.79 ms
map run	Aff	50000	122.70 ms	42.18 ms	92.87 ms	152.52 ms
map build	Eff	100000	30.10 μs	41.19 μs	976.00 ns	59.22 μs
map build	Aff	100000	111.02 μs	131.04 μs	18.35 μs	203.68 μs
map run	Eff	100000	71.39 ms	22.58 ms	55.42 ms	87.35 ms
map run	Aff	100000	220.17 ms	101.51 ms	148.39 ms	291.95 ms
map build	Eff	1000000	39.98 μs	55.14 μs	994.00 ns	78.97 μs
map build	Aff	1000000	55.25 μs	63.86 μs	10.10 μs	100.41 μs
map run	Eff	1000000	699.84 ms	67.03 ms	652.44 ms	747.24 ms
map run	Aff	1000000	1.87 s	1.18 s	1.03 s	2.70 s
map build	Eff	350000	95.63 μs	126.59 μs	6.12 μs	185.14 μs
map build	Aff	350000	58.30 μs	64.13 μs	12.95 μs	103.64 μs
map run	Eff	350000	236.09 ms	61.87 ms	192.35 ms	279.84 ms
map run	Aff	350000	658.45 ms	347.16 ms	412.97 ms	903.93 ms
map build	Eff	700000	101.47 μs	133.48 μs	7.09 μs	195.86 μs
map build	Aff	700000	60.53 μs	70.48 μs	10.69 μs	110.36 μs
map run	Eff	700000	488.74 ms	20.19 ms	474.46 ms	503.02 ms
map run	Aff	700000	1.37 s	735.55 ms	850.74 ms	1.89 s
apply build	Eff	10000	133.85 μs	176.84 μs	8.81 μs	258.90 μs
apply build	Aff	10000	63.14 μs	69.41 μs	14.06 μs	112.22 μs
apply run	Eff	10000	14.33 ms	9.33 ms	7.73 ms	20.93 ms
apply run	Aff	10000	61.29 ms	16.65 ms	49.52 ms	73.06 ms
apply build	Eff	50000	40.66 μs	55.84 μs	1.17 μs	80.14 μs
apply build	Aff	50000	72.56 μs	80.89 μs	15.36 μs	129.75 μs
apply run	Eff	50000	123.31 ms	14.57 ms	113.00 ms	133.61 ms
apply run	Aff	50000	272.31 ms	75.15 ms	219.17 ms	325.45 ms
apply build	Eff	100000	84.90 μs	117.01 μs	2.16 μs	167.64 μs
apply build	Aff	100000	68.79 μs	78.21 μs	13.49 μs	124.09 μs
apply run	Eff	100000	222.97 ms	59.31 ms	181.03 ms	264.91 ms
apply run	Aff	100000	496.86 ms	59.40 ms	454.86 ms	538.86 ms
apply build	Eff	1000000	65.78 μs	89.12 μs	2.76 μs	128.79 μs
apply build	Aff	1000000	52.41 μs	59.99 μs	9.99 μs	94.82 μs
apply run	Eff	1000000	2.67 s	573.44 ms	2.26 s	3.07 s
apply run	Aff	1000000	6.20 s	976.56 ms	5.51 s	6.89 s

[safareli]

@natefaubion I guess the do stack safety issue should be handled in compiler.
like if we have do for Eff and we are doing recursion use bind

[natefaubion]

@safareli That amounts to a whole program optimization due to mutual recursion.

[Cmdv]

I'd call it installation so there is no confusion with native used of install ie if you try use yarn to run this it won't work. yarn install 😞

[toastal]

Or postinstall

[Cmdv]

what are you doing here @toastal ? we've provided the least useful feedback 😂

[safareli]

@safareli That amounts to a whole program optimization due to mutual recursion.

So we can't do much here. But at least it's not a regression. I will add that issue to documentation.

[safareli]

Update:
I have been using Eff from this branch on slamdata during development and everything is fine.

[safareli]

purescript/purescript-minibench#9

[safareli]

Not sure how useful will be this bench project might be in future, we can also remove it too.

This file is result of running ln -s ../node_modules/ ./node_modules in bench folder, so we don't mix bench scripts with root project.