Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: error resolution flow control without exceptions #1095

Conversation

aepfli
Copy link
Member

@aepfli aepfli commented Sep 7, 2024

So far, we used exceptions to handle our flow control, but Exceptions are costly. In the end, we enriched our
evaluation Details with errorCode and errorMessage. If desired, the providers can also handle this to reduce the execution footprint in erroneous cases, which don't have to be exceptions. Till now, we also converted those into exceptions and threw them, with this little code change, we would return already the details, and call the hooks, without exceptions being thrown.

For example, FlagNotFound is such a case; it can be expected and not exceptional. Hence, the exception flow might add too much overhead to performance-critical environments.

This implementation now allows the prover implementors to decide whether to support either or both.

Follow-up Tasks

Most likely this is something we also want to do in all other sdks.

How to test

@aepfli aepfli requested a review from a team as a code owner September 7, 2024 15:21
Copy link

codecov bot commented Sep 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.34%. Comparing base (29901b8) to head (430ba1c).
Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1095      +/-   ##
============================================
+ Coverage     95.09%   95.34%   +0.25%     
- Complexity      400      402       +2     
============================================
  Files            39       39              
  Lines           917      924       +7     
  Branches         56       56              
============================================
+ Hits            872      881       +9     
+ Misses           24       23       -1     
+ Partials         21       20       -1     
Flag Coverage Δ
unittests 95.34% <100.00%> (+0.25%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@aepfli aepfli changed the title Allowing flowcontrol with out exceptions feat: Allowing flowcontrol with out exceptions Sep 7, 2024
So far we used exception to handle our flowcontrol, but
Exceptions are costly. In the end we enriched our
evaluation Details with errorCode and errorMessage.
This can be also handled by the providers if desired,
to reduce the execution footprint in errornous cases,
which do not have to be exceptions.

Eg FlagNotFound - it might be the case, but in performance
critical environments, an exception rather than a normal
return, can cause overhead and can be already too costly.

Signed-off-by: Simon Schrottner <[email protected]>
@aepfli
Copy link
Member Author

aepfli commented Sep 7, 2024

I am not good with performance tests, but I tried the following locally:

        IntStream.range( 0, 1000000 ).parallel().forEach(i -> {
            String key = "key" + i;
            c.getBooleanDetails(key, false);
        });

With the exceptions, it took more than 3s; with details, it took only 1.5s.

Remarks:

  • I changed the implementation of the AlwaysBrokenWithDetailsProvider to set a different message all the time so that we never create the same object if the JIT somehow starts to optimize.
  • I removed the asserts because they took most of the time, and i was only aiming for run speed.
  • I ran those tests for the fun about 5 times, with close results.
  • I know this is not the best performance test, but it gives a little bit of insights

// EDIT: forget about my tests, i might have made a mistake - i cant reproduce this - hence that, those are bad tests.

// EDIT: https://www.baeldung.com/java-exceptions-performance is a nice comprehensive post about this

@toddbaert toddbaert changed the title feat: Allowing flowcontrol with out exceptions feat: error resolution flow control with out exceptions Sep 9, 2024
Copy link
Member

@toddbaert toddbaert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @guidobrei 's point here.

In addition, I'm wondering if we should remove stack traces from some or all Errors, in addition to the FlagNotFound.

I think for a few, like ProviderNotReady and TypeMismatch we can also remove the stack trace, especially since these will be frequently instantiated by the SDK here and it's very obvious what the problem is.

@toddbaert toddbaert requested review from liran2000 and removed request for guidobrei September 10, 2024 17:11
@aepfli
Copy link
Member Author

aepfli commented Sep 10, 2024

In addition, I'm wondering if we should remove stack traces from some or all Errors, in addition to the FlagNotFound.

I think for a few, like ProviderNotReady and TypeMismatch we can also remove the stack trace, especially since these will be frequently instantiated by the SDK here and it's very obvious what the problem is.

I added this via a parent class to the errors mentioned. This way, we can easily identify them later. Wdyt? (we can also try to achieve this with an interface and default methods)

@aepfli aepfli force-pushed the issue/allow_flowcontrol_without_exceptions branch from d523c6b to db6033b Compare September 10, 2024 18:40
@aepfli aepfli force-pushed the issue/allow_flowcontrol_without_exceptions branch from db6033b to 3d4b992 Compare September 10, 2024 19:02
@guidobrei
Copy link
Member

I think for a few, like ProviderNotReady and TypeMismatch we can also remove the stack trace, especially since these will be frequently instantiated by the SDK here and it's very obvious what the problem is.

Not sure for the TypeMismatch. If client.getBooleanValue("flag-key", false); is called from user code for a string flag, I'd like to see the root cause.

@toddbaert
Copy link
Member

I think for a few, like ProviderNotReady and TypeMismatch we can also remove the stack trace, especially since these will be frequently instantiated by the SDK here and it's very obvious what the problem is.

Not sure for the TypeMismatch. If client.getBooleanValue("flag-key", false); is called from user code for a string flag, I'd like to see the root cause.

OK I'm fine with that.

@toddbaert
Copy link
Member

Approved but you can make the TypeMismatch change @guidobrei suggestes.

@jarebudev
Copy link
Contributor

I am not good with performance tests, but I tried the following locally:

        IntStream.range( 0, 1000000 ).parallel().forEach(i -> {
            String key = "key" + i;
            c.getBooleanDetails(key, false);
        });

With the exceptions, it took more than 3s; with details, it took only 1.5s.

Remarks:

  • I changed the implementation of the AlwaysBrokenWithDetailsProvider to set a different message all the time so that we never create the same object if the JIT somehow starts to optimize.
  • I removed the asserts because they took most of the time, and i was only aiming for run speed.
  • I ran those tests for the fun about 5 times, with close results.
  • I know this is not the best performance test, but it gives a little bit of insights

// EDIT: forget about my tests, i might have made a mistake - i cant reproduce this - hence that, those are bad tests.

// EDIT: https://www.baeldung.com/java-exceptions-performance is a nice comprehensive post about this

This has got me wondering if there is value in creating some JMH benchmarks for the sdk...?

@Kavindu-Dodan Kavindu-Dodan changed the title feat: error resolution flow control with out exceptions feat: error resolution flow control without exceptions Sep 13, 2024
Copy link

sonarcloud bot commented Sep 16, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants