Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To add a unit test focusing on
OperationCanceledException
, you could set the AutoML setting to use only FastTree:There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have tried that, but it only throws this exception in 1 specific method and this wasn't a stable repro. Not stable enough to try and have a consistent test for it anyways.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. After discussing, even restricting to only FastTree will not make for a stable unit test.
If the timeout occurs in certain steps of FastTree,
AggregateException
is thrown; while at other steps,OperationCanceledException
is thrown.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you force it with a test-specific Trainer, or set of trainers?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried that. The problem is that there is only 1 specific place that causes this issue when a timeout occurs, and I haven't found a way to get it to repro consistently without modifying the method to always throw there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking more along the lines of a test-only mock trainer. Ignore if we don't already have a framework / pattern for this, that'd be more of a future-looking test strategy piece of feedback.