Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Matthews correlation should be reviewed #381

Closed
Tracked by #573
tlienart opened this issue Jul 27, 2020 · 1 comment · Fixed by #571
Closed
Tracked by #573

Matthews correlation should be reviewed #381

tlienart opened this issue Jul 27, 2020 · 1 comment · Fixed by #571
Assignees
Labels
bug Something isn't working easy measures

Comments

@tlienart
Copy link
Collaborator

Here are some cases where it will fail (e.g. due to a square root over negative value)

cm = MLJBase.ConfusionMatrix{2}([29488 13017; 12790 29753], ["0.0", "1.0"])
cm = MLJBase.ConfusionMatrix{2}([29429 13222; 12849 29548], ["0.0", "1.0"])
cm = MLJBase.ConfusionMatrix{2}([29500 13052; 12778 29718], ["0.0", "1.0"])
cm = MLJBase.ConfusionMatrix{20}([465 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0; 0 192 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0; 0 0 0 21541 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0; 0 0 0 0 0 333 0 0 0 0 0 0 0 0 0 0 0 1 0 0; 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0; 0 0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0 0 0 0; 0 0 0 0 0 0 0 0 0 245 0 0 0 0 1 0 0 0 0 0; 0 0 0 0 0 0 0 0 0 0 0 56084 0 0 0 0 0 0 0 0; 0 0 0 0 0 0 0 0 0 0 0 0 51 0 0 0 0 0 1 0; 0 0 0 0 0 0 0 0 0 0 0 0 0 210 0 0 0 0 0 0; 0 0 0 0 0 0 0 0 0 0 0 0 0 0 39 0 0 0 0 0; 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0; 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 3 0 0 0; 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 214 0 0; 1 0 4 1 2 2 0 3 0 0 2 1 0 0 1 1 0 0 19383 2; 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10; 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0; 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0; 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0; 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0], ["0.0", "1.0", "2.0", "3.0", "4.0", "6.0", "7.0", "8.0", "9.0", "11.0", "12.0", "13.0", "14.0", "16.0", "17.0", "18.0", "19.0", "20.0", "21.0", "22.0"])
@tlienart tlienart added bug Something isn't working easy measures labels Jul 27, 2020
@ablaom
Copy link
Member

ablaom commented Jun 6, 2021

Update: We now need to use ConfusionMatrixObject here. Here's the example of the issue in the first example, updated:

julia> cm = MLJBase.ConfusionMatrixObject([29488 13017; 12790 29753], ["0.0", "1.0"])
              ┌───────────────────────────┐
              │       Ground Truth        │
┌─────────────┼─────────────┬─────────────┤
│  Predicted  │     0.01.0     │
├─────────────┼─────────────┼─────────────┤
│     0.02948813017    │
├─────────────┼─────────────┼─────────────┤
│     1.01279029753    │
└─────────────┴─────────────┴─────────────┘


julia> MatthewsCorrelation()(ans)
ERROR: DomainError with -5.3675251778421e18:
sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
Stacktrace:
 [1] throw_complex_domainerror(f::Symbol, x::Float64)
   @ Base.Math ./math.jl:33
 [2] sqrt
   @ ./math.jl:582 [inlined]
 [3] sqrt(x::Int64)
   @ Base.Math ./math.jl:608
 [4] (::MatthewsCorrelation)(cm::MLJBase.ConfusionMatrixObject{2})
   @ MLJBase ~/Dropbox/Julia7/MLJ/MLJBase/src/measures/finite.jl:357
 [5] top-level scope
   @ REPL[21]:1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working easy measures
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants