Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add precompile workload #900

Closed
wants to merge 2 commits into from
Closed

Conversation

rikhuijzer
Copy link
Member

In response to the comments at JuliaAI/MLJ.jl#1016, this PR suggests adding a precompile workload here to MLJBase.jl.

I'm running a bit into an issue though. I can't run evaluate! because that one calls MLJModelInterface.nrows at src/resampling.jl#1289. As far as I understand, the problem is that the interface mode in MLJModelInterface (INTERFACE_MODE) is not set during precompilation yet. However, I also cannot set the global constant because that will likely "bake in" the constant during precompilation via constant propagation, which then could give problems later on.

Is there a way we could get rid of the global constant? Maybe that could make precompilation easier. This might also have some other benefits down the line. I do understand that it's a big ask, though, so we can also continue without calling evaluate!.

@rikhuijzer rikhuijzer requested a review from ablaom April 24, 2023 13:26
@codecov-commenter
Copy link

Codecov Report

Merging #900 (3e0ed3b) into dev (a7f2814) will decrease coverage by 0.78%.
The diff coverage is 0.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##              dev     #900      +/-   ##
==========================================
- Coverage   85.81%   85.03%   -0.78%     
==========================================
  Files          40       41       +1     
  Lines        3567     3562       -5     
==========================================
- Hits         3061     3029      -32     
- Misses        506      533      +27     
Impacted Files Coverage Δ
src/MLJBase.jl 60.00% <ø> (ø)
src/precompile.jl 0.00% <0.00%> (ø)

... and 7 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@ablaom
Copy link
Member

ablaom commented Apr 25, 2023

Thanks for this. Let me give a little context to see if that helps you see a solution. The idea that INTERFACE_MODE is a switch to allow new model implementations to avoid certain package dependencies (such as Tables.jl) when they depend on MLJModelInterface.jl. (We had tried a Requires.jl solution at the time but got issues.) As we document here, these implementations don't actually function until MLJBase is imported, which changes the value of the constant.

Is there a way we could get rid of the global constant?

Unfortunately, I don't see a way to do this that isn't disruptive. I guess we could use the new optional dependencies feature of 1.9, but that would still be quite a bit of refactoring and would be breaking. I'm not sure breaking changes to MLJModelInterface.jl are even conceivable at this point. That's for MLJ2 ha ha.

A key point is that this: once MLJBase is imported, there is no reason for INTERFACE_MODE to have any value other than FullInterface.

Currently the switch happens in MLJBase.jl/init.jl. Is it possible we move it out of init.jl? I have gotten into trouble trying to modify global constants in other packages outside of init.jl, so I could be missing something here.

@rikhuijzer
Copy link
Member Author

I unfortunately cannot figure out how to get a reasonable speedup without hitting the global constant, so will close this PR

@rikhuijzer rikhuijzer closed this May 8, 2023
@ablaom
Copy link
Member

ablaom commented May 8, 2023

Thanks @rikhuijzer for spending time looking into this. Appreciate it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants