-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
General Discussion regarding Ensemble #1358
Comments
Is there a downside to coupling it to PEtab? The upcoming PEtab Result format could cover ensemble predictions as simulation experiments. This could reduce and shift this part
into a nicer/simpler format. I agree, ensembles themselves can be completely decoupled from AMICI or predictions, and simply serve as a thin wrapper around a NumPy array of parameter vectors. Something useful for such a wrapper would be a nice way specify prediction experiments, e.g. how to tell pyPESTO to "create a new ensemble from the current ensemble that predicts a knockout experiment, by setting this parameter to zero". I guess having the ensemble be a knockout_ensemble = ensemble.copy()
knockout_ensemble["knocked_out_parameter"] = 0 re: supported simulators, if PEtab is used to specify the ensemble predictions, then we could use the [1] https:/PEtab-dev/libpetab-python/blob/90379c41611ea941b9865ba8dd724b406b7a31ef/petab/simulate.py |
Thanks for starting this discussion. To reduce complexity, I would suggest to first tackle the higher level questions. What is generated from a parameter ensemble or model ensemble? Is there a common structure that can/should be represented in pypesto? Is the current
I think this covers the main use case in pypesto already, but once there exists some concept of model in pypesto, it shouldn't be hard to support the more general case. In case of a bigger refactoring, I would preventively rename
It wouldn't be usable for any non-PEtab applications. Nevertheless, it might be better to have some easy-to-use functionality coupled to PEtab, than having some practically unusable general concept. In any case, it should be made clear that it is (supposed to be) tied to PEtab. |
I'd be happy to hear more about the use cases for a model ensemble first. If it's the calibrated models from model selection, it might make more sense to move some of this to PEtab Select, e.g. s.t. a PEtab Select model ensemble can be represented by a collection of pyPESTO
👍 |
The Petab select case was what I mainly thought about. I would think that moving it to PEtab select (or parts of it) makes sense, but would should then clarify what we understand under Ensemble, as Daniel mentioned
But in Petab select we would probably also need some way to create them? 🤔
I really think that a very large portion of predictions boils down to "sbml_id" at given timepoints that might not agree with measurements under specific conditions. And I do think there can/should be a structure to represent this in pypesto.
Looking at it, I feel like the |
Is a prediction result as a (PEtab measurements table)-like dataframe sufficient? This would make handling the predictions for e.g. plotting easier than the current implementation, at least. Then extra AMICI/PEtab-specific things can be optional columns.
This would make handling the predictions for e.g. plotting much easier than the current implementation. Currently, all data given a specific observable and a specific experiment is retrieved like ( pyPESTO/pypesto/visualize/sampling.py Lines 176 to 181 in 34e89b3
|
I am not sure if there is much added value in any of those. So far, the main thing is: 1) creating a parameter ensemble, 2) running simulations and collecting some outputs, and 3) computing and visualizing some statistics. The last step is probably most easily done directly with pandas/seaborn once everything is in a properly organized dataframe.
I'd say so.
Yes. |
We would obviously somehow need to allow for not only observables to be put there, otherwise, I think you are right, would make handling visualization much easier. |
There are a multitude of Issues currently opened regarding ensembles (see for example and further context #1357, #1349, #1296, #1294, #1291). We should use this to have a general discussion on the purpose and reasonability of
Ensembles
. We should use this issue to discuss topics regarding Ensembles to pool future improvements. Here are aspects I think we should consider (based on my own opinion, open PRs and discussion), which is certainly not a complete list:General Purpose Questions for the
Ensemble
classEnsembles
in general, but implicitly need an amici model. We do support other simulators as well. It would be completely fine to only support amici ensembles, but then we should make that clear. Consideration here would be, how general we want ensembles to be or whether we should perhaps change it intoAmiciEnsemble
(which does not necessarily need a complete own module?)General purpose Questions for the "Prediction/Predictor" Class
formula f(parameters, states, observables)
instead of just them. This functionality however is not only used in ensembles, but also for example to check model fits or to explore new conditions/ "interventions". I think it might make sense to think about a Predictor Class, as it would streamline a lot of visualization tasks and facilitate model exploration (there is the amiciPredictor class, but I excluded it for the moment, as I think a more general discussion might be helpful). Things to consider here include:What use should an Ensemble class serve?
as a follow up consideration.Any thoughts on this are very welcome and also any further questions/considerations.
The text was updated successfully, but these errors were encountered: