Version 0.3

This version provides miscellaneous improvements upon the last one. The major changes are as follows.

• Start profiling the solvers under various compilation options (-O1, -O2, -Ofast, etc). This is to ensure that the modernized solvers' good performance does not depend on a specific optimization level.
• Start profiling LINCOA and COBYLA with and without the optimization toolbox of MATLAB. The preprocessing of linearly constrained problems makes use of the toolbox if available. This is to make sure that its availability does not affect the good performance of the modernized solvers.
• Start profiling the LINCOA and COBYLA with multiple penalty parameters in the merit function. This is to make sure that the good performance of the modernized solvers holds for a wide range of penalty parameters.
• Implement start_time and end_time in the MATLAB tests. This helps us to monitor which problem takes excessive time or fails, which is nontrivial to detect if the problems are solved in parallel.
• Implement GAMMA3 in the Fortran code. This is the threshold for testing whether DELTA is close enough to RHO after an update (if yes, DELTA will be reset to RHO).
• Enable automatic spell-check using https:/marketplace/actions/check-spelling
• Switch the default Fortran standard to Fortran 2008, and drop the support for Fortran 2003.
• Use the iso_fortran_env intrinsic module, especially for REAL64, INT32, etc.
• Use the storage_size intrinsic function.
• Use allocated characters for variable-length strings.
• Use ERROR STOP instead of STOP in ERRSTOP. We prefer the former because it is more appropriate in this scenario, and, more importantly, Fortran 2018 allows ERROR STOP in PURE procedures, which enables us to achieve the purity of all functions later when we switch the default Fortran standard to Fortran 2018.

In addition, we also profile BOBYQA, NEWUOAand LINCOA with and without RESCUE / IDZ techniques to see how much difference it makes. See benchmarking/rescue_idz/ for the results. According to the test on 20230406, the techniques affect the performance only marginally. The settings of the test are as follows.

1. Test problems: the CUTEst set.
2. Maximum dimension: 200.
3. Maximum number of linear constraints for LINCOA: 20000.
4. Tested features: plain, randomizex0-eps, perm, single, dnoise-1.0e-6, signif5, noise-1.0e-4.
5. Number of random tests when applicable: 3.
6. RHOEND: 1.0e-8.
7. Maximum number of function evaluations: 500*dimension.
8. Tolerance for the convergence tests in the performance profiles: 10^{-i}, i = 1, ..., 16.

Note that we purposely set the stopping criteria and convergence tolerance to observe the long-term behavior of the algorithms, as the techniques under consideration do not take effect unless we ask for very high precision.

The result: RESCUE and IDZ do not affect the performance of the algorithms significantly. For the classical version (i.e., Powell's implementation), the difference is barely observable. For the modernized version, the difference is more obvious, but still not significant.

Therefore, it seems safe to ignore these two techniques in the first implementation of algorithms based on the derivative-free PSB.

We plan to remove IDZ from NEWUOA and LINCOA, and modify the code of NEWUOA, BOYQA, and LINCOA to use RESCUE as a restarting technique. This will significantly simplify the code, and possibly unify the updating subroutines as well as shiftbase. We will also make sure that the modification will improve the performance of the solvers. If this is implemented, then the current release is the last version that implements Powell's original RESCUE and IDZ techniques.

Update 20230510: Version 0.4 still retains RESCUE and IDZ.