Skip to content

Various CI fixes for a lighter dashboard

Guillaume Pasero requested to merge ci_fixes into develop

Summary

Fixes to enable tests on the new CI, and reduce tests verbosity.

Rationale

Enable the tests on new CI, and reduce the log output (#1867 (closed)).

Implementation Details

There are different types of changes in this branch:

  • CI scripts:
    • enable testing
    • use xvfb to launch test (needed for GUI tests)
    • remove unused WRAP_PYTHON3 option
  • Make all tests pass:
    • can't use Shark on ubuntu-native due to #1871 (closed)
    • allow root to launch mpi-enabled tests (EXTRA_OPT --allow-run-as-root)
    • the RPC estimation/optimization functions are broken on Ossim 2.2.0, the tests don't crash but I had to remove some baselines checks for Ossim >= 2.2.0
    • add some multi-baselines...
    • in otb::WaveletOperatorBase, a weird crash was breaking all wavelet tests.
    • in TestMain, I made sure that the MPIFinalize() is called in all cases, to avoid the application hanging at closure. Removed early return.
    • in ApplicationLauncherCommandLine, I had to disable MPIInit() when -testenv is present: since this option leaves exceptions uncaught, the MPIFinalize() at the end of the application launcher is not called, and the TestDriver hangs on closure (see #1845 (closed)).
    • I made a small refactor of TestMain and TestHelper : better parsing of options, support of the --tolerance-ratio option (more details in the next section).
  • Reduce unnecessary test verbosity:
    • massive use of otbLogMacro(Debug, ... ) to control test verbosity at runtime.
    • I changed the priority level of No kwl metadata found in file ... to Debug, I think it is more relevant to log information when metadata has actually been found.
    • Code factorization in Modules/Learning/Supervised/test/TrainMachineLearningModel.cxx
    • the log size has been reduced from ~12MB to 3MB

Additional notes

In order to reduce the number of baselines, I introduced an new option for the test driver: --tolerance-ratio. It defines what percentage of the data can fail the epsilon check. For the moment it only applies to the image comparisons. For instance, passing the option --tolerance-ratio 0.01 means that the number of pixels with difference should be less than 1% of all pixels. By default, this tolerance is 0.

Copyright

The copyright owner is CNES and has signed the ORFEO ToolBox Contributor License Agreement.

Check before merging:

  • All discussions are resolved
  • At least 2 👍 votes from core developers, no 👎 vote.
  • The feature branch is (reasonably) up-to-date with the base branch
  • Dashboard is green
  • Copyright owner has signed the ORFEO ToolBox Contributor License Agreement
Edited by Guillaume Pasero

Merge request reports