Various CI fixes for a lighter dashboard

Summary

Fixes to enable tests on the new CI, and reduce tests verbosity.

Rationale

Enable the tests on new CI, and reduce the log output (#1867 (closed)).

Implementation Details

There are different types of changes in this branch:

• CI scripts:
  • enable testing
  • use xvfb to launch test (needed for GUI tests)
  • remove unused WRAP_PYTHON3 option
• Make all tests pass:
  • can't use Shark on ubuntu-native due to #1871 (closed)
  • allow root to launch mpi-enabled tests (EXTRA_OPT --allow-run-as-root)
  • the RPC estimation/optimization functions are broken on Ossim 2.2.0, the tests don't crash but I had to remove some baselines checks for Ossim >= 2.2.0
  • add some multi-baselines...
  • in otb::WaveletOperatorBase, a weird crash was breaking all wavelet tests.
  • in TestMain, I made sure that the MPIFinalize() is called in all cases, to avoid the application hanging at closure. Removed early return.
  • in ApplicationLauncherCommandLine, I had to disable MPIInit() when -testenv is present: since this option leaves exceptions uncaught, the MPIFinalize() at the end of the application launcher is not called, and the TestDriver hangs on closure (see #1845 (closed)).
  • I made a small refactor of TestMain and TestHelper : better parsing of options, support of the --tolerance-ratio option (more details in the next section).
• Reduce unnecessary test verbosity:
  • massive use of otbLogMacro(Debug, ... ) to control test verbosity at runtime.
  • I changed the priority level of No kwl metadata found in file ... to Debug, I think it is more relevant to log information when metadata has actually been found.
  • Code factorization in Modules/Learning/Supervised/test/TrainMachineLearningModel.cxx
  • the log size has been reduced from ~12MB to 3MB

Additional notes

In order to reduce the number of baselines, I introduced an new option for the test driver: --tolerance-ratio. It defines what percentage of the data can fail the epsilon check. For the moment it only applies to the image comparisons. For instance, passing the option --tolerance-ratio 0.01 means that the number of pixels with difference should be less than 1% of all pixels. By default, this tolerance is 0.

Copyright

The copyright owner is CNES and has signed the ORFEO ToolBox Contributor License Agreement.

---

Check before merging:

• All discussions are resolved
• At least 2 votes from core developers, no vote.
• The feature branch is (reasonably) up-to-date with the base branch
• Dashboard is green
• Copyright owner has signed the ORFEO ToolBox Contributor License Agreement

CI/main_ci.cmake

@@ -77,6 +77,7 @@ ctest_configure(BUILD "${CTEST_BINARY_DIRECTORY}"
     )
 
 if ( NOT _configure_rv EQUAL 0 )
+  # stop processing here
   ctest_submit()
   message( FATAL_ERROR "An error occurs during ctest_configure.")
 endif()
@@ -87,7 +88,6 @@ ctest_build(BUILD "${CTEST_BINARY_DIRECTORY}"
             )
 
 if ( NOT _build_rv EQUAL 0 )
-  ctest_submit()
   message( SEND_ERROR "An error occurs during ctest_build.")
 endif()
 
@@ -97,7 +97,6 @@ ctest_test(PARALLEL_LEVEL 8
            )
 
 if ( NOT _test_rv EQUAL 0 )
-  ctest_submit()
   message( SEND_ERROR "An error occurs during ctest_test.")
 endif()