Fixes to enable tests on the new CI, and reduce tests verbosity.
Enable the tests on new CI, and reduce the log output (#1867 (closed)).
There are different types of changes in this branch:
- CI scripts:
- enable testing
- use xvfb to launch test (needed for GUI tests)
- remove unused WRAP_PYTHON3 option
- Make all tests pass:
- can't use Shark on ubuntu-native due to #1871 (closed)
- allow root to launch mpi-enabled tests (
- the RPC estimation/optimization functions are broken on Ossim 2.2.0, the tests don't crash but I had to remove some baselines checks for Ossim >= 2.2.0
- add some multi-baselines...
otb::WaveletOperatorBase, a weird crash was breaking all wavelet tests.
TestMain, I made sure that the
MPIFinalize()is called in all cases, to avoid the application hanging at closure. Removed early
ApplicationLauncherCommandLine, I had to disable
-testenvis present: since this option leaves exceptions uncaught, the
MPIFinalize()at the end of the application launcher is not called, and the TestDriver hangs on closure (see #1845).
- I made a small refactor of
TestHelper: better parsing of options, support of the
--tolerance-ratiooption (more details in the next section).
- Reduce unnecessary test verbosity:
- massive use of
otbLogMacro(Debug, ... )to control test verbosity at runtime.
- I changed the priority level of
No kwl metadata found in file ...to Debug, I think it is more relevant to log information when metadata has actually been found.
- Code factorization in
- the log size has been reduced from ~12MB to 3MB
- massive use of
In order to reduce the number of baselines, I introduced an new option for the test driver:
--tolerance-ratio. It defines what percentage of the data can fail the epsilon check. For the moment it only applies to the image comparisons. For instance, passing the option
--tolerance-ratio 0.01 means that the number of pixels with difference should be less than 1% of all pixels. By default, this tolerance is 0.
The copyright owner is CNES and has signed the ORFEO ToolBox Contributor License Agreement.
Check before merging:
- All discussions are resolved
- At least 2
👍votes from core developers, no 👎vote.
- The feature branch is (reasonably) up-to-date with the base branch
- Dashboard is green
- Copyright owner has signed the ORFEO ToolBox Contributor License Agreement