OTB 8.0 alpha 1 (Optical image processing) & alpha 2 (SAR image processing) and validation plan
The main goal of release 8.0 is to remove OSSIM from OTB dependencies. This implies the refactoring of a lot of classes and functions, many of which are in the core modules of the library. See this issue for a detailed plan of the refactoring. Before the actual release and release candidates, it is planned to release two alpha versions:
- OTB 8.0.0 alpha-1 : Optical processing, OTB still depends on Ossim, but only use the library in SAR processing
- OTB 8.0.0 alpha-2 : SAR processing, OTB does not depend on Ossim.
As many users do not use the SAR functionnalities of OTB, the first alpha will give them the opportunity to test the refactored functionalities as soon as possible. This includes:
- The Metadata framework
- The DEM Handler
- The radiometry framework (optical calibration and filters)
- The RPC framework
Given the scale of the refactoring, particular attention should be given to the validation of these changes.
The major part of the validation is done using the OTB CI.
The metadata parsing framework is mainly validated by the tests of the
OTBMetadata module, in particular :
otbImageMetadataTestXXXtests are used to validate the API of the
ioTvImageMetadataInterfaceTesttests are used to validate the
OpticalImageMetadataInterfaceclasses. All the implemented sensors are tested (one test per sensor).
Broadly speaking, a lot of tests are impacted by the changes in metadata parsing, and non regression on these tests is also used for the validation.
The only API changes in classes related to optical calibration is the use of
ImageMetadata objects instead of
KeywordLists. The test have not been modified and the changes are validated by verifying that there are no regression.
DEMHandler class API is validated using the
uaTvDEMHandlerXXX tests, testing DEM request on different configuration (with or without geoids, with or without DEM).
TODO: these tests are not at the right place at the moment, they should be moved from the OSSIMAdapters module to the
The RPC framework
The RPC transforms are now based around the GDAL functions
OTBIOGDAL module) is a wrapper around these classes. It is tested by the
The main entry points for these classes are
otb::InverseRPCTransform. These classes are used by the
GenericRSTransform class. See
GenericRSTransform is used in higher level classes dedicated to geometries class, including ImageWarp filters, stereo filters, orthorectification filters, and vector data projection filters. See the tests from the modules
As of 2020-12-02, while most tests produce the same results, differences are still observed in some cases. See !744 (merged). The regressions could be caused by the different implementations of RPC models in Ossim and GDAL. If it appears to be the case we will need to update the test baselines. In this case it would interesting to validate the refactored RPC models using an external geometry library (which one ?)
RPC validation manual tests:
Verify that the transform
geo->sensor->geo(forward model followed by inverse model), is close to the identity transform, and that the error is negligible compared to the sensor precision
Produce GCPs between an ortho-image and the corresponding sensor Image using the Sift algorithm (
HomologousPointsExtractionapplication) and use the pair of points to validate the model. For each pair compute the distance in pixel between the Forward transform of the ortho point and the sensor point, and the distance in meters between the Inverse transform of the sensor point and the ortho point. The potential biases of this test are:
- the accuracy of SIFT points
- the accuracy of the model used to compute the orthoimage
- the DEM used to compute the orthoimage
Some tests have been disabled. These tests are specific to SAR functionalities that cannot work while the SAR part of OTB has not been refactored.
Code quality :
We should ensure some quality metrics before the release alpha:
- Fix compilation warnings on all platforms
- Fix Bugs and CodeSmells from SonarQube in new code
- Ensure a good code coverage on new code (which value should we aim at ?)
Manual testing :
In addition to CI tests. It could be interesting to make additionnal validation tests manually. While OTB tests cover a large parts of use cases, these tests would have several benefits:
- Human based testing might highlight problems that CI have missed (It was the case or this bug)
- CI tests are done on small images. Several test are done on full product (Large inputs tests). However these tests never produce full output images, because it would take too much testing time and a lot of disk space. This means that OTB scaling is not tested.
- We could measure performances differences, in particular in RPC based tasks.
The test could be done using the
Orthorectifcation applications. The products from the
OTB-LargeInput repository could be used. Alternatively, we could use other products (some sample imagery is often available on providers websites).
Example of test procedure:
OpticalCalibrationon the product
- Visual inspection of the result in a GIS software
- check the metadata of the output product with
gdalinfo -mdd all(projection, no data if relevant, metadata written in the OTB domain)
- Run the application with OTB 7.2.0 (before the refactoring) and verify that there is no regression using the
The following sensors should be tested:
- Spot 5
- Spot 6
- Spot 7
- Worldview 2
- QuickBird 2