otb issueshttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues2024-03-05T09:32:20Zhttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2378Divide OTB modules in independent repositories2024-03-05T09:32:20ZTristan LaurentDivide OTB modules in independent repositories# Resume
The goal of this is to separate each modules located in https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/tree/develop/Modules in a separate repo (except Core). A link will be done from otb main repo and each module with the ...# Resume
The goal of this is to separate each modules located in https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/tree/develop/Modules in a separate repo (except Core). A link will be done from otb main repo and each module with the "submodule" git mechanism.
# Pro and cons
Separating modules have several advantages:
- Improve CI build time
- Improve unittest jobs and runs as only module related features are tested
- Issues will now be module-related
- Developers only download needed source code
The CI jobs may increase in complexity as we introduce dependencies with modules. Thus we need to trigger modules jobs from core.
# Plan
- [ ] According to @ytanguy , create a gitlab group to handle all modules in a separate repo. This will be easier for bug tracking and management
- [ ] Start with Miscellaneous module, use OTB modules https://www.orfeo-toolbox.org/CookBook-9.0/RemoteModules.html
- [ ] Create external git repo, link it as a submodule
- [ ] OTB core must download corresponding submodule if the cmake -DOTBGroup_Miscellaneous is turned on
- [ ] Compile it separately, you may use OTB core artifact
- [ ] Adapt CI to run corresponding build job and tests
- [ ] Adapt otb CI to start module job when there is a new dependency (i.e. OTB core)
- [ ] Adapt bug tracking for each project ?
- [ ] Add a Readme with compilation help, otb integration, and some exemples
- [ ] Repeat for Remote
- [ ] Repeat for FeatureExtraction
- [ ] Repeat for SAR
- [ ] Repeat for Segmentation
- [ ] Repeat for Hyperspectral
- [ ] Repeat for Learning
- [ ] Repeat for StereoProcessing10.0.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1671Remove hard-coded muParser expression library-wise2024-03-20T14:05:37ZJulien MichelRemove hard-coded muParser expression library-wise### What changes will be made and why they would make a better Orfeo ToolBox?
#### High level description
Some filters and classes use the `muParser` based filters with hard-coded expressions. This is bad for at least 2 reasons:
* `muP...### What changes will be made and why they would make a better Orfeo ToolBox?
#### High level description
Some filters and classes use the `muParser` based filters with hard-coded expressions. This is bad for at least 2 reasons:
* `muParser` based filters are not as fast as their functor based counter-parts
* `muParser` is an optional dependency. Using it heavily results in many applications and filters disabled if `muParser` is disabled (see #1670 for instance)
#### Risks and benefits
Only risk might be small changing with floating point baselines.
Benefit is a faster OTB less dependent on muParser.
#### Alternatives for implementations
We need to spot places where `muParser` is mis-used:
```
$ grep -R -l BandMath Modules/ | grep -v BandMath | grep -v Parser
Modules/Applications/AppProjection/app/otbGridBasedImageResampling.cxx
Modules/Applications/AppImageUtils/app/otbCompareImages.cxx
Modules/Applications/AppStereo/app/otbStereoFramework.cxx
Modules/Applications/AppStereo/app/otbBlockMatching.cxx
```
### Who will be developing the proposed changes?
TDB.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2330Application performances when using extended filename2023-05-02T08:08:16ZGermain SalguesApplication performances when using extended filenameWe encountered significant slow down when processing raster and using extended filename to access a specific band. The input raster is a JP2000 with several compressed band:
Band 1 Block=1024x1024 Type=Byte, ColorInterp=Red
Overviews:...We encountered significant slow down when processing raster and using extended filename to access a specific band. The input raster is a JP2000 with several compressed band:
Band 1 Block=1024x1024 Type=Byte, ColorInterp=Red
Overviews: 5490x5490, 2745x2745, 1373x1373, 687x687
Overviews: arbitrary
Image Structure Metadata:
COMPRESSION=JPEG2000
For example, the access to a specific band in a vector image using the OTB extended filename feature, seems to cause the problem, example:
> otbcli_Superimpose -inr DTM.tif -inm "L1CFOLDER/QI_DATA/MSK_QUALIT_B08.jp2?&bands=8" -out "./superimpose.tif" -interpolator linear -ram 6560
This command takes around 30sec, while the same command ran directly on a mono band raster (of the same resolution and content), is almost instantaneous. This is mostly likely cause by the combinaition of the otb extended file band and the raster inner compression (jp2 + internal overviews).https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2294The application RefineSensorModel is missing in version 8.0+2024-02-01T08:58:17Zhello-willyThe application RefineSensorModel is missing in version 8.0+# Refactoring the RefineSensorModel application
## Context
The application RefineSensorModel uses a set of tie points to adjust the parameters of a sensor model.
It uses an optimization algorithm (least-square) to fit the model to the...# Refactoring the RefineSensorModel application
## Context
The application RefineSensorModel uses a set of tie points to adjust the parameters of a sensor model.
It uses an optimization algorithm (least-square) to fit the model to the fit points.
The inputs of the application are
- a geom file containing the sensor model
- a text file containing the tie points
The outputs are
- a geom file containing the refined sensor model
- a text file containing output precision statistics
- a text file containing segments representing residues
Since OTB 8, this application is not available.
We want to reintroduce it, since it is of great interest.
## Details of the refactoring
To reintroduce RefineSensorModel, we need to address multiple issues
### OTB doesn't write geom files anymore
The new version of OTB can't produce a geom file as output of the application.
We propose to change the RefineSensorModel application to work at the product level.
Instead of reading and writing a geom file, it will read and write an image file.
Impact:
- The `ingeom` and `outgeom` parameters become `inImage` and `outImage`, and are of type "ParameterType_InputImage" instead of "ParameterType_InputFilename".
- Use the `SensorTransformFactory` to load the RPC or SAR model from the metadata of the input image.
### Hold the fit points
The fit points used to be stored in the SensorModelAdapter class.
This class was removed with OSSIM, so we need to store the fit points somewhere else.
The proposed class for this task is `SensorTransformBase`.
Impact:
- Add the attribute `m_TiePoints` and implement the function `AddTiePoint` to `SensorTransformBase`.
### Run the optimization
The optimization algorithm was provided by OSSIM.
So we need to implement the optimization algorithm and call it for each sensor model (SAR, RPC).
Impact:
- Implement the Least-Square Optimization algorithm ?
- Add an `Optimize` virtual function to `SensorTransformBase`.
- Implement an `Optimize` function in `RPCTransformBase` class that apply the optimization algorithm to the RPC model (RpcParam).
- Implement an `Optimize` function in `SARTransformBase` class that apply the optimization algorithm to the SAR model (SarParam).8.2.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2180Modernize the code of factories in OTB2022-01-13T13:42:02ZThibaut ROMAINModernize the code of factories in OTBShort summary
The current design of factories in OTB can be reworked to be more efficient, and more C++1x compliant.
At the moment we use an old fashioned code to handle concurrency and we register a new factory each time we request an...Short summary
The current design of factories in OTB can be reworked to be more efficient, and more C++1x compliant.
At the moment we use an old fashioned code to handle concurrency and we register a new factory each time we request an object creation. We could use std::once to register our factories once, or use singletons.
This change will not give a significant performance boost because objects instantiation done using factories is only a small part of objects instantiation in OTB, but it is worth modernizing this code to make it safer and easier to understandThibaut ROMAINThibaut ROMAINhttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2093Evolution of DEM handling in OTB 8.02022-09-20T16:09:44ZCédric TraizetEvolution of DEM handling in OTB 8.0The `otb::DEMHandler` class has been refactored to use GDAL instead of Ossim to compute heights. See #1762, !729 and the [wiki](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/wikis/Remove-OSSIM#re-implement-demhandeler).
The new cl...The `otb::DEMHandler` class has been refactored to use GDAL instead of Ossim to compute heights. See #1762, !729 and the [wiki](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/wikis/Remove-OSSIM#re-implement-demhandeler).
The new class is functional and replaces the old DEMHandler, renamed `ossimDEMHandler`.
Several evolutions are considered :
##### Modification of the `GetHeightAboveEllipsoid` method
The `GetHeightAboveEllipsoid` has been reimplemented to have the same behavior as the ossim DEMHandler. In case no geoid is available, the DEM height is returned. Instead, it would make more sense to return the sum of the DEM height and the default height above ellipsoid.
This change is easy to implement, however it might require to change some baseline (I am not sure how many tests are using a DEM but no geoid. I think it mostly concerns test of the `otb::DEMHandler` class itself).
##### Caching and multi-threading improvements
Currently, when requesting a height value, a 2x2 RasterIO request is made on the dem dataset. This is computationally expensive. Instead we could take advantage of GDAL caching and request data on larger region to reduce the number of request to the raster drivers.
Also, because the DEMHandler is implemented as a singleton containing a single dataset (a vrt of all input DEMs) and because a GDALDataset cannot be shared across threads, the current DEMHandler is monothreaded (using std::locks). A method that return a new vrt dataset with all dems and the geoid in the singleton could be created. This method could be used to process DEMs in a multi-threaded environment, for example in the `DEMToImageGenerator` class.
##### DEM files setup.
Currently, DEMs can be set using two different methods
* OpenDEMFile() : add a single raster file to the list of opened DEMs
* OpenDEMDirectory() : add all raster file in the directory to the list of opened DEMs. this is the same API as the old DEM handler.
while it is important to keep the `OpenDEMDirectory` method, as it is used to setup DEM in applications (elev.dem), and is wrapped in Monteverdi, the qgis-plugin, etc... Opening the whole directory can lead to opening a lot of files, for example when targeting directories with a lot of srtm tiles.
It would be nice to have a OpenDEMFiles(std::vector<std::string>> files) that only opens a list of files. We could also have smarter ways of opening DEMs, for example opening all DEMs in a directory that intersect with an input geographic region.
##### Remove `otb::ossimDEMHandle`
This class is still used to setup DEMs for ossim based RPCs and SAR model. Once they will be removed, we will be able to delete this class.8.2.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2013Improve non-regression testing methods2022-01-07T09:48:48ZYannick TANGUYImprove non-regression testing methodsOTBTestDriver is used to compare results (images, vector files, ascii files, ..) in OTB unit tests.
For ogr data, this driver compares different features (ie : nb of fields, nb of features, geometry, etc.) and then compares each feature ...OTBTestDriver is used to compare results (images, vector files, ascii files, ..) in OTB unit tests.
For ogr data, this driver compares different features (ie : nb of fields, nb of features, geometry, etc.) and then compares each feature by dropping content to an ASCII file and comparing reference and result.
=> There are a lot of temporary files written in temporary/testing folder and it may not be very efficient.
Here are some ideas to improve this behavior (thanks @gpasero for the explanations !) :
- the better solution would be to delegate this comparison to a specific OGR API (see GDAL API ?)
- we could compare md5 of result and reference file, and if there is a difference, compare the features : in most cases, it would save some time writing the temporary files.. But we need to get a MD5 sum API that we could call from the C++ test driver
- instead of writing ASCII files for each feature, we could drop content in two Strings and compare them. If there is a difference, the two Strings could be written to a file, as we do now.
=> This subject should be discussed to find the best solution !https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1917ITK 5 support2022-01-06T15:06:39ZVictor PoughonITK 5 supportIssue of !194
TODO:
* https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/merge_requests/194#note_78887Issue of !194
TODO:
* https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/merge_requests/194#note_78887Cédric TraizetCédric Traizethttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1909Refactor InputImageParameter::GetImage2019-05-27T13:45:27ZVictor PoughonRefactor InputImageParameter::GetImageAfter investigation during the fix of #1899 in !497, it was determined that `InputImageParameter::GetImage` should be refactored:
* Support multiple output types
* Make the getter constAfter investigation during the fix of #1899 in !497, it was determined that `InputImageParameter::GetImage` should be refactored:
* Support multiple output types
* Make the getter consthttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1885Use a test framework for unit tests (Follow-up from "Large refactoring of rad...2022-01-07T11:47:24ZJulien MichelUse a test framework for unit tests (Follow-up from "Large refactoring of radiometric indices")The following discussion from !464 should be addressed:
- [ ] @lhermitte started a [discussion](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/merge_requests/464#note_77961): (+3 comments)
> As this is a new test, what about us...The following discussion from !464 should be addressed:
- [ ] @lhermitte started a [discussion](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/merge_requests/464#note_77961): (+3 comments)
> As this is a new test, what about using a real unit test framework that'll display the bogus line?
In the discussion, `boost.test`, `catch2` and `doctest` are proposed. We should decide which one to pick, and identify current tests that could be refactored using a dedicated test framework.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1698Missing test for SparseWvltToAngleMapperListFilter2019-01-21T15:12:21ZManuel GrizonnetMissing test for SparseWvltToAngleMapperListFilterThe class SparseWvltToAngleMapperListFilter has a test named otbSparseWvltToAngleMapperListFilterTest (previously named otbSparseWvltToAngleMapperListFilterNewTest but renamed in MR !173) but there is no corresponding otb_add_test in the...The class SparseWvltToAngleMapperListFilter has a test named otbSparseWvltToAngleMapperListFilterTest (previously named otbSparseWvltToAngleMapperListFilterNewTest but renamed in MR !173) but there is no corresponding otb_add_test in the CMakeLists (module DimensionnalityReduction).
This bug was spotted when we remove all new test (this one was wrongly prefixed with the keywork New).
A test with proper input and baseline should be added here.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1528Filename VS VectorData2022-01-07T11:47:24ZRémi CressonFilename VS VectorData### What changes will be made and why they would make a better Orfeo ToolBox?
Hello guys,
This is an open question.
Should/could we enforce the following rule of thumb for vector data I/O?
* Use `ParameterType_InputVectorData` rather ...### What changes will be made and why they would make a better Orfeo ToolBox?
Hello guys,
This is an open question.
Should/could we enforce the following rule of thumb for vector data I/O?
* Use `ParameterType_InputVectorData` rather than `ParameterType_InputFilename` for input vector layers.
* Use `ParameterType_OutputVectorData` rather than `ParameterType_OutputFilename` for output vector layers.
Currently we use to mix both in applications, but in my opinion we should use the correct parameter type for a better integration of OTB applications in external projects.
#### High level description
Refactor some applications parameters.
#### Risks and benefits
Benefits: I/O data standardization in applications.
Risks: side effects ? (ParameterType_XXputFilename is traditionally used with ogr datasources)
#### Alternatives for implementations
### Who will be developing the proposed changes?