otb issueshttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues2019-04-16T12:57:37Zhttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1519Add spatialite to Superbuild2019-04-16T12:57:37ZJulien MichelAdd spatialite to SuperbuildSpatialite allows to use vector db with spatial index, resulting in faster spatial filtering operations.Spatialite allows to use vector db with spatial index, resulting in faster spatial filtering operations.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2061Available drivers in Datasource and VectorData2022-04-12T09:42:06ZCédric TraizetAvailable drivers in Datasource and VectorDataThe list of drivers available to open and write vector data is not the same in `otb::VectorData` and `otb::ogr::DataSource`. The lists of (file extensions / driver) are :
* For `otb::ogr::DataSource` :
``` cpp
const ExtensionDriverAss...The list of drivers available to open and write vector data is not the same in `otb::VectorData` and `otb::ogr::DataSource`. The lists of (file extensions / driver) are :
* For `otb::ogr::DataSource` :
``` cpp
const ExtensionDriverAssociation k_ExtensionDriverMap[] = {
{".SHP", "ESRI Shapefile"}, {".TAB", "MapInfo File"}, {".GML", "GML"}, {".GMT", "OGR_GMT"}, {".GPX", "GPX"},
{".SQLITE", "SQLite"}, {".KML", "KML"}, {".CSV", "CSV"}, {".GPKG", "GPKG"}};
```
* For `otb::VectorData` (OGRVectorDataIO)
``` cpp
const std::map<std::string, std::string> m_OGRExtensionsToDrivers = {
{".SHP", "ESRI Shapefile"}, {".TAB", "MapInfo File"}, {".GML", "GML"}, {".GPX", "GPX"}, {".SQLITE", "SQLite"}, {".KML", "KML"},
{".GMT", "OGR_GMT"}, {".GPKG", "GPKG"}, {".JSON", "GeoJSON"}, {".GEOJSON", "GeoJSON"}};
```
This is confusing and limiting for OTB users. Some application are based on `VectorData` and some are base on `otb::ogr::Datasource`.
I think the same list should be used for both classes (the union of the two list). The reading/writing is done by ogr so I don't think there is any limitation on this side.
See this [forum post](https://forum.orfeo-toolbox.org/t/problem-when-saving-geojson-file-with-sampleselection-app/700/2)https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1542Make SampleAugmentationFilter able to work inplace2018-03-28T09:04:17ZJulien MichelMake SampleAugmentationFilter able to work inplace`SampleAugmentationFilter` currently copies sample to a new output. It would be great if it could work inplace (see !25)`SampleAugmentationFilter` currently copies sample to a new output. It would be great if it could work inplace (see !25)https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1678Application improvements2018-08-20T12:03:28ZYannick TANGUYApplication improvementsThis story gathers list of improvements for OTB Application from the old wiki.
# General improvements
* Improve otbgui look and feel (big white spaces, default window size, alignment, parameters ordering, etc.)
* Add a "same as input" ou...This story gathers list of improvements for OTB Application from the old wiki.
# General improvements
* Improve otbgui look and feel (big white spaces, default window size, alignment, parameters ordering, etc.)
* Add a "same as input" output type
* Internationalization of otb applications (challenge: without making otbcli depend on qt)
* Improve application Engine API to allow to mark some parameters as "Advanced". It will allow to provide solution to simplify some of the application GUI (like fold all advanced parameters in a specific widget)
* Allow in-memory connection for OGRDataSource between applications (see TrainImagesClassifier)
* In otb::Wrapper::Application, provide a "GenerateBufferOutput()" function that calls Update() on an output image parameter without writing it.
* Interpolation on complex data
* Save N-dimensional images (where N>2) (see SOM maps)
* Display internal information of trained Machine Learning models (for instance, check the variables mostly used in a random forest model,...)
# Existing applications
* Implement IMORM approach in LSMSSmallRegionsMerging
* ComputeConfusionMatrix: Also compute precision, recall and F1 score and optionally write to a csv file
* Radiometric index: document which bands need to be filled for different indices
* Improve performance of sampling applications :
1. * In SampleExtraction : allow to compute features only on selected samples instead of computing them on the full image.
2. * When using a mask during training, CPU usage is not at 100% but rather 70% : why ?
3. * How to skip the processing of tiles without any sample ?
4. * Better estimation of RAM usage for sampling applications (at the moment, the memory cost of OGRData is not taken into account).
* (from M. Planells) In Orthorectification application: output the projected incidence angle (useful in case of SAR image orthorectification). See what is done in S1 toolbox Range Doppler Correction processing. It should output an optionnal image (1 local incidence angle per angle)
*TrainImageClassifierBetter*: explain what happens when no validation vector is provided in the TrainImageClassifier application-> in this case the rating is used and -> idea: display the number of extracted pixels in the log
*ComputeConfusionMatrix*: Improve the log of the confusion matrix (should be aligned properly)
*OpticalCalibration*: Improve OpticalCalibration doc to explain that in case of TOA To Image the input is not a DN image
# New applications
* Apply a polynomial correction
* Bundle block adjustment
* Morphological profiles and profiles classification
* The part of the object detection framework that can know be plugged in the new classification framework
* DSM shading and other stuff like this
* Proper denoising (the smoothing applications is quite poor and there are other filters in ITK)
* Sharpening (there are filters in ITK)
* HDR compression
* Haze correction
* Histogram application, lots of parameters (see numpy.histogram for inspiration) and which output format?
# C++ API
* Add topographic correction of reflectance. OTB filters can already take into account environment effects but not topo effects. It was requested on Mantis (https://bugs.orfeo-toolbox.org/view.php?id=1146)
* Re-write a decent 'compare-ogr' method for the test driverhttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1679Documentation improvements2022-01-06T14:43:11ZVictor PoughonDocumentation improvementsVarious documentation improvements (this is an evolving list):
##### CookBook
* [ ] Write MPI and parallel processing
* [X] Mention existing remote modules and add link to the list
##### Examples
* [ ] Finish reviewing examples after m...Various documentation improvements (this is an evolving list):
##### CookBook
* [ ] Write MPI and parallel processing
* [X] Mention existing remote modules and add link to the list
##### Examples
* [ ] Finish reviewing examples after migration to sphinx #1865
##### Tutorials
* [ ] Rewrite Segmentation exercises to use all in one app for LSMS, change images...
* [ ] Add a 30 minutes introduction with illustrations about SAR images (find Creative Commons support look at SAREDU project for instance https://saredu.dlr.de/)
##### Build system
* [ ] Remove examples tests?
* [ ] Remove examples tests baselines?
* [ ] Add RunExamples.py script in a new test
* [ ] Fix missing example usages
* [ ] Improve license header filtering in rst render (itk copyright)
* [ ] Handle `ApplicationExample`
* [ ] Move `Documentation/Cookbook/Art/` to Data/ after the LFS migration is complete (!387)https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1731Improve rasterization2022-01-07T11:47:25ZRémi CressonImprove rasterizationIn 7.0 `VectorData` might be replaced with `ogr::DataSource` and also in application parameter input/output (#1716).
Some filters for `VectorData` and `ogr::DataSource` have some overlap...
> - `VectorDataToLabelImageFilter` : `OGRDataS...In 7.0 `VectorData` might be replaced with `ogr::DataSource` and also in application parameter input/output (#1716).
Some filters for `VectorData` and `ogr::DataSource` have some overlap...
> - `VectorDataToLabelImageFilter` : `OGRDataSourceToLabelImageFilter`
> - `LabelImageToVectorDataFilter` : `LabelImageToOGRDataSourceFilter`
> - `VectorDataIntoImageProjectionFilter` : `GeometriesProjectionFilter`
(from !222)
...but some features were added to `VectorDataToLabelImageFilter` and not propagated to `OGRDataSourceToLabelImageFilter`
I propose to discuss a possible improvement for `OGRDataSourceToLabelImageFilter` (or `VectorDataToLabelImageFilter` is `VectorData` would be kept, but I don't feel so)
In my opinion, the following burning mode could be useful:
1. Binary (Burn the same value for each geometry. Params: Foreground value (ImagePixelValueType))
2. Field (Burn the value of the given field. Param: Field name (string))
3. FID (Burn the value corresponding to the (cast of the) geometry ID into ImagePixelValueType. (Param: Starting value (size_t) default could be background value))
4. Container (Burn the N-th value of the given container for the N-th geometry. Param: TContainer (e.g. std::unordered_map...))
For `Mosaic` and `ZonalStatistics` this would be useful, and I think we could improve also some other stuff with that.
What do you think?https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1798Handle no data in BandMath and BandMathX2024-03-27T16:19:19ZVictor PoughonHandle no data in BandMath and BandMathXRequested by multiple users: #1511, #1801, otb-users
Better handling of images with no-data in BandMath and BandMathX:
* A flag to ignore no-data values (maybe even on by default?)
* A nodata token available in the expression that is s...Requested by multiple users: #1511, #1801, otb-users
Better handling of images with no-data in BandMath and BandMathX:
* A flag to ignore no-data values (maybe even on by default?)
* A nodata token available in the expression that is set to the nodata value reported by gdal
* support `nan()` function in BandMathX #1801https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1849otbcli_DimensionalityReduction: Add options to output eigenvalues, apply inve...2022-01-07T07:22:27Zaloboaotbcli_DimensionalityReduction: Add options to output eigenvalues, apply inverse transform for PCA and correctely deal with no-dataAdd options to otbcli_DimensionalityReduction
1. ~~Output eigenvalues along eigenvectors for PCA~~
2. ~~Apply the inverse transform~~ (I understand this is already done using the "-outinv" parameter.
3. Deal with nodata
4. Option to ...Add options to otbcli_DimensionalityReduction
1. ~~Output eigenvalues along eigenvectors for PCA~~
2. ~~Apply the inverse transform~~ (I understand this is already done using the "-outinv" parameter.
3. Deal with nodata
4. Option to estimate stats from a certain % sampled from the image
5. ~~Option to output eigenvalues and eigenvectors only, avoiding calcalating and writing the components (useful to decide how many components to keep in a second run that would perform the actual calculation of the components)~~https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1882Standard verbose in applications2022-01-06T13:56:37ZLuc HermitteStandard verbose in applications# TL;DR:
Have all OTB applications have a `-v` option.
# Rationale
It'd be, very, very nice to have `-v|-verbose` option in all OTB applications in standard. That's what we have everywhere in *nix world. I'm not picky to require that...# TL;DR:
Have all OTB applications have a `-v` option.
# Rationale
It'd be, very, very nice to have `-v|-verbose` option in all OTB applications in standard. That's what we have everywhere in *nix world. I'm not picky to require that long option names should start with double dashes, but really, a `-v` would change life.
I know, there is a (let me search...) `OTB_LOGGER_LEVEL=DEBUG` that we can export. But the trouble is... searching, every time, while `-v` is a _de facto_ standard.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1911Add CMake flag to disable QGIS descriptors2022-01-06T14:40:03ZVictor PoughonAdd CMake flag to disable QGIS descriptorsSee: https://forum.orfeo-toolbox.org/t/otb-without-qgis/281
default value should be ON
related to #1910See: https://forum.orfeo-toolbox.org/t/otb-without-qgis/281
default value should be ON
related to #1910https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1915use libressl instead of openssl in dependency to curl2019-06-06T09:06:38ZRashad Kanavathuse libressl instead of openssl in dependency to curllibressl has cmake and windows support that openssl. This is much easy to manage that having a perl everytime to just configure openssl.libressl has cmake and windows support that openssl. This is much easy to manage that having a perl everytime to just configure openssl.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1917ITK 5 support2022-01-06T15:06:39ZVictor PoughonITK 5 supportIssue of !194
TODO:
* https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/merge_requests/194#note_78887Issue of !194
TODO:
* https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/merge_requests/194#note_78887Cédric TraizetCédric Traizethttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1926Set InputImageList from numpy array2022-01-06T14:44:53ZVictor PoughonSet InputImageList from numpy arrayCurrently in the python API there is a `SetImageFromNumpyArray`. But there is no way to set an input image list from a list of numpy arrays.Currently in the python API there is a `SetImageFromNumpyArray`. But there is no way to set an input image list from a list of numpy arrays.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1945Bring back FCLS2022-01-06T15:04:32ZAntoine RegimbeauBring back FCLS### Bring back FCLS
As pointed out in #1921 the NCLS algorithm we had (unmixing algorithm under Nonnegative Constrained Least Square) was buggy and was doing exactly the same as an UCLS (UnConstrained).
@terudel proposed to implement t...### Bring back FCLS
As pointed out in #1921 the NCLS algorithm we had (unmixing algorithm under Nonnegative Constrained Least Square) was buggy and was doing exactly the same as an UCLS (UnConstrained).
@terudel proposed to implement the FCLS algorithm (Fully Constrained) because:
>[..] it is widely used in the hyperspectral community. It is a linear spectral unmixing algorithm that is based both on the non-negativity constraint of factors (such as NCLS) but also on the fact that the sum of the coefficients of the abundance map must be equal to 1 (which makes sense from a physical point of view).
Moreover, if we believed the code, we used to have the two algorithms FCLS and NCLS in the HyperspectralUnmixing application. So it would be nice to have at least one back.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1967How to handle remote module shipping python scripts2019-09-24T19:33:51ZVictor PoughonHow to handle remote module shipping python scriptsDiapOTB is providing [some python scripts](https://gitlab.orfeo-toolbox.org/remote_modules/diapotb/tree/master/python_src) on top of regular OTB applications. They are not available in the binary packages we provide on the website. Would...DiapOTB is providing [some python scripts](https://gitlab.orfeo-toolbox.org/remote_modules/diapotb/tree/master/python_src) on top of regular OTB applications. They are not available in the binary packages we provide on the website. Would there be an easy way to distribute them to make them easily available for users of the binary packages?https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1972Improving log macro performances2022-01-06T15:01:07ZLuc HermitteImproving log macro performancesThe way logging macros are defined, they could consume a lot of time if passed slow functions. IOW,
```cpp
otbMsgDevMacro(somethingslowandcalledforeachpixel());
```
would yield to inefficient code.
The usual way to fix this would be
...The way logging macros are defined, they could consume a lot of time if passed slow functions. IOW,
```cpp
otbMsgDevMacro(somethingslowandcalledforeachpixel());
```
would yield to inefficient code.
The usual way to fix this would be
```cpp
#define otbLogMacro(level,msg) \
if (otb::Logger::Instance()->ShouldWeLog(level)) \ // <-- the new line
{ \
std::ostringstream itkmsg; \
itkmsg msg << "\n"; \
otb::Logger::Instance()->level(itkmsg.str().c_str()); \
}
```
BTW, the `std::string` shall not be converted into a `char const*` to be converted back into an `std::string` -> get rid of `.c_str()`.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2006BundleToPerfectSensor propagates sensor kwl of Pan image2022-01-06T15:24:22ZRémi CressonBundleToPerfectSensor propagates sensor kwl of Pan imageShort summary of the requested feature
BundleToPerfectSensor propagates sensor keyword list of Pan image. It could be more useful to have the keywordlist of the MS image instead.Short summary of the requested feature
BundleToPerfectSensor propagates sensor keyword list of Pan image. It could be more useful to have the keywordlist of the MS image instead.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2033Dimensionality Reduction: set bv pixels in the output as bv2022-01-06T15:52:52ZaloboaDimensionality Reduction: set bv pixels in the output as bvI observe the manual states:
> Background Value -bv float Background value to ignore in computation of the transformation matrix. Note that all pixels will still be processed when applying the transformation.
I think this a wrong choic...I observe the manual states:
> Background Value -bv float Background value to ignore in computation of the transformation matrix. Note that all pixels will still be processed when applying the transformation.
I think this a wrong choice: pixels with bv values in the input should be set as bv in the output.
Also, bv values should be ignored for calculating the input min,max values for rescaling (not sure they are, just in case)https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2038Training material : implement a Jupyter NoteBook on an open-access platform2020-03-24T09:52:10ZYannick TANGUYTraining material : implement a Jupyter NoteBook on an open-access platformOTB training material has evolved in the last years and is quite rich :
- 6 training modules, for beginners or advanced users. These modules contain slides, guide book and solutions to the exercises. They are adapted to training session...OTB training material has evolved in the last years and is quite rich :
- 6 training modules, for beginners or advanced users. These modules contain slides, guide book and solutions to the exercises. They are adapted to training sessions from 1 day to 3 days, depending on the number of modules we choose to present
- one small introduction to OTB, known as "OTB guided tour". For a small training session (3 hours), it allows to present OTB and start creating a small processing chain with OTB & Python.
We would like to improve our training material and propose to implement a jupyter notebooks on an open-access platform, such as Binder (https://mybinder.org/).
The idea is that new users could try and use OTB in a simple environment. Later, they will install it on their own computer and may improve their skills by using other training material.
This issue is here to discuss this idea, to test these kind of platform, at first with the "OTB guided tour".
@dyoussef : feel free to update this issue and propose your ideas !https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2041Missing ogr tools in binary package2022-01-07T07:51:13ZGuillaume PaseroMissing ogr tools in binary packageSince we already package gdal raster tools in the binary package, it would be nice to also have ogr tools (ogrinfo, ogr2ogr, ...). Should be simple to get.Since we already package gdal raster tools in the binary package, it would be nice to also have ogr tools (ogrinfo, ogr2ogr, ...). Should be simple to get.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2042otb application GetParameterImageBase on InputImageListParameter always retur...2022-01-06T16:10:19ZEsquis Benjaminotb application GetParameterImageBase on InputImageListParameter always return a FloatVectorImageDear,
When calling this function on an InputImageListParameter the return pointer is always casted into a FloatVectorImage while there is an accessor on InputImageParameter to get the real ImageBase* not casted.
This is particularly impo...Dear,
When calling this function on an InputImageListParameter the return pointer is always casted into a FloatVectorImage while there is an accessor on InputImageParameter to get the real ImageBase* not casted.
This is particularly important when chaining the applications with pointers of specifiq types (uint8 ...) that needs to keep their types.
Regardshttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2044Dynamic Convert Choose min/max histogram cuts2022-01-06T15:48:40ZJamesZDonlineDynamic Convert Choose min/max histogram cutsCurrently, Dynamic Convert has a Histogram quantile cutting feature which allows the image histogram to be trimmed, using a quantile value (in percent), before rescaling. It would be useful when using Dynamic Convert to also be able to c...Currently, Dynamic Convert has a Histogram quantile cutting feature which allows the image histogram to be trimmed, using a quantile value (in percent), before rescaling. It would be useful when using Dynamic Convert to also be able to choose the min and max values manually to trim the histogram.
This would be useful because I am working with tiles of very large multispectral images. If I tried to tile it all together before converting it, I'd have an image that is several hundred GB. What I want to do is convert pixel types to int and drop all of the bands except for RGB, while maintaining the same cut values for each tile, so everything will be on a consistent scale and take up much less drive space. The problem is that currently the cuts are calculated on a per-image basis which makes this impossible.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2062Option to remove the progress bar in the Python wrapper2022-12-16T10:38:19ZCédric TraizetOption to remove the progress bar in the Python wrapperIn the command line interface (`otbcli_XXX`), there is an option `-progress` available to hide the progress of an application. For example :
* With progress :
``` cpp
otbcli_EdgeExtraction -in S2A_ROI.tif -out /tmp/out.tif -progress...In the command line interface (`otbcli_XXX`), there is an option `-progress` available to hide the progress of an application. For example :
* With progress :
``` cpp
otbcli_EdgeExtraction -in S2A_ROI.tif -out /tmp/out.tif -progress true
2020-06-17 16:28:59 (INFO) EdgeExtraction: Default RAM limit for OTB is 256 MB
2020-06-17 16:28:59 (INFO) EdgeExtraction: GDAL maximum cache size is 794 MB
2020-06-17 16:29:00 (INFO) EdgeExtraction: OTB will use at most 12 threads
2020-06-17 16:29:00 (INFO): Estimated memory for full processing: 38.8794MB (avail.: 256 MB), optimal image partitioning: 1 blocks
2020-06-17 16:29:00 (INFO): File /tmp/out.tif will be written in 1 blocks of 1000x1000 pixels
Writing /tmp/out.tif...: 100% [**************************************************] (0s)
```
* Without progress :
``` cpp
otbcli_EdgeExtraction -in S2A_ROI.tif -out /tmp/out.tif -progress false
2020-06-17 16:29:05 (INFO) EdgeExtraction: Default RAM limit for OTB is 256 MB
2020-06-17 16:29:05 (INFO) EdgeExtraction: GDAL maximum cache size is 794 MB
2020-06-17 16:29:05 (INFO) EdgeExtraction: OTB will use at most 12 threads
2020-06-17 16:29:05 (INFO): Estimated memory for full processing: 38.8794MB (avail.: 256 MB), optimal image partitioning: 1 blocks
2020-06-17 16:29:05 (INFO): File /tmp/out.tif will be written in 1 blocks of 1000x1000 pixels
```
The Python wrapper does not have the same functionality and it could be interesting to add it.
When creating an application in Python an observer object is added on the pipeline, the progress is then redirected to the python console. Calling `RemoveAllObserver()` on the application will remove the progress reporting. However this API is not very explicit, and it can have side effects as all observers are removed.
A solution would be to store in the application Python object the tag associated to the observer return by ITK when creating the observer, to be able to remove the observer directly from Python with `RemoveObserver(tag)`. This could be encapsulated in a `Application.ReportProgress(bool)` method that removes the observer if there is one, or create another one if it has been previously destroyed.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2073Add fid features in SampleExtraction2021-11-08T10:12:26ZMathieu FauvelAdd fid features in SampleExtractionWhen pixel values are extracted using SampleExtraction app with the optional parameter "out", the columns of the generated vector file by SampleSelection are kept. However, the fid feature (provided by the shapefile or the sqlite file) i...When pixel values are extracted using SampleExtraction app with the optional parameter "out", the columns of the generated vector file by SampleSelection are kept. However, the fid feature (provided by the shapefile or the sqlite file) is lost. As far as I understand, since the file is possibly written using multiples threads, the initial pixel row order of the input sampling positions is lost and without the fid feature it is not possible to recover pixels extracted from different input images using the same input sampling position.
My example to make it clear: I need to extract the raw pixels, their corresponding Theia masks and their gap filled version. I use the same input sampling positions, but the output samples are in different raw orders. And I can't make a one to one affectation between raw, mask and gap filled extracted values. Because my data have a lot of temporal/spectral feature, the final files are CSV.
I solved my problem by creating a new feature in the vector file outputed by the SampleSelection app, before using the SampleExtraction app. I can then join the three CSVs according to this new feature.
But SampleExtraction could be modified to export the fid (or the row number) of the input sampling positions.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2095Consider switching to the C++11 GLIBC ABI2023-08-01T14:56:42ZLaurențiu NicolaConsider switching to the C++11 GLIBC ABIWe might want to enable the C++11 ABI (`-D_GLIBCXX_USE_CXX11_ABI=1`) in the Linux package for better compatibility with newer compilers.
See https://gitlab.orfeo-toolbox.org/s1-tiling/s1tilingsupportapplications/-/issues/4 for an example.We might want to enable the C++11 ABI (`-D_GLIBCXX_USE_CXX11_ABI=1`) in the Linux package for better compatibility with newer compilers.
See https://gitlab.orfeo-toolbox.org/s1-tiling/s1tilingsupportapplications/-/issues/4 for an example.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2101Output vector layers of LargeScaleMeanShift have not a correct geometry accor...2022-01-06T16:08:34ZaloboaOutput vector layers of LargeScaleMeanShift have not a correct geometry according to GEOS### Description
Output vector layers of LargeScaleMeanShift have not a correct geometry according to GEOS, as implemented in rgeos.
The layer can be fixed using geos tools.
Same problem is found if -mode raster is used, followed by gdal...### Description
Output vector layers of LargeScaleMeanShift have not a correct geometry according to GEOS, as implemented in rgeos.
The layer can be fixed using geos tools.
Same problem is found if -mode raster is used, followed by gdal_polygonize
### Steps to reproduce
`otbcli_LargeScaleMeanShift.bat -in QB_1_ortho.tif -spatialr 4 -ranger 80 -minsize 16 -tilesizex 256 -tilesizey 256 -cleanup 1 -mode.vector.out regions.shp`
in R:
```
> require(rgeos)
> require(rgdal)
> v1 <- readOGR(dsn="D:/FLUXPYRetal/Navarra/SegNavarra", layer="regions")
> gIsValid(v2)
[1] FALSE
Warning message:
In RGEOSUnaryPredFunc(spgeom, byid, "rgeos_isvalid") :
Ring Self-intersection at or near point 370624.65625896002 4831278.72278025
> v1f <- gSimplify(v1, tol = 0.00001)
> gIsValid(v1f)
[1] TRUE
```
### Configuration information
OS, OTB version or tag, information related to build (binaries, superbuild, system libs ...)https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2104Remove 6S2020-09-22T08:12:38ZMickael SavinaudRemove 6S### What changes will be made and why they would make a better Orfeo ToolBox?
6S code is outdated and very difficult to maintain. Other version are available but it could be complicated to update the code in OTB due to manual correction ...### What changes will be made and why they would make a better Orfeo ToolBox?
6S code is outdated and very difficult to maintain. Other version are available but it could be complicated to update the code in OTB due to manual correction done after f2c.
Therefore it could be interesting to remove 6S and keep outside the radiative transfer code from OTB.
The idea is to delegate the functionalities to an external code and managed file interface. For example we can use the [SOS](https://github.com/CNES/RadiativeTransferCode-SOS) code to generate the LUT as for MAJA software. Moreover the MAJA software will be released in open source, so we can reuse the code to read the data.
#### High level description
#### Risks and benefits
Mutualize code with MAJA
#### Alternatives for implementations
### Who will be developing the proposed changes?https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2105Add performance test monitoring into OTB CI2022-01-07T09:50:11ZMickael SavinaudAdd performance test monitoring into OTB CIIn order to facilitate the OTB development when large modifications are done. It could be interesting to monitor the performance of OTB.
Currently it is possible to use regression test to do that. Indeed CTest provide test time executio...In order to facilitate the OTB development when large modifications are done. It could be interesting to monitor the performance of OTB.
Currently it is possible to use regression test to do that. Indeed CTest provide test time execution and CDash display it. The idea could be to retrieve this test time and compare the test time of develop with feature branch.
@sdinot has perform some test with CDash API and it is very interesting:
* all test are available here: https://cdash.orfeo-toolbox.org/viewTest.php?buildid=53386
* successful tests are available here: https://cdash.orfeo-toolbox.org/viewTest.php?onlypassed&buildid=53386
* with API we can retrieve the corresponding JSON:
* https://cdash.orfeo-toolbox.org/api/v1/viewTest.php?buildid=53386
* https://cdash.orfeo-toolbox.org/api/v1/viewTest.php?onlypassed&buildid=53386
With [jq](https://stedolan.github.io/jq/) we can filter the JSON to retrieve the desired information and understand the content:
```
curl -s 'https://cdash.orfeo-toolbox.org/api/v1/viewTest.php?onlypassed&buildid=53386' | jq .
```
To retrieve the test time:
```
curl -s 'https://cdash.orfeo-toolbox.org/api/v1/viewTest.php?onlypassed&buildid=53386' | jq '.tests[] | { name: .name, time: .execTimeFull }'
```
or whitout JSON structure:
```
curl -s 'https://cdash.orfeo-toolbox.org/api/v1/viewTest.php?onlypassed&buildid=53386' | jq '.tests[] | .name, .execTimeFull'
```https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2108Add setup.py to install python Bindings2020-10-23T07:48:26ZEsquis BenjaminAdd setup.py to install python BindingsFor now the python binding is only installed in the install folder but not officially in the pythons libraries
It would be interesting to provide a setup.py to install the python bindings in the python workspace:
For example:
```
#!/us...For now the python binding is only installed in the install folder but not officially in the pythons libraries
It would be interesting to provide a setup.py to install the python bindings in the python workspace:
For example:
```
#!/usr/bin/env python
from distutils.core import setup
setup(name='otbApplication',
version='7.0',
description='OTB Python bindings',
author='OTB Team',
author_email='https://forum.orfeo-toolbox.org/',
url='https://www.orfeo-toolbox.org/',
packages=['otbApplication'],
package_data={'otbApplication': ['_otbApplication.so']},
)
```https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2110ConvertSensorToGeoPoint application with Elevation management2020-10-07T08:33:30ZJulien OsmanConvertSensorToGeoPoint application with Elevation managementIt would be [interesting](https://forum.orfeo-toolbox.org/t/convertsensortogeopoint-elevation-management/811) to be able to use a DEM as input of the ConvertSensorToGeoPoint application. The new RPC mechanism that will come with OTB 8.0 ...It would be [interesting](https://forum.orfeo-toolbox.org/t/convertsensortogeopoint-elevation-management/811) to be able to use a DEM as input of the ConvertSensorToGeoPoint application. The new RPC mechanism that will come with OTB 8.0 should make it easy to implement.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2118double precision would be better for ConvertSensorToGeoPoint application output2022-01-07T07:47:39ZJonathan Guinetdouble precision would be better for ConvertSensorToGeoPoint application outputlat/lon output are displayed in float precision https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/blob/develop/Modules/Applications/AppProjection/app/otbConvertSensorToGeoPoint.cxx#L78. Double precision would be better .lat/lon output are displayed in float precision https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/blob/develop/Modules/Applications/AppProjection/app/otbConvertSensorToGeoPoint.cxx#L78. Double precision would be better .https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2119Support otb::Image writers in OutpuImageParameter2020-11-02T14:20:26ZGuillaume PaseroSupport otb::Image writers in OutpuImageParameterIt would be nice if the applications could use `otb::Image` writers (in addition to `otb::VectorImage` writers). When the application produces an `otb::Image<T,2>` , and the output pixel type corresponds to `T`, we can therefore avoid th...It would be nice if the applications could use `otb::Image` writers (in addition to `otb::VectorImage` writers). When the application produces an `otb::Image<T,2>` , and the output pixel type corresponds to `T`, we can therefore avoid the double cast filters, and save some RAM space.
I know that applications using `otb::Image` internally is not generic, but it gives more performances in some projects outside OTB.
The implementation is not difficult:
* separate macros `CAST_IMAGE_BASE` and `CAST_VECTOR_IMAGE_BASE`
* separate functions `SwitchInput` and `SwitchVectorInput`https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2146ImageClassifier to handle a list of images/masks2021-02-11T08:37:32ZThomas TilakImageClassifier to handle a list of images/masksClassifying an image with [ImageClassifier](https://www.orfeo-toolbox.org/CookBook/Applications/app_ImageClassifier.html) could be (at least in my use case) a long operation.
When one need to classify several images with a model, the l...Classifying an image with [ImageClassifier](https://www.orfeo-toolbox.org/CookBook/Applications/app_ImageClassifier.html) could be (at least in my use case) a long operation.
When one need to classify several images with a model, the loading of the model is repeated for each image.
One improvement would be to accept a list of images/masks as input of `otbcli_ImageClassifier`.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2163Support Copernicus DEM2022-01-07T09:40:30ZMickael SavinaudSupport Copernicus DEM[Copernicus DEM](https://spacedata.copernicus.eu/web/cscda/dataset-details?articleId=394198) is provided free at 30m and 90m. It could nice to check if we support this DEM with the new framework.
[EDIT 2022-01-07]
This DEM is supported ...[Copernicus DEM](https://spacedata.copernicus.eu/web/cscda/dataset-details?articleId=394198) is provided free at 30m and 90m. It could nice to check if we support this DEM with the new framework.
[EDIT 2022-01-07]
This DEM is supported in OTB 8.0. We should had some tests.Cédric TraizetCédric Traizethttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2165Hoover Compare Segmentation: simplify output raster2021-03-03T12:22:24ZaloboaHoover Compare Segmentation: simplify output rasterShort summary of the requested feature
In Hoover Compare Segmentation, I would prefer a simpler output as a single-band raster with pixels ranging 1:4 and let the user color code as he/she wishes.Short summary of the requested feature
In Hoover Compare Segmentation, I would prefer a simpler output as a single-band raster with pixels ranging 1:4 and let the user color code as he/she wishes.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2171Support reading products from virtual file system2022-01-07T09:48:59ZMickael SavinaudSupport reading products from virtual file systemAs described by GDAL [here](https://gdal.org/user/virtual_file_systems.html), EO products are delivered as archive and in parallel more and more stored on network (in object storage for example).
Currently (with 8.0.0-alpha1) even if a...As described by GDAL [here](https://gdal.org/user/virtual_file_systems.html), EO products are delivered as archive and in parallel more and more stored on network (in object storage for example).
Currently (with 8.0.0-alpha1) even if a part of metadata are read with GDAL, we use specific code to retrieve the missing metadata. This code don't support virtual file system although the GDAL part yes. For example, you can made otbcli_ReadImage on a Sentinel-1 data in an archive and retrieve some metadata thanks to GDAL but not from the OTB part.
It would be nice to support this virtual file system in the OTB metadata. With that we can run an ortho on the Pleiades data without unarchive the product or run SARCalibration on a Sentinel1 data store in an object storage.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2174Synchronize the writing of several products from Python API.2022-01-27T09:27:05ZLuc HermitteSynchronize the writing of several products from Python API.## Feature description
It would be nice to have a way of using multiwriter from Python API without having to define a new application like `AppImageListWriter` in MAJA.
If we consider a flow like the following:
```
+----+ /-------...## Feature description
It would be nice to have a way of using multiwriter from Python API without having to define a new application like `AppImageListWriter` in MAJA.
If we consider a flow like the following:
```
+----+ /--------\ +-----+ /----------------\ +---------+ /------------\ +----------+ /----------\ +-------+
| S1 | --> | to_xyz | --> | XYZ | --> | extractNormals | --> | Normals | --> | computeLIA | --> | LIA | --> | OrthoLIA | --> | S2LIA |
+----+ \--------/ +-----+ \----------------/ +---------+ \------------/ +----------+ \----------/ +-------+
|
|
v
+------------+ /----------\ +----------+
| Sin | --> | OrthoSin | --> | S2Sin |
+------------+ \----------/ +----------+
```
We don't have any way (as of v7.2) to plug extra applications on separate paths after an application that produces several images.
In an ideal world, we could write something like
```python
import otbApplication as otb
to_xyz = otb.Registry.CreateApplication("to_xyz")
to_xyz.SetParameterString("in", "S1")
extract_normals = otb.Registry.CreateApplication("extractNormals")
extract_normals.SetParameterInputImage("in", to_xyz.GetParameterOutputImage("out"))
compute_lia = otb.Registry.CreateApplication("computeLIA")
compute_lia.SetParameterInputImage("in", extract_normals.GetParameterOutputImage("out"))
ortho_lia = otb.Registry.CreateApplication("OrthoLIA")
ortho_lia.SetParameterInputImage("in", compute_lia.GetParameterOutputImage("out.lia"))
ortho_lia.SetParameterString("out", "S2LIA")
ortho_sin = otb.Registry.CreateApplication("OrthoSIN")
ortho_sin.SetParameterInputImage("in", compute_lia.GetParameterOutputImage("out.sin"))
ortho_sin.SetParameterString("out", "S2SIN")
otb.WriteOutputs([ortho_lia, ortho_sin])
```
## Implementation
Add the static function `WriteOutputs` to `otb::Wrapper::Application`.
```c++
void WriteOutputs(std::vector<otb::Wrapper::Application> apps);
```
This function does the same work than MAJA's application `AppImageListWriter`: instantiate an otbMultiImageFileWriter filter, set its parameters and update it.
Add the signature of this function to `Application.i`https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2178The projection of the output of HomologousPointsExtraction should be the same...2022-04-12T05:45:56ZJonathan GuinetThe projection of the output of HomologousPointsExtraction should be the same than the input image## Requested feature
The application HomologousPointsExtraction systematically sets the `outvector`, which contains left\right matches, to the WGS84 crs.
https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/blob/develop/Modules/Applicat...## Requested feature
The application HomologousPointsExtraction systematically sets the `outvector`, which contains left\right matches, to the WGS84 crs.
https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/blob/develop/Modules/Applications/AppDescriptors/app/otbHomologousPointsExtraction.cxx#L511
This is not very natural. And it can lead to some complication, for instance if input images are not projected in WGS84 then raster and vector are not superimposable.
Expected behavior: Setting the output vector to the same projection than the input image.
## Conception
Instead of hard coding the value of [`projref`](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/blob/develop/Modules/Applications/AppDescriptors/app/otbHomologousPointsExtraction.cxx#L511), we should read it from the input image 1. `std::string projref = image1->GetImageMetadata()->GetProjectedGeometry();`
## Test
A test should be added to assert that the projection of the outputs is correct.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2194MPI is not initialzed when using the python API2022-01-20T16:34:38ZqueyrutoMPI is not initialzed when using the python API### Description
I installed the [SLIC](https://gitlab.orfeo-toolbox.org/remote_modules/otb-slic) remote module following the official OTB [documentation](https://www.orfeo-toolbox.org/CookBook-8.0/RemoteModules.html#installation-and-usa...### Description
I installed the [SLIC](https://gitlab.orfeo-toolbox.org/remote_modules/otb-slic) remote module following the official OTB [documentation](https://www.orfeo-toolbox.org/CookBook-8.0/RemoteModules.html#installation-and-usage). I built it against an installed OTB on the CNES Hal cluster (OTB v7.2 - python v3.7.2) :
```bash
mkdir /Path/to/Module/build && cd /Path/to/Module/build
cmake -DOTB_DIR=/PathTo/OTB/install -DOTB_BUILD_MODULE_AS_STANDALONE=ON -DCMAKE_INSTALL_PREFIX=/theModulePath/install -DCMAKE_INSTALL_RPATH=/theModulePath/install/lib -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=TRUE ../
make install
```
I also updated the following variables:
```bash
export OTB_APPLICATION_PATH=/theModuleInstallFolder/lib:$OTB_APPLICATION_PATH
export PATH=/theModuleInstallFolder/bin:$PATH
```
When I ran the SLIC remote module, I systematically got an MPI error:
```bash
[queyruto@node123 DATA]$ python
Python 3.7.2 (default, Aug 14 2019, 14:00:35)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import otbApplication
>>> app = otbApplication.Registry.CreateApplication("SLIC")
>>> params = {"in": "SENTINEL2X_all.tif", "out": "SLIC.tif" }
>>> app.SetParameters(params)
>>> app.ExecuteAndWriteOutput()
2021-05-07 15:00:04 (INFO) SLIC: Default RAM limit for OTB is 256 MB
2021-05-07 15:00:04 (INFO) SLIC: GDAL maximum cache size is 0 MB
2021-05-07 15:00:04 (INFO) SLIC: OTB will use at most 24 threads
Auto tiling mode selected, available RAM = 128MB
Number of x tiles = 1
Number of y tiles = 1
Starting segmentation
*** The MPI_Barrier() function was called before MPI_INIT was invoked.
*** This is disallowed by the MPI standard.
*** Your MPI job will now abort.
[node123.sis.cnes.fr:24702] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
```
### Steps to reproduce
Follow the steps above.
Note: the OTB that is installed on the CNES HPC is compiled with the MPI support activated.
### Configuration information
OS: Redhat 7
OTB: v7.2 with python 3.7.2
SLIC: clone from the master branch on 2021/05/10.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2222Question about haralick module and in particular the "normalisation" aspect2022-01-27T07:42:28Zaline deprezQuestion about haralick module and in particular the "normalisation" aspect### Description
I become aware with different processing on different S2 images that the values obtained within a same image are not independent.
Indeed when there is snow or cloud,… on a part of the image, it seems that a bias appears ...### Description
I become aware with different processing on different S2 images that the values obtained within a same image are not independent.
Indeed when there is snow or cloud,… on a part of the image, it seems that a bias appears for the values on the all image.
After a discussion bu e-mails with Julien Michel, he assumes that it is perhaps a bug.
Could you please tell me more about the way of the computation of haralick feature computation ?
And perhaps (if the behavior of the module is normal) give me some advices of pre and/or post processing in order to obtain comparable values over time on my area of interest whatever the spectral content of the other parts of the image.
Thanks a lot for your help
Aline Déprezhttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2227OTB develop version name2022-08-30T13:26:19ZCédric TraizetOTB develop version nameThe command line applications display the current OTB version. For example in OTB 7.4.0 the following message appears when calling an application:
```
This is the EdgeExtraction application, version 7.4.0
```
In between version the sam...The command line applications display the current OTB version. For example in OTB 7.4.0 the following message appears when calling an application:
```
This is the EdgeExtraction application, version 7.4.0
```
In between version the same message is displayed, for example the current develop branch also return version 7.4.0. It would be interesting to print the next minor or major version, as well as the current branch name and commit sha for these versions. For example the current develop version would return
```
This is the EdgeExtraction application, version 8.0.0-develop-de58687e
```8.2.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2236Wrap the DEMHandler in Python2022-08-30T13:26:12ZCédric TraizetWrap the DEMHandler in PythonThe DEMHandler is managed by a singleton in OTB. In the Python wrapper the elevation parameters in one application affects the elevation management of the subsequent applications.
It could be interesting to wrap the DEMHandler in Swig s...The DEMHandler is managed by a singleton in OTB. In the Python wrapper the elevation parameters in one application affects the elevation management of the subsequent applications.
It could be interesting to wrap the DEMHandler in Swig so that we can setup the DEMHandler at the wrapper level. I'm thinking of something like:
``` python
import otbApplication as otb
otb.dem.OpenDEMDirectory("/path/to/srtm/dir")
print(otb.dem.GetHeightAboveEllipsoid(lon,lat))
app = otb.Registry.CreateApplication("OrthoRectification")
app.SetParameterString("io.in", "path/to/filename")
app.SetParameterString("io.out", "out.tif")
# we don't have to set the elevation parameters here
app.ExecuteAndWriteOutput()
otb.dem.ClearDEMs()
```8.2.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2268Support metadata option of GDAL as extended filename2022-04-04T14:28:30ZMickael SavinaudSupport metadata option of GDAL as extended filenameAdd the possibility to set metadata key if possible in the output of otb as for GDAL (with `-mo` key cf.https://gdal.org/programs/gdal_translate.html#cmdoption-gdal_translate-mo for gdaltranslate)
It could be useful to set metadata inst...Add the possibility to set metadata key if possible in the output of otb as for GDAL (with `-mo` key cf.https://gdal.org/programs/gdal_translate.html#cmdoption-gdal_translate-mo for gdaltranslate)
It could be useful to set metadata instead of openning the file after writing to set the metadata.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2273Sentinel-1 metadata reader don't support manifest.safe as input2022-04-13T07:21:11ZMickael SavinaudSentinel-1 metadata reader don't support manifest.safe as inputIf you try to read information from the manifest.safe of a product, `otbcli_ReadImageInfo` return the following warning:
``` bash
otbcli_ReadImageInfo -in /eodata/Sentinel-1/SAR/GRD/2020/08/30/S1A_IW_GRDH_1SDV_20200830T025119_20200830T02...If you try to read information from the manifest.safe of a product, `otbcli_ReadImageInfo` return the following warning:
``` bash
otbcli_ReadImageInfo -in /eodata/Sentinel-1/SAR/GRD/2020/08/30/S1A_IW_GRDH_1SDV_20200830T025119_20200830T025144_034130_03F6BD_0EC0.SAFE/manifest.safe
2022-04-11 09:27:58 (INFO) ReadImageInfo: Default RAM limit for OTB is 256 MB
2022-04-11 09:27:58 (INFO) ReadImageInfo: GDAL maximum cache size is 802 MB
2022-04-11 09:27:58 (INFO) ReadImageInfo: OTB will use at most 4 threads
2022-04-11 09:28:00 (WARNING): Unable to parse XML file /eodata/Sentinel-1/SAR/GRD/2020/08/30/manifest.safe
2022-04-11 09:28:00 (INFO): Loading metadata from official product
2022-04-11 09:28:00 (WARNING): The SensorTransform factory could not find a compatible Sensor Transform
2022-04-11 09:28:00 (WARNING): The SensorTransform factory could not find a compatible Sensor Transform
2022-04-11 09:28:00 (WARNING): The SensorTransform factory could not find a compatible Sensor Transform
2022-04-11 09:28:00 (INFO) ReadImageInfo:
...
```
More information in [S1A_IW_GRDH_1SDV_20200830T025119_20200830T025144_034130_03F6BD_0EC0.SAFE.out](/uploads/7a091af0fccd498becf9b63f2309ec3a/S1A_IW_GRDH_1SDV_20200830T025119_20200830T025144_034130_03F6BD_0EC0.SAFE.out)
OTB don't recognize the fact it is a Sentinel-1!
If you pass directly the tiff file, OTB detect the Sentinel-1 product:
```
otbcli_ReadImageInfo -in /eodata/Sentinel-1/SAR/GRD/2020/08/30/S1A_IW_GRDH_1SDV_20200830T025119_20200830T025144_034130_03F6BD_0EC0.SAFE/measurement/s1a-iw-grd-vh-20200830t025119-20200830t025144-034130-03f6bd-002.tiff
2022-04-11 09:35:29 (INFO) ReadImageInfo: Default RAM limit for OTB is 256 MB
2022-04-11 09:35:29 (INFO) ReadImageInfo: GDAL maximum cache size is 802 MB
2022-04-11 09:35:29 (INFO) ReadImageInfo: OTB will use at most 4 threads
ERROR 1: PROJ: proj_create_from_database: ellipsoid not found
ERROR 1: PROJ: proj_create_from_database: ellipsoid not found
2022-04-11 09:35:31 (INFO): Loading metadata from official product
2022-04-11 09:35:31 (INFO) ReadImageInfo:
Image general information:
Number of bands : 1
Data type : unsigned_short
No data flags : Not found
Start index : [0,0]
Size : [25273,16795]
Origin : [0.5,0.5]
Spacing : [1,1]
Estimated ground spacing (in meters): [10.0425,10.1884]
Image acquisition information:
Sensor : SENTINEL-1A
Acquisition time : 2020-08-30T02:51:19.023121Z
...
```
If you compare to `gdalinfo`, the behavior is the opposite: use the manifest.safe detect the product and its sub-dataset and the tiff not (cf. [Sentinel-1 SAFE driver](https://gdal.org/drivers/raster/safe.html))
For example you can run the following command:
```
gdalinfo SENTINEL1_CALIB:UNCALIB:/eodata/Sentinel-1/SAR/GRD/2020/08/30/S1A_IW_GRDH_1SDV_20200830T025119_20200830T025144_034130_03F6BD_0EC0.SAFE:IW_VH:AMPLITUDE
```
It could be interesting to see how to converge on that to avoid misunderstanding between otb and gdal.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2287Missing feature in DEMHandler2023-11-28T09:00:59ZLuc HermitteMissing feature in DEMHandler**TL;DR:** Provide `DEMHandler::GetGeoidHeight()`
## Rationale
`DEMHandler::GetHeightAboveEllipsoid()` has a convoluted behaviour. Depending on what is visible it would return:
- either geoid offset + DEM offset
- or DEM offset -- alm...**TL;DR:** Provide `DEMHandler::GetGeoidHeight()`
## Rationale
`DEMHandler::GetHeightAboveEllipsoid()` has a convoluted behaviour. Depending on what is visible it would return:
- either geoid offset + DEM offset
- or DEM offset -- almost equivalent to `GetHeightAboveMSL()`
- or default height above ellipsoid
- or geoid offset
Sometimes we only need the geoid offset (it's the case of DiapOTB SARDEMProjection application).
We could make sure to have no DEM information configured, but then the application can not be chained into memory with another application that will need DEM information. Indeed `DEMHandler` is still mostly used as a global object (a singleton) (which is another known issue #2093), which makes it impossible to simultaneously have and not have DEM information.
Even if we don't chain such applications _in-memory_, we must not forget to clear DEM configuration if the applications are used through Python API and not isolated in different _subprocesses_. Indeed the memory would be shared between the various application object, and as such all the singletons...
As a workaround, SARDEMprojection detects `$OTB_GEOID_FILE` value to directly read geoid data from that file, and it falls back to `DEMHandler::GetHeightAboveEllipsoid()` if it's not possible. It's much too easy to use incorrectly.
That's why it'd be better to provide a dedicated `DEMHandler::GetGeoidHeight()` that would simplify applications such as SARDEMProjection, and that would avoid troubles regarding whether we configure DEM information or not in the global `DEMHandler`.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2300Allow disconnected builds of remote modules2022-09-05T12:27:17ZStefanBruensAllow disconnected builds of remote modulesAllow/simplify builds of remote modules when no network is available
Currently, OTB requires remote modules to be fetched at cmake configure time. This is problematic for several reasons:
- Build hosts are often isolated for security r...Allow/simplify builds of remote modules when no network is available
Currently, OTB requires remote modules to be fetched at cmake configure time. This is problematic for several reasons:
- Build hosts are often isolated for security reasons
- Fetching git HEAD of a branch makes a build non-reproducible, as HEAD may change
Preferably, OTB would allow to supply remote modules as tarballs (or expanded in the appropriate directory).
Although it is possible to download e.g. https://gitlab.orfeo-toolbox.org/jinglada/temporalgapfilling/-/archive/master/temporalgapfilling-master.tar.bz2 and expand it to the correct directory, this is rejected by `otb_fetch_content` as the tarball does not contain the `.git` subdirectory.
CMake since 3.11 has the [FetchContent module](https://cmake.org/cmake/help/v3.11/module/FetchContent.html) which does everything OTBs homegrown fetch code does, but also allows to disable fetching content using the `FETCHCONTENT_FULLY_DISCONNECTED` cache variable. Using `FetchContent` would require a small bump of the minimum version, from 3.10.2 to 3.11.9.0.0Thibaut ROMAINThibaut ROMAINhttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2304Use GDAL metadata from an input raster gdal_translated from a DIMAP product2022-10-18T10:41:04ZRémi CressonUse GDAL metadata from an input raster gdal_translated from a DIMAP productIt looks like OTB 8.0.1 is not able to use metadata that are stored in Geotiff metadata, when doing some sensor-based operations (at least for optical calibration of Spot-6 and Spot-7 DIMAP products)
This is an example with a DIMAP from...It looks like OTB 8.0.1 is not able to use metadata that are stored in Geotiff metadata, when doing some sensor-based operations (at least for optical calibration of Spot-6 and Spot-7 DIMAP products)
This is an example with a DIMAP from Spot-7. My goal is to convert the XS image to TOA.
Here we go in plain OTB:
```bash
otbcli_OpticalCalibration -in DIM_SPOT7_MS_2021..._1.XML -out toa.tif
```
Everything works fine.
Now suppose I have an intermediate product, for instance a COG for the XS image.
```bash
gdal_translate DIM_SPOT7_MS_2021..._1.XML cog.tif -of COG
```
Now I can do some `gdalinfo` on the `cog.tif`
```
gdalinfo cog.tif
```
We can see that GDAL (3.4.2) has exported the product metadata in the geotiff. It has everything we need for the optical calibration:
```
Metadata:
AREA_OR_POINT=Area
BAND_MODE=PMS
CLOUDCOVER_MEASURE_DESC=Region of interest mask
CLOUDCOVER_MEASURE_NAME=Area_Of_Interest (ROI)
CLOUDCOVER_MEASURE_TYPE=AUTOMATIC
...
...
Band 1 Block=512x512 Type=UInt16, ColorInterp=Gray
Overviews: 5069x5171, 2534x2585, 1267x1292, 633x646, 316x323
Metadata:
RADIANCE_BIAS=0
RADIANCE_CALIBRATION_DATE=2020-09-22T12:00:00Z
RADIANCE_GAIN=8.07
RADIANCE_MEASURE_DESC=Raw radiometric counts (DN) to TOA Radiance (L). Formulae L=DN/GAIN+BIAS
RADIANCE_MEASURE_UNCERTAINTY=4
RADIANCE_MEASURE_UNIT=watt/m2/steradians/micrometers
SOLAR_IRRADIANCE_CALIBRATION_DATE=2011-09-20T12:00:00Z
...
```
However, OTB fails to use this metadata.
My guess is that the metadata factory fails to recognize the sensor.
I believe OTB should be able to do that, it would be a very useful feature.
Maybe that applies to other kind of products as well (e.g. PHR).
Do you think that is easily implementable in OTB 8.x ?
Could someone can point me to the right C++ classes and metadata-classes documentation so I can take a look?
Many thanks,
Rémihttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2309OTB must be able to write Cloud Optimized Geotiff (COG)2022-09-29T11:54:15ZRémi CressonOTB must be able to write Cloud Optimized Geotiff (COG)Cloud Optimized Geotiff (COG) has gained increasing popularity in recent years, thanks to its ability to transit smoothly in HTTP requests, enabling unprecedented rasters interoperability.
OTB is able to read COG. However, it's not able ...Cloud Optimized Geotiff (COG) has gained increasing popularity in recent years, thanks to its ability to transit smoothly in HTTP requests, enabling unprecedented rasters interoperability.
OTB is able to read COG. However, it's not able to write COG with overviews, a crucial feature. IMO the implementation of COG writing in OTB would be huge.
# GDAL does everything
A lot of work has been made in recent GDAL versions to access and write COG more easily.
For instance since GDAL 3.1 we can simply convert any raster into COG using [[1]](https://www.cogeo.org/developers-guide.html):
```bash
gdal_translate input.tif output_cog.tif -of COG -co COMPRESS=LZW
```
A lot of work has also been done in GDAL to read COG files through most kind of online providers based on the `/vsicurl` provider.
Thanks to that, OTB is able to read most of online COG smoothly (there are still some small problems when the input URI has some `?` chars and OTB believes its an extended filename... however that's not the topic of this issue, and I'll submit another one soon, or propose an MR for that since it's not a big issue).
# OTB is not able to write COD with overviews
But right now, OTB (8.1.0) is **not able to write COG with overviews** (the issue has been described on the forum [here](https://forum.orfeo-toolbox.org/t/does-otb-support-cloud-optimized-geotiffs-in-any-manner/1038)).
The following trick enables to write COG, however output raster don't include overviews (which is one crucial feature of COG).
```bash
otbcli_SomeApplication ... -out "/path/to/output.tif?&gdal:co:TILED=YES&gdal:co:COPY_SRC_OVERVIEWS=YES"
gdalinfo /tmp/toto.tif | grep Overviews # Nothing here...
```
And of course using `gdaladdo` on the output after its creation doesn't lead to a valid COG (we can use `dist-packages/osgeo_utils/samples/validate_cloud_optimized_geotiff.py` to know if it's a valid COG).
# What could be done
First we should investigate if the creation of a COG is a streamable process. Given how overviews are computed, I am tempted to answer "yes".
**However** quick hacking in the `otbGDALImageIO.cxx` (replacing 'GTiff' drivers with 'COG') gaves me this message: `(WARNING): The file format of /tmp/cogtest.tif does not support streaming. All data will be loaded to memory`.
I don't know how this should be implemented in term of user interface, but I was thinking maybe with a COG-specific extended filename, and/or an environment variable to default the writing of all output rasters in the COG way.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2310Reduce memory footprint when the output image is not streamable2022-10-03T07:01:21ZRémi CressonReduce memory footprint when the output image is not streamableI propose to implement something in the `ImageFileWriter` to reduce the memory footprint when the output image is not streamable (e.g. when output raster is JPEG2000 or COG (see investigations with `GDALImageIO` in #2309)).
Indeed when t...I propose to implement something in the `ImageFileWriter` to reduce the memory footprint when the output image is not streamable (e.g. when output raster is JPEG2000 or COG (see investigations with `GDALImageIO` in #2309)).
Indeed when the output format `CanStreamWrite` is `False`, the whole image is generated once, without streaming.
The downside is that every pipeline block is triggered over the `LargestPossibleRegion`, which may require a lot of memory, especially when the pipeline has many blocks (or greedy ones!).
A simple enhancement could consist in using an in-memory image object, fully allocated, that is updated with an explicit streaming mechanism that uses the original `m_StreamingManager` parameters.
Of course sometimes we want to keep the original behavior (default).Rémi CressonRémi Cressonhttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2311Refactoring task for the modularization of OTB2023-11-28T09:00:57ZThibaut ROMAINRefactoring task for the modularization of OTB**New modular architecture**
![otb_9_archi](/uploads/219354ce907b36807efaf3a0e0dfac18/otb_9_archi.png)
**1. Simplify CMake options**
In OTB there is almost 40 cmake options when you call ccmake which is a bit confusing for the average...**New modular architecture**
![otb_9_archi](/uploads/219354ce907b36807efaf3a0e0dfac18/otb_9_archi.png)
**1. Simplify CMake options**
In OTB there is almost 40 cmake options when you call ccmake which is a bit confusing for the average user wanting to compile OTB for his project. A lot of option are about the dependencies of OTB, where you can mix system and superbuild dependencies, which can lead to eratic behaviors which the OTB team can't give a lot of time to debug.
- [x] Propose a simplified approach for the dependencies:
The user who wants to compile OTB will be given two options:
- Use all dependencies from the Superbuild which will be built before OTB
- Use all system dependencies, which will be listed in the cookbook for a one line simple installation
This approach permits to hide all USE_SYSTEM_DEPNAME flags which complicates the cmake process. The user will have to enable or disable the only flag `USE_SUPERBUILD_DEPS` which will activate/deactivate the flags USE_SYSTEM_XXX in the background
- [x] Review of all other cmake variables to determine which ones are mandatory, and which ones can be hidden/removed
- [x] Create OTB_BUILD_CORE, OTB_BUILD_LEARNING ... cmake targets to compile thematic modules
Instead of the current OTBGroup_XXX which is not thematic and can lead to misunderstandings for beginners, we propose to add options which will correspond to the future modules of OTB, which will be packaged with the same name.
**2. New source organisation**
The main idea is to reorganize the modules of OTB in coherent groups, which implies a reorganization of the source code.
Here is an example of a module (Core for example):
- Applications
- Functional_Tests : which will call one or a bunch of applications related to real treatments in production chains
- SubModule1... (For example IO)
- src
- include
- unit_tests
- SubModule2... (For example Common)
- src
- include
- unit_tests
- ...
With this organization, we have the applications related to this module, the tests calling those applications, and a list of submodules which contains filters/C++ classes grouped by theme. These filters are unit tested in each sub module.
To do this reorganization, the following guideline has been done:
- [x] Create a folder for each module (Learning, Stereo, SAR...)
- [x] Move src, include, tests and the related CMakeLists of each submodule to its related module
- [x] Connect CMAKE variables (OTB_BUILD_CORE etc) to the right module folder
- [x] Adapt the declared dependencies of each module in the CMakeLists
- [x] Reactivate the unit tests on the CI for each module
- [x] Reactivate the app tests ont the CI for each module
- [x] Package OTB Core and OTB Full with the current packaging method
- [x] Test the compilation of remote modules with otb core
- [x] move "to be deprecated" apps to a miscellaneous folder
**3. Test refactoring**
All the tests (TU + app) for each modules have been reactivated on the CI. Now it is important to see if all those tests are relevant.
- **Unit test + app tests**
A Lot of baselines and input has been added since the beginning of the project, but some tests have been deprecated. In order to make this refactoring relevant, we will follow this guideline :
- [x] Make a listing of currently used baselines/input files, to see what file is not needed anymore, and if some files are not too old to be relevant.
- [ ] See if there are any non relevant tests
- [ ] Determine which largeInput tests can be transformed into functional tests
- **Use cases for validation (fonctional tests)**
The current LargeInput tests are kind of outdated and not really relevant of real production chains.
The idea is to replace thoses tests with functional tests, containint one or a sequence of applications, which is the case in production chains.
We have to determine real use cases with production teams, to create those functional tests. They will be present in each thematic module.
While the unit test + app tests will be run on the CI, the new functional tests will be run on the HAL cluster
**4. New packaging**
The current packaging of OTB contains everything, all dependencies, all apps... Makeself is currently used to deliver a .run.
At the moment, there is an OTB Core package, and a full OTB package produced by the CI.
The idea is to use CPack which is integrated with cmake, to deliver packages of each OTB modules, that can install along the otb-core package.
For example, a user needs to do some machine learning with OTB, he should then install otb-core package which contains the core modules / apps of OTB and its dependencies, and then he will install otb-ML module, which will be installed in the same install tree.
This modularity permits each user to customize its installation, instead of having a big package while using 50% of it.9.0.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2315Separate tool for stitching segments2022-11-17T07:50:45ZAndrew PlowrightSeparate tool for stitching segmentsI use the mean shift segmentation tool a lot, often for large datasets. I just finished running it for 30 hours to generate a 12GB geopackage.
Given the length of time this takes, I'd be very useful to have a workflow that can be interr...I use the mean shift segmentation tool a lot, often for large datasets. I just finished running it for 30 hours to generate a 12GB geopackage.
Given the length of time this takes, I'd be very useful to have a workflow that can be interrupted. More than once I've had unexpected stoppages and lost hours of processing time.
Ideally, given that my data is usually already tiled, it'd run the segmentation tool independently for each tile, **and then** stitch it all together at the end.
Otherwise, thank you very much for developing these excellent tools. They're very useful. Cheers.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2320Eager mode for ReadImageFilter2024-01-09T14:11:43ZRémi CressonEager mode for ReadImageFilterI was wondering if one could enable/disable a specific behavior of the processing workflow.
In particular I am wondering if the reading and the processing operations could be done in parallel in an OTB pipeline.
This could be an interest...I was wondering if one could enable/disable a specific behavior of the processing workflow.
In particular I am wondering if the reading and the processing operations could be done in parallel in an OTB pipeline.
This could be an interesting feature when accessing the input takes a while, e.g. when processing remote images (/vsicurl/http://.../image.tif). Indeed one region could be downloaded while another one is processed.
The typical processing flow in OTB is the following (we consider a simple pipeline composed of 1 reader --> 1 filter --> 1 writer):
1. Propagates the filter's requested output region to the reader. At the end of this step, the filter knows which region of the input it has to fetch.
2. Read the input data over this requested region
3. Process the requested region
4. Write the output
It's sequential.
What if we could run step 2 and 3 in parallel?
I am just wondering if that is somehow possible.
The only thing to do is for the reader, to not wait until the filter ask him the data, to go grab it. But this supposes that the reader knows in advance what the next requested region would be. I am wondering if somehow this is doable in the ITK framework?
I'll ask for toughs here, then I guess I will ask the ITK community...
Thanks,
Rémihttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2321pleiade neo pansharpening2023-10-19T14:02:43Znicodebopleiade neo pansharpeningHello and happy new year,
I tried to use the bundletoperfectsensor application on pleiades neo images and the results seems blurry.
I'm wondering if this issue comes from the input data or the pansharpening algorithm.Hello and happy new year,
I tried to use the bundletoperfectsensor application on pleiades neo images and the results seems blurry.
I'm wondering if this issue comes from the input data or the pansharpening algorithm.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2325SPOT5 orthorectification fails in 8.0.12023-05-02T08:08:16Zadescamps-aslSPOT5 orthorectification fails in 8.0.1### Description
Applying the orthorectification process in monteverdi fails when the input image is loaded with the following error: "(FATAL) OrthoRectification: itk::ERROR: ImageToGenericRSOutputParameters(0000024E4F8AD130): No informa...### Description
Applying the orthorectification process in monteverdi fails when the input image is loaded with the following error: "(FATAL) OrthoRectification: itk::ERROR: ImageToGenericRSOutputParameters(0000024E4F8AD130): No information in the metadata, please set an image with non empty metadata"
I get the same error with the otbcli_Orthorectification program.
It worked in 7.4.1, but fails in 8.0.1 and 8.1.1
### Steps to reproduce
In Monteverdi, open OrthoRectification application and select the SPOT5 image as input. The error should appear in the log tab.
### Configuration information
Tested on :
- 8.1.1 (Windows Monteverdi binary) : fails
- 8.1.1 (Docker cli) : fails
- 8.0.1 (Docker cli) : fails
- 7.4.4 (Windows Monteverdi binary) : ok
SPOT5 image is a Level1A from SPOT World Heritage (002-003_S5_042-247-0_2013-06-07-10-55-23_HRG-1_J_DT_TT)9.0.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2338C++17 ?2023-04-06T07:49:12ZRémi CressonC++17 ?Hi,
I am trying to switch to C++17, to build OTB along with a project that required it.
From what I saw, setting the `set(CMAKE_CXX_STANDARD 17)` in CMakeLists.txt enables to compile the whole library, at least on Ubuntu:22.04, and my ...Hi,
I am trying to switch to C++17, to build OTB along with a project that required it.
From what I saw, setting the `set(CMAKE_CXX_STANDARD 17)` in CMakeLists.txt enables to compile the whole library, at least on Ubuntu:22.04, and my project's functional tests all pass (I don't use all OTB applications though!). So that is great news.
But when I try to enable `BUILD_TESTING=ON`, I got a lot on compilation errors that I dont well understand.
I tried to solve a few of them but unfortunately I am stuck with some `error: invalid use of 'void'` errors. [Here](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/commits/tryto_compile_cpp17) is my working copy.
Moving from C++14 to C++17 doesn't look so complicated, since the library can be compiled (without tests).
Any plans to switch to C++17? (There was !799 but it's old)