otb issueshttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues2022-01-07T11:47:25Zhttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1928LargeScaleMeanshift memory error with OTB6.7.0, works with OTB6.6.12022-01-07T11:47:25ZMichael WessLargeScaleMeanshift memory error with OTB6.7.0, works with OTB6.6.1### Description
We used OTB 6.7.0 for the zonalSTatistics feature, which worked well. When we tried to run LargeScaleMeanshift from the same version, we always got an out of memory error, while LargeScaleMeanshift from OTB 6.6.1 with ex...### Description
We used OTB 6.7.0 for the zonalSTatistics feature, which worked well. When we tried to run LargeScaleMeanshift from the same version, we always got an out of memory error, while LargeScaleMeanshift from OTB 6.6.1 with exactly the same command works without problems. We did not figure out exactly what the problem was, but we used the `-ram` parameter and set `GDAL_CACHEMAX` and `OTB_MAX_RAM_HINT` to exactly the same values in both versions...one worked, the other one didnt.
If important, we ran both tests in an isolated Docker image (docker v1.39) with exactly the same environment, just different OTB versions.
### Configuration information
OTB 6.6.1 and OTB 6.7.0
**edit**: I know 6.7.0 is not final, we used the nightly build; I just thought it mght be still helpful to report this issue!https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1926Set InputImageList from numpy array2022-01-06T14:44:53ZVictor PoughonSet InputImageList from numpy arrayCurrently in the python API there is a `SetImageFromNumpyArray`. But there is no way to set an input image list from a list of numpy arrays.Currently in the python API there is a `SetImageFromNumpyArray`. But there is no way to set an input image list from a list of numpy arrays.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1917ITK 5 support2022-01-06T15:06:39ZVictor PoughonITK 5 supportIssue of !194
TODO:
* https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/merge_requests/194#note_78887Issue of !194
TODO:
* https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/merge_requests/194#note_78887Cédric TraizetCédric Traizethttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1915use libressl instead of openssl in dependency to curl2019-06-06T09:06:38ZRashad Kanavathuse libressl instead of openssl in dependency to curllibressl has cmake and windows support that openssl. This is much easy to manage that having a perl everytime to just configure openssl.libressl has cmake and windows support that openssl. This is much easy to manage that having a perl everytime to just configure openssl.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1911Add CMake flag to disable QGIS descriptors2022-01-06T14:40:03ZVictor PoughonAdd CMake flag to disable QGIS descriptorsSee: https://forum.orfeo-toolbox.org/t/otb-without-qgis/281
default value should be ON
related to #1910See: https://forum.orfeo-toolbox.org/t/otb-without-qgis/281
default value should be ON
related to #1910https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1909Refactor InputImageParameter::GetImage2019-05-27T13:45:27ZVictor PoughonRefactor InputImageParameter::GetImageAfter investigation during the fix of #1899 in !497, it was determined that `InputImageParameter::GetImage` should be refactored:
* Support multiple output types
* Make the getter constAfter investigation during the fix of #1899 in !497, it was determined that `InputImageParameter::GetImage` should be refactored:
* Support multiple output types
* Make the getter consthttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1897Disabling streaming at application level2019-04-29T17:00:55ZLuc HermitteDisabling streaming at application level# TL;DR:
1-
```c++
writer->SetNumberOfDivisionsTiledStreaming(0);
```
gives improper results
2- We need a way to disable streaming at application level
# Rationale
## Request for Enhancement: `app->SetStreamingType(StreamingType...# TL;DR:
1-
```c++
writer->SetNumberOfDivisionsTiledStreaming(0);
```
gives improper results
2- We need a way to disable streaming at application level
# Rationale
## Request for Enhancement: `app->SetStreamingType(StreamingType::none);`
I've a FFTW filter on a complex image that needs to work on the whole image. When integrated in an OTB application, the application shall not stream the image.
While we *can* require the application to disable streaming through extended filenames (e.g. "`-out dest.tif?&streaming:type=none`"), this is not something we can request at application level. We cannot expect the end-user to know the application internally uses a filter that is incompatible with streaming.
We should be able to control the streaming type at application level.
What I don't know then, is the consequence on Python bindings. That's why I don't propose a merge request.
Note: I've tried
```c++
rmFilter->Update(); // work around streamer to compute FFTW results
SetParameterComplexOutputImage("out", rmFilter->GetOutput());
```
but instead of 10s, the application runs in 50s.
## Bug in `SetNumberOfDivisionsTiledStreaming(0)`
I'm not sure why, but on my 2576 x 1690 complex image, `SetNumberOfDivisionsTiledStreaming(0)` misbehaves.
```
(INFO): Default RAM limit for OTB is 128 MB
(INFO): GDAL maximum cache size is 12847 MB
(INFO): OTB will use at most 4 threads
(INFO): Loading kwl metadata from attached geom file ${DATA}/Input/point_target_SWOT/l0_PT_master_pr_ds_rgcpr.geom
(DEBUG): file ${OTB}/include/OTB-6.7/otbNumberOfDivisionsTiledStreamingManager.hxx, line 46, Activating NumberOfDivisionsTiledStreamingManager streaming mode
WARNING: In ${OTB}include/OTB-6.7/otbNumberOfDivisionsTiledStreamingManager.hxx, line 49
NumberOfDivisionsTiledStreamingManager (0x1fa5e90): NumberOfDivisions set to 0 : streaming disabled
(DEBUG): file ${OTB}/include/OTB-6.7/otbImageRegionSquareTileSplitter.hxx, line 57, Tile dimension : 2096
(DEBUG): file ${OTB}/include/OTB-6.7/otbImageRegionSquareTileSplitter.hxx, line 58, Number of splits per dimension : 2 1
(DEBUG): file ${OTB}/include/OTB-6.7/otbNumberOfDivisionsTiledStreamingManager.hxx, line 55, Computed number of split : 2
(INFO): File ${BUILD}/release/Testing/Temporary/FFTWTest.tif will be written in 2 blocks of 2096x1690 pixels
(DEBUG): GDAL reads [0, 2095]x[0, 1689] x 1 bands of type CFloat64 from file ${DATA}/Input/point_target_SWOT/l0_PT_master_pr_ds_rgcpr.tif
```
As you can see, two tiles are found, with an incorrect dimension moreover. I'm not sure why.
If instead I use `SetNumberOfDivisionsStrippedStreaming(1);`, everything works fine.
What I suggest: have `?&streaming:type=none` trigger the execution of `SetNumberOfDivisionsStrippedStreaming(1);` instead of `SetNumberOfDivisionsTiledStreaming(0)`.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1885Use a test framework for unit tests (Follow-up from "Large refactoring of rad...2022-01-07T11:47:24ZJulien MichelUse a test framework for unit tests (Follow-up from "Large refactoring of radiometric indices")The following discussion from !464 should be addressed:
- [ ] @lhermitte started a [discussion](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/merge_requests/464#note_77961): (+3 comments)
> As this is a new test, what about us...The following discussion from !464 should be addressed:
- [ ] @lhermitte started a [discussion](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/merge_requests/464#note_77961): (+3 comments)
> As this is a new test, what about using a real unit test framework that'll display the bogus line?
In the discussion, `boost.test`, `catch2` and `doctest` are proposed. We should decide which one to pick, and identify current tests that could be refactored using a dedicated test framework.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1882Standard verbose in applications2022-01-06T13:56:37ZLuc HermitteStandard verbose in applications# TL;DR:
Have all OTB applications have a `-v` option.
# Rationale
It'd be, very, very nice to have `-v|-verbose` option in all OTB applications in standard. That's what we have everywhere in *nix world. I'm not picky to require that...# TL;DR:
Have all OTB applications have a `-v` option.
# Rationale
It'd be, very, very nice to have `-v|-verbose` option in all OTB applications in standard. That's what we have everywhere in *nix world. I'm not picky to require that long option names should start with double dashes, but really, a `-v` would change life.
I know, there is a (let me search...) `OTB_LOGGER_LEVEL=DEBUG` that we can export. But the trouble is... searching, every time, while `-v` is a _de facto_ standard.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1876No debug build on CI or dashboard2022-01-07T07:21:02ZVictor PoughonNo debug build on CI or dashboard### Description
There's no debug build in the CI, so asserts are not tested. Example of problems that should be caught by testing in debug mode: #1874, !457.
### Configuration information
debug### Description
There's no debug build in the CI, so asserts are not tested. Example of problems that should be caught by testing in debug mode: #1874, !457.
### Configuration information
debughttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1858QGIS plugin, test in OTB2019-02-26T09:33:21ZAntoine RegimbeauQGIS plugin, test in OTB## QGIS
OTB has been added to QGIS as a provider. [See here.](https://github.com/qgis/QGIS/pull/8814)
We need to add the different tests used in QGIS, in OTB so that we will be able to prevent changes in OTB and report it to QGIS.## QGIS
OTB has been added to QGIS as a provider. [See here.](https://github.com/qgis/QGIS/pull/8814)
We need to add the different tests used in QGIS, in OTB so that we will be able to prevent changes in OTB and report it to QGIS.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1849otbcli_DimensionalityReduction: Add options to output eigenvalues, apply inve...2022-01-07T07:22:27Zaloboaotbcli_DimensionalityReduction: Add options to output eigenvalues, apply inverse transform for PCA and correctely deal with no-dataAdd options to otbcli_DimensionalityReduction
1. ~~Output eigenvalues along eigenvectors for PCA~~
2. ~~Apply the inverse transform~~ (I understand this is already done using the "-outinv" parameter.
3. Deal with nodata
4. Option to ...Add options to otbcli_DimensionalityReduction
1. ~~Output eigenvalues along eigenvectors for PCA~~
2. ~~Apply the inverse transform~~ (I understand this is already done using the "-outinv" parameter.
3. Deal with nodata
4. Option to estimate stats from a certain % sampled from the image
5. ~~Option to output eigenvalues and eigenvectors only, avoiding calcalating and writing the components (useful to decide how many components to keep in a second run that would perform the actual calculation of the components)~~https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1817Huge Memory-leaks when in-memory chaining OTB-applications from Python2024-03-26T14:52:03ZStéphane AlbertHuge Memory-leaks when in-memory chaining OTB-applications from Python### Description
Huge memory consumption when in-memory chaining OTB-Applications within loop from _Python_ code.
### Steps to reproduce
There are two processing pipelines defined by chaining OTB-applications in memory from _Python_ co...### Description
Huge memory consumption when in-memory chaining OTB-Applications within loop from _Python_ code.
### Steps to reproduce
There are two processing pipelines defined by chaining OTB-applications in memory from _Python_ code:
1. create_water_mask:
```mermaid
graph TD;
PAN-->ExtractROI_PAN;
XS-->ExtractROI_XS;
ExtractROI_PAN-->SuperImpose;
ExtractROI_XS-->WaterMaskMaker;
WaterMaskMaker-->SuperImpose;
```
2. create_color_image:
```mermaid
graph TD;
PAN-->ExtractROI_PAN;
XS-->ExtractROI_XS;
ExtractROI_PAN-->BundleToPerfectSensor;
ExtractROI_XS-->BundleToPerfectSensor;
BundleToPerfectSensor-->Convert;
```
A main processing function runs alternatively those two pipeline on a set of (PAN ; XS) 2-uplets in a (Python) loop (see [test_wwm_cim.py](/uploads/73d2c9bc174f7d53cb5b4ed431fe7ae0/test_wwm_cim.py))
Memory-leak instrumentation done with [vg-mem.sh](/uploads/fb31fe9ded1f9868cd832a8132fc7961/vg-mem.sh).
#### Instrumentation of the `WaterMaskMaker` C++ OTB-Application
Command: `vg-mem.sh ./bin/otbApplicationLauncherCommandLine WaterMaskMaker -progress true -in /home/salbert/data/bench-2/QB_TLSE_MUL_512x512_1024x1024.TIF -out /home/salbert/tmp/QB_TLSE_WMM.tif`
Valgrind log: \[otbApplicationLauncherCommandLine-20181213-105906.log\](/uploads/7beb36ede3bca565a38fd58dc6d19c18/otbApplicationLauncherCommandLine-20181213-105906.log
Small memory-leaks in _OSSIM_ projection and _RTTI_ systems. Same for TIF and JP2000
#### Instrumentation of the `BundleToPerfectSensor` OTB-Application
Command: `vg-mem.sh ./bin/otbApplicationLauncherCommandLine BundleToPerfectSensor -progress true -inp /home/salbert/data/bench-2/PRIMARY_BUNDLE_P_1024x1024.tif -inxs /home/salbert/data/bench-2/PRIMARY_BUNDLE_MUL_1024x1024.tif -out /home/salbert/tmp/PRIMARY_BUNDLE_BTPS.tif uint16`
Valgrind log: [otbApplicationLauncherCommandLine-20181220-130819.log](/uploads/19f7dfbdda614771744711de8a1c0028/otbApplicationLauncherCommandLine-20181220-130819.log)
L924, L910: Memory-leaks in pipeline `::UpdateOutputInformation()` from https://github.com/InsightSoftwareConsortium/ITK/blob/v4.12.2/Modules/Core/Common/include/itkImportImageContainer.hxx#L188
L896: Also inquire about leak of process-object instance of https://github.com/InsightSoftwareConsortium/ITK/blob/v4.12.2/Modules/Core/Common/include/itkInPlaceImageFilter.hxx#L38.
#### Instrumentation of _SWIG_ OTB-Application wrapper
_SWIG_ wrapper of OTB-applications have been instrumented using the [otb-app-swig-mem-log.patch](/uploads/2f8c6b39793aadb2e3b2f35c4612b3b1/otb-app-swig-mem-log.patch) trace patch and all instances of OTB-Applications (be they composite or not) have been manually counted).
Command: `vg-mem.sh ./test_OTB_application.py` (see [test_OTB_application.py](/uploads/d578087dea8c209fb841166256a0c50e/test_OTB_application.py))
Valgrind log: [test_OTB_application_memory.py-20181214-113339.log](/uploads/becd9109e348354e8001c583a47df1c8/test_OTB_application_memory.py-20181214-113339.log)
Command: `vg-mem.sh ./test_OTB_application_memory_JP2.py` ([test_OTB_application_memory_JP2.py](/uploads/e5003321db00cb3091ed9320c04621af/test_OTB_application_memory_JP2.py))
Valgrind log: [test_OTB_application_memory_JP2.py-20181217-104219.log](/uploads/6bc5de2ae3275759af70c798b17703ae/test_OTB_application_memory_JP2.py-20181217-104219.log)
There's no memory leak due to incorrect reference-counting of the SWIG wrapper. However:
1. the reference-counter wrapper implementation \[[1](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/blob/release-6.6/Modules/Wrappers/SWIG/src/Python.i#L45)\] is out of date and could be replaced by direct _SWIG_ 3 handling \[[2](http://www.swig.org/Doc3.0/SWIGPlus.html#SWIGPlus_smart_pointers)\]
2. `ImageBaseType` (which is a reference-counted instance) is not wrapped, but OTB-Application wrapper provides a `::GetParameterImage()` returning a C++ raw-pointer without reference-counting handling, which may cause some memory leaks (see next section about ) in the _Python_ environment, e.g.:
```python
def main( self ):
app = ...
img = app.GetParameterImage( ... ) # img is not reference-counted
# If not giving img to a wrapped C++ function putting it into an itk::SmartPointer<> will cause a memory-leak which of all image-data.
```
#### Instrumentation of full _Python_ sample code
Same memory-leaks as described above are traces. Moreover, the metadata-dictionary is lost when running multiple-times this instrumentation test without deleting the temporary output file.
Command: `vg-mem.sh ./test_wmm_cim.py` (2 iterations without removing `${HOME}/tmp` files)
Valgrind log: [test_wwm_cim.py-20181219-161831.log](/uploads/f4322d42a25e01cfdd71a26c99ded4f7/test_wwm_cim.py-20181219-161831.log) (see also: [test_wwm_cim.py-20181217-174610.log](/uploads/ff1b413d2c2ca2efcf96937f2dd5d90c/test_wwm_cim.py-20181217-174610.log) L3666 which is related to a suspicious call to `::GetParameterImage()` from `otb::ImageFileReader<>` when initializing pipeline on existing output-data file.)
### Configuration information
OTB-6.6 release and ITK-4.12 on Ubuntu 18.04 LTS.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1798Handle no data in BandMath and BandMathX2024-03-27T16:19:19ZVictor PoughonHandle no data in BandMath and BandMathXRequested by multiple users: #1511, #1801, otb-users
Better handling of images with no-data in BandMath and BandMathX:
* A flag to ignore no-data values (maybe even on by default?)
* A nodata token available in the expression that is s...Requested by multiple users: #1511, #1801, otb-users
Better handling of images with no-data in BandMath and BandMathX:
* A flag to ignore no-data values (maybe even on by default?)
* A nodata token available in the expression that is set to the nodata value reported by gdal
* support `nan()` function in BandMathX #1801https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1786Incorrect estimated memory and number of blocks in DynamicConvert with OTB_MA...2018-12-11T08:19:20ZVictor PoughonIncorrect estimated memory and number of blocks in DynamicConvert with OTB_MAX_RAM_HINT=4096### Description
With the following example in DynamicConvert, setting OTB_MAX_RAM_HINT=2250 results in:
```
2018-12-04 14:31:59 (INFO): Estimated memory for full processing: 2299.47MB (avail.: 2250 MB), optimal image partitioning: 2 bl...### Description
With the following example in DynamicConvert, setting OTB_MAX_RAM_HINT=2250 results in:
```
2018-12-04 14:31:59 (INFO): Estimated memory for full processing: 2299.47MB (avail.: 2250 MB), optimal image partitioning: 2 blocks
2018-12-04 14:31:59 (INFO): Estimation will be performed in 4 blocks of 7168x6144 pixels
```
But increasing to OTB_MAX_RAM_HINT=4096 results in:
```
018-12-04 14:32:36 (INFO): Estimated memory for full processing: 1.109e+07MB (avail.: 4096 MB), optimal image partitioning: 2708 blocks
2018-12-04 14:32:36 (INFO): File out_test.tif will be written in 2916 blocks of 205x205 pixels
```
See full log below.
### Steps to reproduce
```
poughov@pc-victor ~ OTB_MAX_RAM_HINT=2250 time otbcli_DynamicConvert -in T44NNN_20181129T050551_B08.jp2 -out out_test.tif
2018-12-04 14:31:59 (INFO): No kwl metadata found in file T44NNN_20181129T050551_B08.jp2
2018-12-04 14:31:59 (INFO): Default RAM limit for OTB is 2250 MB
2018-12-04 14:31:59 (INFO): GDAL maximum cache size is 383 MB
2018-12-04 14:31:59 (INFO): OTB will use at most 8 threads
2018-12-04 14:31:59 (INFO): Estimated memory for full processing: 2299.47MB (avail.: 2250 MB), optimal image partitioning: 2 blocks
2018-12-04 14:31:59 (INFO): Estimation will be performed in 4 blocks of 7168x6144 pixels
Computing shrink Image for min/max estimation...: 100% [**************************************************] (5 seconds)
2018-12-04 14:32:05 (INFO): Estimated memory for full processing: 1724.62MB (avail.: 2250 MB), optimal image partitioning: 1 blocks
2018-12-04 14:32:05 (INFO): File out_test.tif will be written in 1 blocks of 10980x10980 pixels
Writing out_test.tif...: 100% [**************************************************] (4 seconds)
56.85user 1.54system 0:10.38elapsed 562%CPU (0avgtext+0avgdata 2136324maxresident)k
0inputs+235656outputs (0major+933435minor)pagefaults 0swaps
poughov@pc-victor ~ OTB_MAX_RAM_HINT=4096 time otbcli_DynamicConvert -in T44NNN_20181129T050551_B08.jp2 -out out_test.tif
2018-12-04 14:32:30 (INFO): No kwl metadata found in file T44NNN_20181129T050551_B08.jp2
2018-12-04 14:32:30 (INFO): Default RAM limit for OTB is 4096 MB
2018-12-04 14:32:30 (INFO): GDAL maximum cache size is 383 MB
2018-12-04 14:32:30 (INFO): OTB will use at most 8 threads
2018-12-04 14:32:30 (INFO): Estimated memory for full processing: 2299.47MB (avail.: 4096 MB), optimal image partitioning: 1 blocks
2018-12-04 14:32:30 (INFO): Estimation will be performed in 1 blocks of 10980x10980 pixels
Computing shrink Image for min/max estimation...: 100% [**************************************************] (5 seconds)
2018-12-04 14:32:36 (INFO): Estimated memory for full processing: 1.109e+07MB (avail.: 4096 MB), optimal image partitioning: 2708 blocks
2018-12-04 14:32:36 (INFO): File out_test.tif will be written in 2916 blocks of 205x205 pixels
Writing out_test.tif...: 100% [**************************************************] (10 seconds)
54.74user 6.27system 0:16.18elapsed 377%CPU (0avgtext+0avgdata 1889620maxresident)k
0inputs+235656outputs (0major+776377minor)pagefaults 0swaps
```
### Configuration information
Linux standalone package OTB 6.6.1https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1754GDAL VSI support2024-03-27T14:47:33ZLaurențiu NicolaGDAL VSI supportI've recently heard someone insist that OTB is a bad framework and isn't "cloud-ready" because it doesn't support the "technologies of the future", like object storage. The thing is, GDAL does support cURL, Amazon S3 and other storage dr...I've recently heard someone insist that OTB is a bad framework and isn't "cloud-ready" because it doesn't support the "technologies of the future", like object storage. The thing is, GDAL does support cURL, Amazon S3 and other storage drivers. Unfortunately, OTB doesn't seem to.
I tried a couple of tests:
```
$ CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt otbcli_ReadImageInfo -in "http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif?skipgeom=true"
2018-10-24 13:26:34 (INFO): Default RAM limit for OTB is 128 MB
2018-10-24 13:26:34 (INFO): GDAL maximum cache size is 393 MB
2018-10-24 13:26:34 (INFO): OTB will use at most 4 threads
2018-10-24 13:26:35 (INFO):
Image general information:
Number of bands : 3
No data flags : Not found
Start index : [0,0]
Size : [65536,65536]
Origin : [0.5,0.5]
Spacing : [1,1]
Estimated ground spacing (in meters): [547.072,296.427]
Image acquisition information:
Sensor :
Image identification number:
Image default RGB composition:
[R, G, B] = [0,1,2]
Ground control points information:
Number of GCPs = 0
GCPs projection =
Output parameters value:
indexx: 0
indexy: 0
sizex: 65536
sizey: 65536
spacingx: 1
spacingy: 1
originx: 0.5
originy: 0.5
estimatedgroundspacingx: 547.0715332
estimatedgroundspacingy: 296.4269409
numberbands: 3
sensor:
id:
time:
town:
country:
rgb.r: 0
rgb.g: 1
rgb.b: 2
projectionref:
keyword:
gcp.count: 0
gcp.proj:
gcp.ids:
gcp.info:
gcp.imcoord:
gcp.geocoord
$ CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt gdalinfo http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif
Driver: GTiff/GeoTIFF
Files: none associated
Size is 514, 515
Coordinate System is:
PROJCS["unnamed",
GEOGCS["NAD27",
DATUM["North_American_Datum_1927",
SPHEROID["Clarke 1866",6378206.4,294.9786982138982,
AUTHORITY["EPSG","7008"]],
AUTHORITY["EPSG","6267"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4267"]],
PROJECTION["Cylindrical_Equal_Area"],
PARAMETER["standard_parallel_1",33.75],
PARAMETER["central_meridian",-117.333333333333],
PARAMETER["false_easting",0],
PARAMETER["false_northing",0],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]]]
Origin = (-28493.166784412522247,4255884.543802191503346)
Pixel Size = (60.022136983193739,-60.022136983193739)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( -28493.167, 4255884.544) (117d38'27.05"W, 33d56'37.74"N)
Lower Left ( -28493.167, 4224973.143) (117d38'27.05"W, 33d39'53.81"N)
Upper Right ( 2358.212, 4255884.544) (117d18'28.38"W, 33d56'37.74"N)
Lower Right ( 2358.212, 4224973.143) (117d18'28.38"W, 33d39'53.81"N)
Center ( -13067.478, 4240428.844) (117d28'27.71"W, 33d48'15.38"N)
Band 1 Block=514x15 Type=Byte, ColorInterp=Gray
```
Notice that OTB reports wrong origin, ground spacing and size values.
```
$ CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt strace -s2048 -e trace=file -f otbcli_ReadImageInfo -in "http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif"
[snip]
[pid 6069] access("http://download.osgeo.org/geotiff/samples/gdal_eg/cea.geom", R_OK) = -1 ENOENT (No such file or directory)
[pid 6069] getcwd("/home/user", 4096) = 16
[pid 6069] lstat("/home/user/http:", 0x7ffed725d380) = -1 ENOENT (No such file or directory)
[pid 6069] access("http://download.osgeo.org/geotiff/samples/gdal_eg/cea.TIL", F_OK) = -1 ENOENT (No such file or directory)
[pid 6069] access("http://download.osgeo.org/geotiff/samples/gdal_eg/cea.til", F_OK) = -1 ENOENT (No such file or directory)
[pid 6069] openat(AT_FDCWD, "http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6069] access("http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif", F_OK) = -1 ENOENT (No such file or directory)
[pid 6069] openat(AT_FDCWD, "http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6069] stat("http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif", 0x7ffed725dc50) = -1 ENOENT (No such file or directory)
[pid 6069] readlink("http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif", 0x245a730, 2048) = -1 ENOENT (No such file or directory)
2018-10-24 13:26:57 (INFO): No kwl metadata found in file http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif
```
OTB tries to open HTTP URLs as files.
```
$ CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt strace -s2048 -f otbcli_ComputeImagesStatistics -il "http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif?skipgeom=true"
[snip]
[pid 6110] sendto(4, "GET /geotiff/samples/gdal_eg/cea.tifqqqqqqrr HTTP/1.1\r\nHost: download.osgeo.org\r\nAccept: */*\r\n\r\n", 96, MSG_NOSIGNAL, NULL, 0) = 96
```
OTB gets into an infinite loop trying to request the wrong URL (notice the `qqqqqqrr` suffix).
```
CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt strace -s2048 -f otbcli_Convert -in "http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif?skipgeom=true" -out foo.tif
[snip]
[pid 6171] sendto(4, "GET /geotiff/samples/gdal_eg/cea.tifqqqqqqqq HTTP/1.1\r\nHost: download.osgeo.org\r\nAccept: */*\r\n\r\n", 96, MSG_NOSIGNAL, NULL, 0) = 96
```
Same issue, different suffix. Uninitialized memory?
Of course, I expect performance to be pretty poor when doing network access, but sometimes it's useful, and the alternative is to use a FUSE emulation driver that might have its own problems.
Related: #1746
Related: #1506
Related: #1642https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1731Improve rasterization2022-01-07T11:47:25ZRémi CressonImprove rasterizationIn 7.0 `VectorData` might be replaced with `ogr::DataSource` and also in application parameter input/output (#1716).
Some filters for `VectorData` and `ogr::DataSource` have some overlap...
> - `VectorDataToLabelImageFilter` : `OGRDataS...In 7.0 `VectorData` might be replaced with `ogr::DataSource` and also in application parameter input/output (#1716).
Some filters for `VectorData` and `ogr::DataSource` have some overlap...
> - `VectorDataToLabelImageFilter` : `OGRDataSourceToLabelImageFilter`
> - `LabelImageToVectorDataFilter` : `LabelImageToOGRDataSourceFilter`
> - `VectorDataIntoImageProjectionFilter` : `GeometriesProjectionFilter`
(from !222)
...but some features were added to `VectorDataToLabelImageFilter` and not propagated to `OGRDataSourceToLabelImageFilter`
I propose to discuss a possible improvement for `OGRDataSourceToLabelImageFilter` (or `VectorDataToLabelImageFilter` is `VectorData` would be kept, but I don't feel so)
In my opinion, the following burning mode could be useful:
1. Binary (Burn the same value for each geometry. Params: Foreground value (ImagePixelValueType))
2. Field (Burn the value of the given field. Param: Field name (string))
3. FID (Burn the value corresponding to the (cast of the) geometry ID into ImagePixelValueType. (Param: Starting value (size_t) default could be background value))
4. Container (Burn the N-th value of the given container for the N-th geometry. Param: TContainer (e.g. std::unordered_map...))
For `Mosaic` and `ZonalStatistics` this would be useful, and I think we could improve also some other stuff with that.
What do you think?https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1698Missing test for SparseWvltToAngleMapperListFilter2019-01-21T15:12:21ZManuel GrizonnetMissing test for SparseWvltToAngleMapperListFilterThe class SparseWvltToAngleMapperListFilter has a test named otbSparseWvltToAngleMapperListFilterTest (previously named otbSparseWvltToAngleMapperListFilterNewTest but renamed in MR !173) but there is no corresponding otb_add_test in the...The class SparseWvltToAngleMapperListFilter has a test named otbSparseWvltToAngleMapperListFilterTest (previously named otbSparseWvltToAngleMapperListFilterNewTest but renamed in MR !173) but there is no corresponding otb_add_test in the CMakeLists (module DimensionnalityReduction).
This bug was spotted when we remove all new test (this one was wrongly prefixed with the keywork New).
A test with proper input and baseline should be added here.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1662mandatory parameters: why and how?2022-01-07T07:46:22ZRashad Kanavathmandatory parameters: why and how?### Description
Current state of mandatory parameter is really confusing.
Parameter class has a bool m_Mandatory and is true by default. Unless a parameter is set to off with `MandatoryOff(p)` then they are really required.
This make...### Description
Current state of mandatory parameter is really confusing.
Parameter class has a bool m_Mandatory and is true by default. Unless a parameter is set to off with `MandatoryOff(p)` then they are really required.
This make me tip toe around in finding mandatory flag in [otbQgisDescriptor.cxx](Modules/Wrappers/QGIS/src/otbQgisDescriptor.cxx#L302). Now that doesn't solve issue and ends up in error when using otbExtractROI. Error message from QGIS is `mode.fit.vect` is not set. QGIS sees this parameter as mandatory probably because otb description for ExtractROI said so!
This parameter is only required if `mode = fit`. so it must be optional. But not according to `param->GetMandatory()`. It says parameter is mandatory no matter what value is set for `mode` parameter. This is wrong and is problematic for apps like QGIS and others who rely on descriptor files.
In QGIS description file, there is a csv field to identify that parameter as optional or not. (mandatory or not in OTB terms).
Here is the list of paramters reported by otbcli_ExtractROI
```
in (mandatory)
out (mandatory)
mode.fit.im (mandatory)
mode.fit.vect (mandatory)
mode.extent.ulx (mandatory, default value is 0)
mode.extent.uly (mandatory, default value is 0)
mode.extent.lrx (mandatory, default value is 0)
mode.extent.lry (mandatory, default value is 0)
mode.extent.unit (mandatory, default value is pxl)
mode.radius.r (mandatory, default value is 0)
mode.radius.unitr (mandatory, default value is pxl)
mode.radius.cx (mandatory, default value is 0)
mode.radius.cy (mandatory, default value is 0)
mode.radius.unitc (mandatory, default value is pxl)
startx (mandatory, default value is 0)
starty (mandatory, default value is 0)
sizex (mandatory, default value is 0)
sizey (mandatory, default value is 0)
cl (mandatory, no default value)
elev.dem (optional, off by default)
elev.geoid (optional, off by default)
elev.default (mandatory, default value is 0)
ram (optional, off by default, default value is 128)
inxml (optional, off by default)
```
my suggestion is to use below rule to find out if parameter is mandatory.
A parameter is mandatory if statisfies all of the conditions:
1. Must have parent parameter set. `mode.fit.vect` is mandatory only if `mode = fit`
2. Must not have default a value.
Below code can cover first case in this rule. Before going on to add this in otbQgisDescriptor, I would like to find out if there is a better way to deal with mandatory flag in OTBApplicationEngine so other can rely on Parameter::GetMandatory() and stick to API rather than spaghetti forever.
```
+ std::string optional = "false";
+ ParameterKey pName(name);
+ auto splitName = pName.Split();
+ if (splitName.size() > 2)
+ optional = "true";
+ else
+ optional = param->HasValue() ? "true" : "false";
+
```
we can add a check for if-default-value-exists.
### Steps to reproduce
try ExtractROI application using QGIS 3.2 and OTB 6.6
### Configuration information
OS, OTB version or tag, information related to build (binaries, superbuild, system libs ...)https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1640[DOC] Document alternate Application API Python syntax2024-03-06T14:17:38ZManuel Grizonnet[DOC] Document alternate Application API Python syntax### Target documentation ressources
bandmath.IN = image.tif
bandmath.EXP = "im1b1"
This type of syntax should be described in the otb app section of the Cookbook (explain why upper case parameter is mandatory for instance).
### Chang...### Target documentation ressources
bandmath.IN = image.tif
bandmath.EXP = "im1b1"
This type of syntax should be described in the otb app section of the Cookbook (explain why upper case parameter is mandatory for instance).
### Change requested
Describe precisely the changes that are required.