otb issueshttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues2024-03-20T15:59:44Zhttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2380Separate Miscellaneous in separate repository2024-03-20T15:59:44ZTristan LaurentSeparate Miscellaneous in separate repositoryTo compilate project, we can use OTB modules https://www.orfeo-toolbox.org/CookBook-9.0/RemoteModules.html
- [x] Create external git repo, link it as a submodule
- [x] OTB core must download corresponding submodule if the cmake -DOTBGrou...To compilate project, we can use OTB modules https://www.orfeo-toolbox.org/CookBook-9.0/RemoteModules.html
- [x] Create external git repo, link it as a submodule
- [x] OTB core must download corresponding submodule if the cmake -DOTBGroup_Miscellaneous is turned on
- [x] Compile it separately, you may use OTB core artifact
- [x] Verify that package is correctly built during cpack
- [ ] Adapt CI to run corresponding build job and tests
- [ ] Adapt otb CI to start module job when there is a new dependency (i.e. OTB core)
- [ ] Adapt bug tracking for each project ?
- [ ] Add a Readme with compilation help, otb integration, and some exemples10.0.0Tristan LaurentTristan Laurenthttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2378Divide OTB modules in independent repositories2024-03-05T09:32:20ZTristan LaurentDivide OTB modules in independent repositories# Resume
The goal of this is to separate each modules located in https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/tree/develop/Modules in a separate repo (except Core). A link will be done from otb main repo and each module with the ...# Resume
The goal of this is to separate each modules located in https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/tree/develop/Modules in a separate repo (except Core). A link will be done from otb main repo and each module with the "submodule" git mechanism.
# Pro and cons
Separating modules have several advantages:
- Improve CI build time
- Improve unittest jobs and runs as only module related features are tested
- Issues will now be module-related
- Developers only download needed source code
The CI jobs may increase in complexity as we introduce dependencies with modules. Thus we need to trigger modules jobs from core.
# Plan
- [ ] According to @ytanguy , create a gitlab group to handle all modules in a separate repo. This will be easier for bug tracking and management
- [ ] Start with Miscellaneous module, use OTB modules https://www.orfeo-toolbox.org/CookBook-9.0/RemoteModules.html
- [ ] Create external git repo, link it as a submodule
- [ ] OTB core must download corresponding submodule if the cmake -DOTBGroup_Miscellaneous is turned on
- [ ] Compile it separately, you may use OTB core artifact
- [ ] Adapt CI to run corresponding build job and tests
- [ ] Adapt otb CI to start module job when there is a new dependency (i.e. OTB core)
- [ ] Adapt bug tracking for each project ?
- [ ] Add a Readme with compilation help, otb integration, and some exemples
- [ ] Repeat for Remote
- [ ] Repeat for FeatureExtraction
- [ ] Repeat for SAR
- [ ] Repeat for Segmentation
- [ ] Repeat for Hyperspectral
- [ ] Repeat for Learning
- [ ] Repeat for StereoProcessing10.0.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2140Build: pass `-pipe` to GCC/Clang2021-02-08T17:30:15ZLaurențiu NicolaBuild: pass `-pipe` to GCC/ClangThis can speed up the builds a bunch. Unfortunately, I don't think CMake can do it automatically where it makes sense.This can speed up the builds a bunch. Unfortunately, I don't think CMake can do it automatically where it makes sense.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2136RFC: Enable `zstd` support in SuperBuild2021-02-10T07:39:09ZLaurențiu NicolaRFC: Enable `zstd` support in SuperBuild### What changes will be made and why they would make a better Orfeo ToolBox?
ZSTD is a modern compression format that offers compression ratios comparable with `zlib`, but faster compression and decompression speeds: https://engineerin...### What changes will be made and why they would make a better Orfeo ToolBox?
ZSTD is a modern compression format that offers compression ratios comparable with `zlib`, but faster compression and decompression speeds: https://engineering.fb.com/2016/08/31/core-data/smaller-and-faster-data-compression-with-zstandard/. GDAL supports ZSTD since 2.3.
This is a proposal to include `libzstd` in the SuperBuild package, and enable GDAL's support of it. OTB applications will be able to use ZSTD compression seamlessly, by virtue of GDAL.
Results for a noisy `Float32` 10980x10980 TIFF, no predictor:
| Format | Size | Compression time | Compression time (MT) | Decompression time |
| ------ | ------ | ------ | ------ | ------ |
| Uncompressed | 482 330 028 b | N/A | N/A | N/A |
| DEFLATE | 419 729 334 b | 4.030s | 0.687s | 1.647s |
| ZSTD-9 | 415 737 011 b | 8.514s | 3.787s | 0.903s |
Results for a 15-bit `Int16` 10980x10980 TIFF, no predictor:
| Format | Size | Compression time | Compression time (MT) | Decompression time |
| ------ | ------ | ------ | ------ | ------ |
| Uncompressed | 226 122 884 b | N/A | N/A | N/A |
| DEFLATE | 201 822 038 b | 9.292s | 7.615s | 8.064s |
| ZSTD-9 | 207 553 979 b | 13.921s | 7.801s | 7.696s |
Results for a the file above, converted to 16-bit, no predictor:
| Format | Size | Compression time | Compression time (MT) | Decompression time |
| ------ | ------ | ------ | ------ | ------ |
| Uncompressed | 241 187 444 b | N/A | N/A | N/A |
| DEFLATE | 177 040 501 b | 1.736s | 0.396s | 0.869s |
| ZSTD-9 | 184 242 872 b | 12.715s | 3.077s | 0.558s |
Results for a the file above, converted to 16-bit, horizontal differencing predictor:
| Format | Size | Compression time | Compression time (MT) | Decompression time |
| ------ | ------ | ------ | ------ | ------ |
| Uncompressed | 241 187 444 b | N/A | N/A | N/A |
| DEFLATE | 159 937 804 b | 2.053s | 0.392s | 1.722s |
| ZSTD-9 | 153 871 845 b | 13.487s | 3.058s | 0.735s |
| ZSTD-3 | 154 626 632 b | 1.643s | 0.378s | 0.565s |
Times are best of three runs, taken on my computer, with GDAL from `osgeo/gdal:ubuntu-full-3.2.0`.
It seems that users would need to be very careful selecting the compression level, because the default is less efficient than DEFLATE-6.
#### Risks and benefits
- it makes SuperBuild more complex (one more library to keep track of)
- the SuperBuild package will be slightly larger (`libzstd.so` is about 678 KB)
- files saved with ZSTD will probably be unreadable by most software, but it's not used by default
- it allows users to take advantage of the faster compression and decompression speeds
#### Alternatives for implementations
Do nothing.
### Who will be developing the proposed changes?
Not sure.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2126orfeo-toolbox Conda package WSL2022-01-07T08:55:18Zschwarzcsorfeo-toolbox Conda package WSL### Description
I am able to install the orfeo-toolbox on WSL Ubuntu, using Anaconda and creating a python=3.7 virtual environment (conda install -c conda-forge -c orfeotoolbox otb).
I am able to successfully run the "Smoothing" test ca...### Description
I am able to install the orfeo-toolbox on WSL Ubuntu, using Anaconda and creating a python=3.7 virtual environment (conda install -c conda-forge -c orfeotoolbox otb).
I am able to successfully run the "Smoothing" test case using the above described setup (https://www.orfeo-toolbox.org/packages/nightly/2019-06-28/CookBook-stable/recipes/python.html).
However when I run the command "print(str(otb.Registry.GetAvailableApplications()))", I get the error message:
[...]/oftb/lib/otb/applications/otbapp_TrainVectorClassifier.so: undefined symbol: _ZN5shark6random9globalRngE
I further looked into the problem and it seems some applications are working, e.g. "TrainImageClassifier", while others such as "TrainVector Classifier" produces the same error:
run--> app1= otb.Registry.CreateApplication("TrainVectorClassifier")
error--> 2020-11-17 12:30:03 (WARNING): Failed to load libraries from /home/cschwarz/miniconda3/envs/oftb/lib/otb/applications/otbapp_TrainVectorClassifier.so while trying to create application TrainVectorClassifier
because: -> /home/cschwarz/miniconda3/envs/oftb/lib/otb/applications/otbapp_TrainVectorClassifier.so: undefined symbol: _ZN5shark6random9globalRngE
### Steps to reproduce
- install conda on WSL Ubuntu
1. conda create --name oftb python=3.7 spyder=4
2a. conda activate oftb
2b. conda install -c conda-forge -c orfeotoolbox otb
3. python
4. import otbApplication as otb
5. print(str(otb.Registry.GetAvailableApplications()))
### Configuration information
conda version: 4.9.2
python version: 3.7.8
otb : 7.1.0 / Build: py37h06a4308_1
I am very excited to use OTB for patch-image-processing and would value your advice on how to solve this issue?
Best Regards,
Christianhttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2112Use symlink in main package page2022-01-07T07:30:30ZCédric TraizetUse symlink in main package pageThere are two pages on the website containing release packages:
* `https://www.orfeo-toolbox.org/packages/`: packages corresponding to the current release
* `https://www.orfeo-toolbox.org/packages/archives/OTB/`: all otb packages.
Pac...There are two pages on the website containing release packages:
* `https://www.orfeo-toolbox.org/packages/`: packages corresponding to the current release
* `https://www.orfeo-toolbox.org/packages/archives/OTB/`: all otb packages.
Packages for the current release are duplicated on two different pages. We could simplify it by creating symlinks from the main package page to the archive page.https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2084Status of Phenology and BioVars remote modules2022-01-06T16:04:49ZCédric TraizetStatus of Phenology and BioVars remote modulesOTB is not using the latest versions of [OTBPhenology](https://gitlab.orfeo-toolbox.org/jinglada/phenotb/-/tree/master) and [OTBBioVars](https://gitlab.orfeo-toolbox.org/jinglada/otb-bv/-/tree/master) available.
The remote modules conta...OTB is not using the latest versions of [OTBPhenology](https://gitlab.orfeo-toolbox.org/jinglada/phenotb/-/tree/master) and [OTBBioVars](https://gitlab.orfeo-toolbox.org/jinglada/otb-bv/-/tree/master) available.
The remote modules contain fixes and should be compatible with OTB > 7.0, but the version currently used are not. This means that compiling OTB with the cmake options `Module_OTBPhenology=ON` and `Module_OTBBioVars=ON` will fail.
These remote modules are not tested on CI and are not packaged.
I think we should at least test them on CI, as they are part of official remote modules. If they are not tested I think they should be removed.
They could also be packaged. They are under GPL license, but this is not a problem according to #1691. Note that TemporalGapFilling is also under GPL license and is packaged.Thibaut ROMAINThibaut ROMAINhttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/2069Conda package broken2022-08-30T13:26:40ZCédric TraizetConda package brokenThe conda package is broken. On Ubuntu 18.04, conda 4.8.3 and Python 3.7 and 3.8 environments, the command `conda install -c orfeotoolbox otb` leads :
```
Collecting package metadata (current_repodata.json): done
Solving environment: fa...The conda package is broken. On Ubuntu 18.04, conda 4.8.3 and Python 3.7 and 3.8 environments, the command `conda install -c orfeotoolbox otb` leads :
```
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: \
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:
Specifications:
- otb -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0']
Your python: python=3.7
If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.
```
Also reproduced on Ubuntu Virtual Machine. See this [forum post](https://forum.orfeo-toolbox.org/t/otb-conda-installation-on-wsl/708)8.2.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1967How to handle remote module shipping python scripts2019-09-24T19:33:51ZVictor PoughonHow to handle remote module shipping python scriptsDiapOTB is providing [some python scripts](https://gitlab.orfeo-toolbox.org/remote_modules/diapotb/tree/master/python_src) on top of regular OTB applications. They are not available in the binary packages we provide on the website. Would...DiapOTB is providing [some python scripts](https://gitlab.orfeo-toolbox.org/remote_modules/diapotb/tree/master/python_src) on top of regular OTB applications. They are not available in the binary packages we provide on the website. Would there be an easy way to distribute them to make them easily available for users of the binary packages?https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/issues/1519Add spatialite to Superbuild2019-04-16T12:57:37ZJulien MichelAdd spatialite to SuperbuildSpatialite allows to use vector db with spatial index, resulting in faster spatial filtering operations.Spatialite allows to use vector db with spatial index, resulting in faster spatial filtering operations.