S1Tiling issueshttps://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues2024-03-22T10:08:02Zhttps://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/154Review documentation2024-03-22T10:08:02ZLuc HermitteReview documentation1.1Thierry KoleckThierry Koleckhttps://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/152Support OTB 92024-03-28T15:28:00ZLuc HermitteSupport OTB 9- [ ] Make sure S1Tiling is compatible with OTB 9
- [ ] Update the relevant documentation (e.g. document which modules are required)
- [ ] Update the installation scripts
- [ ] Update the dockers
- [x] Check the baseline is compatible wi...- [ ] Make sure S1Tiling is compatible with OTB 9
- [ ] Update the relevant documentation (e.g. document which modules are required)
- [ ] Update the installation scripts
- [ ] Update the dockers
- [x] Check the baseline is compatible with future (#150) OTB 8 compatible baseline1.1https://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/151Use precise orbit files for establishing grid of ECEF sensor position matchin...2024-03-19T15:23:22ZLuc HermitteUse precise orbit files for establishing grid of ECEF sensor position matching S2 grid (LIA)Knowing which input S1 files to download in order to generate grid of XYZ sensor positions (aligned on S2 grid) can be a bit tricky:
- S1 product footprints seldom cover S2 tiles -- hence S1Tiling, that _concats_ data from 2 S1 products...Knowing which input S1 files to download in order to generate grid of XYZ sensor positions (aligned on S2 grid) can be a bit tricky:
- S1 product footprints seldom cover S2 tiles -- hence S1Tiling, that _concats_ data from 2 S1 products
- We have no documentation that tells how wide is the Orbit State Vector (OSV) series found in S1 products
- Downloading one S1 product just for its embedded orbit information is a bit overkill.
Instead we should use precise orbit files.
Notes: (non-issues)
- As we typically expect the LIA maps to be computed beforehand, we don't need to wait for 3 days before obtaining orbit data for a given pair of _S2 tile_ + _relative-orbit-id_. We can just use any orbit data published long before.
- OTB 9 (/SARDEMProjection2's) `SarSensorModel` can compute sensor positions from the precise orbit data -- we don't need extra data like GCPs, slantRangeTime...
The resolution of this issue can be decomposed as such:
- [ ] Fetch Precise Orbit Files from an/the official provider
- https://github.com/scottstanie/sentineleof may help
- [ ] Change download routine to support this new source of data
- [ ] Create a new OTB application
- that reuses SARDEMProjection2 code related to `-withxyz`, `-withsatpos` and that supports DEM in any spatial reference
- that decodes OSV data from EOF files to construct `FastSarSensorModel` accordingly
- [ ] Adapt the LIA `ComputeGroundAndSatPositionsOnDEM` step to use the new OTB application1.2https://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/123Copy Geoid file with DEM files2023-01-30T14:19:06ZThierry KoleckCopy Geoid file with DEM filesWhen the option `cache_srtm_by:` is set to `copy`, the DEM files are copied to the tmp directory. This issue proposes to copy also the geoid file in the tmp directoryWhen the option `cache_srtm_by:` is set to `copy`, the DEM files are copied to the tmp directory. This issue proposes to copy also the geoid file in the tmp directory1.1https://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/66Write output with UInt16 encoding2024-03-25T10:51:43ZMickael SavinaudWrite output with UInt16 encodingCurrently the output is written as float. To save disk space, it could be interesting to save the output as Uint16 with a specific formula to retrieve the real value.Currently the output is written as float. To save disk space, it could be interesting to save the output as Uint16 with a specific formula to retrieve the real value.1.1Luc HermitteLuc Hermittehttps://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/65Support zip file in input directory2021-03-12T10:17:29ZMickael SavinaudSupport zip file in input directoryWhen I put a zip file in the input directory, the product is not detected. Even I unzip the file, the product is not detected. I need to create a directory with the product name without SAFE suffix to be able to run s1tiling.
I would b...When I put a zip file in the input directory, the product is not detected. Even I unzip the file, the product is not detected. I need to create a directory with the product name without SAFE suffix to be able to run s1tiling.
I would be nice to define that in the documentation and to add the support of a directory which contain zip file.1.1https://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/59Clean automatically older SAFE data in raw directory2024-03-21T17:57:53ZThierry KoleckClean automatically older SAFE data in raw directoryTo save disk space, one can keep only a given number of file (SAFE) in the raw directory.
Before downloading, the code should remove the older SAFE file in order to keep the same number of file in this directory. This number can be fixe...To save disk space, one can keep only a given number of file (SAFE) in the raw directory.
Before downloading, the code should remove the older SAFE file in order to keep the same number of file in this directory. This number can be fixed by the user in the config file.1.1https://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/54Encapsulate OTB process execution into subprocessed2024-03-19T15:31:06ZLuc HermitteEncapsulate OTB process execution into subprocessedIn order to reduce memory leak risks to dask leaks, and alleviate chances for #49 to occur, the calls to OTB applications should be done in subprocesses.
No data, like `np.array`, are exchanged with OTB application through their Python ...In order to reduce memory leak risks to dask leaks, and alleviate chances for #49 to occur, the calls to OTB applications should be done in subprocesses.
No data, like `np.array`, are exchanged with OTB application through their Python bindings, but only filenames. This means there should be no problem to hide calls to OTB python bindings in subprocesses.1.2https://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/131Implement SLC support2023-08-28T07:58:50ZLuc HermitteImplement SLC supportAt the moment, only GRD products are supported. SLC products should also be supported.At the moment, only GRD products are supported. SLC products should also be supported.https://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/130Mirror github repository2023-05-23T16:46:56ZRémi CressonMirror github repositoryGithub repo is not in sync with this repo, which could be confusing for the users.Github repo is not in sync with this repo, which could be confusing for the users.Thierry KoleckThierry Koleckhttps://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/128Incorrect analysis of S2 results regarding with S1 product need to be downloaded2023-04-05T10:04:55ZLuc HermitteIncorrect analysis of S2 results regarding with S1 product need to be downloadedThe current implementation `does_final_product_need_to_be_generated_for()` to tell whether a S1 product need to be downloaded is incorrect.
- if the S2 product is found and the S2 filtered product isn't found on disk, then the S1 produc...The current implementation `does_final_product_need_to_be_generated_for()` to tell whether a S1 product need to be downloaded is incorrect.
- if the S2 product is found and the S2 filtered product isn't found on disk, then the S1 product won't be marked for download.
=> This is not an issue as the filtered product can be generated from the other one
- if the S2 product is not found while the S2 filtered product is found on disk, then the S1 product won't be marked for download.
=> This is an issue if the S2 product is required (`keep_non_filtered_products : True`): we won't have the required S1 data to generate the S2 product
Note: this scenario is unlikely, but not impossiblehttps://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/122Error KilledWorker2023-12-14T07:09:06ZfloveilError KilledWorkerHey,
When I launch S1 Tiling, I have an error on `distributed.scheduler.KilledWorker` :
`distributed.scheduler.KilledWorker: Attempted to run task /home/florent/s1chain_data/data/tmp/S2/36LVQ/s1a_36LVQ_vh_DES_050_20220804t033102.tif o...Hey,
When I launch S1 Tiling, I have an error on `distributed.scheduler.KilledWorker` :
`distributed.scheduler.KilledWorker: Attempted to run task /home/florent/s1chain_data/data/tmp/S2/36LVQ/s1a_36LVQ_vh_DES_050_20220804t033102.tif on 3 different workers, but all those workers died while running it. The last worker that attempt to run the task was tcp://127.0.0.1:34493. Inspecting worker logs is often a good next step to diagnose what went wrong. For more information see https://distributed.dask.org/en/stable/killed.html.`
I also add [S1Processor-main.debug.log](/uploads/fcbda81572b1eaaa5c1b9c6658e843a0/S1Processor-main.debug.log)
If you have an idea of the source of the problem.
Thankshttps://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/120Support option "keep_non_filtered_product" in cfg file2022-11-10T16:33:41ZThierry KoleckSupport option "keep_non_filtered_product" in cfg filehttps://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/118Chain concatenation and spatial despeckling in-memory, when possible.2022-11-10T16:33:41ZLuc HermitteChain concatenation and spatial despeckling in-memory, when possible.Concatenation is a non trival step that:
- recognizes 2 compatibles inputs and changes the output filename in consequence
- rename orthorectification output when there is only one input.
To be chained in-memory SpatialDespeckle would ne...Concatenation is a non trival step that:
- recognizes 2 compatibles inputs and changes the output filename in consequence
- rename orthorectification output when there is only one input.
To be chained in-memory SpatialDespeckle would need to:
- recognize 2 compatibles inputs and change the output filename in consequence
- start from the renamed file (instead of expecting to be chained in-memory) when there is only one input.
Relates to #116 where in-memory chaining of concat+despeckle isn't supported yet.https://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/113Incorrect pre-download filter for LIAMap2022-10-25T14:26:25ZLuc HermitteIncorrect pre-download filter for LIAMapWhen filtering files to download from `LIAMap` application, files to downloads aren't filtered correctly.
We should:
- Select the pair with the best cover in the required time-range
- Don't look at `fname_fmt.concatenation`, but at `f...When filtering files to download from `LIAMap` application, files to downloads aren't filtered correctly.
We should:
- Select the pair with the best cover in the required time-range
- Don't look at `fname_fmt.concatenation`, but at `fname_fmt.lia_product`
- and then download the selected pair if need be.
The issue has been observed with the following scenario, while the last product (S1A_IW_GRDH_1SDV_20201221T044221_20201221T044246_035779_043013_4BE6) is not yet on disk
```
[DataSource]
download : True
first_date : 2020-12-01
last_date : 2020-12-30
[Processing]
calibration: sigma
tiles : 33NVB
fname_fmt.concatenation : {flying_unit_code}_{tile_name}_{polarisation}_{orbit_direction}_{orbit}_{acquisition_stamp}_{calibration_type}.tif
```
See the [log file](/uploads/58708065b12c9eb7c01b78b6ed05ffae/S1Processor-main.debug.log).https://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/106Improve handling of S1Tiling version2022-01-28T16:15:38ZLuc HermitteImprove handling of S1Tiling version- [ ] Version number should be written at the top of log files
- We may have to use version number from `__meta__.py` and/or current commit hash
- [ ] It should be possible to deduce it from git tags when the current commit is tagged- [ ] Version number should be written at the top of log files
- We may have to use version number from `__meta__.py` and/or current commit hash
- [ ] It should be possible to deduce it from git tags when the current commit is taggedhttps://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/102Apply Precise Orbit files before processing2022-09-28T12:59:59ZThierry KoleckApply Precise Orbit files before processingAfter downloading the GRD data, we can download the Precise Orbit files to update the GDR products.
[https://sentinel.esa.int/web/sentinel/-/access-to-sentinel-1-processor-and-orbit-auxiliary-files](https://sentinel.esa.int/web/sentinel...After downloading the GRD data, we can download the Precise Orbit files to update the GDR products.
[https://sentinel.esa.int/web/sentinel/-/access-to-sentinel-1-processor-and-orbit-auxiliary-files](https://sentinel.esa.int/web/sentinel/-/access-to-sentinel-1-processor-and-orbit-auxiliary-files)
- [ ] @koleckt Evaluate the effect of applying orbit file on orthorectifyed products
- [ ] @lhermitte Implement the downloading and process to apply orbit fileThierry KoleckThierry Koleckhttps://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/97Help page deploying job may fail due to "artifacts for pages are too large"2021-10-13T12:03:05ZLuc HermitteHelp page deploying job may fail due to "artifacts for pages are too large"Sometimes page deploying job will fail because of the weight of built help pages.
A workaround is to remove documentation of older versions with `mc rm -r --force minio-otb/s1-tiling/www/0.2.0rc42/` for instance. But this is only a work...Sometimes page deploying job will fail because of the weight of built help pages.
A workaround is to remove documentation of older versions with `mc rm -r --force minio-otb/s1-tiling/www/0.2.0rc42/` for instance. But this is only a workaround, and the problem needs to be fixed.
If we take a look closer at the `_static` directories, we see they occupy 12Mb each. It's not even from the few S1 and S2 images used in the documentation but mostly from `fonts/` and `css/` data.
We need to find a way to factorize there two information (or may be `_static/` directory in order to reduce the weight of _page_ artefacts.https://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/90Integration Gamma Naught - RTC2021-07-20T14:52:46Zfabien contivalIntegration Gamma Naught - RTCIl conviendrait de pouvoir intégrer le module Gamma Naught RTC.
Il s'agit de l'insérer juste avant l'étape de rectification S2 puis de pouvoir l'activer/la désactiver en config.
https://gitlab.orfeo-toolbox.org/s1-tiling/gamma0-rtcIl conviendrait de pouvoir intégrer le module Gamma Naught RTC.
Il s'agit de l'insérer juste avant l'étape de rectification S2 puis de pouvoir l'activer/la désactiver en config.
https://gitlab.orfeo-toolbox.org/s1-tiling/gamma0-rtcfabien contivalfabien contivalhttps://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling/-/issues/76Provide output in dB values2021-05-25T13:53:32ZMickael SavinaudProvide output in dB valuesdB values option can be nice to avoid conversion after S1 tilingdB values option can be nice to avoid conversion after S1 tiling