let-it-snow issueshttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues2024-03-21T14:04:19Zhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/126rastertools default windows_size parameters throws error on some S2 tiles2024-03-21T14:04:19ZGuillaume Eynard-Bontempsrastertools default windows_size parameters throws error on some S2 tilesDepending on elevation and sun angle, Radius automatically computed might be too big.
See following error:
```
INFO:hillshade.py::process_file:Processing file /usr/local/test_lis/dtm/32TLR/dem_20m.tif
2024-02-26T13:43:42.708 ERROR:sn...Depending on elevation and sun angle, Radius automatically computed might be too big.
See following error:
```
INFO:hillshade.py::process_file:Processing file /usr/local/test_lis/dtm/32TLR/dem_20m.tif
2024-02-26T13:43:42.708 ERROR:snow_detector.py::detect_snow:Rastertools hillshade methods encountered an error : The radius (option --radius, value=629) must be strictly less than half the size of the window (option --window_size, value=1024)
2024-02-26T13:43:42.709 ERROR:let_it_snow_fsc.py::let_it_snow_fsc:The radius (option --radius, value=629) must be strictly less than half the size of the window (option --window_size, value=1024)
Traceback (most recent call last):
File "app/let_it_snow_fsc.py", line 166, in let_it_snow_fsc
detect_snow(config, output_dir, h2_chain_version, product_counter)
File "/usr/local/lib/python3.8/site-packages/s2snow/snow_detector.py", line 162, in detect_snow
compute_hillshade_mask_rastertools(relief_shadow_mask, dem, config.metadata, tmp_dir, tg_resolution,
File "/usr/local/lib/python3.8/site-packages/s2snow/hillshade.py", line 309, in compute_hillshade_mask_rastertools
hillshade_str = tool.process_file(dem_path)
File "/usr/local/lib/python3.8/dist-packages/eolab/rastertools/hillshade.py", line 128, in process_file
raise ValueError(f"The radius (option --radius, value={radius}) must be strictly "
ValueError: The radius (option --radius, value=629) must be strictly less than half the size of the window (option --window_size, value=1024)
```
We must:
- increase default windows_size to a sensible value (4096?)
- Check that it works on different conditions, with correct results
- Eventually make rastertools more robust to this case.1.12.0Michael ERBLANGMichael ERBLANGhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/118Tests set parameters (and other part of the files here) are tightly linked to...2024-01-22T15:50:32ZGuillaume Eynard-BontempsTests set parameters (and other part of the files here) are tightly linked to CNES HPC infrastructureA lot of test set configuration files point to CNES HPC infrastructure shared storage, in particular for auxiliary data and saga_gis software.
It can be verified using simple grep commands:
```
grep -R work .
grep -R soft .
```
```
......A lot of test set configuration files point to CNES HPC infrastructure shared storage, in particular for auxiliary data and saga_gis software.
It can be verified using simple grep commands:
```
grep -R work .
grep -R soft .
```
```
...
./Input-Data-Test/SNOW_PRODUCTS/SENTINEL2A_20180131-105416-437_L2A_T31TCH_D_V1-4/param_test.json: "dem": "/work/OT/siaa/Theia/Neige/DEM/S2__TEST_AUX_REFDE2_T31TCH_0001.DBL.DIR/S2__TEST_AUX_REFDE2_T31TCH_0001_ALT_R2.TIF"
...
./Input-Data-Test/S2/shaded_snow/global_parameters_shaded_snow_saga.json: "sagagis_tools_libs": "/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4-4ujdnstfskok34hg3kerp662wwdaezso/lib/saga"
...
```
It would be better to extract all this to be able to run all tests outside of CNES infrastructure.1.12.0Michael ERBLANGMichael ERBLANGhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/106Migration from PBS to SLURM2024-01-12T08:26:36ZPierre TysebaertMigration from PBS to SLURMHPC scripts must be migrated to Slurm Workload Manager
Ajout de scripts :
- [ ] génération d'un produit neige
- [ ] génération de produits neige pour plusieurs tuile ou date
- [x] génération de synthèse (unique)
- [x] génération de synt...HPC scripts must be migrated to Slurm Workload Manager
Ajout de scripts :
- [ ] génération d'un produit neige
- [ ] génération de produits neige pour plusieurs tuile ou date
- [x] génération de synthèse (unique)
- [x] génération de synthèse par tuile
- [x] génération de synthèse par période1.12.0Pierre TysebaertPierre Tysebaerthttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/62Test 40 fails due to timeout issue2024-03-28T16:00:50ZAurore DupuisTest 40 fails due to timeout issueTest 40 : snow_annual_map_muscate_test fails
Timeout has been set to 3000s but the process is stopped with message "Child killed"
However, when launching lis on the json built with test 39 there is no problem.Test 40 : snow_annual_map_muscate_test fails
Timeout has been set to 3000s but the process is stopped with message "Child killed"
However, when launching lis on the json built with test 39 there is no problem.1.12.0Céline RailléCéline Railléhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/130Rastertools shaded snow algorithm is missing some shadow especially on mounta...2024-03-21T15:35:04ZGuillaume Eynard-BontempsRastertools shaded snow algorithm is missing some shadow especially on mountain ridgesWhile comparing results of LIS blue in #21, I observed some shadow detection problem with the new rastetools algorithm used in 1.11.0 version.
The overall observation is that:
- In 1.10.2 version, saga gis algorithm (or the threshold we...While comparing results of LIS blue in #21, I observed some shadow detection problem with the new rastetools algorithm used in 1.11.0 version.
The overall observation is that:
- In 1.10.2 version, saga gis algorithm (or the threshold we use) tends to overestimated shaded zones, and on casted shadow, there seems to be some little shift on the area detected in shadow.
- In 1.11.0 version using rastertools, the shaded areas are much more conservative: casted shadow are really precise, but on slopes and especially mountain ridges, some shaded areas are missed, leading to strange patterns when viewing the results.
The following images illustrate these results and problems near Goulier area in France on the S2 image taken on 19 Nov 2019.
The hillshade mask is green is issued from saga-gis, wherease the one in blue on the top is from rastertools:
![LIS_hillshade_mask_goulier_2019_comparison](/uploads/020555e6fc1ef4b7194c292246483cb7/LIS_hillshade_mask_goulier_2019_comparison.PNG)
On the left, rastertools mask is missing shadow near the mountain ridges. On the top, we can see that saga-gis mask detects also slopes with lower illumination. On the right, rastertools mask is better on casted shadow.
It is hard to argue on some areas which one of the mask is the best, a choice between over estimation and conservative one. But the problem of the ridges of rastertools mask is clearly visible on the final result (LIS 1.11 in blue, LIS 1.10 in green):
![LIS_result_goulier_1.11](/uploads/be12b73c14665a9265f778a74d9db8fd/LIS_result_goulier_1.11.PNG)![LIS_result_goulier_1.10](/uploads/9a26693fc938d9c4073fa52cb009415c/LIS_result_goulier_1.10.PNG)https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/129synthesis code is verbose2024-03-08T16:12:34ZSimon Gascoinsynthesis code is verbose`let_it_snow_synthesis.py` outputs a lot of text to STDOUT even when setting "log_level": "WARNING" in the [config file](`doc/synthesis_launch_example.json`). It seems that the log level of OTB applications is not maintained throughout t...`let_it_snow_synthesis.py` outputs a lot of text to STDOUT even when setting "log_level": "WARNING" in the [config file](`doc/synthesis_launch_example.json`). It seems that the log level of OTB applications is not maintained throughout the code.https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/127NaN in metadata file2024-03-06T09:28:36ZSimon GascoinNaN in metadata fileNot really a "LIS" issue but in LIS*.xml files produced by CNES (/work/THEIA/hesperides/prod/HYMOTEP/) incidence angles copied from L2A product are always NaN
<Incidence_Angles>
<ZENITH_ANGLE unit="deg">NaN</ZENITH_ANGLE>
...Not really a "LIS" issue but in LIS*.xml files produced by CNES (/work/THEIA/hesperides/prod/HYMOTEP/) incidence angles copied from L2A product are always NaN
<Incidence_Angles>
<ZENITH_ANGLE unit="deg">NaN</ZENITH_ANGLE>
<AZIMUTH_ANGLE unit="deg">NaN</AZIMUTH_ANGLE>
</Incidence_Angles>https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/122Ability to configure a tmp working directory different from output directory2023-10-05T10:22:32ZGuillaume Eynard-BontempsAbility to configure a tmp working directory different from output directoryCurrent output for Snow FSC product example layout:
```
fsc_config.json
LIS_METADATA.XML
LIS_S2-SNOW-FSC_T31TCJ_20230204T105857_1.10.2_1.tif
LIS_S2-SNOW-FSC-QCFLAGS_T31TCJ_20230204T105857_1.10.2_1.tif
tmp/
```
At least tmp folder path s...Current output for Snow FSC product example layout:
```
fsc_config.json
LIS_METADATA.XML
LIS_S2-SNOW-FSC_T31TCJ_20230204T105857_1.10.2_1.tif
LIS_S2-SNOW-FSC-QCFLAGS_T31TCJ_20230204T105857_1.10.2_1.tif
tmp/
```
At least tmp folder path should be configurable if needed (typically if we don't want to check the files here). This is especially a good thing when working on an HPC cluster.
Maybe json and XML files should have the product id in their file names.
Should we create a parent folder with product id?
Same should be done for other products.https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/108Difficulties working with Copernicus images2023-07-17T12:05:15ZLUCIA BELEN CANIUNIR LENANDifficulties working with Copernicus imagesI have been trying to work on the LIS application for the Andes-Chile, however, I have had some complications in its execution.
I have some queries, in the script "fsc_config.py" from line 580 onwards in the function >>def retrieve_missi...I have been trying to work on the LIS application for the Andes-Chile, however, I have had some complications in its execution.
I have some queries, in the script "fsc_config.py" from line 580 onwards in the function >>def retrieve_mission(self, absoluteFilename)<<
would it only be possible to load the information of specific products contained in THEIA, would it be feasible to modify this function to be able to extract the data from Copernicus (Sentinel 2 images), what could you recommend me for this situation?https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/107check if LIS Blue solves this case2022-12-15T16:34:29ZSimon Gascoincheck if LIS Blue solves this caseThe shadow mask at the acquisition date of this image can be generated using insol R-package.
```
library(insol)
library(raster)
# sun vector from S2 image metadata
sun_azimuth=166.513749079941
sun_zenith=63.3878130889177
sv=normalvect...The shadow mask at the acquisition date of this image can be generated using insol R-package.
```
library(insol)
library(raster)
# sun vector from S2 image metadata
sun_azimuth=166.513749079941
sun_zenith=63.3878130889177
sv=normalvector(sun_zenith,sun_azimuth)
# DEM
fdem='S2__TEST_AUX_REFDE2_T31TCH_0001.DBL.DIR/S2__TEST_AUX_REFDE2_T31TCH_0001_ALT_R2.TIF'
dem=raster(f)
# intensity
sh=doshade(dem,sv)
grd=cgrad(dem)
hsh=grd[,,1]*sv[1]+grd[,,2]*sv[2]+grd[,,3]*sv[3]
hsh=(hsh+abs(hsh))/2
hsh=raster(hsh,crs=projection(dem))
extent(hsh)=extent(dem)
# threshold intensity and add projected shadows (same threshold as LIS Blue default)
mask = hsh*sh<0.2
fmask = 'relief_shadow_mask_20191119-104916-971_R2_T31TCH.tif'
writeRaster(mask, datatype='INT2U', fmask, options=c('COMPRESS=DEFLATE'))
```
[relief_shadow_mask_20191119-104916-971_R2_T31TCH.tif](/uploads/eb67d41c3a3b254da67c3a4aba6831df/relief_shadow_mask_20191119-104916-971_R2_T31TCH.tif)https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/101Salt classified as snow in Camargue2023-07-31T14:49:02ZAurore DupuisSalt classified as snow in CamargueOn this product : https://theia.cnes.fr/atdistrib/rocket/#/collections/Snow/70210c8c-cd96-518c-a535-7c10e5fb68cf
L2A : https://theia.cnes.fr/atdistrib/rocket/#/collections/SENTINEL2/797303ad-6236-5fc5-bc52-5a647f658bd1
![image](/upload...On this product : https://theia.cnes.fr/atdistrib/rocket/#/collections/Snow/70210c8c-cd96-518c-a535-7c10e5fb68cf
L2A : https://theia.cnes.fr/atdistrib/rocket/#/collections/SENTINEL2/797303ad-6236-5fc5-bc52-5a647f658bd1
![image](/uploads/62028c94a8150521e6645382c7850ce2/image.png)https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/74Error in Build_json.py While Reading Landsat Data2021-03-22T09:11:16Zarnab-muhuriError in Build_json.py While Reading Landsat DataBuild_json.py is unable to read the Landsat zip. The naming template in the dictionary (as declared inside Build_json.py) for the Landsat products is not matching.
I am attaching the snapshots here.
![Build_json_landsat_dict](/upload...Build_json.py is unable to read the Landsat zip. The naming template in the dictionary (as declared inside Build_json.py) for the Landsat products is not matching.
I am attaching the snapshots here.
![Build_json_landsat_dict](/uploads/3611db993497648cb5e1fd8070dc2e9e/Build_json_landsat_dict.png)
![Landsat_LIS_Error](/uploads/8140c3d125c99cf57140f35f2821cf3a/Landsat_LIS_Error.png)
![Landsat8_Naming](/uploads/dff089e301647837ac8f2b435bdc113b/Landsat8_Naming.png)
[LC08_CU_003008_20160417_20190430_C01_V01.xml](/uploads/186905c6321c06da3e3f659ec00c4d0e/LC08_CU_003008_20160417_20190430_C01_V01.xml)
We will have to add a new dictionary.Backloghttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/72Improve snow detection in forest areas2020-09-08T13:58:39ZAurore DupuisImprove snow detection in forest areashttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/45Move compute_psl.py into python package2021-03-22T09:09:56ZAurore DupuisMove compute_psl.py into python packageBacklogAurore DupuisAurore Dupuishttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/38Test 37 enhancement2021-03-22T09:12:39ZAurore DupuisTest 37 enhancementtest 37 failed due to nodata management
sgascoin: The nodata difference could be be fixed by unsetting the nodata in both products. (0 should not be nodata). There remain a difference according to gdalcompare that I cannot identify in Q...test 37 failed due to nodata management
sgascoin: The nodata difference could be be fixed by unsetting the nodata in both products. (0 should not be nodata). There remain a difference according to gdalcompare that I cannot identify in QGIS both products look identical (even the difference map is zero everywhere).
todo : set the nodata value and update the test data.Backloghttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/34handle fluctuating suffix policy of Muscate when preparing annual maps2021-03-22T09:13:06ZSimon Gascoinhandle fluctuating suffix policy of Muscate when preparing annual mapsThe name of the product in the catalogue (as returned by amalthee) is not identical to the one in the datalake (due to suffix change). We can also ignore the product version.The name of the product in the catalogue (as returned by amalthee) is not identical to the one in the datalake (due to suffix change). We can also ignore the product version.BacklogSimon GascoinSimon Gascoinhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/33read Snow products from datalake for annual map2021-03-22T09:08:55ZSimon Gascoinread Snow products from datalake for annual mapThe Snow collection is accessible in the datalake so the annual map should check this first instead of recomputing the Snow product.The Snow collection is accessible in the datalake so the annual map should check this first instead of recomputing the Snow product.BacklogSimon GascoinSimon Gascoinhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/30Remote module2020-03-30T20:16:33ZJulie BrossardRemote moduleLIS_remoteJulie BrossardJulie Brossardhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/28Use ParameterType_Bool instead of ParameterType_Int2021-03-22T09:09:32ZAurore DupuisUse ParameterType_Bool instead of ParameterType_IntVoir les commentaires de la MR !3
* src/ComputeSnowLine.cxx l 60
* src/ComputeContours.cxx l58Voir les commentaires de la MR !3
* src/ComputeSnowLine.cxx l 60
* src/ComputeContours.cxx l58Backloghttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/27Inconsistent snow cover in adjacent tile overlap (Québec)2020-01-20T10:57:27ZSimon GascoinInconsistent snow cover in adjacent tile overlap (Québec)![anim](/uploads/023e6035bedfbb49896432be0827ba0f/anim.gif)
Link to products
https://theia.cnes.fr/atdistrib/rocket/#/collections/Snow/6d9cc086-88ff-578e-b284-f0e5a5fe9642
https://theia.cnes.fr/atdistrib/rocket/#/collections/Snow/28ad16...![anim](/uploads/023e6035bedfbb49896432be0827ba0f/anim.gif)
Link to products
https://theia.cnes.fr/atdistrib/rocket/#/collections/Snow/6d9cc086-88ff-578e-b284-f0e5a5fe9642
https://theia.cnes.fr/atdistrib/rocket/#/collections/Snow/28ad16fd-8e29-58d7-905b-5b790b8aba93
ZS=-1 (south) vs. ZS=110 (north)
Probable cause is that pass2 is not activated for the south product