let-it-snow issueshttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues2024-03-28T16:00:50Zhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/62Test 40 fails due to timeout issue2024-03-28T16:00:50ZAurore DupuisTest 40 fails due to timeout issueTest 40 : snow_annual_map_muscate_test fails
Timeout has been set to 3000s but the process is stopped with message "Child killed"
However, when launching lis on the json built with test 39 there is no problem.Test 40 : snow_annual_map_muscate_test fails
Timeout has been set to 3000s but the process is stopped with message "Child killed"
However, when launching lis on the json built with test 39 there is no problem.1.12.0Céline RailléCéline Railléhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/130Rastertools shaded snow algorithm is missing some shadow especially on mounta...2024-03-21T15:35:04ZGuillaume Eynard-BontempsRastertools shaded snow algorithm is missing some shadow especially on mountain ridgesWhile comparing results of LIS blue in #21, I observed some shadow detection problem with the new rastetools algorithm used in 1.11.0 version.
The overall observation is that:
- In 1.10.2 version, saga gis algorithm (or the threshold we...While comparing results of LIS blue in #21, I observed some shadow detection problem with the new rastetools algorithm used in 1.11.0 version.
The overall observation is that:
- In 1.10.2 version, saga gis algorithm (or the threshold we use) tends to overestimated shaded zones, and on casted shadow, there seems to be some little shift on the area detected in shadow.
- In 1.11.0 version using rastertools, the shaded areas are much more conservative: casted shadow are really precise, but on slopes and especially mountain ridges, some shaded areas are missed, leading to strange patterns when viewing the results.
The following images illustrate these results and problems near Goulier area in France on the S2 image taken on 19 Nov 2019.
The hillshade mask is green is issued from saga-gis, wherease the one in blue on the top is from rastertools:
![LIS_hillshade_mask_goulier_2019_comparison](/uploads/020555e6fc1ef4b7194c292246483cb7/LIS_hillshade_mask_goulier_2019_comparison.PNG)
On the left, rastertools mask is missing shadow near the mountain ridges. On the top, we can see that saga-gis mask detects also slopes with lower illumination. On the right, rastertools mask is better on casted shadow.
It is hard to argue on some areas which one of the mask is the best, a choice between over estimation and conservative one. But the problem of the ridges of rastertools mask is clearly visible on the final result (LIS 1.11 in blue, LIS 1.10 in green):
![LIS_result_goulier_1.11](/uploads/be12b73c14665a9265f778a74d9db8fd/LIS_result_goulier_1.11.PNG)![LIS_result_goulier_1.10](/uploads/9a26693fc938d9c4073fa52cb009415c/LIS_result_goulier_1.10.PNG)https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/126rastertools default windows_size parameters throws error on some S2 tiles2024-03-21T14:04:19ZGuillaume Eynard-Bontempsrastertools default windows_size parameters throws error on some S2 tilesDepending on elevation and sun angle, Radius automatically computed might be too big.
See following error:
```
INFO:hillshade.py::process_file:Processing file /usr/local/test_lis/dtm/32TLR/dem_20m.tif
2024-02-26T13:43:42.708 ERROR:sn...Depending on elevation and sun angle, Radius automatically computed might be too big.
See following error:
```
INFO:hillshade.py::process_file:Processing file /usr/local/test_lis/dtm/32TLR/dem_20m.tif
2024-02-26T13:43:42.708 ERROR:snow_detector.py::detect_snow:Rastertools hillshade methods encountered an error : The radius (option --radius, value=629) must be strictly less than half the size of the window (option --window_size, value=1024)
2024-02-26T13:43:42.709 ERROR:let_it_snow_fsc.py::let_it_snow_fsc:The radius (option --radius, value=629) must be strictly less than half the size of the window (option --window_size, value=1024)
Traceback (most recent call last):
File "app/let_it_snow_fsc.py", line 166, in let_it_snow_fsc
detect_snow(config, output_dir, h2_chain_version, product_counter)
File "/usr/local/lib/python3.8/site-packages/s2snow/snow_detector.py", line 162, in detect_snow
compute_hillshade_mask_rastertools(relief_shadow_mask, dem, config.metadata, tmp_dir, tg_resolution,
File "/usr/local/lib/python3.8/site-packages/s2snow/hillshade.py", line 309, in compute_hillshade_mask_rastertools
hillshade_str = tool.process_file(dem_path)
File "/usr/local/lib/python3.8/dist-packages/eolab/rastertools/hillshade.py", line 128, in process_file
raise ValueError(f"The radius (option --radius, value={radius}) must be strictly "
ValueError: The radius (option --radius, value=629) must be strictly less than half the size of the window (option --window_size, value=1024)
```
We must:
- increase default windows_size to a sensible value (4096?)
- Check that it works on different conditions, with correct results
- Eventually make rastertools more robust to this case.1.12.0Michael ERBLANGMichael ERBLANGhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/21Autumn snow not detected in shaded slopes2024-03-15T13:08:10ZSimon GascoinAutumn snow not detected in shaded slopesSnow on shaded slopes is not detected in this area near Goulier, France on 19 Nov 2019.
[Link L2B Theia](https://theia.cnes.fr/atdistrib/rocket/#/collections/Snow/5f5b56d0-ca36-57ec-8df5-2ff7c559205c)
[Link L1C image](https://services....Snow on shaded slopes is not detected in this area near Goulier, France on 19 Nov 2019.
[Link L2B Theia](https://theia.cnes.fr/atdistrib/rocket/#/collections/Snow/5f5b56d0-ca36-57ec-8df5-2ff7c559205c)
[Link L1C image](https://services.sentinel-hub.com/ogc/wms/b0eec84c-9a8b-4186-8bd1-b79f90329fab?version=1.1.1&service=WMS&request=GetMap&format=image%2Ftiff&crs=EPSG%3A3857&layers=1-NATURAL-COLOR&bbox=161788%2C5268431%2C174591%2C5275148&time=2019-05-01T00%3A00%3A00.000Z%2F2019-11-21T23%3A59%3A59.000Z&width=1360&height=703&showlogo=false&transparent=true&maxcc=100&evalsource=S2&nicename=Sentinel-2%20L1C%20image%20on%202019-11-21.jpg&bgcolor=000000)
![Capture_d_écran_2019-11-21_à_16.40.09](/uploads/7400394bf058bee195aad212f88b23a2/Capture_d_écran_2019-11-21_à_16.40.09.png)
![Capture_d_écran_2019-11-21_à_16.40.02](/uploads/56df66f6257ca196342ff2fc6c1e432d/Capture_d_écran_2019-11-21_à_16.40.02.png)Simon GascoinSimon Gascoinhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/129synthesis code is verbose2024-03-08T16:12:34ZSimon Gascoinsynthesis code is verbose`let_it_snow_synthesis.py` outputs a lot of text to STDOUT even when setting "log_level": "WARNING" in the [config file](`doc/synthesis_launch_example.json`). It seems that the log level of OTB applications is not maintained throughout t...`let_it_snow_synthesis.py` outputs a lot of text to STDOUT even when setting "log_level": "WARNING" in the [config file](`doc/synthesis_launch_example.json`). It seems that the log level of OTB applications is not maintained throughout the code.https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/127NaN in metadata file2024-03-06T09:28:36ZSimon GascoinNaN in metadata fileNot really a "LIS" issue but in LIS*.xml files produced by CNES (/work/THEIA/hesperides/prod/HYMOTEP/) incidence angles copied from L2A product are always NaN
<Incidence_Angles>
<ZENITH_ANGLE unit="deg">NaN</ZENITH_ANGLE>
...Not really a "LIS" issue but in LIS*.xml files produced by CNES (/work/THEIA/hesperides/prod/HYMOTEP/) incidence angles copied from L2A product are always NaN
<Incidence_Angles>
<ZENITH_ANGLE unit="deg">NaN</ZENITH_ANGLE>
<AZIMUTH_ANGLE unit="deg">NaN</AZIMUTH_ANGLE>
</Incidence_Angles>https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/118Tests set parameters (and other part of the files here) are tightly linked to...2024-01-22T15:50:32ZGuillaume Eynard-BontempsTests set parameters (and other part of the files here) are tightly linked to CNES HPC infrastructureA lot of test set configuration files point to CNES HPC infrastructure shared storage, in particular for auxiliary data and saga_gis software.
It can be verified using simple grep commands:
```
grep -R work .
grep -R soft .
```
```
......A lot of test set configuration files point to CNES HPC infrastructure shared storage, in particular for auxiliary data and saga_gis software.
It can be verified using simple grep commands:
```
grep -R work .
grep -R soft .
```
```
...
./Input-Data-Test/SNOW_PRODUCTS/SENTINEL2A_20180131-105416-437_L2A_T31TCH_D_V1-4/param_test.json: "dem": "/work/OT/siaa/Theia/Neige/DEM/S2__TEST_AUX_REFDE2_T31TCH_0001.DBL.DIR/S2__TEST_AUX_REFDE2_T31TCH_0001_ALT_R2.TIF"
...
./Input-Data-Test/S2/shaded_snow/global_parameters_shaded_snow_saga.json: "sagagis_tools_libs": "/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4-4ujdnstfskok34hg3kerp662wwdaezso/lib/saga"
...
```
It would be better to extract all this to be able to run all tests outside of CNES infrastructure.1.12.0Michael ERBLANGMichael ERBLANGhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/106Migration from PBS to SLURM2024-01-12T08:26:36ZPierre TysebaertMigration from PBS to SLURMHPC scripts must be migrated to Slurm Workload Manager
Ajout de scripts :
- [ ] génération d'un produit neige
- [ ] génération de produits neige pour plusieurs tuile ou date
- [x] génération de synthèse (unique)
- [x] génération de synt...HPC scripts must be migrated to Slurm Workload Manager
Ajout de scripts :
- [ ] génération d'un produit neige
- [ ] génération de produits neige pour plusieurs tuile ou date
- [x] génération de synthèse (unique)
- [x] génération de synthèse par tuile
- [x] génération de synthèse par période1.12.0Pierre TysebaertPierre Tysebaerthttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/122Ability to configure a tmp working directory different from output directory2023-10-05T10:22:32ZGuillaume Eynard-BontempsAbility to configure a tmp working directory different from output directoryCurrent output for Snow FSC product example layout:
```
fsc_config.json
LIS_METADATA.XML
LIS_S2-SNOW-FSC_T31TCJ_20230204T105857_1.10.2_1.tif
LIS_S2-SNOW-FSC-QCFLAGS_T31TCJ_20230204T105857_1.10.2_1.tif
tmp/
```
At least tmp folder path s...Current output for Snow FSC product example layout:
```
fsc_config.json
LIS_METADATA.XML
LIS_S2-SNOW-FSC_T31TCJ_20230204T105857_1.10.2_1.tif
LIS_S2-SNOW-FSC-QCFLAGS_T31TCJ_20230204T105857_1.10.2_1.tif
tmp/
```
At least tmp folder path should be configurable if needed (typically if we don't want to check the files here). This is especially a good thing when working on an HPC cluster.
Maybe json and XML files should have the product id in their file names.
Should we create a parent folder with product id?
Same should be done for other products.https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/101Salt classified as snow in Camargue2023-07-31T14:49:02ZAurore DupuisSalt classified as snow in CamargueOn this product : https://theia.cnes.fr/atdistrib/rocket/#/collections/Snow/70210c8c-cd96-518c-a535-7c10e5fb68cf
L2A : https://theia.cnes.fr/atdistrib/rocket/#/collections/SENTINEL2/797303ad-6236-5fc5-bc52-5a647f658bd1
![image](/upload...On this product : https://theia.cnes.fr/atdistrib/rocket/#/collections/Snow/70210c8c-cd96-518c-a535-7c10e5fb68cf
L2A : https://theia.cnes.fr/atdistrib/rocket/#/collections/SENTINEL2/797303ad-6236-5fc5-bc52-5a647f658bd1
![image](/uploads/62028c94a8150521e6645382c7850ce2/image.png)https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/108Difficulties working with Copernicus images2023-07-17T12:05:15ZLUCIA BELEN CANIUNIR LENANDifficulties working with Copernicus imagesI have been trying to work on the LIS application for the Andes-Chile, however, I have had some complications in its execution.
I have some queries, in the script "fsc_config.py" from line 580 onwards in the function >>def retrieve_missi...I have been trying to work on the LIS application for the Andes-Chile, however, I have had some complications in its execution.
I have some queries, in the script "fsc_config.py" from line 580 onwards in the function >>def retrieve_mission(self, absoluteFilename)<<
would it only be possible to load the information of specific products contained in THEIA, would it be feasible to modify this function to be able to extract the data from Copernicus (Sentinel 2 images), what could you recommend me for this situation?https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/1confusion in $f_s$ and $f_t$ in ATBD doc2023-01-16T14:49:21ZManuel Grizonnetconfusion in $f_s$ and $f_t$ in ATBD docmigrate from http://tully.ups-tlse.fr/grizonnet/let-it-snow/issues/39migrate from http://tully.ups-tlse.fr/grizonnet/let-it-snow/issues/39BacklogSimon GascoinSimon Gascoinhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/107check if LIS Blue solves this case2022-12-15T16:34:29ZSimon Gascoincheck if LIS Blue solves this caseThe shadow mask at the acquisition date of this image can be generated using insol R-package.
```
library(insol)
library(raster)
# sun vector from S2 image metadata
sun_azimuth=166.513749079941
sun_zenith=63.3878130889177
sv=normalvect...The shadow mask at the acquisition date of this image can be generated using insol R-package.
```
library(insol)
library(raster)
# sun vector from S2 image metadata
sun_azimuth=166.513749079941
sun_zenith=63.3878130889177
sv=normalvector(sun_zenith,sun_azimuth)
# DEM
fdem='S2__TEST_AUX_REFDE2_T31TCH_0001.DBL.DIR/S2__TEST_AUX_REFDE2_T31TCH_0001_ALT_R2.TIF'
dem=raster(f)
# intensity
sh=doshade(dem,sv)
grd=cgrad(dem)
hsh=grd[,,1]*sv[1]+grd[,,2]*sv[2]+grd[,,3]*sv[3]
hsh=(hsh+abs(hsh))/2
hsh=raster(hsh,crs=projection(dem))
extent(hsh)=extent(dem)
# threshold intensity and add projected shadows (same threshold as LIS Blue default)
mask = hsh*sh<0.2
fmask = 'relief_shadow_mask_20191119-104916-971_R2_T31TCH.tif'
writeRaster(mask, datatype='INT2U', fmask, options=c('COMPRESS=DEFLATE'))
```
[relief_shadow_mask_20191119-104916-971_R2_T31TCH.tif](/uploads/eb67d41c3a3b254da67c3a4aba6831df/relief_shadow_mask_20191119-104916-971_R2_T31TCH.tif)https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/14Ice free Saint Laurent classified as snow2021-06-09T15:09:04ZOlivier HagolleIce free Saint Laurent classified as snowThe saint Laurent does not look too bright, or too turbid, but is classified as snow.
Tuile T19TCN, 20190413
![RGB](https://theia.cnes.fr/data//SENTINEL2/2019/04/13/SENTINEL2A_20190413-155002-190_L2A_T19TCN_D_V2-0/SENTINEL2A_20190413-15...The saint Laurent does not look too bright, or too turbid, but is classified as snow.
Tuile T19TCN, 20190413
![RGB](https://theia.cnes.fr/data//SENTINEL2/2019/04/13/SENTINEL2A_20190413-155002-190_L2A_T19TCN_D_V2-0/SENTINEL2A_20190413-155002-190_L2A_T19TCN_D_V2-0_QKL_ALL.png)
![Mask](https://theia.cnes.fr/data//Snow/2019/04/13/SENTINEL2A_20190413-155002-190_L2B-SNOW_T19TCN_D_V1-7/SENTINEL2A_20190413-155002-190_L2B-SNOW_T19TCN_D_V1-7_QKL_ALL.png)
https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/7Change numpy array calculation2021-03-22T09:16:00ZManuel GrizonnetChange numpy array calculationfrom: http://tully.ups-tlse.fr/grizonnet/let-it-snow/issues/11
Numpy calculations need to be changed or maerged into a single file (utils.py ?) Here is the list where numpy is used:
- compute_HSmin Use numpy boolean filter and min to f...from: http://tully.ups-tlse.fr/grizonnet/let-it-snow/issues/11
Numpy calculations need to be changed or maerged into a single file (utils.py ?) Here is the list where numpy is used:
- compute_HSmin Use numpy boolean filter and min to find the minimum elevation where there is snow.
- compute_HSmax Use numpy boolean filter, min and max to find the elevation where, from this value and above, there is only snow.
- compute_cloudpercent/snowpercent (also used in format_output.py)
- compute_cloud
- compute_stats_internal Use combinations of numpy array adn sum to make stats from image
- format_percent Format stats as percentages as in the publicationBackloghttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/16data_availability_check not used2021-03-22T09:15:15ZSimon Gascoindata_availability_check not usedOption "data_availability_check" in hpc/prepare_data_for_snow_annual_map.py is not usedOption "data_availability_check" in hpc/prepare_data_for_snow_annual_map.py is not usedBackloghttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/34handle fluctuating suffix policy of Muscate when preparing annual maps2021-03-22T09:13:06ZSimon Gascoinhandle fluctuating suffix policy of Muscate when preparing annual mapsThe name of the product in the catalogue (as returned by amalthee) is not identical to the one in the datalake (due to suffix change). We can also ignore the product version.The name of the product in the catalogue (as returned by amalthee) is not identical to the one in the datalake (due to suffix change). We can also ignore the product version.BacklogSimon GascoinSimon Gascoinhttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/38Test 37 enhancement2021-03-22T09:12:39ZAurore DupuisTest 37 enhancementtest 37 failed due to nodata management
sgascoin: The nodata difference could be be fixed by unsetting the nodata in both products. (0 should not be nodata). There remain a difference according to gdalcompare that I cannot identify in Q...test 37 failed due to nodata management
sgascoin: The nodata difference could be be fixed by unsetting the nodata in both products. (0 should not be nodata). There remain a difference according to gdalcompare that I cannot identify in QGIS both products look identical (even the difference map is zero everywhere).
todo : set the nodata value and update the test data.Backloghttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/74Error in Build_json.py While Reading Landsat Data2021-03-22T09:11:16Zarnab-muhuriError in Build_json.py While Reading Landsat DataBuild_json.py is unable to read the Landsat zip. The naming template in the dictionary (as declared inside Build_json.py) for the Landsat products is not matching.
I am attaching the snapshots here.
![Build_json_landsat_dict](/upload...Build_json.py is unable to read the Landsat zip. The naming template in the dictionary (as declared inside Build_json.py) for the Landsat products is not matching.
I am attaching the snapshots here.
![Build_json_landsat_dict](/uploads/3611db993497648cb5e1fd8070dc2e9e/Build_json_landsat_dict.png)
![Landsat_LIS_Error](/uploads/8140c3d125c99cf57140f35f2821cf3a/Landsat_LIS_Error.png)
![Landsat8_Naming](/uploads/dff089e301647837ac8f2b435bdc113b/Landsat8_Naming.png)
[LC08_CU_003008_20160417_20190430_C01_V01.xml](/uploads/186905c6321c06da3e3f659ec00c4d0e/LC08_CU_003008_20160417_20190430_C01_V01.xml)
We will have to add a new dictionary.Backloghttps://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/-/issues/45Move compute_psl.py into python package2021-03-22T09:09:56ZAurore DupuisMove compute_psl.py into python packageBacklogAurore DupuisAurore Dupuis