This files list the identifies open points, and also the actions in terms of data processing and production.
**WARNING: The following applies to LIS version 1.5**
## Densification using Landsat 8 products (old format)
The old format Landsat products are available through Theia. They were not used to densify the input snow products timeseries in the frame of the snow annual map generation (available under /work/OT/siaa/Theia/Neige/SNOW_ANNUAL_MAP_LIS_1.5). This can now be done by using the library [amalthee/0.2](https://gitlab.cnes.fr/datalake/amalthee.git). Among the parameters, it is possible to request products corresponding to a region of interest. This is the best way to retrive products corresponding to the S2 and L8 new format tiles (example: "T31TCH")
On CNES HPC:
```
module load amalthee/0.2
amalthee = Amalthee('oldlandsat')
amalthee.show_params("Landsat")
```
Once the old Landsat products are retrived and processed to obtain the snow products, the snow annual map could
be densified by simply adding them to the densification list. Please refer to [tutorials/prepare_snow_annual_map_data.md]((https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/blob/develop/doc/tutorials/prepare_snow_annual_map_data.md))
for the usage of the script on CNES HPC.
## Modification of the [prepare_snow_annual_map_data.py](https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/blob/develop/hpc/prepare_data_for_snow_annual_map.py)
This script must be modified to at least change the sub-processes that are currently asynchronous and that requires to run it multiple times.
**WARNING: The following tutorial applies to LIS version 1.5 and only on CNES HPC.
However, it could be an example for generating custom preparation script outside of the CNES HPC**
## Prerequisites
The script prepare_snow_annual_map_data.py can only be launch on CNES HPC and with specific modules:
- Python in version 3.5.2
- Amalthee in version 0.2
On CNES HPC:
```
module load python/3.5.2
module load amalthee
```
The script prepare_snow_annual_map_data.py must be located along the following scripts, in order to launch correctly the sub-tasks:
-[run_lis_from_filelist.sh](https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/blob/develop/hpc/run_lis_from_filelist.sh), it is a PBS script dedicated to the production of the snow products
-[run_snow_annual_map.sh](https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/blob/develop/hpc/run_snow_annual_map.sh), it is a PBS script dedicated to the production of the snow annual maps
## Configuration parameters
The scripts prepare_snow_annual_map_data.py does not take additional argument in itself. To edit the configuration,
the script must be modified manually with any text editor. The section of the script to modified is reported below.
```
def main():
params = {"tile_id":"T32TPS",
"date_start":"01/09/2017",
"date_stop":"31/08/2018",
"date_margin":15,
"mode":"DEBUG",
"input_products_list":[],
# path_tmp is an actual parameter but must only be uncomment with a correct path
These parameters are describe in [snow_annual_map_schema.json](https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/blob/develop/doc/snow_annual_map_schema.json)
and correspond to the parameters of the json file to provide to the application run_snow_annual_map.py.
However, the two last parameters are specific to prepare_snow_annual_map_data.py:
- "snow_products_dir", must be filled with the storage path chosen for all the snow products.
- "data_availability_check", must remains at "false" and is only modified by the script itself if all the data required for the snow annual map processing are available.
The only external configuration parameters is the following file:
-[selectNeigeSyntheseMultitemp.csv](https://gitlab.orfeo-toolbox.org/remote_modules/let-it-snow/blob/develop/hpc/selectNeigeSyntheseMultitemp.csv), it is the list of tiles (ex: "T31TCH") for which we want to generate the snow annual map.
## Execution
The script can now simply be launched by the following command.
```
python prepare_snow_annual_map_data.py
```
The generated log allows to monitor the status of the data requires before snow annual map processing.
In the case where all the required snow products are not already available in the **"snow_products_dir"** (which is more than likely!), the script will call two different type of asynchronous sub-process:
- the first one is a request for the generation of the missing snow products under **"snow_products_dir"**, when the L2A products are available.
- the second one is triggered when some L2A products are not available, the sub-process in then in charge for the downloading of the products in to the datalake (see Amalthee documentation for command amalthee_theia.fill_datalake())
Because these sub-processes are asynchronous, it is required to run the prepare_snow_annual_map_data.py multiple time.
For example, if no data is available and that it is the first time you run the script, it would require to run it 3 times:
- the first time to fill the datalake with the requested L2A products.
- the second time to generate all the snow products corresponding to these L2A products
- and finally to check that all the requested data is now available and to trigger the actual computation of the snow annual map.
Between each run it is adviced to monitor off-line the status of the different sub-processes, instead of running the script more than necessary.
