SameFileError when processing with StartMaja
name: General Issue (FA)
about: Use this template for reporting an anomaly.
Initially reported by an external user here: https://github.com/CNES/MAJA/issues/96
System information
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
- Maja installed from (source or binary):
- Maja version (use command below): 4.2
- Maja commit sha (if Maja is installed from source):
- Python version (From binpkg or custom):
- Complementary info (i.e. GCC/Compiler version)
You can collect some of this information using our environment capture script
You can also obtain the Maja version with:
maja --version
Describe the current behavior
Given a workplan
DATE | TILE | MODE | L1-PRODUCT | ADDITIONAL INFO
2021-10-10 10:09:41 | 32UQU | BACKWARD | S2A_MSIL1C_20211010T100941_N0301_R022_T32UQU_20211010T122329.SAFE | Backward of 8 products
2021-10-10 10:09:41 | 32UQU | NOMINAL | S2A_MSIL1C_20211010T100941_N0301_R022_T32UQU_20211010T110450.SAFE | L2 from previous
2021-10-15 10:10:29 | 32UQU | NOMINAL | S2B_MSIL1C_20211015T101029_N0301_R022_T32UQU_20211015T121549.SAFE | L2 from previous
2021-10-23 10:21:01 | 32UQU | NOMINAL | S2A_MSIL1C_20211023T102101_N0301_R065_T32UQU_20211023T110926.SAFE | L2 from previous
2021-10-30 10:11:41 | 32UQU | NOMINAL | S2A_MSIL1C_20211030T101141_N0301_R022_T32UQU_20211030T110921.SAFE | L2 from previous
2021-11-04 10:11:09 | 32UQU | NOMINAL | S2B_MSIL1C_20211104T101109_N0301_R022_T32UQU_20211104T121548.SAFE | L2 from previous
2021-11-14 10:11:59 | 32UQU | NOMINAL | S2B_MSIL1C_20211114T101159_N0301_R022_T32UQU_20211114T105348.SAFE | L2 from previous
2021-11-19 10:13:21 | 32UQU | NOMINAL | S2A_MSIL1C_20211119T101321_N0301_R022_T32UQU_20211119T110331.SAFE | L2 from previous
I got the following error
Traceback (most recent call last):
File "/maja/lib/python/orchestrator/launcher/maja.py", line 100, in <module>
main()
File "/maja/lib/python/orchestrator/launcher/maja.py", line 81, in main
lprocessor.scientific_processing()
File "/maja/lib/python/orchestrator/processor/l2_nominal_processor.py", line 216, in scientific_processing
l2_image_file_writer, checking_conditional_clouds)
File "/maja/lib/python/orchestrator/processor/l2_processor.py", line 828, in process_one_product
self._cams_status, l_IgnoreCurrentLTC)
File "/maja/lib/python/orchestrator/processor/l2_processor_header_writer_setup.py", line 157, in setup_l2_header_writer
l_L2HeaderFileWriter.write()
File "/maja/lib/python/orchestrator/plugins/common/muscate/maja_muscate_l2_header_writer.py", line 75, in write
self.pre_processing()
File "/maja/lib/python/orchestrator/plugins/common/base/maja_l2_header_writer_base.py", line 124, in pre_processing
self.outputl2privateimagefilenamesprovider.get_ltc_image_dir_filename())
File "/maja/lib/python/orchestrator/common/file_utils.py", line 129, in copy_tree
copy_file(src_filename, dst_filename)
File "/maja/lib/python/orchestrator/common/file_utils.py", line 112, in copy_file
shutil.copyfile(source, dest)
File "/maja/lib/python3.7/shutil.py", line 104, in copyfile
raise SameFileError("{!r} and {!r} are the same file".format(src, dst))
shutil.SameFileError: '/mnt/vol/tmp/Start_maja_e73b1f2b4319aeef75c481cf23529f75/SENTINEL2A_20211010-101729-643_L2A_T32UQU_C_V1-0/DATA/SENTINEL2A_20211010-101729-643_L2A_T32UQU_C_V1
-0_PVD_ALL/LTC/20211104.mha' and '/mnt/vol/input/L2A-Products/32UQU/SENTINEL2A_20211010-101729-643_L2A_T32UQU_C_V1-0/DATA/SENTINEL2A_20211010-101729-643_L2A_T32UQU_C_V1-0_PVD_ALL/L
TC/20211104.mha' are the same file
It seems to be happening rather randomly - sometimes reprocessing with the same exact inputs works as intended, sometimes it fails again. Looking at the traceback it seems pretty easy to mitigate (either with try/except block or by checking if the file exists beforehand) - the question is whether it is not a symptom of some underlying issue.
Describe the expected behavior
Urgency/Criticity
- Urgency (Select Low/Medium/High):
- Criticity (Select Low/Medium/High):
Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem. Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
A path to a context folder is also accepted. Please note the Context policy
Correction
(Optional) Provide a version number (Major.Minor.Patch, e.g. 3.2.1) when the correction shall be introduced.