Skip to content

Import OTB-Data test data under git lfs

Stéphane Albert requested to merge 1834-test-data-lfs into develop

Summary

Extract and import testing-data subset of the OTB-Data repository into the the OTB one and track these files using git lfs.

This MR is related to #1834 (closed).

Rationale

Optimize testing and CI by extracting and importing test-data from external repository into the OTB repository, which is critical for CI processing time.

Implementation Details

  • Watched data files (using `inotify) used by ctest command on hulk to extract testing data subset out of OTB-Data
  • Scripted extraction of all binary extensions of OTB-Data to setup git lfs tracking
  • Scripted LZW compression on all TIF files using gdal_translate
  • Scripted renaming of filename and filename.hdr to filename.raw and filename.raw.hdr to be tracked by git lfs
  • Setup git lfs tracking (see https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/blob/1834-test-data-lfs/.gitattributes)
  • Imported OTB-Data subset

Additional notes

  • HDR raw files should also be compressed.
  • Factorization of testing data (using same reduced files for more tests)

Copyright

The copyright owner is CNES and has signed the ORFEO ToolBox Contributor License Agreement.

Check before merging:

  • All discussions are resolved
  • At least 2 👍 votes from core developers, no 👎 vote.
  • The feature branch is (reasonably) up-to-date with the base branch
  • Dashboard is green
  • Copyright owner has signed the ORFEO ToolBox Contributor License Agreement
Edited by Antoine Regimbeau

Merge request reports