Skip to content

Refactor all keypoints tests into a single test

Julien Michel requested to merge refactor-keypoints-tests into develop

Summary

This MR refactors all keypoints tests into a single, more stable test.

Rationale

For years we endured feTvImageToFastSIFTKeyPointSetFilterSceneOutputInterestPointAscii and the like flickering our dasbhoard. They are now gone and testing strategy is stable and easier to adapt to different behavior of compilers / plateform.

Implementation Details

I removed all tests (and test code) related to Sift, SiftFast and Surf (as well as to the matching filter), and wrote a single test that generates a slightly rotated and scaled image from a reference image, and then for each aglorithm in Sift, SiftFast, and Surf do:

  1. Detect keypoints on both images
  2. Match keypoints
  3. Compute statistics on matching performance
  4. Use quality thresholds to decide if test passes or not.

Here is the ouptut of the new test:

 test 559
    Start 559: feTvKeyPointsAlgorithmsTest

559: Test command: /home/mp/michelj/scratch/dev/clang/otb/bin/otbDescriptorsTestDriver "otbKeyPointsAlgorithmsTest" "/work/scratch/michelj/dev/otb-data/Input/QB_TOULOUSE_RpcTag_100_100.tif"
559: Test timeout computed to be: 1500
559: 2019-01-11 16:26:48 (INFO): Default RAM limit for OTB is 128 MB
559: 2019-01-11 16:26:48 (INFO): GDAL maximum cache size is 3208 MB
559: 2019-01-11 16:26:48 (INFO): OTB will use at most 16 threads
559: 2019-01-11 16:26:49 (INFO): Loading kwl metadata from official product in file /work/scratch/michelj/dev/otb-data/Input/QB_TOULOUSE_RpcTag_100_100.tif
559: Secondary image generated by applying a rotation of 2.5 degrees and scaling of 0.989999999999999991118215802999.
559: Checking Surf implementation:
559: =============================
559: Found 405 points in reference image and 411 points in secondary image
559: Found 80 matches with 55 valid matches (tolerance of 0.5 pixels)
559: More than 15% of reference points have a match:    Ok (19.753086090087890625%)
559: More than 15% of secondary points have a match:    Ok (19.4647216796875%)
559: More than 65% of matches are good:                 Ok (68.75% at 0.5 pixel accuracy)
559: Checking Sift implementation:
559: =============================
559: Found 510 points in reference image and 486 points in secondary image
559: Found 241 matches with 217 valid matches (tolerance of 0.5 pixels)
559: More than 45% of reference points have a match:    Ok (47.254901885986328125%)
559: More than 45% of secondary points have a match:    Ok (49.58847808837890625%)
559: More than 85% of matches are good:                 Ok (90.04149627685546875% at 0.5 pixel accuracy)
559: Checking SiftFast implementation:
559: =================================
559: Found 441 points in reference image and 465 points in secondary image
559: Found 317 matches with 316 valid matches (tolerance of 0.5 pixels)
559: More than 65% of reference points have a match:    Ok (71.88208770751953125%)
559: More than 65% of secondary points have a match:    Ok (68.1720428466796875%)
559: More than 95% of matches are good:                 Ok (99.684539794921875% at 0.5 pixel accuracy)
559:  -> Test EXIT SUCCESS.
559: -------------  No control baseline tests    -------------
1/1 Test #559: feTvKeyPointsAlgorithmsTest ......   Passed    7.97 sec

This allowed me to remove:

  • 10 test source code files
  • 16 tests
  • 19 baseline files from OTB-Data

Icing on the cake:

  • Testing of those feature no longer relies on baselines comparisons,
  • Number of test does no longer depends on OTB_USE_SIFTFAST.

Copyright

The copyright owner is CNES and has signed the ORFEO ToolBox Contributor License Agreement.


Check before merging:

  • All discussions are resolved
  • At least 2 👍 votes from core developers, no 👎 vote.
  • The feature branch is (reasonably) up-to-date with the base branch
  • Dashboard is green
  • Copyright owner has signed the ORFEO ToolBox Contributor License Agreement
Edited by Julien Michel

Merge request reports