Refactor all keypoints tests into a single test

Summary

This MR refactors all keypoints tests into a single, more stable test.

Rationale

For years we endured feTvImageToFastSIFTKeyPointSetFilterSceneOutputInterestPointAscii and the like flickering our dasbhoard. They are now gone and testing strategy is stable and easier to adapt to different behavior of compilers / plateform.

Implementation Details

I removed all tests (and test code) related to Sift, SiftFast and Surf (as well as to the matching filter), and wrote a single test that generates a slightly rotated and scaled image from a reference image, and then for each aglorithm in Sift, SiftFast, and Surf do:

1. Detect keypoints on both images
2. Match keypoints
3. Compute statistics on matching performance
4. Use quality thresholds to decide if test passes or not.

Here is the ouptut of the new test:

 test 559
    Start 559: feTvKeyPointsAlgorithmsTest

559: Test command: /home/mp/michelj/scratch/dev/clang/otb/bin/otbDescriptorsTestDriver "otbKeyPointsAlgorithmsTest" "/work/scratch/michelj/dev/otb-data/Input/QB_TOULOUSE_RpcTag_100_100.tif"
559: Test timeout computed to be: 1500
559: 2019-01-11 16:26:48 (INFO): Default RAM limit for OTB is 128 MB
559: 2019-01-11 16:26:48 (INFO): GDAL maximum cache size is 3208 MB
559: 2019-01-11 16:26:48 (INFO): OTB will use at most 16 threads
559: 2019-01-11 16:26:49 (INFO): Loading kwl metadata from official product in file /work/scratch/michelj/dev/otb-data/Input/QB_TOULOUSE_RpcTag_100_100.tif
559: Secondary image generated by applying a rotation of 2.5 degrees and scaling of 0.989999999999999991118215802999.
559: Checking Surf implementation:
559: =============================
559: Found 405 points in reference image and 411 points in secondary image
559: Found 80 matches with 55 valid matches (tolerance of 0.5 pixels)
559: More than 15% of reference points have a match:    Ok (19.753086090087890625%)
559: More than 15% of secondary points have a match:    Ok (19.4647216796875%)
559: More than 65% of matches are good:                 Ok (68.75% at 0.5 pixel accuracy)
559: Checking Sift implementation:
559: =============================
559: Found 510 points in reference image and 486 points in secondary image
559: Found 241 matches with 217 valid matches (tolerance of 0.5 pixels)
559: More than 45% of reference points have a match:    Ok (47.254901885986328125%)
559: More than 45% of secondary points have a match:    Ok (49.58847808837890625%)
559: More than 85% of matches are good:                 Ok (90.04149627685546875% at 0.5 pixel accuracy)
559: Checking SiftFast implementation:
559: =================================
559: Found 441 points in reference image and 465 points in secondary image
559: Found 317 matches with 316 valid matches (tolerance of 0.5 pixels)
559: More than 65% of reference points have a match:    Ok (71.88208770751953125%)
559: More than 65% of secondary points have a match:    Ok (68.1720428466796875%)
559: More than 95% of matches are good:                 Ok (99.684539794921875% at 0.5 pixel accuracy)
559:  -> Test EXIT SUCCESS.
559: -------------  No control baseline tests    -------------
1/1 Test #559: feTvKeyPointsAlgorithmsTest ......   Passed    7.97 sec

This allowed me to remove:

• 10 test source code files
• 16 tests
• 19 baseline files from OTB-Data

Icing on the cake:

• Testing of those feature no longer relies on baselines comparisons,
• Number of test does no longer depends on OTB_USE_SIFTFAST.

Copyright

The copyright owner is CNES and has signed the ORFEO ToolBox Contributor License Agreement.

---

Check before merging:

• All discussions are resolved
• At least 2 votes from core developers, no vote.
• The feature branch is (reasonably) up-to-date with the base branch
• Dashboard is green
• Copyright owner has signed the ORFEO ToolBox Contributor License Agreement

Modules/Feature/Descriptors/test/CMakeLists.txt

@@ -22,54 +22,19 @@ otb_module_test()
 
 set(OTBDescriptorsTests
 otbDescriptorsTestDriver.cxx
-otbImageToSURFKeyPointSetFilterOutputDescriptorAscii.cxx
-otbImageToSIFTKeyPointSetFilterOutputInterestPointAscii.cxx
 otbHistogramOfOrientedGradientCovariantImageFunction.cxx
-otbImageToSURFKeyPointSetFilterOutputInterestPointAscii.cxx
-otbKeyPointSetsMatchingFilter.cxx
-otbImageToSIFTKeyPointSetFilterOutputDescriptorAscii.cxx
-otbImageToSIFTKeyPointSetFilterOutputAscii.cxx
 otbFourierMellinImageFilter.cxx
 otbImageToHessianDeterminantImageFilter.cxx
-otbImageToSIFTKeyPointSetFilterOutputImage.cxx
 otbFourierMellinDescriptors.cxx
-otbImageToSIFTKeyPointSetFilterDistanceMap.cxx
+otbKeyPointsAlgorithmsTest.cxx
 )
 
-if(OTB_USE_SIFTFAST)
-  list(APPEND OTBDescriptorsTests
-       otbImageToFastSIFTKeyPointSetFilterOutputInterestPointAscii.cxx
-       otbImageToFastSIFTKeyPointSetFilterOutputDescriptorAscii.cxx
-       )
-endif()
-
 add_executable(otbDescriptorsTestDriver ${OTBDescriptorsTests})
 target_link_libraries(otbDescriptorsTestDriver ${OTBDescriptors-Test_LIBRARIES})
 otb_module_target_label(otbDescriptorsTestDriver)
 
 # Tests Declaration
 
-otb_add_test(NAME feTvImageToSURFKeyPointSetFilterSceneOutputDescriptorAscii COMMAND otbDescriptorsTestDriver
-  --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvImageToSURFKeyPointSetFilterSceneKeysOutputDescriptor.txt
-  ${TEMP}/feTvImageToSURFKeyPointSetFilterSceneKeysOutputDescriptor.txt
-  otbImageToSURFKeyPointSetFilterOutputDescriptorAscii
-  ${INPUTDATA}/scene.png
-  ${TEMP}/feTvImageToSURFKeyPointSetFilterSceneKeysOutputDescriptor.txt
-  3 3
-  )
-
-
-otb_add_test(NAME feTvImageToSIFTKeyPointSetFilterSceneOutputInterestPointAscii COMMAND otbDescriptorsTestDriver
-  --ignore-order --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvImageToSIFTKeyPointSetFilterSceneKeysOutputInterestPoint.txt
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSceneKeysOutputInterestPoint.txt
-  otbImageToSIFTKeyPointSetFilterOutputInterestPointAscii
-  ${INPUTDATA}/scene.png
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSceneKeysOutputInterestPoint.txt
-  6 3 0.08 10.0
-  )
-
 otb_add_test(NAME feTvHistogramOfOrientedGradientCovariantImageFunction COMMAND otbDescriptorsTestDriver
   --compare-ascii ${EPSILON_8}
   ${BASELINE_FILES}/feTvHistogramOfOrientedGradientCovariantImageFunction.txt
@@ -80,94 +45,7 @@ otb_add_test(NAME feTvHistogramOfOrientedGradientCovariantImageFunction COMMAND
   5 273 64
   )
 
-otb_add_test(NAME feTvImageToSURFKeyPointSetFilterSceneOutputInterestPointAscii COMMAND otbDescriptorsTestDriver
-  --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvImageToSURFKeyPointSetFilterSceneKeysOutputInterestPoint.txt
-  ${TEMP}/feTvImageToSURFKeyPointSetFilterSceneKeysOutputInterestPoint.txt
-  otbImageToSURFKeyPointSetFilterOutputInterestPointAscii
-  ${INPUTDATA}/scene.png
-  ${TEMP}/feTvImageToSURFKeyPointSetFilterSceneKeysOutputInterestPoint.txt
-  3 3
-  )
-
-otb_add_test(NAME feTvKeyPointSetsMatchingFilter COMMAND otbDescriptorsTestDriver
-  --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvKeyPointSetsMatchingFilterOutputAscii.txt
-  ${TEMP}/feTvKeyPointSetsMatchingFilterOutputAscii.txt
-  otbKeyPointSetsMatchingFilter
-  ${TEMP}/feTvKeyPointSetsMatchingFilterOutputAscii.txt
-  0.6 0
-  )
-
-otb_add_test(NAME feTvImageToSIFTKeyPointSetFilterSceneDescriptorAscii COMMAND otbDescriptorsTestDriver
-  --ignore-order --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvImageToSIFTKeyPointSetFilterSceneKeysOutputDescriptor.txt
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSceneKeysOutputDescriptor.txt
-  otbImageToSIFTKeyPointSetFilterOutputDescriptorAscii
-  ${INPUTDATA}/scene.png
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSceneKeysOutputDescriptor.txt
-  6 3 0.08 10.0
-  )
 
-otb_add_test(NAME feTvImageToSIFTKeyPointSetFilterGridOutputAscii COMMAND otbDescriptorsTestDriver
-  --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvImageToSIFTKeyPointSetFilterGridKeysOutput.txt
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterGridKeysOutput.txt
-  otbImageToSIFTKeyPointSetFilterOutputAscii
-  ${INPUTDATA}/damier.png
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterGridKeysOutput.txt
-  7 3 0.01 10.0
-  )
-
-otb_add_test(NAME feTvImageToSIFTKeyPointSetFilterSquareRotatedOutputAscii COMMAND otbDescriptorsTestDriver
-  --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvImageToSIFTKeyPointSetFilterSquareRotatedKeysOutput.txt
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSquareRotatedKeysOutput.txt
-  otbImageToSIFTKeyPointSetFilterOutputAscii
-  ${INPUTDATA}/carre_ori.png
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSquareRotatedKeysOutput.txt
-  7 3 0.01 10.0
-  )
-
-otb_add_test(NAME feTvImageToSIFTKeyPointSetFilterSquare2OutputAscii COMMAND otbDescriptorsTestDriver
-  --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvImageToSIFTKeyPointSetFilterSquareKeysOutput2.txt
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSquareKeysOutput2.txt
-  otbImageToSIFTKeyPointSetFilterOutputAscii
-  ${INPUTDATA}/carre.png
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSquareKeysOutput2.txt
-  1 3 0.2 0.9
-  )
-
-otb_add_test(NAME feTvImageToSIFTKeyPointSetFilterSquareRotated2OutputAscii COMMAND otbDescriptorsTestDriver
-  --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvImageToSIFTKeyPointSetFilterSquareRotatedKeysOutput2.txt
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSquareRotatedKeysOutput2.txt
-  otbImageToSIFTKeyPointSetFilterOutputAscii
-  ${INPUTDATA}/carre_ori.png
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSquareRotatedKeysOutput2.txt
-  1 3 0.215 10.0
-  )
-
-otb_add_test(NAME feTvImageToSIFTKeyPointSetFilterSquareOutputAscii COMMAND otbDescriptorsTestDriver
-  --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvImageToSIFTKeyPointSetFilterSquareKeysOutput.txt
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSquareKeysOutput.txt
-  otbImageToSIFTKeyPointSetFilterOutputAscii
-  ${INPUTDATA}/carre.png
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSquareKeysOutput.txt
-  7 3 0.01 10.0
-  )
-
-otb_add_test(NAME feTvImageToSIFTKeyPointSetFilterQB_SuburbOutputAscii COMMAND otbDescriptorsTestDriver
-  --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvImageToSIFTKeyPointSetFilterQB_SuburbOutputAscii.txt
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterQB_SuburbOutputAscii.txt
-  otbImageToSIFTKeyPointSetFilterOutputAscii
-  ${INPUTDATA}/QB_Suburb.png
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterQB_SuburbOutputAscii.txt
-  2 3 5.0 0.0
-  )
 
 otb_add_test(NAME feTvForwardFourierMellinImageFilter COMMAND otbDescriptorsTestDriver
   --compare-n-images ${EPSILON_6} 2
@@ -190,26 +68,6 @@ otb_add_test(NAME feTvImageToHessianDeterminantImageFilter COMMAND otbDescriptor
   1.5
   )
 
-otb_add_test(NAME feTvImageToSIFTKeyPointSetFilterSquareOutputImage COMMAND otbDescriptorsTestDriver
-  --compare-image ${EPSILON_8}
-  ${BASELINE}/feTvImageToSIFTKeyPointSetFilterSquareImageOutput.png
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSquareImageOutput.png
-  otbImageToSIFTKeyPointSetFilterOutputImage
-  ${INPUTDATA}/carre.png
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterSquareImageOutput.png
-  7 3 0.01 10.0
-  )
-
-otb_add_test(NAME feTvImageToSIFTKeyPointSetFilterGridOutputImage COMMAND otbDescriptorsTestDriver
-  --compare-image ${EPSILON_8}
-  ${BASELINE}/feTvImageToSIFTKeyPointSetFilterGridImageOutput.png
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterGridImageOutput.png
-  otbImageToSIFTKeyPointSetFilterOutputImage
-  ${INPUTDATA}/damier.png
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterGridImageOutput.png
-  7 3 0.01 10.0
-  )
-
 otb_add_test(NAME feTvFourierMellinDescriptorsRotationInvariant COMMAND otbDescriptorsTestDriver
   otbFourierMellinDescriptorsRotationInvariant
   ${INPUTDATA}/poupees.png
@@ -237,41 +95,7 @@ otb_add_test(NAME feTvFourierMellinDescriptors COMMAND otbDescriptorsTestDriver
   ${TEMP}/feTvFourierMellinDescriptors.txt
   )
 
-otb_add_test(NAME feTvImageToSIFTKeyPointSetFilterDistanceMap COMMAND otbDescriptorsTestDriver
-  --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvImageToSIFTKeyPointSetFilterDistanceMap.txt
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterDistanceMap.txt
-  --ignore-lines-with 2 INFO DEBUG
-  otbImageToSIFTKeyPointSetFilterDistanceMap
-  ${INPUTDATA}/scene.png
-  6 3 0.08 10.0
-  15.0 # rotation
-  1.2 # zoom factor
-  10 255
-  ${TEMP}/feTvImageToSIFTKeyPointSetFilterDistanceMap.txt
-  )
-
-if(OTB_USE_SIFTFAST)
-otb_add_test(NAME feTvImageToFastSIFTKeyPointSetFilterSceneOutputInterestPointAscii COMMAND otbDescriptorsTestDriver
-  --ignore-order --compare-ascii ${EPSILON_3}
-  ${BASELINE_FILES}/feTvImageToFastSIFTKeyPointSetFilterSceneKeysOutputInterestPoint.txt
-  ${TEMP}/feTvImageToFastSIFTKeyPointSetFilterSceneKeysOutputInterestPoint.txt
-  otbImageToFastSIFTKeyPointSetFilterOutputInterestPointAscii
-  ${INPUTDATA}/ROI_IKO_PAN_LesHalles_sub.tif
-  ${TEMP}/feTvImageToFastSIFTKeyPointSetFilterSceneKeysOutputInterestPoint.txt
-  6
-  )
-
-# RK: 06/2016. the root cause of this test having different output on platforms comes from libsiftfast (3rd party code)
-# Until there is a fix, that failure cannot be attributed to OTB or dashboard results
-#  --ignore-order --epsilon-boundary 0.01 --compare-ascii ${EPSILON_2}
-#  ${BASELINE_FILES}/feTvImageToFastSIFTKeyPointSetFilterSceneKeysOutputDescriptor.txt
-#  ${TEMP}/feTvImageToFastSIFTKeyPointSetFilterSceneKeysOutputDescriptor.txt
-
-otb_add_test(NAME feTvImageToFastSIFTKeyPointSetFilterSceneOutputDescriptorAscii COMMAND otbDescriptorsTestDriver
-  otbImageToFastSIFTKeyPointSetFilterOutputDescriptorAscii
-  ${INPUTDATA}/ROI_IKO_PAN_LesHalles_sub.tif
-  ${TEMP}/feTvImageToFastSIFTKeyPointSetFilterSceneKeysOutputDescriptor.txt
-  6
-  )
-endif()
+otb_add_test(NAME feTvKeyPointsAlgorithmsTest COMMAND otbDescriptorsTestDriver
+  otbKeyPointsAlgorithmsTest
+  ${INPUTDATA}/QB_TOULOUSE_RpcTag_100_100.tif
+)