Dimensionality reduction algorithms

Summary

This merge request contains new algorithms for Dimensionality Reduction.

Rationale

This is the integration of a remote module made by Cédric Traizet.

Implementation Details

Classes and files

M       CMake/FindShark.cmake
D       Modules/Learning/LearningBase/include/otbSharkUtils.h
A       Modules/ThirdParty/Shark/include/otbSharkUtils.h
M       Modules/ThirdParty/Shark/otb-module-init.cmake

The Shark module has been enhanced to detect if Shark was built with OpenMP, CBLAS, DYNLIB. And the file SharkUtils has been moved to Shark module where it belongs.

M       Modules/Learning/Supervised/src/CMakeLists.txt
D       Modules/Learning/Supervised/src/otbMachineLearningModelFactoryBase.cxx
A       Modules/Learning/LearningBase/src/CMakeLists.txt
A       Modules/Learning/LearningBase/src/otbMachineLearningModelFactoryBase.cxx
M       Modules/Learning/LearningBase/include/otbMachineLearningModelFactoryBase.h
M       Modules/Learning/LearningBase/CMakeLists.txt
M       Modules/Learning/LearningBase/otb-module.cmake

The mutex from MachineLearningModelFactoryBase was in OTBSupervised module whereas the class is declared in OTBLearningBase (don't know why). I moved it at the right place. Also, the dependencies of OTBLearningBase are now much cleaner.

M       Modules/Learning/LearningBase/include/otbMachineLearningModel.h
M       Modules/Learning/LearningBase/include/otbMachineLearningModel.txx

Key point of this merge request: the base class MachineLearningModel now has a dimension setting (allows to control the number of output dimensions with dimensionality reduction models).

A       Modules/Learning/DimensionalityReductionLearning/CMakeLists.txt
A       Modules/Learning/DimensionalityReductionLearning/README.md
A       Modules/Learning/DimensionalityReductionLearning/include/otbAutoencoderModel.h
A       Modules/Learning/DimensionalityReductionLearning/include/otbAutoencoderModel.txx
A       Modules/Learning/DimensionalityReductionLearning/include/otbAutoencoderModelFactory.h
A       Modules/Learning/DimensionalityReductionLearning/include/otbAutoencoderModelFactory.txx
A       Modules/Learning/DimensionalityReductionLearning/include/otbDimensionalityReductionModelFactory.h
A       Modules/Learning/DimensionalityReductionLearning/include/otbDimensionalityReductionModelFactory.txx
A       Modules/Learning/DimensionalityReductionLearning/include/otbImageDimensionalityReductionFilter.h
A       Modules/Learning/DimensionalityReductionLearning/include/otbImageDimensionalityReductionFilter.txx
A       Modules/Learning/DimensionalityReductionLearning/include/otbPCAModel.h
A       Modules/Learning/DimensionalityReductionLearning/include/otbPCAModel.txx
A       Modules/Learning/DimensionalityReductionLearning/include/otbPCAModelFactory.h
A       Modules/Learning/DimensionalityReductionLearning/include/otbPCAModelFactory.txx
A       Modules/Learning/DimensionalityReductionLearning/include/otbSOMModel.h
A       Modules/Learning/DimensionalityReductionLearning/include/otbSOMModel.txx
A       Modules/Learning/DimensionalityReductionLearning/include/otbSOMModelFactory.h
A       Modules/Learning/DimensionalityReductionLearning/include/otbSOMModelFactory.txx
A       Modules/Learning/DimensionalityReductionLearning/otb-module.cmake

New module with dimensionality reduction models. They are designed to fit in the existing machine learning framework. We can train these models, and then compute the prediction (i.e. the variable with reduced dimensions). Among the new models, there are:

• Autoencoders (from Shark)
• PCA (from Shark)
• Self Organizing Maps (from OTB)

Applications

M       Modules/Applications/AppDimensionalityReduction/app/CMakeLists.txt
A       Modules/Applications/AppDimensionalityReduction/app/otbImageDimensionalityReduction.cxx
A       Modules/Applications/AppDimensionalityReduction/app/otbTrainDimensionalityReduction.cxx
A       Modules/Applications/AppDimensionalityReduction/app/otbVectorDimensionalityReduction.cxx
A       Modules/Applications/AppDimensionalityReduction/include/otbDimensionalityReductionTrainAutoencoder.txx
A       Modules/Applications/AppDimensionalityReduction/include/otbDimensionalityReductionTrainPCA.txx
A       Modules/Applications/AppDimensionalityReduction/include/otbDimensionalityReductionTrainSOM.txx
A       Modules/Applications/AppDimensionalityReduction/include/otbTrainDimensionalityReductionApplicationBase.h
A       Modules/Applications/AppDimensionalityReduction/include/otbTrainDimensionalityReductionApplicationBase.txx
M       Modules/Applications/AppDimensionalityReduction/otb-module.cmake

New applications added to use the DR models:

• One application for training (TrainDimensionalityReduction)
• Two applications for prediction (ImageDimensionalityReduction and VectorDimensionalityReduction)

Tests

A       Modules/IO/TestKernel/include/otbReadDataFile.h
M       Modules/Learning/Supervised/test/otbTrainMachineLearningModel.cxx
M       Modules/Learning/Supervised/test/tests-libsvm.cmake
M       Modules/Learning/Supervised/test/tests-opencv.cmake
M       Modules/Learning/Supervised/test/tests-shark.cmake

There was 3 duplicate functions to parse the data file letter.scale. Now there is only one, in the OTBTestKernel module. A lighter version of this data file has been added for faster tests.

A       Modules/Learning/DimensionalityReductionLearning/test/CMakeLists.txt
A       Modules/Learning/DimensionalityReductionLearning/test/otbAutoencoderModelTest.cxx
A       Modules/Learning/DimensionalityReductionLearning/test/otbDimensionalityReductionLearningTestDriver.cxx
A       Modules/Learning/DimensionalityReductionLearning/test/otbImageDimensionalityReductionFilterTest.cxx
A       Modules/Learning/DimensionalityReductionLearning/test/otbPCAModelTest.cxx
A       Modules/Learning/DimensionalityReductionLearning/test/otbSOMModelTest.cxx
M       Modules/Applications/AppDimensionalityReduction/test/CMakeLists.txt

Tests have been added, at DR model level, and also at application level. They also use letter_light.scale.

Documentation

None

Additional notes

None