Render examples in the CookBook

Summary

The next step in the Software Guide migration to the rst CookBook: render examples.

• New CookBook section: C++
• Render examples in the CookBook section C++/Examples
• Rewrite two examples with the new format (see below)
  • Examples/BasicFilters/LeeImageFilter.rst
  • Examples/FeatureExtraction/TextureExample.rst
• clang-format all examples & cleanup old tags

How to write a "new style" example

• Put the .cxx file in Examples/tag/YourExample.cxx
• Use normal C++14 style (no more BeginLatex/BeginCmdLineArgs/etc)
• At the top of the .cxx file there must be a "usage" comment with the following syntax:

/* Example usage:
./TextureExample Input/ADS40RoiSmall.png Output/TextureOutput.tif Output/pretty_TextureOutput.png 2 1 1
*/

• (Optional) Write a description in Examples/tag/YourExample.rst. You may include figures from otb-data/Input or otb-data/Output (e.g. the output of your example) with the "absolute path" syntax, for example:

.. figure:: /Output/pretty_TextureOutput.png

    Result of applying the ScalarImageToTexturesFilter to an image (contrast).

• Run your example and add the figures needed for the description in otb-data/Output.

How to run examples

Contrary to the software guide, examples are not run when building the CookBook. They can be run manually using a new script: Documentation/Cookbook/Scripts/RunExamples.py. You can run all examples or only one. For example, in an OTB build directory do:

RunExamples.py /path/to/otb /path/to/otb-data

or:

RunExamples.py /path/to/otb /path/to/otb-data --name TextureExample

TODO in future work:

• Update all examples from old latex format to new rst format
  • Remove \doxygen and \code macros
  • Replace \includegraphics{} directives with .. figure:: filename.png in the .rst file
  • Add needed .png figures to otb-data/Output
• Remove examples tests
• Remove examples tests baselines
• Add RunExamples.py script in a new test
• Fix missing usages
• Improve RunExamples.py to be able to run in another working directory than otb-data
• Modernize examples to C++14
• Continue migrating Software Guide sections that are not examples
• Improve license header filtering in rst render (itk copyright)
• Handle ApplicationExample
• Move Documentation/Cookbook/Art/ to Data/ after the LFS migration is complete

Rationale

#1679

Implementation Details

Most of the implementation is in the new scripts Documentation/Cookbook/Scripts/otbGenerateWrappersRstDoc.py and Documentation/Cookbook/Scripts/RunExamples.py.

The "usage" command is extracted from the cxx source code with this regex: \/\* Example usage:\n(\.\/[a-zA-Z]+ (.*?))\*\/.

otb-data

Changes in otb-data (otb-data!16 (merged)) are adding the examples outputs that are needed for the examples:

A       Output/GomaSmallLeeFiltered.png
A       Output/pretty_TextureOutput.png

As we improve the old examples or add new ones, figures can be added in Documentation/Cookbook/Art/ if they are static, or otb-data/Output if they are the output of an example.

Copyright

The copyright owner is CNES and has signed the ORFEO ToolBox Contributor License Agreement.

---

Check before merging:

• All discussions are resolved
• At least 2 votes from core developers, no vote.
• The feature branch is (reasonably) up-to-date with the base branch
• Dashboard is green
• Copyright owner has signed the ORFEO ToolBox Contributor License Agreement

added documentation refactoring labels

mentioned in merge request otb-data!16 (merged)

changed the description

Use normal C++14 style (no more BeginLatex/BeginCmdLineArgs/etc)

Does that mean that it is not possible to split an example into multiple parts inserting text between them as it is was done in the SG?

Correct. The idea is to have "standalone" examples: one single page with the input/output figures, a description and the complete code. We feel like this is much better for readability and maintenance than the current software guide system for which we see many shortcomings. If you want to review the final html output you can build the cookbook on this branch (and otb-data branch) with:

cmake -DBUILD_COOKBOOK=ON
make CookBookHTML
firefox Documentation/Cookbook/html/index.html &

I can also upload the html somewhere if you want to review it but can't manage the build. Feedback is welcome!

If you can upload an exampe somewhere, I would be interested in having a look. It may be a pity loosing the "literate programming" aspect of the SG, but if "you" feel this is old school, I am fine with that.

• Example with LeeImageFilter (migration complete):

Develop: https://www.orfeo-toolbox.org/packages/nightly/latest/SoftwareGuide-6.7/SoftwareGuidech6.html#x17-570006.2.4

This branch: https://vpoughon.github.io/otb_cookbook_examples/C++/Examples/BasicFilters/LeeImageFilter.html

• Example with AtmosphericCorrectionSequencement (migration not done yet, so you get the default render without figures for now):

Develop: https://www.orfeo-toolbox.org/packages/nightly/latest/SoftwareGuide-6.7/SoftwareGuidech9.html#x20-790009.2

This branch: https://vpoughon.github.io/otb_cookbook_examples/C++/Examples/Radiometry/AtmosphericCorrectionSequencement.html

I understand that it's a trade off. With this migration we lose:

• Litterate programming (embded latex equations/figures within the code). They now need to be in a separate .rst file.
• Automatic links to the doxygen class documentation with the \doxygen latex macro (might be possible with a sphinx plugin, I need to look into it) (EDIT: I found a solution for this)

But we gain:

• More web friendly documentation: better rendering, better navigation
• A single documentation (eventually no more sg)
• Complete examples: you can copy paste the code and it builds (not the case with current SG)
• Readable examples source code
• Less risk of software guide breaking (see #1794 (closed))
• Simpler build system

OK. Thanks for sharing. I am not really convinced by the separation of code and test, but I am not against either, since I don't think many people use the C++ examples anymore.

If this is easier to maintain, I agree with the trade off.

I do not understand why moving those example from the SG to the Cookbook.
What kind of user are your targeting?
In my opinion CookBook readers are end users: meaning that they do not care about C++ example. Whereas SG readers are people that might want to know how to right stuff in C++. If your idea is to get rid of the SG as its actual form I agree. If you just want to move the examples from the SG to the CB I think it is not a good idea.

PS: it looks good though!

Well this is only the first step in the migration. The plan is to drop the SG entirely. After this MR, what's left in the SG is only pure latex which will be migrated with a simple pandoc + manual review to rst pages in the new "C++" section of the CookBook.

IMHO The current software guide is an absolute mess: out of date examples and concept, broken figure, 700+ pages essentially demonstrating how to use a reader and a writer, mixed up with remote sensing courses or state of the art sections that also needs maintenance, plus the tenth of examples copied from ITK that they do not even have in their software guide anymore. I think it offers a really poor view of OTB C++ API for the interested reader.

As @jinglada stated, SG has a really low audience, we could simply drop it ... Or make a C++ doc that is actually useful to someone trying to learn OTB C++ side.

Having a single documentation is what most projects do, and it is already a lot for us to maintain one guide. @poughov simplification work will allow to have a C++ section in cookbook explaining all the essential concepts, plus an easy to maintain examples galery. We could then work on referencing those example in doxygen (which is a whole story on its own!).

No idea why CI is failing but dashboard is green

dashboard is green, needs review and votes

changed the description

added 7 commits

• b756dc4f - REFAC: review TrainMachineLearningModelFromSamplesExample
• 89c6f3ff - REFAC: cookbook examples: support multiple usage
• 85b5c78c - REFAC: blacklist ApplicationExample from cookbook for now
• b6a7a3f5 - REFAC: fix HyperspectralUnmixingExample usage
• ebda2d5e - REFAC: fix ProsailModel example
• e3c410a2 - REFAC: partial fix for LAIAndPROSAILToSensorResponse example
• 419c1861 - REFAC: blacklist LAIAndPROSAILToSensorResponse example for now

Compare with previous version

I pushed a few commits todays that fix some examples usage and support multiple usage to be compatible with all existing code. Now all examples can be run with:

RunExamples.py ~/cnes/dev/otb ~/cnes/dev/otb-data

the following examples do not have a usage and should be reviewed in future work:

Examples/Classification/ClassificationMapRegularizationExample.cxx
Examples/Classification/SupervisedImageClassificationExample.cxx
Examples/Classification/DempsterShaferFusionOfClassificationMapsExample.cxx
Examples/Classification/SOMImageClassificationExample.cxx
Examples/IO/ImageReadCastWrite.cxx
Examples/IO/ImageReadWrite.cxx
Examples/IO/MultibandImageReadWrite.cxx
Examples/IO/VectorDataIOExample.cxx
Examples/IO/MetadataExample.cxx
Examples/IO/ImageReadRegionOfInterestWrite.cxx
Examples/IO/ComplexImageReadWrite.cxx
Examples/IO/ImageSeriesIOExample.cxx
Examples/Image/Image3.cxx
Examples/Image/Image2.cxx
Examples/Image/Image1.cxx
Examples/Image/Image4.cxx
Examples/Image/ImageListExample.cxx
Examples/Image/VectorImage.cxx
Examples/Image/Image5.cxx
Examples/Installation/HelloWorld.cxx
Examples/Projections/GeometriesProjectionExample.cxx
Examples/Projections/EstimateRPCSensorModelExample.cxx
Examples/Projections/OrthoRectificationExample.cxx
Examples/Projections/PlaceNameToLonLatExample.cxx
Examples/Projections/VectorDataExtractROIExample.cxx
Examples/Projections/VectorDataProjectionExample.cxx
Examples/Tutorials/HelloWorldOTB.cxx
Examples/Tutorials/OrthoFusion.cxx
Examples/Filtering/CompositeFilterExample.cxx
Examples/Filtering/SecondDerivativeRecursiveGaussianImageFilter.cxx
Examples/Iterators/ShapedNeighborhoodIterators1.cxx
Examples/Iterators/ImageLinearIteratorWithIndex2.cxx
Examples/Iterators/ImageRandomConstIteratorWithIndex.cxx
Examples/Iterators/NeighborhoodIterators5.cxx
Examples/Iterators/NeighborhoodIterators3.cxx
Examples/Iterators/ImageLinearIteratorWithIndex.cxx
Examples/Iterators/NeighborhoodIterators2.cxx
Examples/Iterators/ImageSliceIteratorWithIndex.cxx
Examples/Iterators/IteratorsExamples.cxx

changed the description

added 1 commit

• 372b7bde - REFAC: fix ProsailModel example output

Compare with previous version

added 1 commit

• 97a036b7 - REFAC: sort cookbook examples & fix ApplicationExample

Compare with previous version