Remove OSSIM

What changes will be made and why they would make a better Orfeo ToolBox?

The main idea is to remove OSSIM from OTB, in order to reduce the dependency risk, and also to refactor some parts of the library.

High level description

OSSIM is used for geometric sensor modelling and metadata parsing. It has been used for that since the beginning of the OTB. Then adapter classes have been added, to hide OSSIM headers from OTB public API. Now, it is time to plan the removal of this dependency, whose development cycle is difficult to follow. In the current state, only a small portion of OSSIM is used anyway.

This work can be performed in several steps:

Risks and benefits

Risks:

Large refactoring, there is a substantial amount of work needed, mostly for the porting of geometric models.
Impacts on the validation tests, and baselines.
Several components are complex to migrate (DEMHandler, SarSensorModel,...)

Benefits:

4 mandatory dependencies will be removed from OTB: Ossim, OssimPlugins, GeoTiff, OpenThreads
2 modules will be removed: OTBOssimAdapters and OTBOpenThreadsAdapters
Sanitize the code base
Easier support of new sensors models
Better metadata architecture (which allows better metadata processing in our pipelines)
Separation between metadata parsing, and geometric modelling.

Alternatives for implementations

Here are some details and propositions regarding the implementation.

Unused classes

This is a list of classes that could be removed at the beginning of the refactoring without much trouble:

Module OTBOssimAdapters
- otbDEMConvertAdapter
- otbGeometricSarSensorModelAdapter
- otbPlatformPositionAdapter
Module OTBProjection
- old map projections: Eckert, Lambert3CartoSud, LambertConformalConic, Mollweid, SinusoidalMap, SVY21Map, TransMercator. We should rather have a set of WKT strings for the "common" map projections.
Module OTBAppImageUtils
- Application DEMConvert

Metadata parsing

Currently, metadata is parsed from each child class of ossimSensorModel. This metadata is then exported into an ImageKeywordlist. On the other hand, the otbXXXImageMetadataInterface is designed to expose this metadata in a generic way. It would make sense to move the metadata-parsing code from ossimSensorModels into otbXXXImageMetadataInterface ("IMI"), and have a single class that represents a sensor metadata.

I have been thinking about a more generic way to parse metadata. In the end, there are always corner cases where the metadata is read from some file and then transformed before being recorded into the ImageKeywordlist. I propose the following workflow to replace the current function ReadGeometryFromImage():

We use a classic IMIFactory to find if a given IMI (associated to a given sensor) can read a product. For a given input image filename, each IMI can return the list of associated metadata files and their type (DIMAP, SAFE, txt file, ... ). Note that the image itself can also be part of the list if there are metadata embedded in the image.
Reading the metadata files. The purpose of this step is to parse each metadata file associated with the image file and supply it as a (in-memory) XML tree. This tree is given to a IMI that will look for needed information. The parsing can be done by different classes, so far we expect a plain XMLReader, a TextMetadataReader and a GDALMetadataReader. They can all derive from a base class. Depending on the format:
- XMLReader can be simply implemented using TinyXML
- TextMetadataReader will try to parse 'key=value' pairs and format it in a XML tree.
- GDALMetadataReader will use GDALDataset::GetMetadata() to extract 'key=value' pairs and format them into a XML tree.
Parsing in IMI. This step consists in finding the relevant metadata in the different XML trees and mapping them into a KeywordList (map<string,string>) with the usual metadata keys (used in geom files). At this step, we keep the metadata as strings, but we may also do specific conversions to check the numerical range. This parsing can be nicely written with generic functions like add() used in ossimSentinel1Model.cpp. If the parsing returns successfully, the generated ImageKeywordlist is given to the input image metadata dictionary.
Instanciation of a Transform. I propose to separate the classes used to store and access the product metadata, and the classes used to represent a transform associated with a sensor model. The truth is that among all the sensor models currently handled by Ossim, we mostly use RPC based models. There are few physical sensor models, but we could implement them with a generic "PushBroomSensorModel". Maybe complex physical models will actually deserve a separate implementation. Regarding SAR sensors, we now tend to rely on SarSensorModel which is generic. This separation would also be consistent if we want to handle both physical and RPC models associated with a given sensor (no need to clone the metadata class). An other good point is that pipelines that don't do any geometry or projection stuff will not have to instanciate and configure a sensor transform that they won't use anyway. The SensorTransform will be initialized from the ImageKeywordList obtained in step 3.

The following diagram sums up the different steps.

In the case of a single image file with a geom, the steps 1, 2 and 3 are not needed, the geom is directly injected into the ImageKeywordlist.

In a nutshell, the IMI classes will be in charge of identifying the correct sensor and parsing all the relevant metadata. The SensorTransform classes will operate various geometric transforms, configured from an ImageKeywordList.

The role of IMI classes can also be extended, for instance give the list of metadata keys that carry per-band values. A concatenate file will then be able to gather these metadata together.

Note from RKH: use RapidXML instead of TinyXML.

Geometric Models

We move the code related to geometric transforms into several generic SensorTransforms. For now we can expect:

a RPCSensorTransform (maybe based on GDALRPCTransform())
a SarSensorTransform (a port of ossimSarSensorModel)
(optional) a GeometricSarSensorTransform ? (for deprecated SAR models)
(optional) a PushBroomSensorTransform ? (for Spot5)
(optional) a MatrixSensorTransform ? (for airborne sensors)

For geometric transform, all the structures like ossimDpt, ossimGpt should be replaced with ITK classes. Ideally, the new SensorTransform could derive from otb::SensorModelBase, but the current classes SensorModelBase, ForwardSensorModel and InverseSensorModel are not really usefull. The difference between forward and inverse transform could very well be implemented with a template specialization on SensorModelBase. I propose to remove ForwardSensorModel and InverseSensorModel, refactor SensorModelBase, and derive new SensorTransforms from SensorModelBase.

Like with ossimSensorModel, we can implement tuning parameters in otb::SensorModelBase to reproduce the ossimAdjustableParameterInterface. The adjustment parameters are often used to configure a residual affine transform that is composed with the geometric model.

DEMHandler

Role:

Open a DEM directory (with SRTM tiles, single GeoTIFF file, ...)
Open a geoid file (*.egm)
Handle a default elevation setting
Provide the elevation at any coordinates (lon/lat)

Actions: there is nothing really equivalent in GDAL. If we go for a custom development, we could also plan a different use of this DEM:

Point-based: this is the current implementation. We query the DEMHandler for an elevation at each position independently. This implementation requires some caching to be efficient (done by Ossim, can also be done by GDAL).
Grid-based: this is a new strategy that could be implemented. For instance, during an orthorectification process, a deformation grid is computed, using calls to the DEMHandler at each point. It would be more efficient to extract the elevation values block by block. In this case, we can also foresee some relevant features for DEM processing (like no-data filling, reprojection, smoothing,...).

With a grid-based approach, the DEMHandler would rather return an image pointer (probably the output of a pipeline) than a single float. After a quick review of the calls to itk::Transform::TransformPoint(), it seems difficult to avoid the point-based service in the DEMHandler.

The best example is the computation of an ortho extent. This requires (at least) 4 forward transforms for each image corner. Assuming a RPC model, this sensor-to-ground transform uses an iterative approach, because the RPC gives the transform from ground to sensor. The DEMHandler will have to supply several elevations for each image corner and the (x,y) coordinates of theses elevations can't be easily predicted. Also, since the corners may represent a large extent on the ground, it is risky to assume that the whole region can be buffered.

Then, there is also the question of the input to the DEMHandler. At the moment, we give a directory and an (optional) path to a geoid file. Giving a path to a DEM file could be easier for the users, but keeping the existing behaviour would ensure nothing is broken on user side. Since a DEM is often composed of several tiles, the VRT solution seems the best to gather the tiles in a single entry point. We can even create it in-memory to deal with backward compatibility.

The solution proposed is to work with 3 classes:

DEMHandler: part of OTBTransform, singleton, holds the defaults settings, and an internal DEM reader
DEMReaderInterface: part of OTBTransform, defines the interface to request point-based and grid-based elevation
GDALDEMReader: part of OTBImageIO, implements the DEMReaderInterface, with point-based and grid-based services as a standard image reader.

This is the proposed skeleton for the future DEMHandler class:

  class DEMHandler : public itk::Object
  {
  public:
    /** Retrieve the singleton instance */
    static Pointer Instance();
    
    /** Default DEM settings  */
    string GetDEMPath();
    void   SetDEMPath(string);
    
    string GetGeoidPath();
    void   SetGeoidPath(string);
    
    double GetDefaultHeight();
    void   SetDefaultHeight();
    
    void SetReader(reader);
    DEMReaderInterface* GetReader();

  private:
    static Pointer m_Singleton;
    
    string m_Geoid;
    string m_DEM;
    double m_DefaultHeight;
    
    /** internal DEMReaderInterface */
    DEMReaderInterface::Pointer m_Reader;
  };

The skeleton of DEMReaderInterface will look like:

  class DEMReaderInterface
  {
  public:
    /** Point-based elevation */
    virtual double GetHeight(double x, double y);
    
    /** Grid-based elevation */
    virtual otb::Image<double,2>* GetHeightGrid();
    
    /** setup function */
    void SetSource(dem, geoid, defaultHeight);
  };

The GDALDEMReader class will be a composite filter, containing the following boxes:

[optional] DirectoryToVRT: small tool used if the DEM path is a directory. It will:
- find the first VRT file in that directory
- or create an in-memory VRT using image present in the directory
ImageFileReader: actually reads an input VRT (we may also use GDALImageIO directly)
DEMResampler: can be used to support interpolation, resampling the DEM to a different grid and a different SRS.
[optional] GeoidReader: reads the input geoid file, this will likely be an ImageFileReader, it may use the full buffer if the size allows it.
[optional] AddImageFilter: combine DEM and geoid heights

This reader will be used during the point-based calls: a small patch of DEM will be extracted around the requested location. During grid-based calls, it will return an otb::Image connected to a pipeline.

The DEMResampler can be really simple at the start. The only mandatory feature to be implemented is the interpolation on a different grid (using the same SRS). Indeed, this is how the point-based method GetHeight() will interpolate the correct elevation. Later on, we can improve it to support:

transformation to a different SRS
gap-filling
smoothing
...

It could be a nice feature in the long term.

We will probably need to modify the GDAL_CACHE_SIZE variable for better performances. Also, this cache size should be subtracted from our RAM parameter.

Who will be developing the proposed changes?

TBD

Edited Oct 07, 2021 by Julien Osman