GitLab

This guide documents the changes required to migrate a code base which uses OTB v7 to use OTB v8.

• Context
  • How OSSIM was used until OTB v7
  • How we replaced OSSIM in OTB v8
• The new ImageMetadata object
  • Presentation
  • Migration example
• Metadata parsing
  • Presentation
• DEM Handler
  • Presentation
  • Migration example
• The new Sensor Model mechanism
  • Presentation
    • RPC sensor model
    • SAR sensor model
  • Migration example

Context

OSSIM is used for geometric sensor modelling and metadata parsing. It has been a dependency of the OTB since it's beginning. Then adapter classes have been added, to hide OSSIM headers from OTB public API. With the version 8 of OTB, it is time to remove this dependency, whose development cycle is difficult to follow. Until OTB v7, only a small portion of OSSIM was used anyway.

How OSSIM was used until OTB v7

OSSIM comes with a complete framework to read and write metadata from image files. It is able to read the metadata and store them in an object callel KeywordList. The OTB is then in charge of channeling this object through the processing pipeline so every filter can access them. When one needs to output an image file, OSSIM takes the KeywordList object and write it's content to the file. OSSIM use the GEOM file format for the metadata.

OSSIM also comes with a framework for sensor modelling. It includes a factory that instantiate the correct model according the to content of the metadata KeywordList. It is the able to perform forward and invers transform. Depending on the sensor, the model will be based on RPCs, or other sensor specific model.

How we replaced OSSIM in OTB v8

The two main work for this version 8 have been:

• A complete refactoring of the metadata management. We replaced OSSIM by GDAL to access the metadata in the image files, and we developed classes called ImageMetadataInterfaces (IMI) to parse the metadata and organize them in an ImageMetadata object.
• A re-implementation of the sensor model framework. We replaced OSSIM's RPC model by GDAL's. We implemented our own SAR model. A whole new SensorModel factory as developed.

A new DEM Handler was also developed, based on what GDAL proposes.

The new ImageMetadata object

Presentation

To replace OSSIM's MetadataDictionary, a new ImageMetadata object is introduced. It inherits from ImageMetadataBase.

classDiagram
  class ImageMetadataBase
  ImageMetadataBase : - NumericKeys
  ImageMetadataBase : - StringKeys
  ImageMetadataBase : - LUT1DKeys
  ImageMetadataBase : - LUT2DKeys
  ImageMetadataBase : - TimeKeys
  ImageMetadataBase : - GeometryKeys
  ImageMetadataBase : - ExtraKeys
  ImageMetadataBase : - [] operator(key)
  ImageMetadataBase : - Add(key, value)
  ImageMetadataBase : - Remove(key)
  ImageMetadataBase : - Has(key)
  ImageMetadataBase : - ToKeywordList(kwl)
  ImageMetadataBase : - FromKeywordList(kwl)
  ImageMetadataBase : - Fuse(imd)

  class ImageMetadata
  ImageMetadata : - Bands
  ImageMetadata : - slice(start, end)
  ImageMetadata : - append(imd)
  ImageMetadata : - compact()
  ImageMetadata : - Merge(imd)
  ImageMetadata : - AppendToKeywordLists(kwlv)
  ImageMetadata : - AppendToBandKeywordLists(kwlv)
  ImageMetadata : - FromKeywordLists(kwlv)

  ImageMetadataBase <|-- ImageMetadata
  ImageMetadata "1" *-- "0..*" ImageMetadataBase

ImageMetadataBase encapsulates seven std::map to store seven different kind of metadata:

• Numeric metadata for the metadata that can be stored as a double
• String metadata for the metadata that can be stored as a std::string
• LUT 1D metadata the metadata that can be stored as a one dimension table
• LUT 2D metadata for the metadata that can be stored as two dimensions table
• Time metadata for the metadata that can be stored as a time object
• Geometry metadata for the metadata that represent a model
• Extra metadata for non generic metadata stored as std::string

The keys of the maps are described in the file otbMetaDataKey. This file also defines the time object.

The ImageMetadataBase class also provides 4 methods:

• the [] operator for a read-only access the metadata from the key
• the Add method to set a metadata value
• the Remove method to delete a metadata value
• the Has method to test if a key has a value

The ImageMetadata class is used to store the metadata. It contains a std::vector Bands that contains one ImageMetadataBase for each band of the product. The metadata that are common to all the bands are stored in the ImageMetadata object itself. It also provides some useful methods:

• the slice method to access the metadata of a range of bands
• the append method to concatenate two ImageMetadata objects
• the compact method to put to the top level the metadata that are common to all the bands
• the Merge method to merge with another ImageMetadata

Migration example

Before, OSSIM worked with KeywordLists that transit through the pipeline with ITK's MetadataDictionary object.

ImageKeywordlist kwl = inputImg->GetImageKeywordlist();
if (kwl.GetSize() > 0)
  outputImg->SetImageKeywordlist(kwl);

After, those objects are not used anymore for the metadata, replaced by the ImageMetadata class.

ImageMetadata imd = inputImg->GetImageMetadata();
if (imd != nullptr)
  outputImg->SetImageMetadata(imd);

One can simply access the data in the ImageMetadata using its public functions:

ImageMetadata imd = inputImg->GetImageMetadata();
if (imd != nullptr && imd.Has(MDNum::NumberOfLines))
{
  int nLines = imd[MDNum::NumberOfLines];
  imd[MDStr::BandName] = "NIR";
}

Metadata parsing

Presentation

There is a new workflow to replace the current function ReadGeometryFromImage():

• The GDAL input/output capabilities are encapsulated in the GDALImageIO class, which derivates from ImageIO. This class is in charge of fetching the metadata from the product (supplier capabilities inherited from the class MetadataSupplierInterface) and storing them in memory as a keywordlist. It is also in charge of writing the metadata to the product (storage capabilities inherited from the class MetadataStorageInterface).

• An ImageMetadataInterface (IMI) is then called to parse the metadata. There is one IMI per sensor. We use a classical Factory to find which one can parse the metadata of a product. The IMI's parse method will pick the metadata from the ImageIO and fill an ImageMetadata object.

• Some metadata are not read by GDAL. To parse those metadata, the IMI can call other suppliers, depending on the file format:

  • to parse XML files, XMLMetadataSupplier uses GDAL's XML parsing mechanism ("ReadXMLToList" method from the "GDALMDReaderBase" class) to convert the XML file into a GDAL ListString, which is a succession of 'key=value' pairs.
  • to parse text files, TextMetadataSupplier tries to parse 'key=value' pairs.

  Other suppliers can be added if needed. Those classes (including GDALImageIO) all implement the method GetMetadataValue which returns the value of the metadata from a given key. The base class also implements the methods GetAs and GetAsVector which are used by the IMI.

• The IMI finds the relevant metadata in the different Metadata Suppliers and use the Add() method of the ImageMetadata object to store the metadata. If the parsing returns successfully, the generated ImageMetadata is given to the ImageCommon that propagate through the pipeline.

DEM Handler

Presentation

Before the refactoring, otb::DEMHandler class was an adapter class for OSSIM DEMs. It provided an API to retrieve height from DEM and/or geoid using OSSIM, and was used to set up DEM and geoids to be used by OSSIM internally (which is used in OSSIM RPCs for example).

The otb::DEMHandler class is a singleton, a reference to the otb::DEMHandler object can be obtained via the GetInstance() method. This is a change of API, the Ossim otb::DEMHandler returned a ITK smart pointer to the singleton object. This was not a very good design, and a singleton object does not need to use the ITK memory management system.

The new approach is based on RasterIO from GDAL. A 2x2 window is extracted around the point of interest, and the height, above ellipsoid or above mean sea level. The following methods are provided:

• GetHeightAboveEllipsoid(lon, lat):
  • SRTM and geoid both available: dem_value + geoid_offset
  • No SRTM but geoid available: default height above ellipsoid + geoid_offset
  • SRTM available, but no geoid: dem_value
  • No SRTM and no geoid available: default height above ellipsoid
• GetHeightAboveMSL(lon, lat):
  • SRTM and geoid both available: dem_value
  • No SRTM but geoid available: 0
  • SRTM available, but no geoid: dem_value
  • No SRTM and no geoid available: 0

Several DEM tiles can be provided at the same time, using the OpenDEMDirectory method. All raster from the input directory will be opened by GDAL. A mosaic of all DEM tiles is then created as a virtual dataset (vrt).

Geoid are managed with the GDALOpenVerticalShiftGrid and GDALApplyVerticalShiftGrid function from GDAL API. The former opens a 1D raster grid as a GDAL Datasource, and the latter creates a new datasource from the raster grid (geoid) and a raster (the DEM). Vertical datums (shifts from the reference ellipsoid) are applied on the fly.

All raster that can be opened by gdal can be used as a geoid. In Ossim it was common to use the egm96.grd file as geoid, this file cannot be opened by GDAL. However it is still possible to use it by using the following egm96.grd.hdr file :

ENVI
samples = 1441
lines   = 721
bands   = 1
header offset = 24
file type = ENVI Standard
data type = 4
interleave = bsq
sensor type = Unknown
byte order = 1
wavelength units = Unknown
map info = {Geographic Lat/Lon, 1, 1,-0.125, 90.125, 0.25, 0.25,WGS-84}
coordinate system string = {GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137,298.257223563]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]]}
band names = {
Band 1}

With this file attached, GDAL will be able to read the egm96.grd file as a ENVI dataset

Migration example

We introduced a slight change in the DEMHandler manipulation. Before, we would manipulate a smart pointer:

otb::DEMHandler::Pointer demHandler = otb::DEMHandler::Instance();

After, we manipulate the handle to the singletron:

otb::DEMHandler & demHandler = otb::DEMHandler::GetInstance();
OR
auto & demHandler = otb::DEMHandler::GetInstance();

The new Sensor Model mechanism

Presentation

The class otb::SensorTransformBase is the new base class for sensor models. It inherits from otb::Transform, which inherits from itk::Transform. It is templated over the data type, and input and output dimentions. All sensor model classes should inherit from it, and implement the methods:

• SetMetadataModel that takes a boost::any object representing the model;
• IsValidSensorModel that returns true if the model is correctly set;
• TransformPoint that process the transformation.

RPC sensor model

classDiagram
  class Transform~TScalarType, NInputDimensions, NOutputDimensions~{
  -TransformPoint(Point)* Point
  }

  class SensorTransformBase~TScalarType, NInputDimensions, NOutputDimensions~{
  -SetMetadataModel(boost_any)* bool
  -IsValidSensorModel()* bool
  }

  class RPCTransformBase~TScalarType, NInputDimensions, NOutputDimensions~{
  -SetMetadataModel(boost_any) bool
  -IsValidSensorModel() bool
  #Projection_RPCParam m_RPCParam
  #GDALRPCTransformer m_Transformer
  }

  class RPCForwardTransform~TScalarType, NInputDimensions, NOutputDimensions~{
  -TransformPoint(Point) Point
  }

  class RPCInverseTransform~TScalarType, NInputDimensions, NOutputDimensions~{
  -TransformPoint(Point) Point
  }

  Transform <|-- SensorTransformBase
  SensorTransformBase <|-- RPCTransformBase
  RPCTransformBase <|-- RPCForwardTransform
  RPCTransformBase <|-- RPCInverseTransform

The RPC model is stored in the otb::ImageMetadata object, using the key MDGeom::RPC. The classes otb::RPCForwardTransform and otb::RPCInverseTransform inherit from otb::RPCTransformBase and are used to perform forward and inverse transformation using this model.

The abstract class otb::RPCTransformBase contains the implementation of the SetMetadataModel method, which receives the RPC description from the otb::ImageMetadata and instantiates an otb::GDALRPCTransformer.

The classes otb::RPCForwardTransform and otb::RPCInverseTransform each implement there version of the TransformPoint method which uses the otb::GDALRPCTransformer.

SAR sensor model

classDiagram
  class Transform~TScalarType, NInputDimensions, NOutputDimensions~{
  -TransformPoint(Point)* Point
  }

  class SensorTransformBase~TScalarType, NInputDimensions, NOutputDimensions~{
  -SetMetadataModel(boost_any)* bool
  -IsValidSensorModel()* bool
  }

  class SarTransformBase~TScalarType, NInputDimensions, NOutputDimensions~{
  -SetMetadataModel(boost_any) bool
  -IsValidSensorModel() bool
  #SARParam m_SarParam
  #SarSensorModel m_Transformer
  }

  class SarForwardTransform~TScalarType, NInputDimensions, NOutputDimensions~{
  -TransformPoint(Point) Point
  }

  class SarInverseTransform~TScalarType, NInputDimensions, NOutputDimensions~{
  -TransformPoint(Point) Point
  }

  Transform <|-- SensorTransformBase
  SensorTransformBase <|-- SarTransformBase
  SarTransformBase <|-- SarForwardTransform
  SarTransformBase <|-- SarInverseTransform

The SAR model works on the same pattern than the RPC model.

Migration example

Before, the sensor model relying on OSSIM would be instantiated like this:

typedef otb::ForwardSensorModel<double, InputSpaceDimension, InputSpaceDimension> ForwardSensorModelType;
typename ForwardSensorModelType::Pointer sensorModel = ForwardSensorModelType::New();
sensorModel->SetImageGeometry(m_InputKeywordList);
if (sensorModel->IsValidSensorModel())
  [...]

After, the new sensor model framework uses a simple factory that needs a pointer to the ImageMetadata object, and the transform direction:

auto sensorModel = otb::SensorTransformFactory::GetInstance().CreateTransform
  <double, InputSpaceDimension, OutputSpaceDimension>(*imd, TransformDirection::FORWARD);
if (sensorModel != nullptr)
  [...]

The GenericRSTransform filter is also impacted by this refactoring. The methods getting and setting the KeyWordList are replaced by methods getting and setting ImageMetadata objects:

typedef otb::GenricRSTransform<double, InputSpaceDimension, OutputSpaceDimension> TransformType;
TransformType::Pointer transform = TransformType::New();
transform->SetInputKeywordList(kwl);

Becomes

 transform->SetInputImageMetadata(imd);
 transform->SetOutputImageMetadata(*(image->GetImageMetadata));