Huge Memory-leaks when in-memory chaining OTB-applications from Python (#1817) · Issues · Main Repositories / otb

Huge Memory-leaks when in-memory chaining OTB-applications from Python

Description

Huge memory consumption when in-memory chaining OTB-Applications within loop from Python code.

Steps to reproduce

There are two processing pipelines defined by chaining OTB-applications in memory from Python code:

create_water_mask:

graph water_mask_maker;
  PAN-->ExtractROI_PAN;
  XS-->ExtractROI_XS;
  ExtractROI_PAN-->SuperImpose;
  ExtractROI_XS-->WaterMaskMaker;
  WaterMaskMaker-->SuperImpose;

create_color_image:

graph color_image_maker;
  PAN-->ExtractROI_PAN;
  XS-->ExtractROI_XS;
  ExtractROI_PAN-->BundleToPerfectSensor;
  ExtractROI_XS-->BundleToPerfectSensor;
  BundleToPerfectSensor-->Convert;

A main processing function runs alternatively those two pipeline on a set of (PAN ; XS) 2-uplets in a (Python) loop (see test_wwm_cim.py)

Memory-leak instrumentation done with vg-mem.sh.

Instrumentation of the `WaterMaskMaker` C++ OTB-Application

Command: vg-mem.sh ./bin/otbApplicationLauncherCommandLine WaterMaskMaker -progress true -in /home/salbert/data/bench-2/QB_TLSE_MUL_512x512_1024x1024.TIF -out /home/salbert/tmp/QB_TLSE_WMM.tif

Valgrind log: [otbApplicationLauncherCommandLine-20181213-105906.log](/uploads/7beb36ede3bca565a38fd58dc6d19c18/otbApplicationLauncherCommandLine-20181213-105906.log

Small memory-leaks in OSSIM projection and RTTI systems. Same for TIF and JP2000

Instrumentation of the `BundleToPerfectSensor` OTB-Application

Command: vg-mem.sh ./bin/otbApplicationLauncherCommandLine BundleToPerfectSensor -progress true -inp /home/salbert/data/bench-2/PRIMARY_BUNDLE_P_1024x1024.tif -inxs /home/salbert/data/bench-2/PRIMARY_BUNDLE_MUL_1024x1024.tif -out /home/salbert/tmp/PRIMARY_BUNDLE_BTPS.tif uint16

Valgrind log: otbApplicationLauncherCommandLine-20181220-130819.log

L924, L910: Memory-leaks in pipeline ::UpdateOutputInformation() from https://github.com/InsightSoftwareConsortium/ITK/blob/v4.12.2/Modules/Core/Common/include/itkImportImageContainer.hxx#L188

L896: Also inquire about leak of process-object instance of https://github.com/InsightSoftwareConsortium/ITK/blob/v4.12.2/Modules/Core/Common/include/itkInPlaceImageFilter.hxx#L38.

Instrumentation of SWIG OTB-Application wrapper

SWIG wrapper of OTB-applications have been instrumented using the otb-app-swig-mem-log.patch trace patch and all instances of OTB-Applications (be they composite or not) have been manually counted).

Command: vg-mem.sh ./test_OTB_application.py (see test_OTB_application.py)

Valgrind log: test_OTB_application_memory.py-20181214-113339.log

Command: vg-mem.sh ./test_OTB_application_memory_JP2.py (test_OTB_application_memory_JP2.py)

Valgrind log: test_OTB_application_memory_JP2.py-20181217-104219.log

There's no memory leak due to incorrect reference-counting of the SWIG wrapper. However:

the reference-counter wrapper implementation [1] is out of date and could be replaced by direct SWIG 3 handling [2]
ImageBaseType (which is a reference-counted instance) is not wrapped, but OTB-Application wrapper provides a ::GetParameterImage() returning a C++ raw-pointer without reference-counting handling, which may cause some memory leaks (see next section about ) in the Python environment, e.g.:

  def main( self ):
      app = ...
      img = app.GetParameterImage( ... ) # img is not reference-counted
      # If not giving img to a wrapped C++ function putting it into an itk::SmartPointer<> will cause a memory-leak which of all image-data.

Instrumentation of full Python sample code

Same memory-leaks as described above are traces. Moreover, the metadata-dictionary is lost when running multiple-times this instrumentation test without deleting the temporary output file.

Command: vg-mem.sh ./test_wmm_cim.py (2 iterations without removing ${HOME}/tmp files)

Valgrind log: test_wwm_cim.py-20181219-161831.log (see also: test_wwm_cim.py-20181217-174610.log L3666 which is related to a suspicious call to ::GetParameterImage() from otb::ImageFileReader<> when initializing pipeline on existing output-data file.)

Configuration information

OTB-6.6 release and ITK-4.12 on Ubuntu 18.04 LTS.

### Description

Huge memory consumption when in-memory chaining OTB-Applications within loop from _Python_ code.

### Steps to reproduce

There are two processing pipelines defined by chaining OTB-applications in memory from _Python_ code:
1. create_water_mask:
```mermaid
graph water_mask_maker;
  PAN-->ExtractROI_PAN;
  XS-->ExtractROI_XS;
  ExtractROI_PAN-->SuperImpose;
  ExtractROI_XS-->WaterMaskMaker;
  WaterMaskMaker-->SuperImpose;
```
2. create_color_image:
```mermaid
graph color_image_maker;
  PAN-->ExtractROI_PAN;
  XS-->ExtractROI_XS;
  ExtractROI_PAN-->BundleToPerfectSensor;
  ExtractROI_XS-->BundleToPerfectSensor;
  BundleToPerfectSensor-->Convert;
```
A main processing function runs alternatively those two pipeline on a set of (PAN ; XS) 2-uplets in a (Python) loop (see [test_wwm_cim.py](/uploads/73d2c9bc174f7d53cb5b4ed431fe7ae0/test_wwm_cim.py))

Memory-leak instrumentation done with [vg-mem.sh](/uploads/fb31fe9ded1f9868cd832a8132fc7961/vg-mem.sh).

#### Instrumentation of the ```WaterMaskMaker``` C++ OTB-Application

Command: ```vg-mem.sh ./bin/otbApplicationLauncherCommandLine WaterMaskMaker -progress true -in /home/salbert/data/bench-2/QB_TLSE_MUL_512x512_1024x1024.TIF -out /home/salbert/tmp/QB_TLSE_WMM.tif```

Valgrind log: [otbApplicationLauncherCommandLine-20181213-105906.log](/uploads/7beb36ede3bca565a38fd58dc6d19c18/otbApplicationLauncherCommandLine-20181213-105906.log

Small memory-leaks in _OSSIM_ projection and _RTTI_ systems. Same for TIF and JP2000

#### Instrumentation of the ```BundleToPerfectSensor``` OTB-Application

Command: ```vg-mem.sh ./bin/otbApplicationLauncherCommandLine BundleToPerfectSensor -progress true -inp /home/salbert/data/bench-2/PRIMARY_BUNDLE_P_1024x1024.tif -inxs /home/salbert/data/bench-2/PRIMARY_BUNDLE_MUL_1024x1024.tif -out /home/salbert/tmp/PRIMARY_BUNDLE_BTPS.tif uint16```

Valgrind log: [otbApplicationLauncherCommandLine-20181220-130819.log](/uploads/19f7dfbdda614771744711de8a1c0028/otbApplicationLauncherCommandLine-20181220-130819.log)

L924, L910: Memory-leaks in pipeline ```::UpdateOutputInformation()``` from https://github.com/InsightSoftwareConsortium/ITK/blob/v4.12.2/Modules/Core/Common/include/itkImportImageContainer.hxx#L188

L896: Also inquire about leak of process-object instance of https://github.com/InsightSoftwareConsortium/ITK/blob/v4.12.2/Modules/Core/Common/include/itkInPlaceImageFilter.hxx#L38.

#### Instrumentation of _SWIG_ OTB-Application wrapper

_SWIG_ wrapper of OTB-applications have been instrumented using the [otb-app-swig-mem-log.patch](/uploads/2f8c6b39793aadb2e3b2f35c4612b3b1/otb-app-swig-mem-log.patch) trace patch and all instances of OTB-Applications (be they composite or not) have been manually counted).

Command: ```vg-mem.sh ./test_OTB_application.py``` (see [test_OTB_application.py](/uploads/d578087dea8c209fb841166256a0c50e/test_OTB_application.py))

Valgrind log: [test_OTB_application_memory.py-20181214-113339.log](/uploads/becd9109e348354e8001c583a47df1c8/test_OTB_application_memory.py-20181214-113339.log)

Command: ```vg-mem.sh ./test_OTB_application_memory_JP2.py``` ([test_OTB_application_memory_JP2.py](/uploads/e5003321db00cb3091ed9320c04621af/test_OTB_application_memory_JP2.py))

Valgrind log: [test_OTB_application_memory_JP2.py-20181217-104219.log](/uploads/6bc5de2ae3275759af70c798b17703ae/test_OTB_application_memory_JP2.py-20181217-104219.log)

There's no memory leak due to incorrect reference-counting of the SWIG wrapper. However:
1. the reference-counter wrapper implementation [[1]] is out of date and could be replaced by direct _SWIG_ 3 handling [[2]]
2. ```ImageBaseType``` (which is a reference-counted instance) is not wrapped, but OTB-Application wrapper provides a ```::GetParameterImage()``` returning a C++ raw-pointer without reference-counting handling, which may cause some memory leaks (see next section about ) in the _Python_ environment, e.g.:
```python
  def main( self ):
      app = ...
      img = app.GetParameterImage( ... ) # img is not reference-counted
      # If not giving img to a wrapped C++ function putting it into an itk::SmartPointer<> will cause a memory-leak which of all image-data.
```

#### Instrumentation of full _Python_ sample code

Same memory-leaks as described above are traces. Moreover, the metadata-dictionary is lost when running multiple-times this instrumentation test without deleting the temporary output file.

Command: ```vg-mem.sh ./test_wmm_cim.py``` (2 iterations without removing ```${HOME}/tmp``` files)

Valgrind log: [test_wwm_cim.py-20181219-161831.log](/uploads/f4322d42a25e01cfdd71a26c99ded4f7/test_wwm_cim.py-20181219-161831.log) (see also: [test_wwm_cim.py-20181217-174610.log](/uploads/ff1b413d2c2ca2efcf96937f2dd5d90c/test_wwm_cim.py-20181217-174610.log) L3666 which is related to a suspicious call to ```::GetParameterImage()``` from ```otb::ImageFileReader<>``` when initializing pipeline on existing output-data file.)

### Configuration information

OTB-6.6 release and ITK-4.12 on Ubuntu 18.04 LTS.

[1]: https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/blob/release-6.6/Modules/Wrappers/SWIG/src/Python.i#L45
[2]: http://www.swig.org/Doc3.0/SWIGPlus.html#SWIGPlus_smart_pointers