Resolve "Release the Python Global Interpreter Lock (GIL) during execution"
Summary
closes #2078
Implementation Details
See https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock, http://www.swig.org/Doc4.0/Python.html#Python_multithreaded and https://github.com/InsightSoftwareConsortium/ITK/pull/1065
In SWIG file otbApplication.i, setting module's parameter thread
to "1" allows application wrappers to release Python GIL before entering wrapped code.
PyCommand
has been modified to reacquire the GIL before calling its callback function. Patch taken from ITK : https://github.com/InsightSoftwareConsortium/ITK/commit/3eb1f6a6e59344a756dece628833b0214145057b
PythonLogOutput
has not been modified, as setting thread=1
seems sufficient to solve bug #2078 (comment 91988). The difference with PyCommand is that we are using director classes in this case.
However, in the scope of this merge request the patch should be applied.
Additional notes
-
In ITK, they also add
%feature("nothreadallow");
. However if we do the same in otbApplication.i it seems to prevent the GIL to be correctly released. In the end, settingthread
to "1" seems enough. -
PyCommand
is not very useful in OTB. It is not used in the wrapper. However it is part of the API, so we should solve the GIL bug in this case.
This class can be used in combination with AddObserver
(from itk objects) to call Python code when a specific event is invoked by the object, for example
import otbApplication as otb
app = otb.Registry.CreateApplication("EdgeExtraction")
app.IN = input.tif
app.OUT = output.tif
def Update():
print("hello")
iterationCommand = otb.itkPyCommand_New()
iterationCommand.SetCommandCallable(Update)
app.AddObserver(MyEvent, iterationCommand.GetPointer())
app.ExecuteAndWriteOutput()
will print "hello" when MyEvent is invoked.
This is useful in ITK where filters are wrapped in python. For example it is possible to print information at each iteration of a filter
In OTB the only wrapped class where this could be used is the Application class. This class does not invoke a lot of event, only AddProcessToWatchEvent
in otb::Wrapper::Application, subclasses might invoke event when overriding methods (e.g. in DoExecute), this is never the case for otb Applications. Note that AddProcessToWatchEvent
is already observed to trace the progress of the application (see the logger classes).
If someday we want to simplify the SWIG wrapper maybe PyCommand should be removed.
Copyright
The copyright owner is CNES and has signed the ORFEO ToolBox Contributor License Agreement.
Check before merging:
- All discussions are resolved
- At least 2
👍 votes from core developers, no👎 vote. - The feature branch is (reasonably) up-to-date with the base branch
- Dashboard is green
- Copyright owner has signed the ORFEO ToolBox Contributor License Agreement
- Optionally, run
git diff develop... -U0 --no-color | clang-format-diff.py -p1 -i
on latest changes and commit