OTB-Ice asynchronous loading
What changes will be made and why they would make a better Orfeo ToolBox?
This issue is about implementing asynchronous loading of tiles of image-data by the OTB-Ice rendering engine in order to improve responsiveness and interactivity of Monteverdi, such as for Google Maps.
Currently, the OTB-Ice rendering engine divides image-data of each layer into a grid of tiles, thus decreasing the disk-to-RAM loading when scrolling the image-view. Also, it provides a fast move feature which postpones loading of tiles after the scrolling has finished. The OTB-Ice rendering of the image-layer is blocked while new tiles are being loaded.
The idea is to delegate loading of image-data tiles to a background task which signals when each tile is loaded so that the OTB-Ice rendering is not blocked. Loading of tile image-data would start at lowest resolution first, be rendered then be updated to requested resolution and be rendered.
High level description
Loading of image-data tiles for each image-layer is currently done by the ::LoadTile()
function of the otb::GlImageActor
. It could be delegated to an otb::ImageTileLoader
which manages a background loading thread:
- the
otb:GlImageActor
enqueues a tile request into theotb::ImageTileLoader
which asynchronously loads the lowest resolution first - The
otb::ImageTileLoader
calls back theotb::GlImageActor
when image-data of the tile has been loaded - The
otb::GlImageActor
then renders the lowest resolution of image-data of the tile - When all lowest resolution tiles have been processed, the
otb::ImageTileLoader
automatically starts loading image-data of tile at requested region and calls back theotb::GlImageActor
when ready. - Whenever all tiles have been processed, it sleeps.
Whenever the requested resolution has changed, already loaded tiles are kept and the otb::GlImageActor
enques a new tile request. When image-data has been loaded, the highest resolution is released, thus freeing memory.
The ::CleanLoadedTiles()
function of the otb::GlImageActor
would need only minor changes to handle that. So does the ::UnloadTile()
function.
The otb::GlImageActor::Tile
structure contains tile regions and settings alongside with image-data. It should be split in two separate parts:
- the former being used to enqueue tile requests and check tile state
- the latter being used for image-data exchange between the
otb::GlImageActor
and theotb::ImageTileLoader
.
The otb::ImageFileReader<>
ownership would need to be transfered to the otb::ImageTileLoader
.
A cancel tile request should be added to the otb::ImageTileLoader
and appropriate driving code in the ::UpdateData()
function of the otb::GlImageActor
in order to handle changes of resolution and/or scrolling when image-data of tiles are still being loaded.
Note: A flag could be implemented to switch asynchronous loading ON/OFF.
An additional idea is to implement a thread-pool manager from which the otb::ImageTileLoader
could borrow a thread resource. The number of thread managed could be limited or not. It the latter case, it is equivalent to the aforementioned implementation except that ownership of thread resources is granted to the thread-pool manager.
Risks and benefits
When importing image-data in Monteverdi, the rendering is blocked until all image-data for the viewport has been loaded. Asynchronous loading could help to reduce this delay since each tile would be rendered as fast as possible without waiting for all tiles to be loaded. Likewise, when scrolling or zooming.
The OTB-Ice renders layer back-to-front in order to handle translucency or view-through effects. So, when image-data of a tile has finished loading, a render call-back will cause the rendering of all the scene to be redone. But, this OpenGL-based hardware-accelerated rendering is faster than disk-to-RAM loading, especially when decompressing image-data.
One otb::ImageTileLoader
would be associated to each otb::GlImageActor
which means one background thread for each otb::GlImageActor
running concurrent disk access (on different image-data files). Usually, the number of image-data layers is low. So, the number of threads is kept low but concurrent disk access could kill the performances and is hardware dependent, which should be benchmarked. Also, a system thread context increases RAM usage.
Note: Loading image-data from different disks would improve parallelization.
The use of a thread-pool manager could be useful to limit number of simultaneous threads, thus limiting resources usage, but will limit parallelization of image-data loading.
Alternatives for implementations
Not Applicable.
Who will be developing the proposed changes?
(TBD) Development of this feature needs good knowledge of the OTB-Ice rendering engine, graphics, OpenGL rendering and Monteverdi.