OTB-Ice asynchronous loading
What changes will be made and why they would make a better Orfeo ToolBox?
This issue is about implementing asynchronous loading of tiles of image-data by the OTB-Ice rendering engine in order to improve responsiveness and interactivity of Monteverdi, such as for Google Maps.
Currently, the OTB-Ice rendering engine divides image-data of each layer into a grid of tiles, thus decreasing the disk-to-RAM loading when scrolling the image-view. Also, it provides a fast move feature which postpones loading of tiles after the scrolling has finished. The OTB-Ice rendering of the image-layer is blocked while new tiles are being loaded.
The idea is to delegate loading of image-data tiles to a background task which signals when each tile is loaded so that the OTB-Ice rendering is not blocked. Loading of tile image-data would start at lowest resolution first, be rendered then be updated to requested resolution and be rendered.
High level description
Loading of image-data tiles for each image-layer is currently done by the
::LoadTile() function of the
otb::GlImageActor. It could be delegated to an
otb::ImageTileLoader which manages a background loading thread:
otb:GlImageActorenqueues a tile request into the
otb::ImageTileLoaderwhich asynchronously loads the lowest resolution first
otb::ImageTileLoadercalls back the
otb::GlImageActorwhen image-data of the tile has been loaded
otb::GlImageActorthen renders the lowest resolution of image-data of the tile
- When all lowest resolution tiles have been processed, the
otb::ImageTileLoaderautomatically starts loading image-data of tile at requested region and calls back the
- Whenever all tiles have been processed, it sleeps.
Whenever the requested resolution has changed, already loaded tiles are kept and the
otb::GlImageActor enques a new tile request. When image-data has been loaded, the highest resolution is released, thus freeing memory.
::CleanLoadedTiles() function of the
otb::GlImageActor would need only minor changes to handle that. So does the
otb::GlImageActor::Tile structure contains tile regions and settings alongside with image-data. It should be split in two separate parts:
- the former being used to enqueue tile requests and check tile state
- the latter being used for image-data exchange between the
otb::ImageFileReader<> ownership would need to be transfered to the
A cancel tile request should be added to the
otb::ImageTileLoader and appropriate driving code in the
::UpdateData() function of the
otb::GlImageActor in order to handle changes of resolution and/or scrolling when image-data of tiles are still being loaded.
Note: A flag could be implemented to switch asynchronous loading ON/OFF.
An additional idea is to implement a thread-pool manager from which the
otb::ImageTileLoader could borrow a thread resource. The number of thread managed could be limited or not. It the latter case, it is equivalent to the aforementioned implementation except that ownership of thread resources is granted to the thread-pool manager.
Risks and benefits
When importing image-data in Monteverdi, the rendering is blocked until all image-data for the viewport has been loaded. Asynchronous loading could help to reduce this delay since each tile would be rendered as fast as possible without waiting for all tiles to be loaded. Likewise, when scrolling or zooming.
The OTB-Ice renders layer back-to-front in order to handle translucency or view-through effects. So, when image-data of a tile has finished loading, a render call-back will cause the rendering of all the scene to be redone. But, this OpenGL-based hardware-accelerated rendering is faster than disk-to-RAM loading, especially when decompressing image-data.
otb::ImageTileLoader would be associated to each
otb::GlImageActor which means one background thread for each
otb::GlImageActor running concurrent disk access (on different image-data files). Usually, the number of image-data layers is low. So, the number of threads is kept low but concurrent disk access could kill the performances and is hardware dependent, which should be benchmarked. Also, a system thread context increases RAM usage.
Note: Loading image-data from different disks would improve parallelization.
The use of a thread-pool manager could be useful to limit number of simultaneous threads, thus limiting resources usage, but will limit parallelization of image-data loading.
Alternatives for implementations
Who will be developing the proposed changes?
(TBD) Development of this feature needs good knowledge of the OTB-Ice rendering engine, graphics, OpenGL rendering and Monteverdi.