Deep Learning Streaming Image Data Generator
Training DL models like conv nets needs the generation of batches of image data over which the training algorithm can loop.
Keras seems to be the most widely used DL framework in Python and it provides an ImageDataGenerator class which provides batches of tensor image data with real-time data augmentation; the data is be looped over (in batches). Assuming you have a set of image patches in a directory, you do:
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator()
train_generator = train_datagen.flow_from_directory(
train_dir,
target_size=(150, 150)
batch_size=20)
#Fit the DL model using the generator
model.fit_generator(
train_generator,
steps_per_epoch=100,
epochs=30,
validation_data=validation_generator,
validation_steps=50)
Right now, to train a DL model with Keras using remote sensing imagery needs to extract patches from the images and save them as individual images so that the standard Keras generator can be used. This also needs to rasterize the reference data set (the shapefile with the labelled training data) so that it fits the image geometry.
This procedure is not efficient and implies lots of I/O.
Using OTB tools developed for the sampling framework would allow to provide a patch image generator which is fully streamed (no need to write data to disk) and does not need to rasterize the reference data. And the code could be:
from otbApplication import otbPatchImageGenerator
train_datagen = otbPatchImageGenerator()
train_generator = train_datagen.flow_from_directory(
train_image_list,
reference_vector_list,
target_size=(150, 150)
batch_size=20)
Similar generators could be provided for time series using images or for pixel-based classification starting from the SQLite files generated by the SampleExtraction application.
Although the examples above use Keras and Python, the underlying C++ code could provide in memory tensors for Tensorflow so that once a model is trained in python it can be deployed effciently by otbImageClassifier.