GDAL VSI support
I've recently heard someone insist that OTB is a bad framework and isn't "cloud-ready" because it doesn't support the "technologies of the future", like object storage. The thing is, GDAL does support cURL, Amazon S3 and other storage drivers. Unfortunately, OTB doesn't seem to.
I tried a couple of tests:
$ CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt otbcli_ReadImageInfo -in "http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif?skipgeom=true"
2018-10-24 13:26:34 (INFO): Default RAM limit for OTB is 128 MB
2018-10-24 13:26:34 (INFO): GDAL maximum cache size is 393 MB
2018-10-24 13:26:34 (INFO): OTB will use at most 4 threads
2018-10-24 13:26:35 (INFO):
Image general information:
Number of bands : 3
No data flags : Not found
Start index : [0,0]
Size : [65536,65536]
Origin : [0.5,0.5]
Spacing : [1,1]
Estimated ground spacing (in meters): [547.072,296.427]
Image acquisition information:
Sensor :
Image identification number:
Image default RGB composition:
[R, G, B] = [0,1,2]
Ground control points information:
Number of GCPs = 0
GCPs projection =
Output parameters value:
indexx: 0
indexy: 0
sizex: 65536
sizey: 65536
spacingx: 1
spacingy: 1
originx: 0.5
originy: 0.5
estimatedgroundspacingx: 547.0715332
estimatedgroundspacingy: 296.4269409
numberbands: 3
sensor:
id:
time:
town:
country:
rgb.r: 0
rgb.g: 1
rgb.b: 2
projectionref:
keyword:
gcp.count: 0
gcp.proj:
gcp.ids:
gcp.info:
gcp.imcoord:
gcp.geocoord
$ CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt gdalinfo http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif
Driver: GTiff/GeoTIFF
Files: none associated
Size is 514, 515
Coordinate System is:
PROJCS["unnamed",
GEOGCS["NAD27",
DATUM["North_American_Datum_1927",
SPHEROID["Clarke 1866",6378206.4,294.9786982138982,
AUTHORITY["EPSG","7008"]],
AUTHORITY["EPSG","6267"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4267"]],
PROJECTION["Cylindrical_Equal_Area"],
PARAMETER["standard_parallel_1",33.75],
PARAMETER["central_meridian",-117.333333333333],
PARAMETER["false_easting",0],
PARAMETER["false_northing",0],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]]]
Origin = (-28493.166784412522247,4255884.543802191503346)
Pixel Size = (60.022136983193739,-60.022136983193739)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( -28493.167, 4255884.544) (117d38'27.05"W, 33d56'37.74"N)
Lower Left ( -28493.167, 4224973.143) (117d38'27.05"W, 33d39'53.81"N)
Upper Right ( 2358.212, 4255884.544) (117d18'28.38"W, 33d56'37.74"N)
Lower Right ( 2358.212, 4224973.143) (117d18'28.38"W, 33d39'53.81"N)
Center ( -13067.478, 4240428.844) (117d28'27.71"W, 33d48'15.38"N)
Band 1 Block=514x15 Type=Byte, ColorInterp=Gray
Notice that OTB reports wrong origin, ground spacing and size values.
$ CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt strace -s2048 -e trace=file -f otbcli_ReadImageInfo -in "http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif"
[snip]
[pid 6069] access("http://download.osgeo.org/geotiff/samples/gdal_eg/cea.geom", R_OK) = -1 ENOENT (No such file or directory)
[pid 6069] getcwd("/home/user", 4096) = 16
[pid 6069] lstat("/home/user/http:", 0x7ffed725d380) = -1 ENOENT (No such file or directory)
[pid 6069] access("http://download.osgeo.org/geotiff/samples/gdal_eg/cea.TIL", F_OK) = -1 ENOENT (No such file or directory)
[pid 6069] access("http://download.osgeo.org/geotiff/samples/gdal_eg/cea.til", F_OK) = -1 ENOENT (No such file or directory)
[pid 6069] openat(AT_FDCWD, "http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6069] access("http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif", F_OK) = -1 ENOENT (No such file or directory)
[pid 6069] openat(AT_FDCWD, "http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6069] stat("http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif", 0x7ffed725dc50) = -1 ENOENT (No such file or directory)
[pid 6069] readlink("http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif", 0x245a730, 2048) = -1 ENOENT (No such file or directory)
2018-10-24 13:26:57 (INFO): No kwl metadata found in file http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif
OTB tries to open HTTP URLs as files.
$ CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt strace -s2048 -f otbcli_ComputeImagesStatistics -il "http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif?skipgeom=true"
[snip]
[pid 6110] sendto(4, "GET /geotiff/samples/gdal_eg/cea.tifqqqqqqrr HTTP/1.1\r\nHost: download.osgeo.org\r\nAccept: */*\r\n\r\n", 96, MSG_NOSIGNAL, NULL, 0) = 96
OTB gets into an infinite loop trying to request the wrong URL (notice the qqqqqqrr
suffix).
CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt strace -s2048 -f otbcli_Convert -in "http://download.osgeo.org/geotiff/samples/gdal_eg/cea.tif?skipgeom=true" -out foo.tif
[snip]
[pid 6171] sendto(4, "GET /geotiff/samples/gdal_eg/cea.tifqqqqqqqq HTTP/1.1\r\nHost: download.osgeo.org\r\nAccept: */*\r\n\r\n", 96, MSG_NOSIGNAL, NULL, 0) = 96
Same issue, different suffix. Uninitialized memory?
Of course, I expect performance to be pretty poor when doing network access, but sometimes it's useful, and the alternative is to use a FUSE emulation driver that might have its own problems.
Related: #1746 (closed)
Related: #1506 (closed)
Related: #1642