Add performance test monitoring into OTB CI
In order to facilitate the OTB development when large modifications are done. It could be interesting to monitor the performance of OTB.
Currently it is possible to use regression test to do that. Indeed CTest provide test time execution and CDash display it. The idea could be to retrieve this test time and compare the test time of develop with feature branch.
@sdinot has perform some test with CDash API and it is very interesting:
- all test are available here: https://cdash.orfeo-toolbox.org/viewTest.php?buildid=53386
- successful tests are available here: https://cdash.orfeo-toolbox.org/viewTest.php?onlypassed&buildid=53386
- with API we can retrieve the corresponding JSON:
With jq we can filter the JSON to retrieve the desired information and understand the content:
curl -s 'https://cdash.orfeo-toolbox.org/api/v1/viewTest.php?onlypassed&buildid=53386' | jq .
To retrieve the test time:
curl -s 'https://cdash.orfeo-toolbox.org/api/v1/viewTest.php?onlypassed&buildid=53386' | jq '.tests[] | { name: .name, time: .execTimeFull }'or whitout JSON structure:
curl -s 'https://cdash.orfeo-toolbox.org/api/v1/viewTest.php?onlypassed&buildid=53386' | jq '.tests[] | .name, .execTimeFull'