Skip to content

Refac tests

Right now, since INPUT is only initialized once, during our different tests, the INPUT object is not in a clean state because of previous function calls.
For our tests to be solid, we should use only clean instances when testing properties and exports, file writing, etc.
We can use a "fixture" function that generates a new inp object for each function test.
Edit: I abandoned this because it makes the whole test process much longer due to multiple file downloads.

We have many isolated tests, it would be better to make it fewer functions, to avoid initializing INPUT 127 times. We can do several assert statements in the same function.

Also, right now we have way too many tests results in the junit.xml.

BTW I also want to do something about this with test_pipeline.

We do not need so many functions to run at each commit, this is clearly outrageous CPU usage for nothing. Instead of testing 1200 possible combinations and for each unit test we check (shape, etc), the best would be to test max 10 to 100 pipelines test, with selection and intelligent choice on which kind we're testing. The generative thing is really nice code, but I believe we are testing too many duplicated routines.

When one feature is broken, we just read "500 failed tests" in the report, this is not helpful 😅

TODO :

  • Use fixtures ?
  • Write tmp files in /dev/shm (I'll make sure this won't change anything to the test behavior)
  • Less reports of pipeline tests in XML, but print every failed pipeline pytest stdout
  • Grouptest_* in funcs that makes more sense, with multiple assertions (e.g. test_dtype, test_shape, test_transform could be grouped in test_properties)
  • Regarding operators, it would be nice to have a result for each kind of operator in the junit xml.

What I mean for this last one is that we should split or group some of those :

def test_operator_mult:
    _test(lambda x: x * x, "im1 mult im1")
    _test(lambda x: x * FILEPATH, "im1 mult im1")
    _test(lambda x: FILEPATH * x, "im1 mult im1")
    _test(lambda x: x * 2, "im1 * 2")
    _test(lambda x: x * 2.0, "im1 * 2.0")
    _test(lambda x: 2 * x, "2 * im1")
    _test(lambda x: 2.0 * x, "2.0 * im1")

Also: With pyest we can pass the -k option with substring of test func to runs, it's really handy.
So we should reface our test name functions with this in mind.
For example I used pytest -vv -k "test_pipeline" test_core.py to test my latest MR.

Edited by Vincent Delbar