Similar to #351 Any test runner that doesn't need live updating shouldn't have to aggregate results manually.