Supply our own runner

[LujunWeng]

Is that possible I supply my own runner and control the beginning of each testcase?
I find in the default main function run(argc, argv) is directly called as follows

// Standard C/C++ main entry point
int main (int argc, char * const argv[]) {
    return Catch::Session().run( argc, argv );
}

I think it better if there is another run method with some runner passed, like this one
Catch::Session().run( argc, argv, new somerunner() );

[dvirtz]

You can use test fixtures for set up code:
https://github.com/philsquared/Catch/blob/master/docs/test-fixtures

On 30 July 2015 at 07:27, Lujun Weng notifications@github.com wrote:

Is that possible I supply my own runner and control the beginning of each
testcase?
I find in the default main function run(argc, argv) is directly called as
follows

// Standard C/C++ main entry pointint main (int argc, char * const argv[]) {
return Catch::Session().run( argc, argv );
}

I think it better if there is another run method with some runner passed,
like this one
Catch::Session().run( argc, argv, new somerunner() );

—
Reply to this email directly or view it on GitHub
#475.

[LujunWeng]

Thank you for reply. But that seems not what I want.
I am trying to write a Catch adapter for Visual Studio, but I don't know how to cooperate with Catch.

[eloff]

I'm also looking to add setup and teardown code around every test case. GTest allows this through defining a TestEventListenerProxy and overriding the virtual methods OnTestStart and OnTestEnd.

My use case is I have a lot of custom logging, and I want to buffer it during a test run and then either dump the buffer to stderr on test failure or erase it on success. There's no point cluttering the output for passing tests, but it's helpful to see for failures.

[philsquared]

Ok, there's two - possibly three - things here:

1. @LujunWeng's original question was about "control[ling] the beginning of each test case". I'm not 100% sure what "control" means here. But assuming it's just being able to run some code before, and possible after, each test case then I have some news that will help (more shortly).
2. @eloff's request asks for something like OnTestStart and OnTestEnd methods that can be overridden (a lá GTest). My news relates to this. But before we get to that the use case referred to "custom logging" and only wanting to "dump the buffer to stereo on test failure". That's exactly what INFO does (https://github.com/philsquared/Catch/blob/master/docs/logging.md). Is there a reason that INFO cannot work for your use case, @eloff?

Ok, so having concluded that neither interested party may, in fact, be interested after all - here's the interesting news :-)

I've added the ability to add listener objects. From the way @eloff describes them they sound similar to GTest's listeners - but I've not used those, so I don't know how similar they are.

In Catch a listener is really just a reporter. The differences are:

1. They are registered using CATCH_REGISTER_LISTENER (actually there's an INTERNAL_ prefix on there at time of writing - I'll remove that).
2. All registered listeners are used - in addition to whatever reporter is being used at the time.
3. Listeners should derive from Catch::TestEventListenerBase they don't have to - it just makes it easier by default implementing all the events so you only have to implement the ones you're interested in).
4. You don't need to provide a getDescription() method.

Here's an example (this needs to go in the same file as #define CATCH_CONFIG_MAIN or #define CATCH_CONFIG_RUNNER:

struct Listener : Catch::TestEventListenerBase {

    using TestEventListenerBase::TestEventListenerBase;

    virtual void testCaseStarting( Catch::TestCaseInfo const& testInfo ) override {
        std::cout << testInfo.name << std::endl;
    }
};
INTERNAL_CATCH_REGISTER_LISTENER( Listener )

(note this example uses C++11 features. In particular you'll have to write your own constructor to forward the ReporterConfig argument down if you don't have forwarding constructors available).

This is a new, currently undocumented (other than here), feature - available from catch-1.2.1-develop.12. It's possible I'll iterate this a bit before locking it down - so bear in mind the interface may change (at least the way it is registered).

[philsquared]

Oh, I should say also: it's not just test case starting that you can monitor, you can implement event handlers for:
testRunStarting/Ended, testGroupStarting/Ended, testCaseStarting/Ended, sectionStarting/Ended and even assertionStarting/Ended. Also skipTest for test that are not run due to being "hidden".

[eloff]

@philsquared Thanks for responding so quickly, I did not expect that after my experiences with other GitHub open-source projects. It seems I am the tardy one here.

I will try replacing my logging with INFO when running under the tests, it may be enough. I've already replaced my assertions with REQUIRE when under tests, so it seems workable (with REQUIRE I discovered care must be taken not to call assertions in e.g. global setup code that runs outside of the tests.)

I'm also very happy to see the event listener. That was the only remaining feature of GTest that I miss. I'm now very happy that I settled on catch for this project!

From my POV you can close this issue as resolved.

[philsquared]

Thanks @eloff.
As for "responding so quickly": in truth you just got lucky ;-) I struggle to keep up and am not able to focus on Catch full-time so I miss a lot. Glad it's working out for you though.
I didn't get the fact, before, that your logging was not part of your test code. Redirecting it to INFO when building for test could work, but I'm not so sure that's the best way, now.
In the reporters there is a means for them to specify that stdout should be captured (in the same way as the INFO macros) - but, currently, that behaviour is hard-coded per-reporter (e.g. see:
https://github.com/philsquared/Catch/blob/master/include/reporters/catch_reporter_console.hpp#L30).
I forget what my reasoning was there now. You might want to experiment by changing that false to true and see if it gives you the behaviour you want. I might look at making that behaviour configurable.

[LujunWeng]

Thank you for responding and sorry for my poor English. ：）

My original intention is to find a way to enumerate all test cases, start all test cases and collect information through some simple interface. Therefore, I can generate a DLL file or something similar which I can use GUI to find and run it.

After the issue was posted, I found I can use the command line to tackle this. But, I still hope there is easy way to find Catch tests, enumerate all test cases and run them.

Again, thank you for this project. I will continuously study this project and learn from it. I hope I can make some contribution to it in future.

[eloff]

@philsquared I tried setting shouldRedirectStdOut = false to true (I changed it everywhere in the code where I found it, not just the indicated line) and it works a treat for hiding all output, but it also hides the output when the test fails, which I don't want. I think the event listener, plus manually buffering/flushing log output should work. It's similar to what I used to do with GTest.