Repository navigation
BeforeSuite/AfterSuite run even if there are no tests #199
Description
Hi,
A few days ago I found some behaviour of which I'm not sure whether it's a bug or a feature. In my behat.yml I have defined multiple suites, and in my contexts I have also defined a @BeforeSuite hook. When I run a test passing the path of the feature file and no --suite= option, Behat will cycle through all my suites executing the @BeforeSuite (and I presume, but I did not test, also the @AfterSuite) for every suite in my behat.yml. Even though via the paths: configuration the feature I was running is only defined for one suite.
If running without --suite, but with a filter (whether it's passing a specific feature file or using something like --tags) I would expect Behat to ignore any suites that don't have any matching features or scenario's. And if a suite is ignored, you also don't need to run any @BeforeSuite and @AfterSuite.
Can this be considered a bug, or it is an intentional feature? For example is it possible to manipulate the filters or paths or something in the @BeforeSuite that might cause Behat to run scenario's for the suite even though initially nothing would have matched?
If this can be considered a bug (or unintentional behaviour) I don't mind diving into the code to see if I can create a PR for it, but I'm not sure where to start looking where something like this can be changed.
Activity
ciaranmcnulty commentedon Jul 1, 2022
Hm, not sure what the correct behaviour here would be
carlos-granados commentedon Oct 30, 2024
I think this should be considered a bug. I really don't think that Behat should be calling any of the suite hooks if it is not running any feature or scenario in the suite. We probably need to confirm if a similar thing is happening with the feature hooks, if they are being called for features even if no scenario is run. @acoulton what do you think?
stof commentedon Oct 30, 2024
IIRC, Behat is not eagerly loading the list of scenarios that will be run. It relies on an iterator, which allows it to scale to a huge amount of features without consuming tons of memory (assuming you don't use a formatter that buffers huge amount of memory).
Because of that, by the time the
BeforeSuitehook is due to be run, Behat has no idea how many features will be run in the suite (which can be0). And if we runBeforeSuitefor empty runs, it makes sense thatAfterSuitealso runs.note that this is also what happens in PHPUnit when you have a testsuite with 0 tests: https://github.com/sebastianbergmann/phpunit/blob/f3d49bb34256407cbcd256aa0e71f1079b4d08b5/tests/end-to-end/event/data-provider-exception.phpt#L23-L29
carlos-granados commentedon Oct 30, 2024
@stof you are probably right. Let me confirm this and if it is as you explain I'll close this ticket
acoulton commentedon Oct 31, 2024
If we close this maybe we should just mention the behaviour in the hooks documentation.
carlos-granados commentedon Nov 2, 2024
Confirmed that with the current implementation it is not possible to know in the
beforeSuitehook if there are any tests to run. Changed the type of the issue to documentation so that we can at least document this behaviour