chore: Mark slow pytests as 'slow' so they can be excluded. #1720
Conversation
|
@royjacobson does this PR will exclude slow tests from running? And if yes, when will we run them? |
|
Should at least run all tests on release IMO |
Doesn't look like it Did we actually decide to decrease out regtest running times? They run every 3hrs and take 30mins, so we have a window of about 2.5hrs 🤔 I thought we wanted to tackle CI tests |
|
true, but its probably worthwhile to also tackle regression tests because it's only a matter of time until they become too slow. maybe its a bit too early to establish such slow/fast framework though. |
| @@ -5,3 +5,5 @@ log_date_format = %Y-%m-%d %H:%M:%S | |||
| log_file_level=DEBUG | |||
| log_cli = true | |||
| asyncio_mode=auto | |||
| markers = | |||
| slow: marks tests as slow (deselect with '-m "not slow"') | |||
There was a problem hiding this comment.
So by default we don't run slow tests. Right?
That's actually a good catch. I think for reg-tests we should also now include '-m "not slow`. From my understanding, this disables all slow tests by default so you have to be explicit when to run those. |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
|
This doesn't disable them be default, it's just marking them for exclusion with the flag. I would like to add the fast tests on the pre-merge CI runs and leave the regression tests as-is, but we don't have to decide it here :) |
Marked every test running over 10s as 'slow'.
All the tests together - 10:13, only the fast tests - 2:40.