You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Bluepill flags have grown in number and complexity over time. It is time to simplify the logic, deprecate some, merge behaviors. Here are a few proposals.
Some proposals:
Flags that have grown complicated over time especially those that got deviated from its original meaning (Eg. only-retry-failed used for other purposes too)
Flags whose defaults are good enough and are rarely overridden.
Merge flags with similar or confusing purposes. (Eg. error-retries and failure-tolerance can be merged.)
Rarely used flags, if any. (printf-config, test-bundle-path, additional-unit-xctests, additional-ui-xctests)
I think what makes it hard to reason about the flags is that they are non-orthogonal (that is you can't reason about them independently of the other flags). How about focusing on the tests themselves and the possible results?
We have individual tests that might end up in one of four states: success, failure, crash, and timeout. Are there others?
And on each run, we want to be able to specify how much tolerance we have for each of those states right? We could do something like:
--timeout-tolerance=4 --failure-tolerance=2 --crash-tolerance=0 and mean allow 4 timeouts 2 failures and 0 app crashes. Would that make it simpler?
Bluepill flags have grown in number and complexity over time. It is time to simplify the logic, deprecate some, merge behaviors. Here are a few proposals.
Some proposals:
only-retry-failed
used for other purposes too)error-retries
andfailure-tolerance
can be merged.)printf-config
,test-bundle-path
,additional-unit-xctests
,additional-ui-xctests
)Comments and suggestions are welcome.
\cc @ob @RainNapper @jerrymarino @chenxiao0228 @oliverhu
The text was updated successfully, but these errors were encountered: