Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate and simplify Bluepill config flags #457

Open
ravimandala opened this issue Aug 12, 2020 · 1 comment
Open

Consolidate and simplify Bluepill config flags #457

ravimandala opened this issue Aug 12, 2020 · 1 comment

Comments

@ravimandala
Copy link
Contributor

ravimandala commented Aug 12, 2020

Bluepill flags have grown in number and complexity over time. It is time to simplify the logic, deprecate some, merge behaviors. Here are a few proposals.

Some proposals:

  1. Flags that have grown complicated over time especially those that got deviated from its original meaning (Eg. only-retry-failed used for other purposes too)
  2. Flags whose defaults are good enough and are rarely overridden.
  3. Merge flags with similar or confusing purposes. (Eg. error-retries and failure-tolerance can be merged.)
  4. Rarely used flags, if any. (printf-config, test-bundle-path, additional-unit-xctests, additional-ui-xctests)
  5. Non-functional flags, if any.
  6. Optionally consolidate internal-only flags.

Comments and suggestions are welcome.

\cc @ob @RainNapper @jerrymarino @chenxiao0228 @oliverhu

@ob
Copy link
Member

ob commented Aug 12, 2020

I think what makes it hard to reason about the flags is that they are non-orthogonal (that is you can't reason about them independently of the other flags). How about focusing on the tests themselves and the possible results?

We have individual tests that might end up in one of four states: success, failure, crash, and timeout. Are there others?

And on each run, we want to be able to specify how much tolerance we have for each of those states right? We could do something like:

--timeout-tolerance=4 --failure-tolerance=2 --crash-tolerance=0 and mean allow 4 timeouts 2 failures and 0 app crashes. Would that make it simpler?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants