Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added first Necessary And Sufficient EraRule tests #4423

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

TimSheard
Copy link
Contributor

@TimSheard TimSheard commented Jun 18, 2024

Adds a new style of tests.

One writes a specification for a triple (Environment, State, Signal).
This specifucation specifies the minimal constraints to make the tests pass.
Tests can be run in 3 Modes 1) Apply, Invert, Remove
In Apply Mode, we test that the given tests are Sufficient to pass the STS rules.
in Invert Mode, we test that the given tests are Neccssary, as we invert them, and then Expect the test to Fail.
if the test doesn't fail, the test is Not necessary.
in Remove Mode, we remove the constraint. It may or may not pass, as one can get lucky. We use classify
to record which constraints can get lucky.

Checklist

  • Commit sequence broadly makes sense and commits have useful messages
  • New tests are added if needed and existing tests are updated
  • When applicable, versions are updated in .cabal and CHANGELOG.md files according to the
    versioning process.
  • The version bounds in .cabal files for all affected packages are updated. If you change the bounds in a cabal file, that package itself must have a version increase. (See RELEASING.md)
  • All visible changes are prepended to the latest section of a CHANGELOG.md for the affected packages. New section is never added with the code changes. (See RELEASING.md)
  • Code is formatted with fourmolu (use scripts/fourmolize.sh)
  • Cabal files are formatted (use scripts/cabal-format.sh)
  • hie.yaml has been updated (use scripts/gen-hie.sh)
  • Self-reviewed the diff

@MaximilianAlgehed
Copy link
Collaborator

I haven't had time yet to look in deep detail. However, what I'm a little bit concerned about here is that assertWith takes only a Status. The problem with that as I see it is that it means we alter all calls to assertWith with the same status flag. I would suggest doing something like this:

data Status a = Status { alterations :: Map a Alteration }
data Alteration = Apply | Invert | Remove
assertWith :: Alteration -> Term fn Bool -> Pred fn
assertWith Apply = assert
assertWith Remove = const TruePred
assertWith Invert = assert . not_

assertWithTag :: (IsBaseUniverse fn, Ord a)=> a -> Status a -> Term fn Bool -> Pred fn
assertWithTag tag status = assertWith (Map.lookupWithDefault Apply tag $ alterations status)

This way it would be possible to control the behaviour of different assertions in a neat way given we write some convenient syntax for constructing Status. Then comes the question of what type to use for the tag, we could simply do Status String for most use cases, or one could introduce a little enum type for one's specs.

@TimSheard
Copy link
Contributor Author

I completely agree with this. One thing missing is communicating, back to the test, what extra information is being used. That will be necessary to make good error messages if one of them fails.

@TimSheard TimSheard force-pushed the ts-failingPropTests branch 3 times, most recently from 397bbb3 to c3d134e Compare June 19, 2024 15:19

module Test.Cardano.Ledger.Constrained.Conway.NecessaryAndSufficient where

-- hiding (Pred,match,reify)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
-- hiding (Pred,match,reify)

Comment on lines +78 to +79
-- , ("voteDelegated", ppMap pcCredential pcDRep (dRepMap um))
-- , ("ptrs", ppMap ppPtr pcCredential (ptrMap um))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
-- , ("voteDelegated", ppMap pcCredential pcDRep (dRepMap um))
-- , ("ptrs", ppMap ppPtr pcCredential (ptrMap um))

Comment on lines +368 to +370
prop "GOVCERT Invert 1" $ govProp (Invert 1)
prop "GOVCERT Invert 2" $ govProp (Invert 2)
prop "GOVCERT Invert 3" $ govProp (Invert 3)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having to number each predicate that we want to flip and having to make a separate test for each number is a bit too much. Could this numbering be done behind the scenes somehow so that whoever is writing these constraints doesn't need to keep track of that?

assertExplain
["Status Nonmatching Invert (" ++ show n ++ " /= " ++ show m ++ " (" ++ show x ++ ")"]
x
assertWith n Remove x = assertExplain ["Status Remove " ++ show n ++ " (" ++ show x ++ ")"] TruePred
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it make sense to also remove the constraints one-by-one like we do with Invert?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants