Changes between Version 10 and Version 11 of CIVL-C

Timestamp:

05/16/23 12:10:04 (3 years ago)

Author:

Alex Wilton

Comment:

--

CIVL-C

@@ -65 +65 @@
 * Most JUnit tests should be focused on one specific class. Currently a large amount of our tests are more like "integration" tests rather than "unit" tests.
 * Integration tests running `civl verify` should be more fine-grained than "assert true" and "assert false." Perhaps utilize the return code suggestion made in the Features section of this list.
-* JUnit tests should minimize external prover calls as much as possible so that they don't take a lot of time to run.
+* JUnit tests should minimize external prover calls as much as possible because they can cause our testing framework to be unstable:
+  * Tests may fail if timeout is too low, and they may take too long if it is too high.
+  * External provers get updates which may change their results and hence the results of our tests.
+  * Different developers may have different prover versions, causing discrepancies between their test results.
 
 == Continue Doing