We now count non-convergence (i.e., error message contains 'did not converge') as a sub-type of failures.
Additionally, count skipped export-runs and skipped duplicate runs as sub-types of skipped tests.
Print test statistics also in the (not particularly useful) case that the timeout is set to 0 by testing against None instead to see if a timeout was set.
The occurence of a line with 'Error:' does not necessarily imply a
failed test result, e.g., for test cases that test against the error
messages.
So we revert the previous change related to the printing of 'Error' lines
and tweak the handling in verbose-test mode some more.
In non-test mode, PRISM and prism-auto both write to stdout, without
prism-auto seeing/processing the output of PRISM. If the output of
prism-auto is piped to another program or to a file, the prism-auto
output is buffered. Then, the output by prism-auto (e.g., printing the
command lines) is not properly synchronized with the output of the
PRISM instances.
So, we flush stdout at appropriate locations.
Additionally, on timeout we prepend a '\n' to ensure that the timeout
message starts at a new line (in particular for the common case of a
timeout during explicit model building, where there is no newline from
PRISM until the model is fully built).
When using -x to add additional options, i.e., to force a specific engine,
often runs of PRISM are effectively duplicated, e.g., if there are .args files
that select multiple engines.
Using the --skip-duplicate-runs, prism-auto tries to clean up the argument list,
remove switches that don't have an effect and to detect duplicate runs, only actually
executing the first one.
Option to allow skipping PRISM runs that do exports.
This is useful when overriding the engine, as the exports differ slightly
between engines and some export options are not supported.
In nailgun mode, if --ngprism is not specified, we derive the location of the ngprism
binary from the --prog setting: We simply replace the file part with ngprism, as usually
the ngprism binary is located in the same directory as the prism startup script.
Sometimes, if the prism-auto scripts gets interrupted, an existing
nailgun server is not properly shut down and might break a subsequent
prism-auto run.
(1) use dedicated printColoured method, which suppresses
colouring if isColourEnabled() is false (e.g., when piping to a file)
(2) use dedicated colour (light red) for warnings, as before
git-svn-id: https://www.prismmodelchecker.org/svn/prism/prism/trunk@12003 bbc10eb1-c90d-0410-af57-cb519fbb1720
The previously used filecmp.cmp opens the files to be compared in 'rb'
mode, i.e., it will tell us that two files that differ only in the
line-ending encoding (CRLF vs LF) are not equivalent. However, we'd
like to get the export tests to succeed on Windows, regardless of the
line endings. So, we provide our own file comparison method that opens
the file in 'rU' mode (universal newline mode), which converts all the
newline encodings to '\n' transparently.
git-svn-id: https://www.prismmodelchecker.org/svn/prism/prism/trunk@11880 bbc10eb1-c90d-0410-af57-cb519fbb1720
Somehow, PRISM can not open a NamedTemporaryFile created on Windows
(see issue prismmodelchecker/prism#11) when passed the filename via
the -mainlog parameter.
So, on Windows, we fall back on the old method of capturing stdout
directly via the Popen call. As this does not work with nailgun (the
C printfs go to the nailgun server stdout), we currently don't allow
nailgun use on Windows.
git-svn-id: https://www.prismmodelchecker.org/svn/prism/prism/trunk@11879 bbc10eb1-c90d-0410-af57-cb519fbb1720
PRISM recognizes the filename 'stdout' as special and exports
to standard output, so prepending a directory by prism-auto
does not make sense.
git-svn-id: https://www.prismmodelchecker.org/svn/prism/prism/trunk@11592 bbc10eb1-c90d-0410-af57-cb519fbb1720
Joachim Klein