If there are exceptions during the iteration, we want to have a working HTML file with the iterations up to that point.
By flushing after each vector, we can ensure that the HTML is in a working state.
Switch from
... took X iterations, Y MV-multiplications and Z seconds.
to
... took X iterations, Y multiplications and Z seconds.
as MV-multiplications does not have a terribly intuitive meaning.
Previously, we'd print an 'Approximate result' in addition to the
exact rational number in exact mode:
Result (probability): 1/6
Approximate result: 0.16666666666666666
Result (probability): 1/6
Approximate result: 0.16666666666666666
Now, we rename that to 'As floating point' and
use BigRational.toApproximateString to print an
approximate result with ~ prefix, and an exact
floating point result without ~:
Result (probability): 1/6
As floating point: ~0.16666666666666666
Result (probability): 1/2
As floating point: 0.5
Now aligns with the default implementation in explicit.MDP: If there is a choice consisting of a self-loop,
produces value 0 for zero-reward state/choice and infinite value for positive-reward state/choice
(should be catched before in precomputations).
The previous behaviour (self-loops have infinite value) messes with maximal total reward computations with
Gauss-Seidel, e.g.
prism functionality/verify/mdps/rewards/total-reward-2.nm functionality/verify/mdps/rewards/total-reward-2.nm.props -explicit -gs -test -prop 3
for the PRISM test suite fails.
Also, align comments in MDP.mvMultRewJacMinMaxSingle.
For numerical results, support e.g.
// RESULT: ~0.1244123
to indicate that the result is known to be imprecise.
For the exact engine, provide better error reports:
- If result is not exactly as expected, check if it's numerically close
- If the expected result is marked as imprecise, don't consider as an error
The parametric engine works under the assumption that models
are well-defined, i.e., that parameter valuations will lead
to graph-preserving probabilities, etc.
If we encounter a point in a region where these assumptions
don't hold, we now treat this by splitting the region. Regions
with such an improper midpoint that are below the precision
threshold are considered to be undefined and further processing
is aborted.
In particular, this treatment ensures that we never perform
policy iteration on a model with parameter instantiation
that is not well defined / supported. E.g., if there
were negative rewards for such a point, policy iteration
would previously not necessarily terminate.
To check that, for all parameter valuations in a region,
(1) some threshold property holds or (2) that a given
scheduler is optimal, a set of constraints is checked.
Currently, only heuristic checking is implemented, i.e., determining
via sampling if the constraints hold. This can produce false positives
in the region computations, i.e., a region is not sufficiently split.
We add a warning if the region computation relied on the heuristic
constraint check and thus can be imprecise.
In particular, align output format with other engines and treat exact mode
specially (print rational numbers instead of functions, approximate values)
For parametric, output results per region.
Print not only the state index, but also the variable valuations for that state
(similar to the other engines).
Example for exact:
prism prism-examples/dice/dice.pm -pf 'filter(print, P=?[ F d=1])' --exact
=== Previously ===
Results (non-zero only) for filter true:
([0.0,1.0])0:={ 1 | 6 }1:={ 1 | 3 }2:={ 0 }3:={ 2 | 3 }4:={ 0 }5:={ 0 }6:={ 0 }7:={ 1 }8:={ 0 }9:={ 0 }10:={ 0 }11:={ 0 }12:={ 0 }
==================
=== Now ==========
Results (non-zero only) for filter true:
0:(0,0)=1/6 (~0.16666666666666666)
1:(1,0)=1/3 (~0.33333333333333337)
3:(3,0)=2/3 (~0.6666666666666666)
7:(7,1)=1 (1.0)
==================
Example for parametric:
=== Previously ===
Results (non-zero only) for filter true:
([0.0,0.5])0:={ ( -1 ) p + 1 }1:={ 1 }2:={ 0 }([0.5,1.0])0:={ p }1:={ 1 }2:={ 0 }
==================
=== Now ==========
Results (non-zero only) for filter true:
([0.0,0.5]):
0:(0)={ ( -1 ) p + 1 }
1:(1)={ 1 }
([0.5,1.0]):
0:(0)={ p }
1:(1)={ 1 }
==================
To ensure that the policy iteration (performed on an MDP where parameters have been replaced
by a parameter valuation) converges and converges on the right result in the presence of
zero-reward end components, we initialise the policy iteration for Rmin[F] with a
proper scheduler, i.e., that reaches the goal with probability one (see [BertsekasTsitsiklis91]).
Together with the previous commit, this fixes#4 and #15.
In the state eliminator, i.e., solving for a DTMC, a state that has probability
zero of reaching the target states (i.e., that can not reach the target state)
should get infinite reward.
Previously, the check for this looked at the returned set of collectStatesBackward(),
which always returns the whole state space (used for backward elimination order).
Now, we use the variant of collectStatesBackward() that only returns the states
that can reach the target set.
Additionally, we now compute the set of infinity states for Rmin/Rmax using
prob0a / prob0e, respectively, and ensure that they get value infinity.
As well, remove getNumTotalChoices() in favour of the getNumChoices() method
as used in the explicit models.
This will allow using the explicit model checkers prob0/prob1 methods.
Sometimes, we want to use the prob0/1... methods to generate schedulers instead of doing
the standard precomputations and don't want to have the normal log output.
Not inherited using inheritSettings()
In Java 9, there is a new system class java.lang.Module that is implicitly imported
everywhere and which clashes with the parser.ast.Module class, resulting in
compilation errors, as javac is not able to disambiguate between the two
automatically.
We are therefore more specific when referencing the PRISM parser's 'Module' class,
by using the full 'parser.ast.Module' name.
Switching between max and min when removing the negation was missing for MDPs.
Test case:
prism functionality/verify/mdps/ltl/simple_ltl.nm functionality/verify/mdps/ltl/simple_ltl.nm.props -exact -prop 3
from prism-tests.
Dave Parker