Skip to content

Latest commit

 

History

History
549 lines (445 loc) · 42.1 KB

UserGuide.md

File metadata and controls

549 lines (445 loc) · 42.1 KB

EvaluationLibrary v1.0.0 — User Guide

English · 简体中文 · Project overview

Installation, modelling exercises, result interpretation and complete class reference for EvaluationLibrary.

Contents

1. Start with the library

EvaluationLibrary measures simulation signals and events, checks requirements and reports results with explicit data qualification. It contains 75 public evaluation blocks, supporting result interfaces and functions, and nine self-contained examples. The only library dependency is Modelica Standard Library 4.0.0.

This guide follows a practical sequence: run an existing example, assemble a small model, vary its parameters, then interpret the result. The exercises use top-level within; clauses; save each in a separate .mo file, or adjust within when placing it in a named package. The English and Chinese manuals use the same Modelica class names and parameter names.

Three decisions before connecting a block

  • Quantity: which signal or event is being observed, and in which unit?
  • Observation: which window, sample grid and validity condition are appropriate?
  • Interpretation: is the output a descriptive metric, a requirement decision or a finite-set summary?

2. Load and run an example

  1. Extract the archive. Keep the EvaluationLibrary directory, its package.mo and all subdirectories together.
  2. Load Modelica 4.0.0 in your Modelica environment, then open EvaluationLibrary/package.mo. Avoid loading a second copy of the same package name.
  3. Open EvaluationLibrary.Examples.MultiSignalAssessment. Inspect its Diagram view, then use its stored experiment: start 0 s, stop 2.2 s, output interval 0.01 s and tolerance 10⁻⁷.
  4. Run the model and read the console summary. In the result browser, inspect assessment.y, assessment.minimum, assessment.maximum and report.overall.
Channel Input and criterion Expected interpretation
angle Sine amplitude 0.1 rad; absolute value ≤ 0.2 rad Satisfied
position Ramp to 2 m; absolute value ≤ 1 m Violated once the ramp exceeds 1 m
current Constant 0.3 A; absolute value ≤ 0.5 A Satisfied

The observation ends at 2 s and excludes that endpoint. The simulation runs slightly longer so the closed results can be inspected. The table gives analytical expectations from the source definitions; compare them with your simulator output.

OpenModelica command-line equivalent

Run these commands from the directory that contains the EvaluationLibrary folder. The packaged Resources/Scripts/export_smoke.mos follows the same working-directory convention.

loadModel(Modelica, {"4.0.0"});
loadFile("EvaluationLibrary/package.mo");
checkModel(EvaluationLibrary.Examples.MultiSignalAssessment);
simulate(EvaluationLibrary.Examples.MultiSignalAssessment,
  startTime=0, stopTime=2.2, numberOfIntervals=220,
  tolerance=1e-7);
getErrorString();

Check that the library loads and that the model check succeeds in your chosen Modelica environment before running the examples.

3. Build a signal assessment

Create a new model named FirstAssessment. Add a sine source, Statistics.SignalAssessment and Results.ConsoleWriter. In a graphical editor, connect signal.y to assessment.u[1], then connect the complete assessment.y array to report.u. The following complete model provides the same connections.

within;
model FirstAssessment "Assess a sine signal against an acceptance band"
  Modelica.Blocks.Sources.Sine signal(amplitude=0.8, f=1);
  EvaluationLibrary.Statistics.SignalAssessment assessment(
    n=1,
    metricId={"signal"},
    unitName={"1"},
    lower={-1},
    upper={1},
    windowStop=2,
    samplePeriod=0.01);
  EvaluationLibrary.Results.ConsoleWriter report(
    n=1, runId="first_assessment");
equation
  connect(signal.y, assessment.u[1]);
  connect(assessment.y, report.u);
  annotation(experiment(StartTime=0, StopTime=2.1,
    Interval=0.01, Tolerance=1e-6));
end FirstAssessment;
Setting Choice in this exercise Effect
n 1 One interpreted signal and one complete result.
metricId, unitName {"signal"}, {"1"} Identify the result and declare its unit.
lower, upper −1, +1 Inclusive acceptance band.
windowStart, windowStop 0, 2 s Observe [0, 2) s.
samplePeriod 0.01 s Sample the reported extrema on a 200-point grid.
sampled false (default) Use continuous event relations for bound checks.

Inspect, then vary

  1. Plot assessment.evaluated[1], assessment.minimum[1] and assessment.maximum[1]. The expected extrema are −0.8 and +0.8.
  2. After collection closes, inspect assessment.y[1].valueAvailable, dataStatus and verdict. With the stated setup, expect complete evidence and a satisfied requirement.
  3. Change signal.amplitude to 1.2 and rerun. The band is violated; inspect everViolated, firstViolationTime and violationDuration[1].
  4. Set sampled=true and compare the result. Only sample ticks determine bound violations in this mode; violationDurationAvailable becomes false.

Add channels or change the interpretation

Set n to the number of channels and supply arrays of the same length for identifiers, units and criteria. Each channel can use SignedValue, AbsoluteValue, Difference or WrappedAngleDifference. Use the wrapped form only for angle differences in radians. It evaluates atan2(sin(u-reference), cos(u-reference)). Unit labels are metadata and do not convert degrees to radians.

4. Check a response deadline

The request and response are distinct observations. For a deadline check, connect the request event and the measured action feedback to Response.EventAssessment. The default mode is RequestResponse.

within;
model FirstDeadline "Check the delay between a request and its response"
  Modelica.Blocks.Sources.BooleanStep request(startTime=0.2);
  Modelica.Blocks.Sources.BooleanStep response(startTime=0.3);
  EvaluationLibrary.Response.EventAssessment assessment(
    n=1,
    metricId={"response_delay"},
    deadline={0.2},
    windowStop=1);
  EvaluationLibrary.Results.ConsoleWriter report(
    n=1, runId="first_deadline");
equation
  connect(request.y, assessment.request[1]);
  connect(response.y, assessment.response[1]);
  connect(assessment.y, report.u);
  annotation(experiment(StartTime=0, StopTime=1.1,
    Interval=0.01, Tolerance=1e-6));
end FirstDeadline;

The request occurs at 0.2 s and the response at 0.3 s. The expected matched delay is 0.1 s, within the 0.2 s deadline. Inspect assessment.matched[1], timedOut[1], censored[1] and y[1].verdict. Change the response start to 0.5 s to observe an expired request.

Mode Observed inputs Interpretation
Occurrence Rising edges at request; connect all declared inputs. Event count; a descriptive metric.
RequestResponse Separate request and response rising edges. First-in, first-out matching with one response per request.
Recovery Danger at request, recovery-band condition at response. First danger followed by uninterrupted valid recovery dwell.
  • Requests at the same instant are registered before responses, so zero-delay matching is possible. A response exactly at its deadline is accepted inside the observation window.
  • The window stop is excluded. An unresolved request whose deadline is at or after the exclusive window stop is censored; increase the observation window when longer follow-up is required.
  • An invalid observation gap breaks the event history; queue overflow makes the evidence invalid. Set capacity for the expected number of pending requests.
  • Before recovery is confirmed, loss of validity, return of danger or departure from the recovery band restarts the dwell interval. The first confirmed recovery is latched. With useRecoverySignal=true, configure recoveryReference, recoveryTolerance and connect recoverySignal. The always-declared Boolean inputs still need bindings.

After a complete observation, EventAssessment marks a RequestResponse or Recovery task with no qualified request as NOT_APPLICABLE. An Occurrence task remains a descriptive event count.

ResponseDelay, ResponseTime and ResponseDeadline expose different specialised definitions. In particular, ResponseTime accepts Boolean conditions, and ResponseDeadline does not use the same no-demand outcome as EventAssessment. Read their individual documentation when selecting them.

5. Choose the right class

Task Start with Connection or configuration
Related numeric signals Statistics.SignalAssessment Real inputs to u; result array y to a writer.
Related event tasks Response.EventAssessment Request and response arrays; per-task mode and deadline/dwell.
Continuous-time mean or RMS Statistics.TimeMean, TimeRMS Set the actual window parameters and validity input.
Sampled moments Statistics.SampledMoments One sample grid for mean, RMS and population variance.
Single requirement Requirements.UpperBound, WithinBand Configure limits; optionally enable useValidInput and connect valid; inspect verdict and margin separately.
Diagnostic labels Comparison.LabelPerformance Bind observed and reference labels; connect the eight-result y array.
Finite sampled spectrum Frequency.DiscreteSpectrum Match nSamples to the window and sample grid.
Final report Results.ConsoleWriter Connect complete results; set n and optional required.

Library structure

Package Scope
Signals Differences, relative errors, angle wrapping, finite differences and validity gates.
Statistics Continuous and sampled statistics, shared multi-channel assessment and accumulated quantities.
Response Event counts, request–response matching, recovery, dwell and transient response measures.
Requirements Numeric and Boolean conditions, bands, persistence, deadlines and resource margins.
Comparison Paired signals, channel consistency, finite-run summaries and classification performance.
Frequency Finite sampled spectra, single tones, harmonic distortion, gain/phase and decay ratios.
Results Aggregate qualified results, calculate scores and write console, CSV or text output.
Utilities Result connectors, records, enumerations, formatting functions and internal observation helpers.
Examples Nine small, self-contained examples of the public API.
UsersGuide Overview, getting started, conventions and limitations inside the modelling tool.

Nine existing EvaluationLibrary model icons

Original model icons, exported from their Modelica graphics.

The complete class reference contains every class, including packages, example models, types, functions and internal helpers. Internal helpers support the public blocks and are not the normal entry point for model assembly.

6. Interpret qualified results

MetricOutput connects directly to MetricInput; both carry 30 fixed result and evidence fields. MetricResult represents the same information as a record. Configure metricId, unitName and any requirement metadata at the producer, and match the result-array size to the receiving block’s n.

A result separates four questions: is a number available, was the observation complete and valid, has the requirement been decided, and has collection closed? Read the corresponding fields independently.

Field Read it as
valueAvailable Whether value has a meaningful numerical interpretation.
dataStatus Coverage and validity of the observation.
verdict Requirement decision; descriptive statistics can remain undecided.
complete Collection is closed; this does not imply complete evidence or a satisfied requirement.
everViolated A qualified violation witness has been recorded.
eventCountsAvailable, levelAvailable Whether the event counters or optional engineering category may be interpreted.

Numerical values and observation metadata

Fields Interpretation
metricId, requirementId, requirementVersion Stable identities and optional requirement metadata supplied by the producer.
value, valueAvailable, unit Numerical result, its availability and its stated unit. Unit labels do not convert values.
windowStart, windowStop, observedDuration, validDuration Observation extent and the duration available to the producer.
samplePeriod, expectedSamples, receivedSamples, validSamples Sample-grid definition and coverage for sampled evidence.
everViolated, firstViolationTime A retained violation witness; interpret the instant only when the flag is true.
eventCountsAvailable, eventCount, matchedCount, timedOutCount, censoredCount Event counters supplied by producers that support them.
levelAvailable, level, level0Upper, level1Upper Optional engineering category and its configured thresholds.
evidenceKind, message Evidence interpretation and a readable explanation.

Status codes in connectors

MetricInput and MetricOutput carry integer enumeration codes. MetricResult uses the enumeration types. The code order is not the aggregation priority.

Code DataStatus Verdict
1 PENDING UNDECIDED
2 COMPLETE SATISFIED
3 INCOMPLETE VIOLATED
4 INVALID INVALID
5 NOT_APPLICABLE NOT_APPLICABLE

A result can be COMPLETE / UNDECIDED when it is a statistic with no requirement. An unavailable number is not zero evidence. A closed collection can be incomplete when the requested observation could not be finished.

Aggregate only the intended requirements

VerdictAggregator and the writers use required to select entries for the overall decision; individual entries remain visible. Non-applicable entries are excluded. Among included entries, invalid data dominates a violation, which dominates an undecided result. All included entries must be satisfied to obtain SATISFIED; no included entries gives NOT_APPLICABLE.

If a descriptive statistic is included as required, its undecided verdict can keep the overall result undecided. Set the corresponding required[i] entry to false when it should be reported without contributing a requirement decision.

7. Write console, CSV and text output

All writers accept complete results through u[n]. Configure metadata at the producing block, then connect the whole result or result array. A scalar producer uses connect(metric.result, writer.u[1]). Collection is written once when every result has closed, or at normal simulation termination.

EvaluationLibrary.Results.CSVWriter csv(
  n=1,
  runId="trial",
  resultSetId="signals",
  outputDirectory=".");
// Add to the equation section:
connect(assessment.y, csv.u);

This declaration and connection belong inside an existing model, such as FirstAssessment; they are not a complete standalone model. The destination is trial_signals.csv in the simulator working directory.

Writer Output What to inspect
ConsoleWriter Console summary in declaration order overall, written
CSVWriter Header and rows with runId plus all 30 result fields filePath, written, skippedExisting
TextWriter One formatted line per result filePath, written, skippedExisting
  • Create the destination directory before simulating. Relative paths are resolved by the simulator working directory.
  • The default WritePolicy.RejectExisting preserves an existing file and sets skippedExisting. Select WritePolicy.Overwrite only when you intend to replace that file.
  • Use different runId, resultSetId or filePath values when retaining multiple runs.
  • Unavailable values are formatted as N/A where appropriate. Export success is independent of the requirement verdict. A file-system error can stop a simulation through the Modelica Standard Library file functions.

8. Configure windows and sampling

Use the parameters of the selected class

Interface pattern Typical parameters Validity
Shared assessment blocks windowStart, windowStop; signal sampling uses sampleStart Optional valid via useValidInput
Window-configurable individual blocks windowMode, windowStart, windowStop Optional validity; ExternalSingle exposes a window input
Fixed-window specialised blocks windowStart, windowEnd; sampled classes may expose sampleOffset A declared valid input must be bound

The windows are half open. A tick exactly at the start is included; a tick exactly at the stop is excluded. Blocks with ExternalSingle accept the first contiguous external window. A later reopening invalidates that observation. Sample-grid origins and external window boundaries are independent settings.

Match frequency buffers to the sample grid

For SingleFrequency, DiscreteSpectrum and HarmonicDistortion, explicitly configure the finite sample count. For a zero-based grid, nSamples=200, samplePeriod=0.01 and windowStop=2 give 200 samples in [0, 2) s. Their default combination with windowStop=5 would provide 500 ticks and exceed a 200-sample buffer. Do not leave that combination unadjusted.

EvaluationLibrary.Frequency.SingleFrequency tone(
  frequency=2,
  nSamples=100,
  samplePeriod=0.01,
  sampleStart=0,
  windowStart=0,
  windowStop=1,
  metricId="tone_amplitude",
  unitName="V");

This is the configuration used by the small frequency example. It estimates a finite-window tone; the result depends on the sample grid, window function and coherence with the tone. The simulator output interval is not the evaluator sample period.

Read validity rules locally

Different helpers observe validity differently: WindowEvidence latches invalid input during the active window, whereas SampledEvidence checks scheduled ticks. Public blocks can add their own finite-value and denominator checks. Their dataStatus and availability fields are the authoritative qualification for that producer.

9. Complete class reference

The following tables cover all 147 Modelica classes, including the 75 public evaluation blocks, packages, example models, result types, functions and internal helpers. Each class name links to its source file. Read the class documentation in your modelling environment for its exact inputs, parameters and numerical definition.

9.1 Library

Class Kind Purpose
EvaluationLibrary Package Signal metrics, requirement checks and qualified result reporting

9.2 Signals

Class Kind Purpose
AbsoluteError Block Absolute difference between two signals
Difference Block Signed difference between two signals
FiniteDifference Block Backward difference of consecutive valid samples
RelativeError Block Signed relative error with a reference magnitude limit
ValidityGate Block Signal pass-through with range and age qualification
WrappedAngleError Block Signed angular error wrapped to one revolution
EvaluationLibrary.Signals Package Signal transformations and data validity

9.3 Statistics

Class Kind Purpose
ControlEffort Block Integrated normalised actuator effort and rated-load duration
EnergyEfficiency Block Energy efficiency over an observation interval
GatedPeak Block Sampled peak severity during a hazard condition
Integral Block Signed integral over an observation interval
LinearTrend Block Least-squares linear trend of uniformly sampled values
SampleExtrema Block Sampled extrema, occurrence times and peak-to-peak range
SampleMean Block Arithmetic mean of uniformly sampled values
SampleRMS Block Root mean square of uniformly sampled values
SampleVariance Block Sample variance with a configurable degrees-of-freedom correction
SampledMoments Block One-pass sampled mean, population variance and RMS
SignalAssessment Block Multi-channel bounds assessment with shared sampling
TimeMean Block Time-weighted mean of a continuous signal
TimeRMS Block Time-weighted root mean square of a continuous signal
TimeVariance Block Time-weighted population variance of a continuous signal
TotalVariation Block Sum of absolute changes between consecutive samples
ZeroCrossingOscillation Block Mean period between observed upward level crossings
EvaluationLibrary.Statistics Package Continuous and sampled signal statistics

9.4 Response

Class Kind Purpose
ConditionOccupancy Block Measure condition duration and occupancy fraction
EventAssessment Block Assess multiple event tasks in a shared observation window
EventIntervals Block Measure intervals between successive event rising edges
FallTime Block Measure the 10% to 90% progress time of a falling step
FirstOccurrence Block Record the first valid true condition in a window
MovingAverage Block Compute a continuous mean over a trailing window
OccurrenceCount Block Count valid rising events within an observation window
OvershootUndershoot Block Measure sampled overshoot and undershoot of a transition
RecoveryTime Block Measure recovery delay after a sustained valid return
RepeatedResponse Block Measure repeated request-response delays using a bounded FIFO
ResponseDelay Block Measure the delay between the first request and response edges
RiseFallTime Block Measure the elapsed time between two directed level crossings
RiseTime Block Measure the 10% to 90% progress time of a rising step
SampleHistory Block Store a bounded sequence of samples and scheduled times
SettlingTime Block Confirm the final sustained entry into an error band
SteadyStateError Block Evaluate sampled error bias and residual spread
TrackingError Block Compute continuous integral and RMS tracking errors
ViolationDuration Block Measure total and longest continuous true durations
EvaluationLibrary.Response Package Event timing, response measurements and duration statistics

9.5 Requirements

Class Kind Purpose
BooleanRequirement Block Check a Boolean condition throughout an observation window
CircularErrorBound Block Check an inclusive bound on a wrapped angular error
LabelEpisodeDelay Block Measure detection latency within the first labelled episode
LowerBound Block Check an inclusive lower bound
NumericRequirement Block Apply numerical acceptance bounds to an existing metric
ObjectiveCompletion Block Check objective completion with a failure veto
PersistenceLimit Block Limit the duration of each continuous true interval
ResponseDeadline Block Check repeated request-response deadlines using FIFO pairing
ResponseTime Block Check the response latency after the first request
SignedMargin Block Compute a normalised resource margin and accumulated deficit
UpperBound Block Check an inclusive upper bound
VectorBounds Block Check independent numerical bounds for multiple channels
WithinBand Block Check an inclusive interval
EvaluationLibrary.Requirements Package Numerical, logical and timing requirements

9.6 Comparison

Class Kind Purpose
ChannelConsistency Block Instantaneous range of valid simultaneous channels
Improvement Block Absolute and relative improvement between paired results
LabelPerformance Block Sampled binary classification counts and performance ratios
MultiRunSummary Block Summary of a configured set of comparable result values
PairedArrivalSkew Block Time difference between two first observed arrivals
PairedDifference Block Signed difference between comparable completed results
SampleCorrelation Block Pearson correlation of paired uniform samples
EvaluationLibrary.Comparison Package Comparison of signals, event times and result sets

9.7 Frequency

Class Kind Purpose
DecayRatio Block Mean decay ratio of sampled positive peaks
DiscreteSpectrum Block Single-sided discrete Fourier amplitude spectrum
HarmonicDistortion Block Total harmonic distortion over selected harmonics
SingleFrequency Block Amplitude and phase at a selected frequency
ToneResponse Block Finite-record input-output gain and phase difference
EvaluationLibrary.Frequency Package Finite-record frequency and oscillation analysis

9.8 Results

Class Kind Purpose
CSVWriter Block Write connected result records to a CSV file
ConsoleWriter Block Print one summary of connected result records
EvidenceMonitor Block Observation coverage and data-validity monitor
GradeByThresholds Block Engineering grade from a normalised score
MinimumMargin Block Minimum signed margin across a finite set
SatisfactionFraction Block Fraction of satisfied verdicts in a finite set
TextWriter Block Write connected result records to a text file
VerdictAggregator Block Conjunction of required verdicts with retained result records
WeightedScore Block Weighted mean of normalised scores
WorstCase Block Value with the greatest absolute magnitude in a finite set
EvaluationLibrary.Results Package Result aggregation, qualification and reporting

9.9 Utilities

Use the public evaluation blocks for normal model assembly. The Internal classes support their implementation.

Class Kind Purpose
Functions.Aggregate Function Combine required verdicts with an explicit priority
Functions.EvidenceMessage Function Describe result availability and qualification
Functions.FormatRow Function Format a result as a CSV record or concise text
Functions.FormatSummary Function Format a result set and its overall verdict
Functions.QualifiedCode Function Qualify a verdict using its data status
Functions.QuoteCSV Function Quote and escape a CSV field
Functions.SafeIdentifier Function Check a filename-safe identifier
Functions.SampleCount Function Count uniform samples in a half-open interval
Functions.StatusFromCode Function Convert an integer code to a data status
Functions.StatusText Function Return the text label of a data status
Functions.ToVerdict Function Derive a verdict from data status and a violation flag
Functions.Tone Function Project uniformly spaced samples onto a sinusoid
Functions.VerdictFromCode Function Convert an integer code to a verdict
Functions.VerdictText Function Return the text label of a verdict
Functions Package Functions for result qualification, formatting and spectral analysis
Interfaces.MetricInput Connector Input connector for a structured evaluation result
Interfaces.MetricOutput Connector Output connector for a structured evaluation result
Interfaces Package Directional connectors for structured evaluation results
Internal.AdvanceFIFO Internal helper Update a bounded request-response queue
Internal.ContinuousAccumulator Internal helper Integrate a signal and its squared value while enabled
Internal.ContinuousEvidence Internal helper Track a fixed continuous observation window
Internal.LogCompletion Internal helper Print the outcome of a completed output operation
Internal.SampleAccumulator Internal helper Accumulate sample moments and extrema
Internal.SampledEvidence Internal helper Track validity and coverage on a uniform sample grid
Internal.SpectrumBuffer Internal helper Store a bounded sequence of samples
Internal.ToneSpectrum Internal helper Calculate a finite one-sided amplitude and phase spectrum
Internal.WindowEvidence Internal helper Track one observation window and its data coverage
Internal.WriteRows Internal helper Write a result set to a file or the console
Internal.csvQuote Internal helper Quote and escape an internal CSV field
Internal.oneSidedDFT Internal helper Calculate non-negative discrete Fourier transform bins
Internal Internal helper Internal accumulation, observation and output helpers
Types.AssessmentDirection Type Direction of inclusive numerical acceptance bounds
Types.DataStatus Type Observation coverage and data-validity status
Types.EventMode Type Event-assessment task
Types.EventRole Type Descriptive role of an observed event
Types.EvidenceKind Type Basis of an evaluation result
Types.Grade Type Engineering grade assigned to a qualified score
Types.InputInterpretation Type Interpretation of a real signal before assessment
Types.MetricResult Record Structured metric result with qualification and observation metadata
Types.NumericComparison Type Comparison used to interpret a numerical requirement
Types.ReportFormat Type Format of a result report
Types.Sense Type Direction of improved numerical performance
Types.SpectralWindow Type Weighting for a finite discrete Fourier transform
Types.Taper Type Weighting for a finite sinusoidal projection
Types.Verdict Type Requirement outcome independent of observation coverage
Types.WindowMode Type Fixed or single externally gated observation window
Types.WritePolicy Type Policy for an existing output file
Types Package Result record and enumeration types
EvaluationLibrary.Utilities Package Result types, connectors and shared evaluation functions

9.10 Examples

Class Kind Purpose
ContinuousAndSampleStatistics Model Compare continuous and uniformly sampled statistics
DiagnosticLabelPerformance Model Evaluate Boolean diagnostic decisions against reference labels
FrequencyOrFileOutput Model Estimate a sinusoidal amplitude and write its result
MultiResultSummary Model Combine required verdicts while retaining individual results
MultiSignalAssessment Model Assess three numerical signals in a shared window
RecoveryAndDwell Model Compare successful and unsuccessful recovery
RepeatedEventMatching Model Match repeated request and response edges in FIFO order
RequestResponseDeadline Model Compare timely, late, missing and undemanded responses
WrappedAngleAndSeverity Model Evaluate wrapped angular errors and configured severity levels
EvaluationLibrary.Examples Package Runnable examples of evaluation and result reporting

9.11 UsersGuide

Class Kind Purpose
Conventions Class Units, observation windows and result interpretation
GettingStarted Class Load the library and run a first assessment
Limitations Class Observation and numerical limits
Overview Class Library scope and model groups
EvaluationLibrary.UsersGuide Package User guide

10. Continue with the examples

The example models are small entry points. Each includes its sources, evaluation and reporting, so no external input connections are needed. Run the stored experiment and inspect the quantities named in the model documentation.

Example Focus
ContinuousAndSampleStatistics Compare continuous and uniformly sampled statistics
DiagnosticLabelPerformance Evaluate Boolean diagnostic decisions against reference labels
FrequencyOrFileOutput Estimate a sinusoidal amplitude and write its result
MultiResultSummary Combine required verdicts while retaining individual results
MultiSignalAssessment Assess three numerical signals in a shared window
RecoveryAndDwell Compare successful and unsuccessful recovery
RepeatedEventMatching Match repeated request and response edges in FIFO order
RequestResponseDeadline Compare timely, late, missing and undemanded responses
WrappedAngleAndSeverity Evaluate wrapped angular errors and configured severity levels

FrequencyOrFileOutput includes an active CSV writer. It attempts to write frequency_tone.csv in the simulator working directory and preserves an existing file by default.

11. Troubleshooting and scope

Symptom Check
A model reports an unbound Boolean input. Some classes require valid explicitly. Connect a Boolean validity source; use a constant true only when that assumption is appropriate. Bind all always-declared event inputs.
A complete statistic has an undecided verdict. It may have no requirement. Use a requirement block or the assessment criteria when a pass/fail decision is needed.
Results stay pending or become incomplete. Check window parameters, simulation stop time, sample count, unresolved requests and censored follow-up.
A spectrum is invalid with default parameters. Match nSamples and the exact number of ticks inside the half-open window.
No new CSV file appears. Inspect filePath, the simulator working directory, directory existence and skippedExisting.
An instantaneous comparison is unavailable after the window. ChannelConsistency reports the current range and does not retain the maximum range. Select a separate statistic if a final retained measure is required.
An improvement result has an unexpected sign. Improvement.result is the inherited candidate-minus-reference value. Direction-adjusted outputs are absoluteImprovement and relativeImprovement.
A summary is complete although some runs were excluded. MultiRunSummary summarises accepted inputs. Compare nValid with nInputs.
  • Finite samples do not prove strict continuous extrema. Finite candidate sets do not establish a continuous-domain global optimum or an occurrence probability.
  • The optional levels 0, 1 and 2 in SignalAssessment use sampled absolute interpreted peaks and configured thresholds. They are engineering categories. GradeByThresholds exposes its separate A/B/C/D grade through grade.
  • SampleHistory compacts accepted values after invalid ticks while stamps describes the planned grid. Do not pair those arrays as valid observations when the history is invalid.
  • Numerical behaviour depends on the chosen solver, event handling, sample grid and input model. Recheck example outputs when moving to a different Modelica environment.

12. Documentation and further reading

Use the README for a project overview, the in-tool UsersGuide for a short introduction, and each class’s Documentation for its precise behaviour. The following resources provide additional Modelica guidance.