Release Notes for TestWeaver 3.5.20

Since 3.5.0 Windows XP is no longer supported. Since 3.3.0 all QTronic installers are signed with a trusted certificate. If Windows shows an „unknown publisher“ warning dialog when trying to install, please download a new installer from https://resources.qtronic.de.

Changes in 3.5.20 relative to 3.5.10 (September 2018)

  • The experiment setup accepts now definition of setup/teardown commands. If such commands are defined, they are executed before the simulation starts, respectively after the simulation was ended. Possible use cases: start Matlab/Simulink with the sim_loop.m to run (passive) simulations in Simulink or start a build script, etc. Usage example in project_car/exp_simulink_python.
  • A plugin for Jenkins has been published on the Jenkins web page.
  • The performance of the RML watchers used with the Excel-connection in Silver has been improved.
  • If you change the zoom-level of the TestWeaver browser the change will be saved as global setting.
  • Test-database scenarios exported from an experiment can be imported back in the same experiment now (was prohibited until 3.5.20).
  • Beside the .mdf files, now also files with the .dat extension can be applied as MDF-stimulus (scripting SUT) to Silver.
  • Solved a problem related to network licenses used on terminal server.
  • Documented in the user guide the steps required to do a manual installation of network licenses.
  • Fixed a bug that reported a failing license check when checking experiments with TestWeaver light.
  • Several other bug-fixes and small improvements.

Changes in 3.5.10 relative to 3.5.0 (March 2018)

  • Network Licenses
    • Improved the robustness of the license checks in networks with slow communication rates, such as when using VPN.
  • Bugs fixed
    • In some cases, the scenario generation could have been aborted in 3.5.0 with the error message „Number of parameter changes is not yet known“.
    • When a SUT Parameter Definition file was not specified, the test reports for Jenkins reported failed consistency checks for the experiment Configuration Parameters.
    • The Python instrumentation was added with a wrong path in cases when the Silver .sil file and the .sil running directory were not at the same location.
    • A few more smaller bugs have been fixed as well.
  • Requirement Modelling Language
    • min(v1, ... vn)max(v1, ... vn) mathematical functions with a variable number of arguments, have been added.
    • Warn in the RML editor if the currently displayed watcher is not used by the current experiment.
    • Warn before attempting to rebuild the watchers for an experiment that already stores test results.
    • Documented supported VisualStudio C++ compilers.
    • Experiments can specify to lock a certain compiled RML module without further rebuilds or @config checks. This can be used to specify an experiment that adds a compiled RML module to the test, without the need to distribute the RML source files of the watchers.
  • User Interface
    • Preserve the scroll position after adding a comment in a scenario report.
    • Display as tool-tip in the embedded editors the path to the currently edited file.
  • Command Line Interface
    • Use --import-silver-parameters file to import Silver configuration parameters (for experiments that use Silver as execution platform). Use, for instance, in conjunction with --unit-test to pass Silver parameters in tests that run with Jenkins.
  • Python instrumentation
    • var.stepCountvar.prevStepCountvar.prev2StepCount – measure the duration of a variable value in macro-steps since the previous value change. Similar to the duration measurement in seconds with var.durationvar.prevDurationvar.prev2Duration. For accessing the measurement in macro-steps the variable’s history size must be configured above 1 – for instance with var.ensure_history_size(1).
    • chain, cycle, repeat from itertools are now automatically imported when you import from qtronic.testweaver.instruments. The functions are useful for defining generators, e.g. with a Measurement instrument.

What’s new in TestWeaver 3.5 (December 2017)

A presentation of some of the new features can be viewed here (HTML5) or here (PDF). For a more complete list of changes please consult the text below.

  • Requirement Modelling Language
    • This is our main new feature of this release. It comes in the aid of development and test engineers by providing an easy to use, yet unambiguous, requirement specification language that can be translated automatically into executable code (C/C++). The following features are supported by RML:
    • Easy to write and understand: RML does not require any prior programming background. The language is simple and naturally easy to use. The code is almost self-explanatory.
    • Real time monitoring: The language is designed to be computationally inexpensive and allows to monitor thousands of requirements in real time or faster. The generated watcher code can be compiled for simulation with Silver or can be built as a Functional Mock-up Unit (FMU) that can be run with other simulators as well.
    • Use on-line or off-line: Because the watchers are independent from the test stimulus and because their evaluation can be done in real-time, many use-cases can be supported, from off-line measurement evaluation to automated large coverage testing. A new use case supported by RML is the online requirement verification: interactively guiding a car driver about the manoeuvres necessary to verify the requirements in the car. The demo rml_to_Excel released with Silver 3.5.0 illustrates this use-case.
  • Demo Projects
    • New: project_regression illustrates how to define regression tests that compare the simulation results with pre-recorded reference measurements using absolute and relative tolerances for the signal comparison.
    • New: project_parameter_study illustrates how to define parameter studies with TestWeaver and how to vizualize the evaluation results. If you check this project, be prepared to find a nice surprise inside!
    • RML inside: all experiments with Silver from project_carproject_tank and project_lights demonstrate the usage of the new RML requirement watchers that replace the previously used Python watchers. You might also note that the Predefined Testsreport gives now more details about the evaluation results.
  • User Interface
    • Live report update: A new button on the tool bar allows to trigger the automatic update of the displaied reports during the simulation – this includes, of course, the heatmaps and all the other diagrams that can be displaied by TestWeaver.
    • Templates for RML/Python/CSV files: The browser that shows the contents of a directory allows you to add or delete files. New files can be generated with a filled-in template code, depending on their type: RML, Python or CSV.
    • Autocomplete SUT variable names: When Silver is used as execution platform we can automatically extract a context file with all SUT names, as well as enumeration names for discrete variables. This can be used for autocompletion in the embedded editors for Python and RML. Use:
      • CONTROL-SPACE to get quickly the autocomplete list
      • SHIFT-SPACE to get a list in which you can search for names with wildcard expressions, using the same conventions as in Silver.
    • Embedded RML editor: Featuring: syntac highlighting, syntax checking, auto-completion for functions and variable names, quick help, connection to build commands.
    • Plotter: The plotter received several improvements.
    • Links and images in the user assessments: Right-click inside an editable comment field opens a menu with which you can select an object that is added in the comment, for instance as an HTML link. This can either be an instrument (e.g. a watcher or an alarm), a report, an image or some other ressource from the file system.
  • Interface to Jenkins
    • We export now a bit more details about the failed checks.
    • All reports built by TestWeaver can be exported in HTML format to Jenkins.
  • Experiment Specification
    • Parameter exploration: in the Focus section of you can control the number of k-combinations of parameter changes that are allowed. Consult the User Guide for more details.
    • The specification of the Requirement Watchers in the SUT Configuration changed slightly to accomodate the new RML watchers.
    • The Additional Measurements section received a new configuration field for a macro-step multiplier – this can help to reduce the size of the written measurements files in setups with very small macro-steps.
  • SUT Configuration Parameters
    • New: hex_file=*.hex – wildcard expressions are now accepted in the Parameter Definitions. The @config file annotations accept checks expressed by wildcard expressions as well. For instance: hex_file=diesel*.hex. Configuration parameter values containing spaces can be used if enclosed between double quotes, for instance hex_file="dir1/my file.hex".
  • Python Instrumentation
    • New: tw.set_dynamic_instrument_definitions(True) can be used in your Python test scripts in case you want to associate a varying set of watchers and reporters with your used-defined test script – this inhibits the generation of warnings related to the changing set of instruments.
    • New: chooser.post_choose = function. This is a hook for executing code after a value is set by choose().Use this when you need to propagate the chosen value to other dependent signals, for instance when implementing fault models. Set the property of a chooser to a lambda or to a function.
    • Improved validation: a couple of cases where the access to a variable’s value is incorrect due to a mising .Value raise now runtime exceptions.
    • Fixed bugs: the labels of BitSet partitions were generated wrongly in some cases.
  • Report Query Language
    • The table CoverageStatistics was renamed as TestResults. The old name is still available for backward compatibility. The table supplies now more precomputed properties for every scenario. For instance:
      • min/max values for selected reporter instruments. Use, for instance, x.max.valuex.max.time to retrieve the max value of the instrument x in a given scenario. The project_parameter_study shows a use-case for these values.
      • passed/failed/warning evaluation verdicts for each verdict source, e.g. scriptVerdictwatcherVerdictreporterVerdictterminationVerdict. The report Predefined Tests from the shipped demos illustrates how these can be used in a report.
    • New: (list1 subsetOf list2) – predicate to check set-subset relations.
    • New: customize the colors used in the scatterplot and the barchart diagrams, i.e. for the background colors and for the plotted series/bars.
    • New: more compacting options for merging adjacent cells in un-grouped columns (`min_count`, `union` and others).
  • Others
    • An improved connection between TestWeaver and the ETAS LABCAR OPERATOR driving hardware-in-the-loop systems is now available. A new demo, i.e. exp_etas_labcar from project_hil illustrates the setup.
    • Several performance improvements and bug fixes have been added to this release.

Changes in 3.4.20 relative to 3.4.10 (June 2017)

  • Report Query Language
    • the functions set(N, state) spread the sets of collected states accross several scenarios (a kind of „randomizing“ the collected states). Currently a distance of about 100 scenarios between the collected states is attempted, whenever possible. Note: this randomization function is applied only to sets of states, it is not working for complex objects – in particular it will not work for sets like set(N, (state, time)).
  • SUT Configuration Parameters
    • the jUnit report built from the command line with the parameter --unit-test now exports also the result of checking the consistency of the experiment configuration paremeters.
    • the Current Parameter Values used by an experiment can be set from the command line with --import-parameter-values, for instance when running batch tests for continuous integration with --unit-test.
  • Python instrumentation
    • The WatcherFSM accepts a new way to specify the time intervals when the pass_condition is expected to be true. You can use the following two alternatives (check the user guide for more details):
      • tolerance_time=dt – the timer is started only once when the watcher becomes active (when the while_condition and the wait_event become true)
      • tolerance_delay=dt (new since this version) – when the watcher is active the timer is restarted each time the pass_condition becomes false.
  • A couple of other small improvements and bug-fixes have been added to this release, such as: set the encoding of the .CSV files saved by the embedded editor to ASCII, so that Silver’s CSVReader can process them; using UTF-8 encoding for the jUnit reports exported by --unit-test and others.

Changes in 3.4.10 relative to 3.4.0 (March 2017)

  • Code coverage
    • The MC/DC code coverage is now displayed in the TestWeaver reports, whenever available. The Demo TestAutomation_01 from project_lights and the exp_silver_coverage_ctc from project_car illustrate its usage. More details can be found in the User Guide.
    • Note: Since 3.4.10 Silver can add also the MC/DC code coverage measurement to the vECUs built with sbsBuild and CTC++ – see the ctc_flag command of sbsBuild.
  • User interface
    • Click on a link to an instrument source file opens the Python editor positioned at the (chooser / reporter) instrument definition – the last line of the instrument definition is highlighted.
    • Two improvements have been added to the experiment Statistics tab:
      • a new trace plots the number of new problems discovered in every scenario
      • navigate to a scenario directly from the statistics tab: click on a point of a trace navigates to the associated scenario.
    • The links to .CSV and .TXT files displayed in the TestWeaver browser open now the files in the embedded TestWeaver editor.
    • Fixed bug: the Python editor could enter a state with a very long idle time; moreover, files with more than 6000 lines could not be processed before.
    • Measurement plotter
      • The behavior of the measurement and of the reference cursors in the plotter has changed slightly. The behavior is now more intuitive, namely, the evaluation at the cursor is now in one of three exclusive modes:
        • evaluate at mouse position („Crosshair“)
        • evaluate where the user positioned the measurement cursor or
        • evaluate differences between the measurement and the reference cursor.
      • A memory leak has been solved. This could have caused an „Out of Memory“ in certain situations.
  • Python instrumentation
    • A new Severity level has been added: Severity.ImplausibleValue. If a partition with such a severity level is met the scenario simulation is abandoned with a warning. Intended usage: to mark numerical dirt produced by solvers. Its usage is demonstrated now in the experiments from project_car – for instance to mark negative heat values and negative engine speeds.
    • Two functions that query the state of the instrumentation, useful sometimes when defining fixed startup / shutdown sequences have been added: tw.simulates_generated_scenarios()tw.is_set_to_record_mode().
  • Shipped demos
    • The demos from project_car have been improved considerably: demonstrate the MC/DC code coverage, diagrams that depict the coverage of the powertrain states, illustrate the usage of Severity.ImplausibleValue, plotters in the experiments with Silver use the „night“ color scheme which is easier to read.
  • Report generation
    • Added asInteger(number) to RQL. It is used now to build the diagrams with the powertrain state coverage in the project_car.
    • The time and memory required for processing very large scenario reports has been reduced. Certain reports that could lead to an „Out of Memory“ before can now be processed without trouble.
    • However, if the available memory is not enough to process a query the results added to the report are truncated to avoid an „Out of Memory“ exception.
    • Fixed a bug related to the export of reports as User Tables.
    • Some other corner-cases related to report processing have been improved.
  • SUT Configuration Parameters
    • Collisions between the names of the SUT Configuration Parameters and the additional Silver configuration parameters are detected and reported as errors.
    • Numbers or strings that start with a digit, such 2wd | 4wd, are now accepted as valid values of a configuration parameter.
    • Duplicate values are cleaned away from the definition of the SUT Configuration Parameters.
  • Documentation
    • Some parts of the documentation have been updated.

What’s new in TestWeaver 3.4 (December 2016)

A presentation of some of the new features can be viewed here (HTML5) or here (PDF). For the complete list of changes please consult the text below.

  • Experiment Specification
    • Simplified setup for Silver experiments – a dedicated execution platform has been created for Silver.
    • Simplified setup for MATLAB/Simulink experiments – a dedicated execution platform has been created for MATLAB/Simulink.
    • Support for specifying tests for products with many variants: A new section for SUT Configuration Parameters has been added to the specification. This supports the selection of test scripts and requirement watchers that depend on a specific configuration of the SUT (e.g. the engine being for gasoline or for diesel, the transmission being a 2-WD or a 4-WD one, with or without a „sports“ option, etc.). Details are given in the User Guide and examples in project_car.
    • The location of the user defined scripts and of the requirement watchers is now declared in the experiment specification. This supports the automatic loading of the scripts and watchers matching a configuration during the test.
  • Editor for Python Source Files
    • New editor for Python source files added in TestWeaver. Features:
    • Syntax checking.
    • Keyword colouring and keyword completion.
    • Insertion of template code snippets for frequently used functions and objects, such as: SUT variables, ChooserReporterMeasurementWatcher, and many others.
    • Editing actions: copy, cut, paste, search and replace, increase & decrease indentation of selected lines and others.
  • User Interface
    • The Experiment Specification editors changed to accommodate the new features supported by the experiment specification.
    • A „trial run“ of the instrumented SUT can be started from the tool bar (under the „Check“ button). This does not record scenarios in the experiment and is useful during the setup and the debugging of the instrumentation and of the test scripts.
    • Links to the experiment’s test scripts, requirement watchers and the instrumentation file are provided on the experiment report.
    • The plotter supports several new features, for instance, setting reference and measurement cursors for detailed analysis of signal flows.
    • Opening the log files is now easier: either from the Help menu, or using a new button on the Console, respectively on the SUT Console tabs.
  • Demo Projects
    • All delivered examples have been updated to use the new (much simpler) platform specifications for Silver and MATLAB/Simulink.
    • New demo: project_moduletest_simulink simple test automation for Simulink modules.
    • New demo: exp_simulink_python in project_car, a more elaborate demo; same Python instrumentation that is used with Silver is used for MATLAB/Simulink as well.
    • The scenario report templates use now #scenarioInformation (instead of #scenario) – this gives information about the currently displayed fragment when long scenarios are split into fragments.
    • Additional features demonstrated by the experiments from project_car: usage of the SUT Configuration Parameters to filter scripts and watchers, heat maps to display the coverage of engine states, rate of change limit for continuous and integer choosers, mixing Python and CSV-files for specifying test stimulus. The file run_tests_and_export_junit.bat from project_car shows hot to automate the test execution and the export in JUnit format, e.g. for Jenkins.
  • The Silver Execution Platform
    • A dedicated execution platform for Silver has been added. This can replace in most of the cases the older and more complex specification via the custom SiL/MiL execution platform. Instructions for converting old experiments to the new simpler format are given in the User Guide (see AppendixCompatibility with older versions).
    • The platform specification requires a Silver project (.sil file) that contains the SUT without the instrumentation for test. TestWeaver will create automatically a modified tw_sut.sil file with additional Silver modules that are required for testing, namely:
      • a Python module for the Python scripts and instrumentation will be added automatically by TestWeaver
      • an MDFWriter module is added if in the Measurements section of the Experiment Specification a file with additional measurements is checked by the user
      • depending on the scripts defined by the user, if necessary, other Silver modules that apply test stimulus are added automatically to the instrumented .silMDFReader for stimulus from MDF files, CSVReader for stimulus from CSV files, and so on.
    • Setting the command line parameters for Silver or silversim is no longer necessary. TestWeaver applies the right parameters according to the selections from the GUI.
    • Several enhancements and simplifications apply to the Python instrumentation (described below).
    • Silver run/replay configuration parameters can be defined in a table in the experiment specification.
    • Beside the above Silver configuration parameters, also the current SUT Configuration Parameter values defined for the experiment are available in Silver – these can be used to configure the modules loaded in a simulation. These parameters are also available in Python.
  • Python Instrumentation
    • The Python instruments detect automatically their source file and report it to TestWeaver. A „manual“ detection (previously stored in source_ref) is no longer necessary.
    • Several other operations that required in the past user-code can now be performed automatically by TestWeaver. In order to be backward compatible these new features are only activated if in the definition of the WeaverConnection you set an additional parameter as: context=globals() – details in the User Guide and in the delivered examples. If you do this the following operations will be performed automatically and need not be part of your scripts anymore:
      • the scripts and the watchers will be loaded automatically; moreover, only the scripts and the watchers that are compatible with the current SUT Configuration Parameters will be loaded
      • tw.choose_parameters() will be performed automatically in Silver’s pre_init(time)
      • tw.terminate() will be performed automatically in Silver’s cleanup(time)
    • New functions that check the variable history:
      • toggle-detection: detect_toggles
      • detect of (frequent & significant) changes of a signal around the mean value: value_changed_around_avg.
      • many of the existing functions that check the variable history now support an additional optional parameter that constrains the amplitude of the value changes, such as abs_tol in value_changed(number_of_times, time_window, abs_tol).
    • New tw.ping(): intended to signal TestWeaver to wait longer during simulation without giving up due to TIMEOUT. Used with operations that require long waiting times, for instance starting measurement postprocessing.
    • New tw.get_configuration_parameter(name): get the value of a parameter defined in the SUT Configuration Parameters or in the Silver Additional Silver Configuration Parameters.
    • Floating-point comparison functions from fpconst: old functions have been deprecated: fp_comparefp_lessfp_leqfp_equalfp_geqfp_greater. Use instead the new comparison functions with (optional) absolute and relative tolerance: compareis_lessis_leqis_closeis_geqis_greater.
  • Report Generation and RQL
    • New heatmaps: display the reached states during a simulation or during the entire test using heat-map diagrams. See demos from project_car.
    • The UserAssessments accept now a new editable parameter: errorSource. Intended use: document the error cause after analysis of the symptoms.
    • New Variable properties: sourceFilesourceLinesourceLinkdescriptionAndSourceLink

Changes in 3.3.31 relative to 3.3.30 (February 2017)

No changes

Changes in 3.3.30 relative to 3.3.23 (October 2016)

Small bugs fixed and improvements in the Python instrumentation:

  • user-defined scripts can change parameter values at any time during the simulation (before: parameters could be changed only at time 0 in pre_init)
  • an error is raised if the Python module cannot connect to TestWeaver when a connection is specified
  • else-partitions for bit set reporters allowed.

Changes in 3.3.23 relative to 3.3.22 (September 2016)

No changes

Changes in 3.3.22 relative to 3.3.21 (September 2016)

No changes

Changes in 3.3.21 relative to 3.3.20 (August 2016)

No changes

Changes in 3.3.20 relative to 3.3.1 (July 2016)

Several bugs have been fixed:

  • exporting a scenario in an empty test database caused an exception in 3.3.1
  • the fp_compare from qtronic.util.fpconst delivered counter-intuitive results in some cases
  • the generate_cos from qtronic.util.generators did not apply the offset argument
  • avoid to run „out of memory“ when exporting the HTML reports of very large experiments
  • the signal plotter used sometimes too rough approximations in order to optimize the time to plot large measurement files

The exp_silver_fault_sim from project_car demonstrates now also the usage of user defined scripts.

Changes in 3.3.1 relative to 3.3.0 (May 2016)

  • source code coverage with CTC++: in case CTC++ is not installed or the CTC++ license check fails then the commands CTC_ON, CTC_FULL behave like CTC_OFF – the simulation runs without code coverage measurement; new: the detection of the cases when the license check for CTC++ fails
  • export to Jenkins: the termination message, documenting the normal/abnormal scenario termination reason, is now exported for every script
  • test databases: generate warnings in case the stimulus from an imported test database cannot be set due to changes in the instrumentation
  • running external commands in RTL with #exec: we automatically insert HTML new lines (BR) where ASCII new lines are found in the generated output
  • scenarioSource: new property available for the states tables; gives information about the source of the scenario that generated the current state, such as: a link to the script file, or a link to a test database scenario or the string with the source type
  • TESTWEAVER_SUT_EXPERIMENT: new environment variable, set to the experiment that owns the currently running SUT – usually same as TESTWEAVER_EXPERIMENT, unless you replay a test database scenario with the SUT of some other experiment
  • several bugs have been fixed, for instance merging of A2L range and coverage measurements

What’s new in TestWeaver 3.3 (March 2016)

  • New
    • Continuous integration and test automation:
      • Support to automate the run of TestWeaver experiments and to export the results in the JUnit XML format, for instance for continuous integration and test automation with Jenkins.
  • GUI
    • Measurement plotters:
      • A measurement viewer is now automatically opened in a tab beside the console. An  configuration, when no specific configuration is provided by the user, attempts to populate the plotters with meaningful signals from the selected scenario: inputs and states on the left and watchers and alarms on the right.
      • The HTML export of the experiment results includes now also the plotter images displayed below the scenario reports.
    • Experiment overview:
      • The experiment overview reports have an improved summary information: number of passed / failed scenarios and instruments, coverage information, and more.
      • The experiments where the automatic scenario generation was aborted due to an error are now marked with a warning icon; the tool-tip gives the termination error message.
    • Instrument overview:
      • The instruments in the project tree and in the instrument overview report are colored (e.g. green, yellow, red) to reflect the passed/failed/warning or inconclusive status reached during the simulation.
      • The instrument overview report accommodates in its header a link to a new instrument issues report. This report lists the instruments having the highest communication rates. These might be chattering reporters (alarms or watchers) that can significantly slow-down the simulation and, thus, are candidates for a redesign in case you want to improve the simulation speed. These might also be indicators of functional problems in your system, for instance toggling control signals.
      • The instrument reports display now not only the hit count for every partition, but also the number of scenarios that reached every partition. Also the number and the percentage of scenarios that reached a failure partition of that instrument.
    • Requirement watchers:
      • The overview report for the requirement watchers has been improved – it lists now also for every watcher the number of failed and success scenarios (replaces the hit-count used before).
    • User-defined test scripts:
      • The scripts can now be grouped into sub-directories. A flag in the scripting SUT configuration controls the search of the scripts in sub-directories.
      • The scripts can now be run also in PASSIVE mode. This is particularly useful when the simulating with a tool that requires a long start-up time, for instance with Simulink. An example showing user-defined scripts for Simulink is given in exp_simulink_python from project_tank.
    • Report editors:
      • The auto-completion lists displayed in the RQL and RTL report editors display now a short help for every listed keyword.
      • Wizard for now report definition: easier to define reports for source code coverage (with CTC++), vECU discrete state coverage (via A2L measurements) and vECU signal range monitoring (also via A2L measurements).
  • Interface with the system under test
    • CTC++ code coverage instrumentation:
      • TestWeaver can now receive specific code coverage information from modules instrumented with CTC++. The coverage information is used as feed-back by TestWeaver’s strategy for automatic scenario generation in order to further improve the cumulated code coverage achieved during a test.
      • Silver targets and FMUs can be easily instrumented for code coverage with CTC++, either for the default code coverage measurement, or for the enhanced code coverage measurement connected to TestWeaver. Built-in support for code coverage instrumentation is now available for both:
        • virtual ECUs exported either as Silver targets or as FMUs with sbsBuild
        • Simulink models exported either as Silver targets or as FMUs with simBuild
    • MATLAB/Simulink instruments:
      • The TestWeaver instruments can now be used also from MATLAB m-scripts. This is useful for parameter initialization in certain setups, for instance when starting vehicle simulations with CarMaker or PreScan in Simulink.
      • It is now particularly easy to run module tests for Simulink models using user-defined Python scripts. The same Python syntax and libraries for MiL and SiL that are used with Silver can be used in Simulink as well, for instance the TestWeaver choosers, reporters and the requirement watchers. Either a Silver license or a TestWeaver license is sufficient in order to be able to connect Python scripts to Simulink models. An example is given in exp_simulink_python from project_tank.
    • Python instruments:
      • The Python interpreter from Silver does not generate by default the .pyc files anymore. This avoids to clutter the directories with compiled versions of the scripts.
      • In order to reduce the number of import statements required in common scripts some more packages are now imported by default by qtronic.testweaver.instruments, namely:
        • math.*ostypesitertools
        • qtronic.silver.*
        • from qtronic.utilfpconst.*scheduler.*generators.* and files.*

        You still need to import separately from qtronic.testweaver.fsm

      • The watchers allow the usage of an additional optional parameter: until_event. This predicate is checked in every state after the wait_event is detected and causes a restart of the watcher state machine from the top level. For use cases and examples check the User Guide.
      • The scenario termination request has now an optional third argument that can request also the termination of an experiment. Useful for the cases when TestWeaver drives a (hardware) test-bench and a problem in the test bench preventing the continuation of the experiment is detected.
  • RTL/RQL extensions
    • The built-in RQL tables supply many new properties that give statistical information about passed/failed instruments, scenarios and experiments, for instance:
      • Table Variables: nominalPartitionCoverage, instrumentSeverity, successScenarioCount, abnormalScenarioCount, failedScenarioCount, warningScenarioCount, hasUnreachedSuccess, etc.
      • Table Experiments: scenarioCount, abnormalScenarioCount, failedScenarioCount, failedScenarioPercent, warningScenarioCount, warningScenarioPercent, injectedFaultScenarioCount, injectedDeviationScenarioCount, instrumentCount, failedInstrumentCount, warningInstrumentCount, injectedFaultInstrumentCount, injectedDeviationInstrumentCount, nominalPartitionCoverageAvg, nominalPartitionCoverageMin, decisionCodeCoverage, statementCodeCoverage, multiconditionCodeCoverage, etc.
    • There are new built-in RTL parameters available that give statistical information about instruments, scenarios and experiments:
      • to be used in any report: #experiment.defaultStatistics, #experiment.instrumentStatistics, #experiment.scenarioCount, #experiment.abnormalScenarioCount, #experiment.warningScenarioCount, #experiment.failedScenarioCount, #experiment.failedScenarioPercent, #experiment.instrumentCount, #experiment.failedInstrumentCount, #experiment.warningInstrumentCount and others
      • to be used in instrument reports: #instrument.shortStatistics

Changes in 3.2.5 relative to 3.2.4 (November 2015)

This release adds some small improvements and several bug fixes, e.g. related to report processing performance and export to HTML.

Changes in 3.2.4 relative to 3.2.3 (October 2015)

This release adds some small improvements and bug fixes:

  • Test descriptions read from CSV and Py file comments: the sequence of lines that starts with a comment sign (‚#‘) from the beginning of a CSV or of a Python script file is used by TestWeaver as default test description and will be displayed in the test reports. You can use HTML tags to format the comments. HTML line separators (
    ) are added between the comment lines. Example for a CSV file used as script stimulus:

            # this CSV script changes in1 and in2 simultaneously...
            time, in1, in2
            0.0,  0.0,  1.0
            1.0,  1.0,  0,0
            2.0,  0.0,  1.0
    
  • Skip test scripts that do not match the required SUT configuration: use in the Python instrumentation the call tw.skip_scenario(message) to skip a scenario. The scenario will not be added to the test reports. This functionality is only supported for the user-defined scripts (the scripting SUT), not for the automatic test case generation. Example for a Silver Python script:
            def pre_init(time):
              if not Variable756.isWritten or Variable756.Value < 2:
                tw.skip_scenario("non-matching configuration")
                return DLL_END
              else:
                tw.choose_parameters()
                return DLL_OK
    

    More details are given in the User Guide for the Python instrumentation, cf. „Defining test scripts that depend on the SUT configuration“. For an example see: project_lightsDemo TestAutomation_01script04.py.

    
    
  • Several bugs fixed: memory leak in measurements plotter, reports marked dirty without reason in some situations, and others.

Changes in 3.2.3 relative to 3.2.2 (July 2015)

Beside bug-fixes this release adds a couple of new features and small improvements:

  • Easier generation of ah-hoc reference signals with the Python instrumentation: the update property of the reporters and measurements (also of the other instruments) can now be set also to a (signal) generator or to a list of values that are repeated periodically. For instance, the following setting will generate a signal with a period of 5 steps var.update = (0, 0, 1, 1, 1). More examples are given in the documentation of the Measurement instrument.
    • Moreover, until now the update-functions were used only for their side effects, a return value was not used. This has changed now: the update function need not, but can return a value. When an update function/generator returns a non-None value then that value will be set as the current value of the variable or instrument instance that owns the update function.
  • More functions that analyse the variable history: checking the number of value increases (rising edges), value decreases (falling edges) or sign changes in a time window.
  • Improved documentation for the Python requirement watchers: the documentation has been improved with many new watcher examples.

Changes in 3.2.2 relative to 3.2.1 (June 2015)

Beside bug-fixes this release adds a couple of new features and small improvements:

  • Splitting the requirement watchers and the Python instrumentation in separate files: this is supported now by the package qtronic.testweaver.instruments; it also supports the addition of hyper-links from the instrument descriptions to their source files.
    • Related to this feature: Silver 3.2.2 has updated the wizard that exports the SUT interface as a Python script in order to ease the separation of the watchers from the interface description.
  • Code coverage reports can merge the coverage measurements from several experiments: the wizard for adding a new report supports a new type of code coverage report. You can select the experiments to merge by enumerating their names and / or with wildcard expressions.
  • The WatcherFSM and the WatchersFSM classes support in the instantiation methods an additional optional parameter req_id. Its intended usage is to specify the requirement ID of the watcher as a string. In TestWeaver this will be set as an alternative (alias) name of the instrument. You can add it in an instrument/watcher reports with displayName.
  • The project_lights demo has been re-worked to demonstrate the usage of the above new features.
  • The history of past values associated with the TestWeaver Variable and instrument instances supports now the access to values that were valid k macro-steps ago using var.getPrevValue(k). The last two macro steps can always be accessed directly with the properties: var.prevValue and var.prev2Value (also their durations, e.g. var.duration, var.prevDuration, var.prev2Duration). The access beyond the last two steps with code>var.getPrevValue(k) requires to dimension the variable history, for instance using the method: var.ensure_history_size(k).
  • The entries in the variable history support a new property stepCount – measures the value duration in macro-steps. This means, any HValue supports now the properties ( accessible with the dot notation): (Value, startTime, endTime, duration, stepCount).
  • Like the instances of the qtronic.silver.Variable, now also the instances of the TestWeaver variable (from qtronic.testweaver.instruments) support the properties that query if a variable is connected to some other modules in Silver, namely: (isRead, isWritten, isModified). This can be used to instantiate code that is configuration dependent. Frequent use-case: disable a watcher that compares a signal with a reference signal if the reference-signal is not written by any module.
  • Support for the 2014 version of Veristand has been added (via Silver).

Changes in 3.2.1 relative to 3.2.0 (April 2015)

  • Improved robustness of the license checks
  • Improved documentation of the license management
  • More properties made available in the Python instrumentation, e.g. tw.get_scenario_number()tw.is_exploring()tw.get_script(), and others
  • Several small bugs fixed

What’s new in TestWeaver 3.2 (March 2015)

A presentation of the most significant new features can be viewed here (HTML5) or here (PDF). For the complete list of changes please consult the text below.

  • New packaging: TestWeaver LIGHT
    • TestWeaver LIGHT offers a subset of the functionality of TestWeaver for a fraction of the price. The targeted application domain is classical test automation with user defined test scripts
    • an easy to use Python scripting interface is available, see the demo project_lights
    • can also be used to browse, replay and analyse scenarios generated by TestWeaver
  • The End-User License Agreement has been modified, please review carefully. Most notable change:
    • the maximal number of parallel TestWeaver instances started on a computer is limited by the license you acquire – default is 4 parallel instances on a PC
  • New license type: network license. The following license types are available:
    • computer license – license bound to a single PC
    • dongle license – portable, bound to an USB-stick
    • network license – floating, checked out via network from your license server.A computer must check out a free license, if one is available, from your network license server. After the application is closed on that computer, the license is returned to the server and some other computer can acquire the license via the network. Acquired floating licenses have only a short „grace time“ validity if the network connection to your license server is interrupted – after the grace time, if the network connection is not re-established, the local license expires and the server can allocate it to some other computer that is requesting a license. The grace time validity is set to 5 minutes in 3.2.0.

      Network licenses can, however, also be „borrowed“ to a computer, i.e. made locally and exclusively available to that computer for a longer time. During the „borrowed time“ the local computer license does not loose validity even if there is no network connection. After the borrowed time expires the local license expires and the server can allocate the license anew to a requesting computer.

  • GUI
    • improved GUI for the definition and debugging of report templates
      • the source text editors for reports and queries (RTL/RQL) are now embedded in the browser. This allows an improved syntax validation and highlighting, as well as an easier interaction from the user’s perspective during report specification and debugging.
      • improved wizard for new reports: allows now to create also source code coverage reports (based on the CTC++ code coverage tool), as well as other state coverage and overview reports
    • improvements in the measurement plotter: drag-and-drop signals among plotter views or to the trash, filter signals using wildcard characters, change signal coloring and other settings in the plotter „Preferences“
    • scenario reports can be automatically split into smaller fragments in case of „long“ scenarios. Use #ScenarioFragment instead of #Scenario in queries – see the scenario report templates from the shipped demos as illustration
    • instrument reports display the min and the max values for continuous instruments
    • export of reports and tables – new export format options:
      • CSV file – useful, for instance, if you wish to import tables in MS Excel
      • „as measurements“ – exports the measurement files and optionally any the associated plotter images
    • export scenarios to test database offers more options to control the selection of the exported scenarios
    • a couple of changes have been done in order to harmonize the GUI for TestWeaver (with automatic test generation), and TestWeaver LIGHT (without automatic generation):
      • the „Coverage“ node has been removed from the project tree, instead, the coverage reports are depicted with a specific icon
      • the „Focus“ section of the experiment setup has been restructured. TestWeaver LIGHT does not include any „generation“-specific experiment settings
      • simplified „Preferences“
  • Interface with the system under test
    • Simulink instruments
      • simulation speed improved
      • access properties of other instruments to control the chooser partitions, for instance: var.Value, var.prevValue, var.duration, var.prevDuration – similar to the property names available in the Python instrumentation
      • in the ‚Select Partitions‘ field you can enter m-expressions such as:offset = voltage_A.Value - voltage_B.Value; if offset ~= 0.0 {'pull'}; else {Partitions.All}; end
      • the Python interpreter from Silver can be embedded in Simulink – this makes it possible to use the same Python instrumentation in Simulink as in Silver
    • Python instruments
      • simulation speed improved
      • choosers can generate „smooth signals“ by limiting the maximal rate of change
        • the maximal rate of change per second can be dynamically set and modified for any chooser. Several methods are available, for instance: slope.set_max_changing_rate(5)
        • setting the maximal rate to 0 means the value is not allowed to change; setting the maximal rate of change to None removes the rate of change limitation
        • the degree of „smoothness“ depends also in other parameters – see the user guide for details
      • reporters for bit-sets – used to report values of integer types that encode sets as bit positions in the binary representation
      • mix predefined (starting / ending) sequences with generated sequences in a scenario by dynamically changing the chooser mode to RECORD or CHOOSE
      • access past values using the variable’s history
        • the size of the history must be individually configured for every variable
        • the history size can be configured using two features: a number of differing past values and/or the size of a sliding time-window
        • several functions are available to query properties of the history, for instance: var.value_changed(number_of_times=3, time_window=1)
      • new instrument properties
        • var.derivative, var.prevDerivative – defined based on var.Value, var.prevValue, var.prev2Value and time.prevDuration, time.prev2Duration
        • reporter.severity – the severity rate of the partition matching the value of reporter.Value; you can also access prevSeverity, prev2Severity
        • chooser.occurrence, – the occurrence rate of the partition matching the value of chooser.Value; you can also access prevOccurrence, prev2Occurrence
      • add_partitions, remove_partitions can be used to incrementally modify the chooser allowed partitions
      • used-defined properties can be stored and accessed for any variable, instrument or FSM instance (using a dictionary of key:value associations); this is sometimes useful for storing values in lambda-expressions – since the Python syntax does not allow assignments in lambda-expressions
      • the WeaverConnection instance has now built-in support for test scripts, see the utility methods test_description, test_step, test_condition
      • the FSM instances support an update-hook like the instruments. The update property can be set to a function that will be executed in every step before the top-level FSM is evaluated. This is sometimes more convenient than attaching on-entry, during and on-exit actions to a top-level FSM.
      • comparison functions using absolute and relative tolerance added in qtronic.util.fpconst
  • RTL/RQL extensions
    • avoid displaying empty tables: use the hideIfEmpty format option for tables
    • expand the elements of for all variables: use the new expand section in the query to expand the sets collected by for all variables. This allows to generate better reports that iterate over all alarms, for instance. See the All Alarms report templates used by the demos.
    • new keywords for available for reports:
      • #ScenarioFragment, #fragmentLinks, #fragmentInformation, #experiment.weaverVersion
    • new instrument properties available in queries
      • minValue, maxValue, minState, maxState, reachedSeverity, unreachedSeverity(value) for the #Variable table
      • testDescription and testStep are now always defined as message instruments
      • passedMessagesfailedMessages can be applied to any instrument (previously only supported by message instruments)
      • further new instrument properties: isWatcherisMessageisMessageQueuetargettargetValue
  • Report templates – many of the report templates delivered with the demos have been improved and make use of new features available in 3.2. For instance:
    • the scenario templates use now the #ScenarioFragment in order to support automatic splitting for long scenarios
    • moreover, the scenario templates, e.g. as used in project_lights, have been improved to be applicable for both scripting scenarios and for generated scenarios
    • the All/Other Alarms template uses the new expand feature that avoids to display also sets of alarms in a table row
    • the report template for watcher instruments uses now the new isWatcher instrument property
    • the Toggling Signals template used in project_car illustrates a report that displays all instruments that have frequent or toggling partition changes. The usage of the „toggling“ property is also illustrated in the scenario template used in project_car
  • Delivered examples – the examples with Silver from project_lights and from project_car have been improved:
    • the virtual ECUs are built using the new Silver sbsBuild
    • the car power-train model is based on the latest version of the Functional-Mock-up Interface standard, i.e. FMI version 2.0
    • exp_silver_coverage_ctc demonstrates how tunable parameters can be changed from the Python script driven by TestWeaver – check the wheel_radius parameter in the vECU/egs.dll and in the 2.0 FMU/silver_car.fmu.

Changes in 3.1.6 relative to 3.1.5 (December 2014)

  • Performance improvements in the Python instrumentation.
  • Several small bugs fixed.

Changes in 3.1.5 relative to 3.1.4 (August 2014)

  • The Parallel and the ParallelRace generators from qtronic.util.scheduler support now also cascading generators, i.e. generators that return other generators – they are executed recursively
  • The command line options support starting several experiments in batch mode
  • Beside the $SCRIPT configuration variable, bound to the current scripting file in the scripting SUT configuration, you can now also use the $SCRIPT_NAME variable – this is bound to the file name without the file extension
  • The default measurements splitting time has been reduced to 60 seconds (was 240 seconds before)
  • The length of the scenarios imported from a test database is not extended anymore to the scenario length used for generation, unless the database scenarios are allowed to be used as seeds for generation – this allows an easier assessment of the regression tests. The Focus section of the experiment specification contains now an extra flag that can enable or disable the usage of the database scenarios as seeds.
  • The export of scenarios to a test database has been improved to avoid certain classes of redundant or potentially conflicting scenarios
  • Other bug fixes and small improvements: improved SUT configuration check, improved messages for fatal severity errors, and others.

Changes in 3.1.4 relative to 3.1.3 (May 2014)

  • More useful report templates in the project_car demo:
    • report for shift statistics: minimal, maximal, average shift durations
    • report for toggling signals and signals with frequent partition changes
  • Easier customization of the CTC++ generated reports: define configuration options accepted by ctcpost in the sut directory of an experiment in the file ctcpost_options.txt; for instance:
    • use -f "*xyz.c" to generate the code coverage only for the „xyz“ module
    • check project_lights for an example
  • New RQL property for instruments: isDiscrete
  • Simplified syntax for the RTL exec(...) – the prefix cmd /c is no longer needed – see the code coverage report templates for examples
  • Fixed a bug that prevented the deletion of files from the %TEMP% directory, where these files were a result of unpacking measurement recordings (.zip, .7z) for plotting or export
  • Experiments that cannot be loaded due to corrupted or unsupported format are displayed and marked with an error flag in the tree
  • Simulink instrumentation library: added support for the GBK code page
  • Other bugs fixed: the ‚lock‘ icons were sometimes missing when multiple running TestWeaver instances were used; start via command line was not working sometimes when multiple TestWeaver instances were used

Changes in 3.1.3 relative to 3.1.2 (March 2014)

  • The scenario generation can be configured to use (or deny the usage) of the test database scenarios as seeds for generation
  • Fixed a bug that prevented to generate the CTC++ code coverage reports when the path where TestWeaver was installed contained white space

Changes in 3.1.2 relative to 3.1.1 (March 2014)

  • The instrument overview report shows now also the „ok/failed“ state as instrument background color, i.e. red for reached failures
  • Some demo experiments from project_car failed to start the simulation process in 3.1.1 – fixed.
  • Bug fix for rt-proxy.exe – HiL connection

Changes in 3.1.1 relative to 3.1.0 (March 2014)

  • The directory where TestWeaver saves the plotter configurations used by a certain experiment can now be specified by the user in the experiment configuration in the „Measurements“ section; default: ${PROJECT_PATH}\${EXPERIMENT}\plotter
  • The delays in several functions from the qtronic.testweaver.fsm package can now be not only constants, but also lambda/functions that return a number (the delay in seconds), namely the parameters:
    • tolerance_timewait_delay and pass_duration in WatcherFSM
    • wait_delay in WatchersFSM
    • delay in WaitDelay
    • n in state.after(n)state.before(n)
    • timeout in fsm.timeout_transition(...)
    • ad_hoc_timeout in FSM.init(...)
  • The CTC++ helper scripts CTC_ON.bat, CTC_OFF.bat, CTC_FULL.bat, generateCTCHTMLReports.bat, generateScenCTCHTMLReports.bat have been moved from the experiment sut directory in the TestWeaver installation directory, i.e. in %WEAVER_HOME%/bin. This makes possible a much easier setup of experiments that make use of the CTC++ coverage measurement – see the delivered demos for setup examples
  • The environment variable %TESTWEAVER_PROJECT_PATH% is now available to the SUT when the process is started by TestWeaver – i.e. run/replay in ACTIVE mode
  • Bug fix in ramp generators in qtronic.util.generators
  • Bug fix in chained watchers WatchersFSM: a false while_condition failed to reset the chain
  • Bug fix in the scenario export to CSV and user tables.

What’s new in TestWeaver 3.1? (February 2014)

  • GUI
    • the TestWeaver browser does not depend any longer on a Microsoft Internet Explorer installation – now we use a Mozilla browser that is shipped with the installer
      • from now on, when including local images and other local resources into a RTL/HTML, you must prefix them with ‚file:///
    • new statistics tab displays the progress of the scenario generation (scenario relevance for the experiment’s focus, new events discovered, code coverage indicators when available)
    • the instrument’s description is shown as a tool-tip in the generated reports for all instrument links
  • Signal plotting and measurement recording
    • garbage-collect measurement files: you can specify now more restrictive strategies that define the measurements that should be kept for an experiment, particularly useful for big experiments – check the new options available in the experiment configuration
    • annotate scenarios with comments and criticality assessments: this is now possible directly from the plotter – press „s“ in the plot area at a time point for which you want to associate the assessment
    • measurements in compressed MDF format can be exported to a target directory from the plotter – press „s“ in the plot area
  • Automatic instrumentation of Silver setups
    • the wizard and the functionality integrated in Silver that allows to export ready-to-use Python instrumentation files for your system or module under test has been improved
  • RQL functions
    • for the message queues of instrumented modules, e.g. msgModuleName there are now two filters available: msgModuleName.passedMessages for messages with severity zero, respectively msgModuleName.failedMessages for messages with severity above zero – check in the project_lights the scenario RTL template for usage examples
    • new table with search progress indicators: CoverageStatistics
    • instrument names colliding with predefined state column names are reported as errors in RQL queries
  • Simulink and C instrumentation
    • the instrumentation library is available now also for 64-bit applications
  • Python instrumentation
    • variables and instruments can access the past value two steps behind the current time, use the accessors: prev2Value, prev2Duration. This is particularly useful in order to detect value change events, for instance:
      • u.Value != u.prevValue – to check if output u changed at the end of the current step
      • i.prevValue != i.prev2Value – to check if input i changed at the start of the current step
    • framework for scripting with Python for Silver has been simplified and improved – the project_lights demonstrates it:
      • use a single sil-configuration file for both the generation and the scripting SUT configurations
      • the Python scripts share a sut-interface.py file with the SUT interface and global definitions and properties checked by all scenarios
      • the following three methods new for WeaverConnection are useful in scripted scenarios: tw.test_description(description), tw-test_step(description), tw.test_condition(condition, description, req_id)
    • new utility package qtronic.util.generators: contains a few generator-functions useful for stimulus definition, e.g. wait, ramp, sine
    • functions for comparing float numbers with a defined precision are available in the qtronic.util.fpconst package, check for instance fp_compare(a, b, ndigits)
    • the documentation for the qtronic.util packages has been improved, see under Appendix D – API Reference / Python
  • Measurement of code coverage
    • the integration with the CTC++ code coverage measurement tool has been improved and simplified:
      • simply start the SUT simulation using one of the predefined helper commands CTC_ON.bat, CTC_OFF.bat, CTC_FULL.bat to control the start of simulation with / without coverage measurement
      • the cumulated code measurement is now displayed in the statistics tab of TestWeaver
      • the coverage measurement can be maintained and displayed for the entire experiment (as cumulated measurements) or for each individual scenario
      • check the experiments from project_lights or the exp_silver_dymola_c_coverage_ctc from project_car for examples

Changes in 3.0.3 relative to 3.0.2 (December 2013)

  • Fixed small bugs in the Python fsm module for: FSM, WatcherFSM, WatchersFSM
  • Documentation of the state properties firstFaultTime, firstDeviationTime was inaccurate
  • Removing reports from the project tree now offers the option to delete the RTL source file from the file system as well

Changes in 3.0.2 relative to 3.0.1 (November 2013)

  • Small improvements to some browsing functions, e.g. positioning table headers, etc.
  • Small bug-fixes

Changes in 3.0.1 relative to 3.0.0 (October 2013)

  • The RTL report editors are opened in a modal (i.e. blocking) window since 3.0.0 in order to support redo and undo for the editing actions. You must close the external editor in order to return to TestWeaver. This is especially for experienced users inconvenient. In 3.0.1 we have added a shortcut to open also non-blocking RTL report editors: press shift-F4 when a report is displayed in the TestWeaver browser. The undo/redo stack maintained by TestWeaver does not cover the editing actions performed in a non-blocking editor – use the editor’s undo/redo stack (e.g. Ctrl-Z, Ctrl-Y) in this case.
  • Documentation of the measurement settings in the experiment setup has been improved
  • Fixed a bug in the Simulink Measurement block – depending on the setup, in seldom situations, this could have provoked an exception during simulation
  • Fixed a couple of non-critical bugs in the GUI

What’s new in TestWeaver 3.0.0? (September 2013)

Click here to open a presentation of the new features with screenshots and graphics.

  • Report editors
    • graphical editors: frequent report editing actions are now supported directly by clicks on the HTML reports, e.g. add, remove, reorder and edit table columns and queries
    • new wizards: help to define new reports and queries in an experiment
    • undo/redo: for all report editing actions
  • Signal plotting and measurement recording
    • the instrumentation library adds built-in support for measurement recording in compressed MDF format – available from C, ModelicaSimulink and Python
    • a versatile signal plotter allows a convenient display and analysis of measurements; SHIFT-click on an HTML state/scenario link opens a plotter for that scenario
    • comfortable scaling and value labeling: signal min-max boundaries and discrete value definitions, as declared by the instrumentation, are also used by the plotter
    • intelligent configuration management: you can define and use several plotter configurations; for every HTML table, when navigating with SHIFT-click, TestWeaver recalls the last plotter configuration used by you for that table
    • reuse of plotter configurations: the duplicate experiment action exports the plotter configurations defined in the source experiment also in the new one
    • garbage-collect measurement files: this configurable action allows to control and reduce the disk space occupied by the measurement files by removing measurements that are not „essential“, e.g remove the measurement files that are not referenced by links by the coverage or by the overview reports
    • auto-splitting of long measurement files: this feature allows to record and analyze measurements even for very long scenarios, such as those used for endurance testing; the feature also helps to reduce the disk space used by the measurement files in conjunction with the garbage-collection of measurements
  • Automatic instrumentation of Silver setups
    • export ready-to-use Python instrumentation files for your system or module using a wizard integrated in Silver
    • export Python instrumentation snippets for individually selected Silver variables using SHIFT-copy – then paste them into your favorite text editor
    • the instrument definitions reuse any min-max range or definition of discrete values that is available in Silver, e.g. from ASAP2/A2LCAN/DBCAUTOSAR/ARXMLFunctional Mock-up Units (FMU) or any other module that specifies domain information for signals
  • Email notifications
    • you can specify and configure notifications that will be sent by email when an experiment is finished in a normal or an abnormal way
  • Control of generated scenarios
    • you can now control in the experiment focus if the instrument reports and the SUT coverage information (e.g. from A2L via Silver) are enabled or disabled as information source guiding the scenario generation
    • the solver used for the focus constraints was improved for certain cases
  • RQL functions
    • new format option for value queries: ifnone specifies output in case of empty result tables
    • new state properties: scenarioNumber, isScriptState, isTDBState, isGeneratedState
    • new state-related functions: isAfterChange(expression), isBeforeChange(expression), isAfterIncrease(expression), isBeforeIncrease(expression), and others
    • new fields added to the Variables table that supply coverage statistics for instrument’s partitions:
      • based on the severity / occurrence rates: nominalHitCount, abnormalHitCount, nominalReached, abnormalReached, nominalUnreached, abnormalUnreached
      • based on the partition names: reached(partitionPattern), unreached(partitionPattern), hitCount(partitionPattern)
  • Python instrumentation
    • new subclasses of VariableSUTInput, SUTOutput, SUTModified – verify that a signal source or sink really exists in the SUT configuration. These checks allows the instrumentation to detect problems caused by changes of the SUT interface
    • beside Chooser and Reporter, also Measurement instances are recorded in measurement files
    • new utility package qtronic.testweaver.fsm: defines classes that implement hierarchical state automata, using states, transitions, entry/exit/during actions, etc. – e.g. see the FSM class.
    • new class used for monitoring functional requirementsWatcher replaces the functionality of the previously used StateWatcher class. The Watcher has also an equivalent implementation in the fsm package – the WatcherFSM. This fsm-compatible implementation can be furthermore linked and embedded, if needed, in even more complex monitors using the WatchersFSM class.
    • for a more convenient control over the order in which the instruments are evaluated during simulation with the method tw.report_and_choose(), each instrument can specify at declaration an eval_index. If not specified explicitly, this will be initialized by default in a meaningful way. The default order is: reporters, choosers, FSMs (state machines), measurements, watchers.
    • new instrument functions that simplify certain expressions, example: prnd.was_in("P") and prnd.is_in("D", "R")
  • Simulink, Modelica and C instrumentation
    • new class/blocks & API for signals that will be stored in measurement files
  • HiL and real-time simulation environments
    • for the connection to HiL systems you can use now also PROVEtech:TA from MBtech. The connection is based on Silver’s PythonRTT. This ensures that the same Python API can be used to instrument scripts for dSPACE HiLs, NI HiLs, ETAS HiLs and others.
  • GUI
    • Help->Open log file opens the log file with detailed information about past actions and problems
    • File->Open demo projects allows quick navigation to the example projects delivered with TestWeaver
    • the functionality of the combo-box used to select the scenario report template in the tool-bar has been moved in the context-menu of the scenario node in the project tree
    • new button in the tool-bar to plot the measurements of the currently selected scenario
    • SHIFT-click on a state/scenario link opens a plotter with the scenario measurements (when available)
    • new button in the tool-bar to switch to the interactive report editing mode; when this is active you can modify the queries and the table formatting and you have the undo/redo buttons enabled in the tool-bar

Changes in 2.5.5 relative to 2.5.4 (April 2013)

  • Fixed a bug in the high-level C instrumentation
  • Fixed a bug in rt-proxy.exe that could cause problems on some computer configurations used for dSPACE HiL with Real-Time Testing

Changes in 2.5.4 relative to 2.5.3 (February 2013)

  • Fixed a bug that caused in seldom cases problems at displaying scenarios or setting scenario comments
  • Fixed a bug related to CodeMeter dongles that kept showing the following message in the console: „Could not lock the CodeMeter concurrent access mutex. Continuing without it.“

Changes in 2.5.3 relative to 2.5.0 (December 2012)

  • Fixed bugs:
    • ‚Replay Multiple Scenarios‘ failed for configurations older than 2.5
  • User Guide updated with the new fields added in 2.5 in the Variables and the InstrumentPartitions tables

What’s new in TestWeaver 2.5? (September 2012)

General

  • Scenario generation with improved coverage
    • coverage measurements from other tools, such as Silver’s A2L coverage (more below), can be fed back to TestWeaver to improve the coverage of the generated scenarios
    • overall improved coverage of system states, especially for experiments where the user’s coverage reports were not very well tuned
  • Coverage reports for instruments
    • instrument reports enhanced with coverage and statistical information
  • Coverage reports for A2L signals
    • Silver’s coverage measurement for discrete A2L signals (available since version 2.5.0) has been integrated in TestWeaver. A2L is an ASAM/ASAP2 standard widely used in the automotive industry to describe the configuration parameters and the signals of electronic controllers. For examples check the Silver experiments from project_car.
  • Requirement monitoring
    • new StateWatcher instrument available in Python. Many frequently used system requirements, including ones that monitor temporal dependencies between events, can be conveniently encoded using this instrument. For examples check exp_silver_dymola_c_python in project_car and exp_silver_simulink_python in project_tank.

User interface

  • double-click on the project tree elements to open their associated configuration editors
  • the batch replay of scenarios allows replay in PASSIVE mode
  • the link to the installer download page integrated in the Help menu
  • the communication port settings have been moved from the „Generation SUT“ page to its parent, the „SUT Configuration“ page
  • integer allowed as field type in the user table editor
  • bugs fixed: the retrieved partition comments were wrong in some cases; the keyword completion for Notepad++ did not work properly on some configurations under Win7

Instrumentation libraries

  • more comfortable syntax for the definition of instruments (Python, Simulink, Modelica, C):
    • partition names can be missing, Testweaver generates default names from the min/max values
    • severity/occurrence rates can be missing, Testweaver generates default nominal rates
    • partition descriptions can be missing, defaults are empty descriptions
    • integer typed instruments and partitions: intervals of instruments with an integer type automatically declare all contained integers as ‚instrument partitions‘ in TestWeaver
    • symbolic constants Partitions.All and Partitions.Last can be used to specify the chooser allowed partitions
  • StateWatcher in Python supports an easier encoding of requirements

Report specification and processing (RQL/RTL)

  • scatter plots can be used to depict query outputs. Examples: ClutchHeat in project_car.
  • new formatting option for numeric values: asTime
  • the InstrumentPartitions table defines additional columns reporting the partition reachability, e.g. hitCount
  • the Variables table supports now dot-notation in order to access partition fields, such as in SUCCESS.hitCount
  • expression != null and expression = null are now allowed in RQL
  • not like is now allowed in RQL

Compatibility notes when upgrading from version 2.4.x

  • experiments created with TestWeaver 2.4.x will have to be upgraded to the current format: right click and select Convert to Current Format...
  • experiment setups created with TestWeaver 2.5 cannot be used in TestWeaver 2.4.x unless they are empty – before loading the such experiments in older TestWeaver versions, the content of the ${experiment directory}\scenarios directory must be deleted, with the exception of the UserAssessments.xml file

Changes in 2.4.4 relative to 2.4.3 (August 2012)

  • Fixed several bugs:
    • exporting report scenarios to a new test database fails
    • various errors while running several TestWeaver instances
    • concurrent access from multiple user sessions to CodeMeter.exe

Changes in 2.4.3 relative to 2.4.2 (June 2012)

  • TestWeaver installer now supports silent mode
  • Fix virtual machine false positive bug in license system

Changes in 2.4.2 relative to 2.4.1 (June 2012)

  • RQL operator like now works on multi-line texts
  • Fixed a bug in the console that caused it to stop showing new messages

Changes in 2.4.1 relative to 2.4.0 (June 2012)

  • Demonstrate range monitoring: Silver’s monitoring of signal ranges (via A2L, an ASAM/ASAP2 standard) is demonstrated by two examples, see project_lights/exp_lights and project_car/exp_silver_dymola_c_python
  • Modelica instrumentation:
    • instruments now available also for the version 3.2 of the Modelica Standard Library
    • new demo using Modelica and ITI’s SimulationX in project_tank/exp_simulationx
  • Simulink instrumentation: more robust initialization of the severity/occurrence constants
  • RQL query processing: simulatedTimerunningTime, and testedTime from the Experiments table are now measured in seconds (no longer in milliseconds)
  • other bug fixes and small improvements: new experiment using an other experiment as prototype, license checking

What’s new in TestWeaver 2.4? (April 2012)

General

  • Run multiple TestWeaver instances concurrently – allows better use of multi-core processors
    • resources of running experiments are locked to prevent concurrent modification
    • status of locked experiments displayed in the project tree
    • simplified handling of TCP/IP connections: automatic selection of free ports (for ACTIVE SUT mode)
  • HiL and real-time simulation environments
    • besides dSPACE HiLs and National Instruments HiLs, now also ETAS HiLs can be connected to TestWeaver: via LABCAR and Silver’s PythonRTT. The same Python API can be used to instrument scripts for dSPACE HiLs, NI HiLs and ETAS HiLs.
    • the Silver module used to run the emulation of the RTT-Python API has been renamed to PythonRTT (previously instantiated as Python qtronic.silver.emulation.rtt)
    • Silver’s PythonRTT now supports also an init_sequence in the configuration file (previously only supported by the dSPACE RTT)
    • the SUT configuration editor checks if the SUT is started using the rt-proxy.exe utility
  • Experiment focus: a maximal scenario duration is defined also for the scripting SUT
  • Performance: the memory required to process reports for large experiments has been improved
  • User guide: significantly improved
  • User interface: several improvements, e.g. resizable window for experiment comment
  • Windows 7: several issues changed for a better Win 7 compatibility, e.g. installation, license checking, license update

Report specification and processing (RQL/RTL)

  • New RQL functions: minState, maxState, minPrevValue, maxPrevValue

Python instrumentation

  • Easier partition definition: the labels in a partition definition are now optional. If missing they will be set by default to a string as below:
    • „value“ – if value = min = max
    • „min..max“ – if min != max
  • The testweaver.instruments module now defines a Variable class for TestWeaver. The class extends the dSPACE API of rttlib.variable with the calls supported by the Silver qtronic.silver.Variable, namely:
    • variable.Value – read/write access to current value
    • variable.prevValue – read access to the variable’s value from previous macro step
    • variable.duration, variable.prevDuration – duration (in seconds) of the value / prevValue
  • Choosers and Reporters also support the API of the Variable class, namely: Value, prevValue, duration, prevDuration
  • Platform independent access to the time of a TestWeaver scenario. Use:
    • time = TimeVariable() # instantiate a variable to access the time value (read-only)
    • print time.Value, time.prevValue # access to the current and previous time value

    the time for the TestWeaver instrumentation is now set automatically by the Silver and the RTT simulation framework. Setting „tw.time = ...“ is no longer necessary. The old API that allowed to set the time in the choose and report calls is no longer supported

Changes in 2.3.8 relative to 2.3.6

  • fixed bugs: missing scenario template for test databases; fixed drag and drop of scenarios to test databases
  • improved forward compatibility issues for the instrumentation library
  • improved robustness of Python detection for HiL / rt-proxy.exe

Changes in 2.3.6 relative to 2.3.5 (January 2012)

  • a few bugs fixed for the HiL instrumentation; improved error handling and reporting in rt-proxy.exe

Changes in 2.3.5 relative to 2.3.4

  • a few bugs fixed for the C and the Simulink instrumentation, e.g. CHOOSER_LATENCY. Documentation slightly improved
  • fixed: bug from 2.3.4, causing early termination of recording and scripting scenarios

Changes in 2.3.4 relative to 2.3.3 (November 2011)

  • several bugs fixed
  • heuristics for scenario generation slightly re-tuned

Changes in 2.3.3 relative to 2.3.0 (November 2011)

  • minor bugs fixed
  • the instrumentation library for Simulink supports now also the Japanese Shift_JIS and the Korean EUC-KR character encodings
  • ECU access via UDS supports now the ECU_RESET primitive

What’s new in TestWeaver 2.3?

User interface

  • double click on the project.testweaver file from Windows Explorer starts TestWeaver in the associated project
  • RQL/RTL editor with syntax highlighting and keyword completion, also for the defined instrument names of an experiment
  • improved keyword completion for the constraint editor of the experiment specification
  • support for using non-ASCII characters / character encodings (e.g. UTF-8 and others) in the RQL/RTL reports
  • links from the generated table headers to related instrument definitions
  • messages related to the running SUT separated from other messages in a second console tab
  • highlighting of the warning/error status of the console messages
  • support for using alternative display names for the instruments
  • batch replay of scenarios with support for recording scenario traces in Silver, for instance in MDF format
  • improved feedback at scenario export to test databases
  • improved navigation from the user assessments table to the associated scenarios
  • a log file is written at the location %APPDATA%\QTronic\TestWeaver\log\TestWeaver.log – where %APPDATA% is a Windows environment variable, usually it points to c:\Documents &Settings\\Application Data in Windows XP, or to c:\Users\\AppData\Roaming in Windows 7.

Engine

  • improved heuristics for scenario generation, improved coverage
  • support for non-deterministic SUT simulations
  • improved support for HiL and real-time simulation environments:
    • library for ECU EEPROM initialization and for diagnostic fault code management available – using UDS via CAN
    • beside dSPACE HiL systems now also National Instruments HiL systems are supported (via Silver and NI VeriStand 2009/2010)
  • support for using non-ASCII characters / character encodings (e.g. UTF-8 and others) in the instrumentation libraries
  • new instrument functions available in Python instruments: has_partitionin_partition

Report specification and processing (RQL/RTL)

  • new RQL function, patternCount, for detecting patterns in scenario signals
  • the precision RQL formatting option can now be used also for barchart labels
  • use of aliases is no longer restricted in grouping constructs. E.g., this is now allowed:
    • select min(heatA) group max(heatB) as alias, alias group set(scenario)

Changes in 2.2.1 relative to 2.2.0

  • fixed a bug that was causing experiment corruption in case recording was canceled
  • improved memory detection when launching Java VM in testweaver.bat;
    new: use testweaver-cfgmem.bat for manual control of the JVM memory
  • fixes for the C instrumentation: .h files added, fixed exported functions in instr.dll

What’s new in TestWeaver 2.2?

User interface

  • more accurate/informative progress reporting
  • the coverage and reports node of each experiment show navigable links to reports
  • loading time for very large reports is now significantly improved

Report specification and processing (RQL/RTL)

  • the order by clause is now supported in RQL queries to customize the ordering of the rows in the result
  • the semantic of the state accessor function prevState(expr1, ..., exprn) has been slightly changed:
    • Before: return the state immediately before the change event of one of the expressions
    • Now: return the state immediately after the change event of one of the expressions
  • new access function for the States and Scenario tables, duration(expr1, ..., exprn):
    • measures the period of time for which the argument expressions have held their current values
    • is a short notation for state.endTime - prevState(expr1, ..., exprn).time
  • new RTL parameter #scenario_navigation used to facilitate navigation in very long scenario reports, e.g. for scenarios recording several hours of simulation
  • RQL query output, until now formatted as table or barchart, can now be also formatted as a simple value using format value

Engine

  • support for batch recording of scenarios

Changes in 2.1.1 relative to 2.1.0

  • you can easily run now multiple experiments in parallel on the same computer – particularly useful if you have a multi-core processor. For doing this you have to:
    1. assign different TCP/IP port numbers for the SUT of each experiment,
    2. start several TestWeaver instances,
    3. start an experiment in each TestWeaver instance
  • Simulink and Dymola chooser instruments support the chooser latency now
  • new icons in the Simulink instrumentation library
  • new functions in the Python instrumentation API:
    • Instrument.update() for hooking user defined actions to an instrument, executed before the instrument choose/report request is evaluated
    • Instrument.prevValue() for accessing the value of an instrument at the previous request time
  • simplified Python instrumentation scripts in the shipped examples, already using the new instrument update API
  • added HiL support for scripting; for an example see project_tank\exp_hil_emulation
  • reporter events with severity=ERROR_FATAL stop simulation also in recording and scripting modes now
  • when TestWeaver starts the SUT in active mode the following variable definitions are added to the environment of the SUT process, e.g. for logging or other purposes:
    • TESTWEAVER_EXPERIMENT = the name of the current experiment
    • TESTWEAVER_SCENARIO_NUMBER = the number of the scenario currently being generated / replayed
  • bug fixes in the qtronic.util Python package and in the HiL support functionality

What’s new in TestWeaver 2.0 / 2.1 ?

Remark: 2.0 vs. 2.1. During the beta release of TestWeaver 2.0 we received several new feature requests from our customers, such as: batch export of scenarios as CSV files, batch run of multiple experiments, high-level version of the C instrumentation library, easier support of UNICODE in RTL, and others. We have been able to react fast to these feature requests and they are now available. Therefore, the first official release of a TestWeaver 2.x has the version number 2.1.

HiL Support

  • new TestWeaver function that allows to generate and evaluate test scenarios running on HiL simulators. The interaction with the HiL is done via Python scripts, generated by TestWeaver, and executed in real-time on the HiL. The results are reported back to TestWeaver and guide the subsequent scenario generation. The real-time Python is distributed by dSPACE with AutomationDesk.
  • the HiL Python scripts can also be tested and run on PCs with Silver 2.0 – because Silver 2.0 also supports the dSPACE Python HiL API now. This makes the same test scripts compatible for SIL and HiL: tests can be developed and tested on SIL (PCs) and then run on HiL.
  • the HiL-Support is available under a new license option that has to be purchased in addition to the basic TestWeaver license.

Engine

  • interactive scenario recording
  • support for scripting – check examples\project_lights\exp_lights for usage examples
  • improved control over the generated scenarios:
    • number of generated scenarios
    • frequency of chooser changes per chooser per scenario
    • enabling/disabling of scripting / test databases / generation in the experiment focus
  • scenarios of an experiment can be replayed using the SUT of other experiments in the project
  • Python instrumentation library is more powerful and easier to use, for better MIL-SIL-HiL compatibility we support the dSPACE Python HiL API
  • a new, high-level C/C++ instrumentation is available
    • easy to use high-level instruments that readily support the new features of TestWeaver 2.0: scenario recording, scripting, etc.
    • the API is very similar to the Python instrumentation API
    • the experiment exp_silver_simulink_c in project_tank demonstrates the use of the new library
  • the user assessments and comments are now stored in a separate XML file
    • easier to exchange (e.g., by email), compare or merge
    • allows much faster query updates via the new #UserAssessments table
  • the experiment reports can be exported in HTML format, reconfigured for use outside TestWeaver
  • a package with Python utilities is supplied with TestWeaver:
    • monitoring durations, value changes, oscillations; for a usage example check examples\project_car\exp_fault_sim\sut\instruments.py
    • PID-controllers and filtering

User interface

  • more powerful console, with filtering functions for easier debugging
  • existing experiments can be used as prototypes when creating new ones
  • function for running a batch of experiments, optionally setting limits on the number of scenarios to generate and/or on the running time of each experiment
  • several/all scenarios of an experiment can be exported to CSV files in one action, for instance for use in HiL or other testing environments
  • TestWeaver tables can be exported as/imported from CSV files
    • individual RQL queries from an RTL report can be exported by clicking on their anchors
  • HTML links to detailed reports may now be part of the result of an RQL query (see the reportLink function)
  • drag & dropping reports and scenario templates automatically copies the files referred by these as well
  • hyperlinks and context menus are available in many places, facilitating direct access to files/folders
  • project and experiment directories now have special icons, making them easy to spot in Windows Explorer
  • many other usability improvements throughout the GUI
  • TestWeaver now supports command line options (run testweaver --help for details)

Report specification and processing (RQL/RTL)

  • RTL no longer requires escaping for Unicode characters (you can write &#xxx; instead of &#!xxx;)
  • more accessors in the #State and #Scenario tables; for usage examples check the report templates from examples\project_lights\exp_lights
    • state.severity or severity delivers the maximum severity of some alarm valid in that state
    • state.maxSeverity or maxSeverity delivers the maximum severity that occurred in this state or in any preceding state from the scenario; for instance, scenario.maxSeverity delivers the maximum alarm severity that was raised in a scenario
    • scriptLink returns an empty string or a string containing an HTML-link to the file passed as argument to the SUT for script evaluation
  • aggregation functions may be combined in expressions
    • e.g., select min(duration) < 1.0 ? avg(time) : avg(2 * time)
  • state accessors can now prefix any expression, not just variable names
    • e.g., prevState.(currentGear, targetGear)
  • aliases may be used much more liberally
  • any expression legal in the select clause is now allowed in the having clause as well
  • partition names with special characters may now be referred from RQL
    • e.g., carSpeed.'-20..-1'.min
  • columns marked as hidden may be freely mixed in the select clause with normal, visible columns
  • the prevState function now accepts expressions, not just variable names
    • e.g., prevState(heatA.severity)
  • the separator formatting option now also supports parentheses specification
    • e.g., separator="(comma), [space]" will format ((1, "a"), (2, "b")) as ([1 "a"], [2 "b"])
  • RTL now supports the form #{IDENT} in addition to #IDENT
  • error messages have improved in key places
  • faster query and report processing

Compatibility notes when upgrading from version 1.2.x

  • experiments created with TestWeaver 1.2.x will have to be upgraded to the current format (right click and select Convert to Current Format...)
  • experiment setups created with TestWeaver 2.0 can be used in TestWeaver 1.2.x, however scenarios generated with TestWeaver 2.0 cannot be loaded by TestWeaver 1.2.x
  • TestWeaver 1.2.x experiment setups that use both silver.bat and silversim.exe have to be modified in order to ensure a consistent use of the -o0 flag: it has to be specified either for both or for none
  • your Dymola environment and/or instrumented model may need to be updated in order to use the new version (2.0) of the Dymola instrumentation library
  • due to changes in the Dymola and Simulink instrumentation libraries, your instrumented models will be left with some unused components that should be cleaned up
  • in rare cases RQL queries that work in 1.2.x do not work in 2.0:
    • the def keyword/construct is no longer supported
    • the column name t is no longer supported as an alias for time
    • the at keyword is deprecated, please replace it with stateAt

Changes in 1.2.8 relative to 1.2.5

  • minor bugs fixed

Changes in 1.2.5 relative to 1.2.4

  • larger editable area for the field state.comment, e.g., in the scenario reports
  • improved protocol for automatic license update
  • correction in the replay script of example exp_simulink from project_tank

Changes in 1.2.4 relative to 1.2.3

  • correction in the User Guide

Changes in 1.2.3 – 1.2.2 relative to 1.2.1

  • the communication between TestWeaver and Silver / A2lAccess has been improved in order to support the range check of all A2L signals

Changes in 1.2.1 relative to 1.2.0

  • the reaction of TestWeaver at time-out of SUT communication was fixed
  • multiple restarts of replay are now supported also for the PASSIVE SUT mode

What’s new in TestWeaver 1.2.0?

Performance

As an effect of many optimizations of the report generation engine, the speed of report generation has increased significantly for large databases.

Connection to code coverage tools

A couple of new features facilitate the connection with code coverage measurement tools, such as gcov or Testwell CTC++. For instance, CTC++ performs various coverage measurements, such as MC/DC, relevant for the certification of safety-critical software. The coverage reports generated by CTC++ can be connected to the overview reports displayed by TestWeaver.

More examples

The examples delivered with TestWeaver have been improved. Several new features and use cases are now covered by examples, for instance:

  • Fault simulation: experiment exp_fault_sim in project_car shows how faults of sensors & actuators can be modeled, and how TestWeaver can be used to automate failure analysis
  • Code coverage: experiments exp_silver_dymola_c_coverage_ctc and exp_silver_dymola_c_coverage_gcov in project_car can be used as starting points for code coverage analysis with TestWeaver
  • Simulink & RealTime Workshop SUT: experiment exp_simulink_rtw in project_tank shows how to use RTW-compiled Simulink models for improving runtime performance

User interface

  • accurate progress information is shown for all actions
  • drag & drop support has been extended to allow copying of reports and scenario templates among experiments as well as exporting scenarios to test databases
  • navigation to specific tables in RTL reports is now possible using automatically generated „anchors“, both in the project tree and in the document itself — table headers and bar chart images are now hyperlinks which can be clicked on for optimal positioning in the browser
  • support for zoom in/zoom out has been added to the TestWeaver browser
  • searching for text is now possible in the browser and in the console
  • experiments now have a scenario node showing the last generated scenario
  • the interface has been simplified by merging the run experiment and replay scenario buttons into a single button
  • TestWeaver responds to cancel requests promptly now

Extensions to the report specification language RQL/RTL

  • a more flexible macro system for passing arguments from RTL to RQL is now in place, allowing to define parametric queries
  • the prevState state accessor can be used as a function as well, allowing to jump back in a scenario to states where certain instrument partitions changed
  • new state accessor functions, minState and maxState, allow the selection of a past state where a numeric expression had a minimum or a maximum value
  • the firstState state accessor is available for reaching the starting state of a scenario
  • new aggregation functions facilitate more statistical measurements
    minAllmaxAllsumAllproductAllavgAllcountAll & setAll
  • RTL can now execute external commands and include their output in the result:
    #exec("command and arguments")
  • RTL can now produce links that trigger the execution of a command:
    #exec_link("label", "command and arguments")
  • new formatting option for setting the direction of bar chart labels
    label_orientation
  • the functions occurrenceToColor and severityToColor (in the formatting section) accept now any kind of expression as argument, not just column names

Extensions to the instrumentation libraries

A new type of instrument, the MESSAGE instrument, has been added.

Portable license

QTronic now offers support for dongle licenses. As a result, you can acquire a license that can be written on a dongle (CodeMeter stick) and used to run TestWeaver on any machine.

Compatibility notes when upgrading from version 1.1.x

Experiments created with TestWeaver versions 1.1.1 and above require no conversion when opened with TestWeaver 1.2.0. A minor issue here is caused by the usage of a new format for encoding links in the HTML reports. We recommend regenerating all the reports (right click on experiment name → Update all reports) when migrating an experiment from one version of TestWeaver to another. For experiments created with TestWeaver versions before 1.1.1, please refer to earlier compatibility notes.

TestWeaver 1.2.0 adds a series of new features. Experiments created with 1.2.0 that make use of these new features cannot be opened or processed with earlier versions of TestWeaver.

Changes in 1.1.7 – 1.1.2 relative to 1.1.1

A few minor bugs fixed

Changes in 1.1.1 relative to 1.1.0

There are no compatibility issues. Improvements:

  • in the instrument view the partitions are now ordered by value
  • better diagnosis and error messages in case of configuration errors
  • experiments saved in an older (or newer) format are shown to the user with a distinct icon. The format conversion is done only at the explicit request of the user.

Changes in 1.1.1 relative to 1.0.x

Compatibility notes when upgrading from version 1.0.x

The most important changes that affect compatibility are:

  • experiment specification: the path format for resources, e.g., for starting the system under test or for specifying the location of the RQL/RTL files, has changed. The experiments built under 1.0 will be depicted with a distinct icon in the experiment browser.TestWeaver 1.1 can read and convert on demand the old experiment specifications to the new format. Please note that the experiment specifications for 1.1.x cannot be loaded in 1.0.y

    We recommend to make a backup copy of the experiment data from older versions.

  • license file: additional information has been added to allow for a more comfortable license update directly from the TestWeaver Help menu.

    Please generate a new hardware key and generate a new license file after installing the new version in order to take advantage of the new features.

  • SUT and instrumentation: there have been some changes in the Modelica/Dymola instrumentation library and in the Simulink instrumentation library.

    Please recompile the SUT modules that contain Modelica or Simulink instruments (e.g., executables, such as dymosim.exe or .dll modules for Silver generated with dymbuild or simbuild).

  • demos: the structure and default location of the demos has changed. Previously, they used to be installed in %WEAVER_HOME%\demos. As of this version, you can find under %WEAVER_HOME%\examples. Two projects are available there with example setups for, e.g., Dymola, C, Simulink (see project_car and project_tank).

    Please remove the old demos manually; they are not removed by the 1.0 uninstaller completely, leading to potential errors!

  • report specification and constraint language: the state mutator prevState, that facilitates access to the previous state of a given state, has changed behavior for the first state of a scenario. Specifically, if before we had prevState is null for the first scenario state, in 1.1 we have prevState=state for the first scenario state. See also the new state mutators available in 1.1.

    Please adapt your old experiment constraints as well as your old report specifications so that they use the check prevState = state instead of prevState is null.

  • instrumentation library for Modelica: In the Instruments library (version=“1.1.0″) two small changes have been made.

    You might have to edit the source of your instrumented Modelica systems in order to accommodate these changes: 

    • „Occurence“ has been renamed / corrected to „Occurrence“;
    • the „enabled“ boolean flag in the WeaverConnection has been removed; the Instruments now attempt to establish a connection with TestWeaver at initialization – if that fails then the simulation continues with default values, as if the previous WeaverConnection.enabled flag were set to false.

Summary of new features

Test databases

It is now possible to collect certain „interesting“ scenarios generated by TestWeaver in test databases, e.g., for performing regression tests. The collection of scenarios from one or several test databases can be imported in other experiments where they will be performed before any scenario generation is done. This feature supports a more „deterministic“ procedure of evaluating the system under test, with foreseeable execution time and coverage.

Graphical diagrams and support for statistical test evaluation

There have been several significant extensions in the report specification language. It is now possible to generate reports that contain not only tables, but also graphical diagrams in the form of bar charts. Several language additions such as computation of averages, min, max, counts, percentage with „where“ and „group“ clauses support various statistical evaluations of test results.

More information accessible in the experiment results

  • New experiment information is available in the following tables: States, Scenario, Variables, SutModules:

    experiment.name
    experiment.comment
    experiment.simulatedTime
    experiment.testedTime
    experiment.runningTime
    experiment.numberOfScenarios
    experiment.numberOfStates
    experiment.timestamp

  • New state mutators are available in the States and Scenario tables and in the experiment constraints:
    prevState
    accesses the previous state of a state; for the first scenario state we have state = prevState.
    nextState
    accesses the next state of a state; for the last state of a scenario we have state = nextState.
    scenario
    accesses the last state of a scenario, we have: (state = nextState) iff (state = scenario).
    state
    does not move the state at all.
  • New editable fields in the scenarios: beside the existing editable fields state.comment and state.criticality a new field is available. state.toBeCheckedBy with string values can be used for supporting the work-flow of the test results‘ evaluation.

Experiment focus

  • In the experiment focus, the generation strategy can restrict the scenarios of an experiment to the scenarios from the specified test databases. Alternatively, after the importing of scenarios from one or more test databases is completed, the usual test generation of TestWeaver can be enabled.
  • The performance of handling the upper bounds for the maximal number of allowed faults and deviations has been significantly improved.
  • Improved usability of experiment constraints:
    • by the availability of the state mutators: stateprevState;
    • by the introduction of the new implication logical operator => that allows the writing of more readable constraints, such as
      (endTime < 2.0 => accelPedal = "0%");
    • by changing the way the constraints that restrict the chooser values are evaluated. Namely, the constraints that involve a chooser (input, fault or parameter) are not evaluated at the time points where no choose request is issued by the system under test;
    • by providing better user feedback for scenarios that violate the experiment constraints: the terminationEvent now contains information about the violated constraint, if any.

Extensions to the report specification language RQL/RTL

  • conditional evaluation now possible:
    ? :
  • implication operator:
    =>
  • new aggregation functions available:
    percentage & product
  • individual where filters for aggregation functions, such as in:
    select avg(x) where x > 0, avg(x) where x < 0 ...
  • support for having clauses in the RQL statements that use group by
  • formatting options for bar charts

User interface

  • the Help menu now provides simplified access to the QTronic infrastructure, allowing users to submit problem reports and update their license directly from TestWeaver
  • improved user feedback in case of inconsistent experiment specifications
  • the instrument definitions are now checked for several possible errors, such as overlapping partitions
  • user tables now support data import from CSV files and data export to CSV files
  • the specifications for starting the system under test now use relative paths. ${PROJECT_PATH} and ${EXPERIMENT} are available as symbolic references. Environment variables from the OS can also be used
  • the reports can be dragged and dropped from reports to coverage and vice-versa in the project browser
  • experiment constraints can be edited in a separate window. We have also improved the auto completion available in the constraint editor (trigger auto completion using Ctrl+Space)

User Manual

The TestWeaver User Manual is now also available as pdf-file in TESTWEAVER_HOME\help.

Release Notes of Older Versions

Changes in 1.2.7 relative to 1.2.5

  • minor bug fixes, mainly in a2l-handling
  • fixed gui-update issues
  • allow Silver to continue in case of Div0
  • a2lAccess now supports vcs files
  • can module shipped, without documentation and addition to utility module list

Changes in 1.2.5 relative to 1.2.4

  • improved protocol for automatic license update
  • assignment and breakpoint widgets: signal names can contain dots, e.g. sig15.alfa.beta; names can be quoted in order to include special characters in signal names, e.g. 'var8[15].field name'. Note: the modifier module already supported this syntax
  • significant improvements in dymbuild (dll-module creation for Dymola 6.x):
    • numerical solver displays errors, warnings and statistics in Silver
    • Modelica-conform interpretation of the nominal attribute: atol(x)=rtol*nominal(x)/100; until 1.2.4. the following interpretation was used: atol(x)=rtol*nominal(x)
    • the maximum solver step-size is now restricted to the Silver macro step width
  • bug fix in DymolaReader: at time t, x(t-dt) was plotted
  • a few minor bugs have been fixed

Changes in 1.2.4 relative to 1.2.2

  • after solver failures, such as „Dassl failed with -7“, simulation stops. TestWeaver will be informed about these problems now
  • a few minor bugs have been fixed

Changes in 1.2.2 relative to 1.2.0

  • Silver plotter: better handling of large real numbers (scaling, legends, etc.)
  • communication between Silver / A2LAccess and TestWeaver improved in order to support the range check of all A2L signals

What’s new in Silver 1.2.0?

  • Silver
    • Supports dongle-bound licenses now
    • Console Tab that displays errors, warnings and other messages produced by Silver or the loaded modules
    • Bundle wizard that exports a Silver configuration (all modules, plugins, etc) into a portable, easy-to-distribute folder
    • New lights-control demo
    • Crash handling for multi-threaded modules
    • Keyboard shortcuts
    • Individual modules can be activated and deactivated
    • Create backups of .sil files before overwriting
  • Widgets
    • Plotter
      • Point-to-value when the mouse hovers above the chart
      • Traces are now antialiased
      • Added support for predefined colorschemes (black-on-white and white-on-black)
      • Signal scaling (user-defined or automatic) is much more predictable
      • Selecting multiple traces is supported
      • Snapshots include the (simple) legend
      • Synchronized crosshair among multiple Plotter widgets
    • Diff
      • New widget that allows users to compare the values produced by 2 Silver modules.
    • BreakpointAssignment
      • Content assist when typing functions and variable names
  • Modules
    • SilverDiff (beta)
      • Compare two Silver configurations running them side-by-side. Use the Gui of the Master-Silver and it controls the Slave-silver. Receive all Slave-signals in the master, compare them and get a warning when signals differ too much. Sync both Silver using one selected signal.
    • DllWriter_a2l
      • New module that allows one to use Silver as a debugger
    • MdfDiff
      • New module that compares signals from two mdf files
    • MdfReader
      • Major speed improvement
    • CsvReaderCsvWriter
      • User-defined writing step width
      • Smart warning if the selected decimal point doesn’t match the file contents
      • Select the signals to read/write via -l <signal_list.txt>
  • Discontinued and Issues
    • Windows 2000: We have stopped supporting Windows 2000. Silver does not work on Windows 2000 anymore, since Windows 2000 is missing vital debugging support we rely on in Silver. The previous fix was to reduce debugging support for Windows 2000.
    • For an up-to-date list of known issues, bugs and workarounds, please click here.

What’s new in Silver 1.1.6?

  • A2LAccess now interprets CHARACTERISTIC and MEASUREMENT selection file entries as regular expressions
  • A2LAccess can now read data types FLOATXX_IEEE
  • A2LAccess module initialization more robust, to allow a more flexible placement in the sil-configuration
  • fixed a print format bug in the variable-info strings (was decimal decorated with 0x, now really hex)
  • SimBuild now supports sub-models embedded via ModelReference blocks

What’s new in Silver 1.1.5?

  • the Python module now allows the user to specify an alternate name to be used in Python code if the Silver variable’s syntax requires it (e.g. a.b[0])
  • A2LAccess now stores range information about CHARACTERISTICS and MEASUREMENTS in a file which is incrementally updated from run to run
  • A2LAccess now communicates range violations to TestWeaver
  • fixed a severe bug introduced in 1.1.4 in the A2LAccess module

What’s new in Silver 1.1.4?

  • very large set-ups (> 10000 variables) are loaded much faster and have smaller memory footprint
  • the XCP module now supports DAQ lists (static configuration only)
  • enhanced the FileWriter widget
  • minor bug fixed when recovering from an exception in the GUI

What’s new in Silver 1.1.3?

  • Silver’s graphic interface is much faster in some situations
  • a few minor bugs have been fixed

What’s new in Silver 1.1.2?

  • improved module export from Dymola with dymbuild
  • extended command set for the XCP module
  • minor bug fixed – for silver option -c

What’s new in Silver 1.1.1?

  • improvement: the CSVWriter supports the -l option
  • some minor bugs fixed

What’s new in Silver 1.1.0?

Silver

  • Silver User Manual now also available as pdf-file in SILVER_HOME\help
  • Users can develop custom widgets using the plug-in mechanism
  • Improved reporting and handling of errors generated by modules
  • Statistics tab to display module initialization time and module run time
  • GUI widgets are much faster
  • The Module configuration tab has been enhanced:
    • New types of predefined fields
    • Detection of errors in configured fields
    • Parameter support in fields
    • In the preview line, file names are hyperlinks
  • Drag&Drop of variables from one widget to another also transfers variable properties
    (minimum and maximum value, color, tick size, etc.)
  • Drag widgets directly from the toolbar to the desired place on the screen
  • Assignment widget: new math functions added
  • Data Plotter widget
    • Plotting is much faster with large amounts of data points
    • The trace color, minimum and maximum values can now be set in the widget configuration dialog
    • Improved legend behavior with respect to scaling

Modules

  • Python module
    • Object oriented language, easy to learn, commonly used for test automation context
    • PID controller and filtering (IIR, limiter, gradient limiter) available
    • Easy automation of previously manual tasks now possible
    • Includes instrumentation library for comprehensive test automation with TestWeaver
  • XCP module
    • Implements an XCP slave and access to simulation data
    • Allows Silver to act as if it were an ECU towards data acquisition tools connected as XCP master (e.g. Canapé)
  • A2LAccess module
    • Utility module for access to ECU descriptions in A2L format – an ASAM (ASAP2) standard
    • Improved consistency checks for the A2L information (min-max bounds and address consistency checks)
  • CSV Reader and Writer module
    • Useful for easy automation of processes
    • Simple format for recording signals from the simulation or for driving the simulation with recorded input signals
Zurück zu Support