Professional testing done playfully easy!




Model-based testing is an innovative test approach to improve effectivity and efficiency of the test process. A model-based tester on a project uses models to drive test analysis and design, and keeps advantage of the models for other testing activities such as test implementation and reporting.


( ISTQB® MBT certification )



This is the starting point of our testing and consulting activities:

Our strength is the automated, model-based generation of test suites using graphical usage models for hardware and software systems.

Our highly efficient tool TestPlayer© guides you quickly and accurately to an effective test suite generation for your IT applications.

The TestPlayer© is independent of special application domains, nevertheless, we can refer to expert's report in the areas of Information and Communication Technology, Medicine Technology and Automotives.





Scroll To Top

Model-driven Test Case Generation


An approach, where the test cases are generated in parts or completely from a model is called Model-driven Testing.

The V Model


In general, a distinction is made between

  • System specifications, which model the functional behavior of the SUT, i.e. the system under test and
  • Usage models that model the usage behavior of the future users of the system.

Test cases, which are generated from a system specification are mostly used in the so-called component- or unit test. Usage models are applied for the generation of test cases for the system- and acceptance test, respectively.

The widespread V Model clarifies this relation by distinguishing between the development and the test of a system.

Scroll To Top

Statistical Test Case Generation


With the help of statistical usage models, so-called Markov chains, individual test cases or complete test suites are automativally provided by simple push button operations.

Markov Chains


Markov chains are graphical models and represent


  • usage states for modelling the user behavior during the interaction with the system, as well as
  • state transitions to specify the reaction of the system on a user's interaction.

For example, the input of a search item in the search field of an Internet browser is an interaction of the user with the browser and the output of a list of web pages fulfilling the searching criterion represents the expected reaction of the system. Both usage states <input of a search item> and <output of a list with web pages> as well as the state transition between these usage states that is triggered by the interacting user after pressing the Enter key are described in the Markov chain.



Usually there are many alternatives to reach from a given usage state possible successor states. For example, instead of pressing the Enter key in our example a different action may be performed, such as switching to another window by using the mouse. This will take the user to another usage state.



The probability that a particular action can be executed is called transition probability between the usage states.

By adjusting the probability values it is very easy to define different user classes. In this way, you can create automatically different test cases for different system users.



Scroll To Top

Automatic Test Case Generation


The TestUS TestPlayer© is a versatile tool for the automatic generation of test cases and enables the user-friendly analysis, evaluation and visualization of the resulting test suites.

The TestPlayer©


Using the TestPlayer©, it is possible to assess the quality of the generated test cases on the basis of graphical representations early before the actual test suite execution.


The TestPlayer© shows very fast basic characteristic properties of a given test suite, such as


  • the maximium length of a test case , i.e. the number of single test steps between the start and the end state of the markov chain,
  • the mean length of the test cases in a test suite,
  • the accordance between the usage profile of the Markov chain and the usage frequencies of the generated test suite,
  • the coverage of all usage states after having executed the test suite and
  • the coverage of all state transitions after having executed the test suite.

Our expertise and experience from various national and international projects show that statistical usage models based on Markov chains are ideally suited for the testing of applications in different domains, such as


  • Information and communication technology,
  • Medical technology,
  • Automotive , as well as
  • Automation technology.

The concept of the TestPlayer© is very general and independent of specific application domains. In this way, test models are created that can be used in any domain.



Scroll To Top


Automatic Test Suite Generation


Our strength is the automated, model-based generation of test suites using graphical usage models for hardware and software systems. Test suites can be stored as JSON documents for the development of executable test cases or can be visualized in a user-friendly way.

The Test Player© Dashboard enables the test developer to set configuration parameters in a convenient way to start the generation of the required test cases per mouse click.

After a few seconds the TestPlayer© has automatically generated an abstract test suite, which can be downloaded by the filehandler as a JSON document.

By inspecting characteristic features of the test suite, such as coverage properties or mean length of a test case, the test developer can quickly assess required quality criteria.

The following graphs are single frames of a dynamic coverage diagram and show the visualization of accumulated test cases. This means that all usage states of the test suite are covered at least once during the test execution.

Each test case begins in the start state [ and stops in the end state ].

Tab MCUM (Markov Chain Usage Model) represents in the first frame the usage model, which is given by the test developer. Succeeding frames show the next accumulative test case until a complete coverage of all usage states is reached. Univisited states are colored in blue, states of the accumulated test case and already visited states are colored in orange.

Usage models are edited by means of a graphical editor, which is available in the Model editor section.

The remaining tabs allow the choice between several visualization algorithms and topologies, which are provided by the TestPlayer© for selecting divers representation options of a given test suite. The first frame always represents the usage model, each succeeding diagram shows the next accumulative test case until a complete coverage of all usage states is reached.


The TestPlayer© allows to perform an efficient and targeted analysis and visualization of generated test suites. For this purpose divers analysis and visualization algorithms are provided that possess different characteristic properties.



Scroll To Top


Overview


Our strength is the automated, model-based generation of test suites using graphical usage models for hardware and software systems.

The TestUS TestPlayer© is a versatile tool for the automatic generation of test cases and enables the user-friendly analysis, evaluation and visualization of the resulting test suites.

Using statistical usage models, so-called Markov chains, individual test cases or complete test suites are automatically provided by simple push button operations. Markov chains are graphical models and represent


  • usage states for modelling the user behavior during the interaction with the system, as well as
  • state transitions to specify the reaction of the system on a user's interaction.

The probability that a particular action can be executed is called transition probability between the usage states.

By adjusting the probability values it is very easy to define different user classes. In this way, one can automatically create various test cases for different system users.




Using the TestPlayer©, it is possible to assess the quality of the generated test cases on the basis of graphical representations early before the actual test suite execution.

The TestPlayer© shows very fast basic characteristic properties of a given test suite, such as


  • the maximium length of a test case , i.e. the number of single test steps between the start and the end state of the markov chain,
  • the mean length of the test cases in a test suite,
  • the accordance between the usage profile of the Markov chain and the usage frequencies of the generated test suite,
  • the coverage of all usage states after having executed the test suite and
  • the coverage of all state transitions after having executed the test suite.



The concept of the TestPlayer© is very general and independent of specific application domains. In this way, test models are created that can be used in any domain.

Our expertise and experience from various national and international projects show that statistical usage models based on Markov chains are ideally suited for the testing of applications in different domains, such as


  • Information and communication technology,
  • Medical technology,
  • Automotive , as well as
  • Automation technology.



Our highly efficient tool TestPlayer© guides you quickly and accurately to an effective test suite generation for your IT applications.



Scroll To Top


General Approach


The Testus TestPlayer© is a versatile tool for the automatic generation of test cases and the analysis, visualization and evaluation of generated test suites.

For the automatic generation of test cases a statistical usage model is required, which is also called MCUM (Markov Chain Usage Model). Usage models are created by applying the graphical editor, which is available in the Model editor section.




In the following we explain the main steps for the automatic generation of a test suite and the visualization of the resulting test cases by using a simple example from [1]. The goal is to create test cases for validating the correct behavior of a window manager, which is considered as the SUT (System under Test) for our example.

The usage model for the window manager contains the following usage states that can be reached by individual test steps and their correct behavior should be checked in the course of the test:

  1. After starting the test case from the initial state Invocation a window will open and the SUT is in the state Window.
  2. Starting from the state Window the following states are available:
    1. Maximize: maximizing the size of the window
    2. Minimize: minimizing the size of the window and reduction to an icon
    3. Move: moving of the window
    4. Size: arbitrary change of the window by using the mouse
    5. Close: closing the window
  3. Starting from states Window, Move und Size the state DragMouse is reachable.
  4. Starting from state Windowthe following states are reachable:
    1. Up: moving the mouse upwards
    2. Down: moving the mouse downwards
    3. Left: moving the mouse to the left
    4. Right: moving the mouse to the right
  5. In the state Close the window manager will terminate and the test case is terminating in the final state Termination.



By applying the graphical editor from section Model editor the following incomplete usage model for testing the window manager is created first:



In order to automatically generate test cases using the above usage model, additional events and transition probabilities that trigger state changes have to be annotated on the edges. This task can be achieved by the TestPlayer© Dashboard without major effort, as shown in the section Completion of missing graph elements. The result is an expanded usage model:




Via the TestPlayer© Dashboard [2] we can automatically generate test cases that can be sorted by various criteria. Exact details are explained in the section Automatic generation of test cases. By means of specific metrics, the characteristic properties of a test suite can be analyzed:


  • SSP: Comparison of the steady state probability distribution of the usage model and the relative frequencies of the corresponding states in the generated test suite. As seen in the diagrams, the values for individual usage states are in very good agreement.








  • SSV: Comparison of the mean number of test cases needed to visit a certain usage state once in the usage model and during the test execution. As seen in the diagrams, the values for individual usage states are in very good agreement.






Other interesting metrics are discussed in the section Graphical output of test suite metrics.




After having analyzed the test suite, the resulting test cases can be visualized as explained in Section Graphical visualization of test cases. Edge labels represent the trigger event and how often the edge is visited during the test case execution. The following four test cases show how the coverage of all nodes of the usage model can be achieved by executing the test suite:



  • Test case number 1:


  • Test case number 2:


  • Test case number 3:


  • Test case number 4:


  • Test case number 5:


  • Test case number 6:


  • Test case number 7:


  • Test case number 8:


  • Test case number 9:


As seen the marking (orange colour) of previously tested usage states is increasing test case by test case until all states are marked.




Publications:


  1. J. A. Whittaker and J. H. Poore. Markov Analysis of Software Specifications. ACM Transactions on Software Engineering and Methodology, 2(1):93{106, 1993.
  2. W. Dulz. A Comfortable TestPlayer© for Analyzing Statistical Usage Testing Strategies. Proceedings of the 6th ICSE/ACM Workshop on Automation of Software Test (AST '11), 2011.



The concept of the TestPlayer© is very general and independent of specific application domains. In this way, test models are created that can be used in any domain.



Scroll To Top


References


The TestUS TestPlayer© is a versatile tool for the automatic generation of test cases and enables the user-friendly analysis, evaluation and visualization of the resulting test suites.

The following list contains selected puplications that cover various testing techniques and application domains that can be implemented with the Test Player:


  1. Vitali Schneider, Anna Yupatova, Winfried Dulz, Reinhard German. How to Avoid Model Interferences for Test-driven Agile Simulation based on Standardized UML Profiles. Proceedings of the 4th IEEE/ACM Workshop on Model-driven Approaches for Simulation Engineering, 2014.
  2. W. Dulz. Model-based Strategies for Reducing the Complexity of Statistically Generated Test Suites. Proceedings of the 2013 Software Quality Days, 2013.
  3. A. Djanatliev, W. Dulz, R. German, and V. Schneider. VeriTAS - A Versatile Modeling Environment for Test-driven Agile Simulation . Proceedings of the 2011 Winter Simulation Conference, 2011.
  4. W. Dulz. A Comfortable TestPlayer© for Analyzing Statistical Usage Testing Strategies. Proceedings of the 6th ICSE/ACM Workshop on Automation of Software Test (AST '11), 2011.
  5. I. Dietrich, F. Dressler, W. Dulz, and R. German. Validating UML Simulation Models with Model-Level Unit Tests. Proceedings of the 3rd International ICST Conference on Simulation Tools and Techniques (SIMUTools '10), 2010.
  6. W. Dulz, R. German, Stefan Holpp and Helmut Götz. Calculating the usage probabilities of statistical usage models by constraints optimization. Proceedings of the 5th ICSE/ACM Workshop on Automation of Software Test (AST '10), 2010.
  7. W. Dulz, S. Holpp and R. German. A Polyhedron Approach to Calculate Probability Distributions for Markov Chain Usage Models . Electr. Notes Theor. Comput. Sci. 264(3), 2010.
  8. W. Dulz. On-the-fly Interpretation of Test Cases in an Automatically Generated TTCN-3 Test Suite. Evaluation of Novel Approaches to Software Engineering, in Communications in Computer and Information Science, Volume 69, I, 17-30, 2010.
  9. S. Siegl, W. Dulz, R. German, G. Kiffe. Model-Driven Testing based on Markov Chain Usage Models in the Automotive Domain . Proceedings of the 12th European Workshop on Dependable Computing, 2009.
  10. F. Prester and W. Dulz. markov@.mzT - Model Centric Testing Using Markov Chain Theory. Proceedings of the embeddedWorld 2009.
  11. M. Beyer, W. Dulz and K.-S. Hielscher. Performance Issues in Statistical Testing. Proceedings of 13th GI/ITG Conference Measuring, Modelling and Evaluation of Computer and Communication Systems, 2006.
  12. M. Beyer and W. Dulz. Scenario-Based Statistical Testing of Quality of Service Requirements . International Workshop "Scenarios: Models, Transformations and Tools", Revised Selected Papers, Springer LNCS 3466, 2005.
  13. W. Dulz and F.-H. Zhen. MaTeLo - Statistical Usage Testing by Annotated Sequence Diagrams, Markov Chains and TTCN-3. Proceedings of the Third International Conference on Quality Software (QSIC '03) , 2003.
  14. M. Beyer, W. Dulz and F.-H. Zhen. Automated TTCN-3 test case generation by means of UML sequence diagrams and Markov chains. Proceedings of the 12th Asian Test Symposium (ATS '03), 2003.



The concept of the TestPlayer© is very general and independent of specific application domains. In this way, test models are created that can be used in any domain.



Scroll To Top

Site Notice

TestUS Consulting for Information and Communication Systems


Firm Owner:
Heidi Klotz

Mailing Address:
Cimbernstr. 11, 90402 Nuremberg
Phone: (+49) 911 4809783
Telefax: (+49) 911 46206312
info@testus.eu
http://www.testus.eu

Commercial Register Nürnberg-IHK-Nr. 1058509
Tax number 240/237/21151
Sales tax ID-DE 260 808 350




Liability for content

As a service provider we are responsible for our own content on this site under the general law according to § 7 Abs.1 of the German TMG (Telemedia Act). However, according to §§ 8 to 10 TMG as a service provider we are not obliged to monitor transmitted or stored information or to investigate circumstances that refer to an illegal activity. Obligations to remove or block the use of information under the general laws remain unaffected. However, a liability relating to this is only possible from the date of knowledge of a concrete infringement. Becoming aware of such violations, we will remove the content immediately.




Liability for links

Our site contains links to external websites over which we have no influence. Therefore for these foreign contents we can assume no liability. For the content of linked pages always the respective provider or operator is responsible. Linked Sites were checked at the time of linking for possible legal violations. Illegal contents were not recognizable at the time of linking. A permanent control of all linked pages without concrete evidence of a infringement is not reasonable though. Upon notification of violations, we will remove such links immediately.




Copyright

The contents and works created by the site operators on these pages are subject to German copyright law. Reproduction, adaptation , distribution or any kind of exploitation outside the limits of copyright require the written consent of the author or creator. Downloads and copies of these pages are only for private, non- commercial use. Insofar as the content on our web site was not created by the operator, the third-party copyrights are respected. In particular, third parties contents are marked as such. Should you become aware of copyright infringement despite, we ask you for a reference accordingly. Upon notification of violations, we will remove such contents immediately.

Note:
Despite careful control, we accept at no time the liability for the content of external sites. The web site maintainers are solely responsible for the content of those external sites.

Scroll To Top

TestUS contact data

Dr.-Ing. Winfried Dulz
Cimbernstr. 11
90402 Nürnberg
info@testus.eu
(+49) 171 7422626
(+49) 911 46206312


TestPlayer© Login


An approach, where the test cases are generated in parts or completely from a model is called Model-driven Testing.



Please input account name and password for your personal TestPlayer© access here!

Scroll To Top

TestPlayer© Sign-up


The TestUS TestPlayer© is a versatile tool for the automatic generation of test cases and enables the user-friendly analysis, evaluation and visualization of the resulting test suites.



Please fill in the sign-up form below to get a free TestPlayer© trial account!


The concept of the TestPlayer© is very general and independent of specific application domains. In this way, test models are created that can be used in any domain.



Scroll To Top

© 2017  ​TestUS Consulting Inc.