Our strength is the automated, model-based generation of test-suites using graphical usage models for hardware and software systems. Test suites can be further processed as JSON documents for the development of executable test cases or can be visualized in a user-friendly way.

Pet Clinic - Model-based testing of Spring applications

The following example shows how Spring applications can be tested using the TestPlayer© framework. After installing via Github the Spring application Pet Clinic is started via the local URL http://localhost:9966/petclinic/ and presents the following view:

A detailed description of the Spring Application Pet Clinic can be found on this site.

A correspondent usage model, which serves as a basis for the automatic generation of test cases, has the following structure:

After the initial start state [, which is executed when the test case is started, only the Home usage state is directly reachable, then the other main states Find owners, Veterinarians and Error can also be selected from the toolbar. Depending on the selected usecase, the Find owners and Veterinarians usage states take you to the other sub states. A test case is completed when the final stop state ] has been reached.

State transitions between the 34 usage states are triggered by generic events e1 to e124, which were automatically generated by the TestPlayer© together with the associated transition probabilities p1 to p124 during test-suite generation.

Using the TestPlayer© you can automatically generate test cases for the model-based testing of the Spring application Pet Clinic based on the usage model above. The following test-suite contains 27 test cases for the complete coverage of all usage states, which are generated according to the sort criterion sort_l , i.e. the test cases have been sorted out of 500 statistically randomly generated test cases according to their length:

  • Test suite state coverage contains 27 (out of 500) test cases
The test cases are animated (marked by a bright orange coloration) and show the coverage of the usage states (characterised by a slight orange colouring) as well as the transitions between the states. The number after the colon of the respective click event shows how often the respective state transition was executed during the test case.

While generating the test-suite, the TestPlayer© produces several diagrams, which help to understand the characteristic properties of the test-suite better. This includes e.g.

  • deviation of the statistical frequencies of usage states from the theoretical distribution (if no probability values are specified, this is equivalent to a uniform distribution of all state transitions)

  • frequencies of the usage states

  • frequencies of the transition events

In addition to the graphical representations, the TestPlayer© also generates text files that provide the test cases of the test-suite in a compact JSON format for the test execution. File test_cases_for_state_coverage.json contains all 27 test cases of the test-suite State Coverage for the test of the Spring application Pet Clinic:

For automatic testing of the Spring application Pet Clinic you need additional drivers to automate the access to the web browser. For this purpose we are using the automation framework SeleniumHQ, which can be easily integrated into a testing environment that is based on Eclipse :

Python script TestPetClinic.py contains a description for each usage state which actions, i.e. realizations of the abstract test steps e1 to e124 that are to be executed during the test step. Test step

for example defines what to do when reaching usage state New Vist in order to create a new visit appointment during the test execution. In detail, the following actions must be performed:

  1. Checking pre-condition <New Visit> ensures that the actions of the test step are executed in the New Visit state. In case of failures, the else-branch is performed and a corresponding error message is displayed via the assert statement.
  2. Enter a description in the description input field and click the submit button Add Visit:

  3. Checking post-condition <Owner Information> ensures that the correct state transition to the usage state Owner Information takes place. In addition, the assert statement controls the execution of the test step and assigns the test verdict pass, fail or error depending on the test result.

Run Configurations is used in Eclipse to pass call parameters for test execution:

File option -f test_cases/PetClinic/500_sort_l/test_cases_for_state_coverage.json.txt loads the automatically generated test-suite to cover all usage states during test execution.

Pressing the Run button starts the web browser in automatic mode which performs all actions of the test-suite:

During test execution, the individual test cases are logged in the console and their execution times are displayed at the end:

At the end of the test execution, the result is recorded in a detailed test report. The following data is provided for each test case in the test-suite:

  • Start Time : timestamp when the test case was started
  • Duration : execution time of the test case
  • Result : test verdict ( pass , fail , error ) and the number of test steps
  • test case # : number of the test case in test-suite (1, ... )
  • file name of the test-suite
Further details can be set for each test case in the Show area:
  • Summary : short summary of all test steps
  • Failed : detailed presentation of all test steps having test verdict fail
  • All : detailed presentation of all test

The test report shows that all other test cases of the test-suite except test case #20 passed the test with test verdict pass . The Show section of test case #20 allows you to analyze the failed test step more thoroughly:
  • Test step AddNewVisit has test verdict fail
  • Test steps AddOwner und NewVisit are the immediate predecessors. This explains why test step AddNewVisit failed: a new owner doesn't yet have a pet, so you can't create a visit>!

The following video shows the described procedure and all essential steps once again in a compact representation:

The individual sections in the video are arranged in chronological order as follows:
  • 0:0-1:07: using Spring Application Pet Clinic
  • 1:08-2:48: using the TestPlayer© Dashboards to create test-suites automatically
  • 2:49-2:55: usage model of the Spring application Pet Clinic
  • 2:58-3:44: Eclipse/Python test environment
  • 3:48-4:20: start of the automated execution of the test-suite to cover all usage states
  • 4:21-4:49: shortened test execution and output of test steps in the Eclipse console
  • 4:50-5:23: display of the error free test steps #1 to #19 of the test report
  • 5:24-5:36: analysis of the failed test step AddNewVisit
  • 5:37-6:11: display of the error free test steps #21 to #27 of the test report
  • 6:13-6:20: changes due to the automated test execution of the Spring application Pet Clinic
  • 6:23-6:33: newly generated user Peter Black who triggers the failed test step #20 AddNewVisit