Hide menu

TDDD04 Software Testing

Lab 5: Model-based testing


Purpose

So far in the course, we have seen several techniques that may be useful for testing units of an application, integrating components of an application and generating regression tests for verifying the correct behavior of applications. Testing an application through the use of either static test scripts or exploratory testing may prove insufficient though. Exploratory testing allow us to test all functionality that is possible to reach from the main interface of an application, and has the potential of revealing new bugs, but is time-consuming and costly. Script-based testing is of course much cheaper, but also much more restricted in what it tests, and by their definition, scripts only ever follow a certain path through an application, and sometimes only a restricted part of an application.

If we want to explore how an application behaves under all conditions in which we have stated that it should work, and efficiently find sources of errors during testing, we may have to explore other options as well. One option is to use models of how a certain piece of software behaves, and then use such a model to generate possible interaction patterns with the software, and ascertain certain invariants or facts about our system during such generated interactions.

A model may be a UML state diagram , or some other formalism to describe sets of interactions that are possible, along with their prerequisites and consequences in our system. The general idea is to ensure that there is an abstract model of your software system that you keep in sync with your implementation, and whenever the software changes, your model should reflect these changes and enable you to test many different paths along your application, given that you have defined how to move from one state to the next, such as moving from the "Logged out" state to the "Logged in" state by, well, logging in.

In this lab assignment, you will use the Model-based testing tool GraphWalker developed by people at Spotify in order to run tests and verify the behavior of a system, create changes to your application and ensure that GraphWalker can detect those changes, share your changes with your peers, and work together to resolve the issues that arise.

Preparation

Preparations In this lab, you will work together with other pairs of students within your team, which corresponds to a group in WebReg. Arrange yourselves within those teams so each of you share code in a circular fashion (pair 1 shares changes with pair 2, pair with pair 3, ... and pair 5 with pair 1).

Start by reading how GraphWalker works, and try the smallest Maven-generated GraphWalker example using boiletplate code to understand how Maven and Graphwalker work. Maven is a tool for building applications, as well as maintaining dependencies and a build environment for you, and possibly generating boilerplate code, which is what the example above will help you with. To do the second part of this lab assignment, you will need Firefox version 47.0.1 (available locally for the course in the folder ~TDDD04/firefox) and yEd, a graph editor. Do not allow Firefox to update to a later version than 47.0.1 as this will cause compatibility problems with the rest of the tools needed in the lab. Yed is for editing the model files used to describe the models in PetClinic application. You will get access to both the correct version of Firefox and the yEd graph editor by using the software module /home/TDDD04/module through the command

module add /home/TDDD04/module

and, for ensuring that the module is loaded when you log in again

module initadd /home/TDDD04/module

After running these commands, the command will give your firefox version 47.0.1, and firefox is the command used to launch the yEd graph editor.

Make sure that you can run the Maven-run Graphwalker test suite in the mini project by issuing the command mvn graphwalker:test from the command line in the boilerplate project.

Part 1

In the project, you will see a small example of how to exercise the edges and the vertices in this model in the file src/main/java/com/company/SomeSmallTest.java.

Imagine that the state transition graph described in the GraphML file (src/main/resources/com/company/SmallTest.graphml) really described a state machine, such as a vending machine, and the test code exercised this state machine somehow. Some of the transitions in the model may even be invalid, as in not properly reflecting the actual machine.
  1. Introduce a change in the test code so that, when executing one of the edges twice, it will invalidate an assertion in the test program. GraphWalker may not detect this, so you may need to modify the annotation of the test class so that the generator and stop condition allow you to traverse each edge of the graph more than once. Does GraphWalker detect the error now?
  2. In general, there are several things that could happen as you develop your application and model for testing it, and several interesting questions regarding the relationship between models and implementations:
    • What is the relationship between the model coverage, as in edge coverage, vertex coverage or path coverage when GraphWalker executes the tests that implement the model steps, and code coverage of the application code?
    • Does 100% edge coverage imply 100% code coverage? Does it imply 100% branch coverage?
Also, given that you have described all different paths through an application in your model, what would happen if some paths are invalidated by further development of your application, where new conditions are introduced for transitions?

In the next part, you will explore what happens when such changes take place, when working with your team to share code changes.

Part 2

UPDATE 2018: There seems to be a problem with testing adding the new pets, therefore we will be removing that part from the tests.

If you are having problems running the model, in the graphwalker pet clinic example: remove new pet in the model for OwnerInformationSharedState and update the OwnerInformation.java to remove the associated methods and rebuild the proejct.

In this assignment, you will work with the GraphWalker PetClinic example which is a somewhat larger, but still small web application. In the tests, you will use a web browser automation tool called Selenium, which enables you to automate browser interactions with a web application, such as clicking on links, filling out forms, etcetera. It will not help you assess the visual appearance of an application, but you can traverse the application from the perspective of the web browser. The tests that GraphWalker uses in this case include browser interactions and verification of the outcome of such interactions. The edges are actions, and the nodes are where you verify the outcome of your actions, and the validity of the current application state.
  1. Make sure you can run the test suite in the PetClinic example successfully before you proceed.
  2. Next, you will introduce changes in the application that will require your model to change. Either, you may add constraints to the domain model, as in the pets, owners and veterinarians, such as that names should have a certain limited length, or that the age of owners should be in a given range. Also ensure that the tests fail after the introduction of your changes unless you change the tests. For instance, if you change the required length of a telephone number for a pet owner, your model should fail to verify the current model of the PetClinic application state in which you add a new owner. Introduce changes to the PetClinic application that in turn require changes to either
    • the model edges, including guards and actions
    • the model states,
    • the model step definitions, or
    • the generator or stop conditions
    to be detected. Introduce two such changes to your local PetClinic application.
  3. After you have introduced these changes, you shall share them within your team. Use Gitlab for this purpose, and share through a common repository of your modified PetClinic applications. After you have cloned the PetClinic repository to your own machine and local repository, add a new remote repository on Gitlab. Start by creating a bare git repository for your shared code, and then pushing your application. We suggest you use separate branches for your pairs, so you can switch between them.
  4. Make sure you can detect each others' modifications to the application in your Graphwalker tests, and resolve the issues you encounter.

Reporting your results

See the general instructions for successfully completing the lab assignment. Note that you will have to upload the results on Gitlab before demonstrating to your assistant. Each pair shall provide a 2 page report where you describe:
  • answers to the questions in part 1
  • your changes to the PetClinic application (refer to commits and describe them)
  • the changes that you received from your peers, how your model-based tests detected them, and how you could revise your model to correctly reflect the new functionality introduced
Also, you shall provide your modified code to the PetClinic application as part of your results.

Page responsible: Lena Buffoni
Last updated: 2018-10-10