TDDD04 Software Testing

Lab 1: Unit Testing

Purpose

In this first lab assignment, you will make your first acquaintance with basic unit testing techniques, using frameworks for testing code at the unit level, and some techniques for integration testing. For this lab you will be working in Java and using Eclipse as your development environment. You are expected to install and try some additional software during these tasks, for the purpose of building and testing small code examples. Testing software is an art as well as a craft, with practices and techniques that you will learn about iteratively through these assignments. Software testing is not a practice that we should engage in only at the end of a development cycle, but should be an integral part of development. Therefore, some parts of the lab assignment will require you to write some software in order to practice testing during development. Software testing itself also changes the mindset of you as a developer, and forces you to consider, just as a scientist, how you can verify that your software works as you think it should. Indeed, it makes it paramount that you can phrase clearly what you mean that your application should do and why.

Preparation

In lab assignments in this course, you are assumed to use pair programming in the course, as in, taking turns to program, and providing active support to one another if you are not sitting at the keyboard. If you are not used to this from previous courses, now is a great time to practice!

You will use Eclipse as part of this assignment. If you have not seen this development environment before, it is a good idea to follow some a short tutorial. We will assume that you are familiar with not only the basic functionality of this development environment, but also its facilities for managing extensions, for the IDE itself as well as for applications built with Eclipse. When installing plugins for Eclipse, you do so from within Eclipse using the Update Manager. Additionally, you may need to read a tutorial on Java if you have not encountered it before. Hopefully, with a background in other object-oriented languages, there should not be any major difficulties. It is expected that you spend some time getting to know both the language and development environment before you start. In part 1 below, you will be introduced to the basic techniques for testing an application, and learn how to use unit testing frameworks to test programmatically.

Part 2 focuses on how to use testing as a practice during development, so that we can proactively write tests, even testing parts of our application without having all functionality implemented. Here, we practice creating testable, loosely coupled code. Finally, in part 3 you will inspect a real application, in this case an open source Java game, and review the extent to which the code is testable at the unit level, and possible consequences this may have.

Part 1, Unit testing: xUnit basics

The first exercise uses a small application that determines whether three integers represent possible side lengths for an isosceles, scalene or equilateral triangle. You may download the application as a zip archive or, preferrably, clone it from Gitlab if possible. Open the solution file in Eclipse, and you will see two projects: one main project for the application itself, and one test project.

Start by running the application and test it (use the "Run As" button and choose Java Application). It should look something like the screenshot below.

Write down a number of test cases that you believe will exercise the application and trigger all behavior that you consider interesting to test. Do not mind the details of the implementation just yet, but record your test cases in a file.

When trying the application with different values for the side lengths, you will notice that certain combinations of values will cause the application to crash and throw a runtime exception. There may also be other errors in the application. Take note of these errors as you test it. Note your expectations along with the actual outcomes.

To understand what is wrong, and how we would like the application to behave, we can devise separate tests that do not run the application from the Main method, but rather use special classes and methods in those classes to run specific methods methods of our program. Frameworks for running standalone tests like this are often called xUnit frameworks, as they share a number of features with the original SmallTalk test framework SUnit, and usually only differ in name by the first letter of their names, often corresponding to the language or runtime environment for which they are written (JUnit, Nunit and so on). In this part, we will make use of JUnit which is Java xUnit test framework that is natively supported by Eclipse. On the JUnit website, there are introductory exaples that show how JUnit works, but in general, it works like most other xUnit frameworks:

• There is a test runner, as in a separate executable that loads your tests as a library together with your main application, analyzes the test library to detect test classes and specially annotated methods in them.
• It conducts one-time initialization procedures to setup the test environment, followed by per- test initialization for each test method invocation.
• It invokes test methods in isolated threads, and listens for abnormal termination through exceptions to detect failing tests.
• Finally, it reports the results of running tests in a separate view or in connection with either tests or code.

As a developer, you may invoke the test runner from within your IDE, or use build-tool-specific commands for running your test suites from the command line. In this exercise, we will only look at what happens when we use the test runner from within the IDE (Eclipse). Here is what it looks like when you run the tests in the separate test project:

and the results appear in the JUnit view:

In the test suite called TriangleTest, an assertion can be made through the use of assertion methods from the JUnit class Assert. Uning assertions add additional test methods so that there are test methods for all behaviors that you deemed interesting to test, based on the test cases you recorded previously. You should have at least one test method for each type of triangle, and assertions relevant to establishing that each type of triangle is correctly detected. Also, write methods to tear down your test fixture. In this case, you should ensure that Sides property does not have a non-null value.

As you have created all the tests you believe should pass, modify the original application so that it conforms to your tests, and does not crash on any input.

Part 2, Unit/Integration testing: Stubs and Mocks

Writing good quality tests is an integrated practice in contemporary software development, and testing can even be done before you write your main application code. This practice, Test-Driven Development (TDD), has a number of potential benefits:

• You force yourself to be specific about what you see as the desired functionality of every component before you write it, which helps you think clearly about the functionality.
• You will write testable code, which substantially helps in making your code better by avoiding high coupling between components.
• Your code will be in a working state earlier with smaller tests, which will help you gain confidence in your product, and reduce the time to debug later. See paper by George and Williams from 2004 on the results from an experiment where they studied the effects of TDD.

Start by forking a Web Application, specifically a REST-based web service written using Spring.

The application that you forked has a first controller ValuesConroller set of tests for it. Now we want to start to develop this into a proper web application, beginning with tests.

We would like to expand the current application with some kind of persistent storage, as in either a database or a file to store and retrieve values using our ValuesController.

The ValuesController should be separate from the mechanism for managing the persistent store, and the model class for the types of values that we wish to store and retrieve. An interface LoginService containing the methods for managing the persistent store that correspond to the operations of our ValuesController is provided.

This is important, as we will start to modify the tests so that they reflect the expected behavior of our application when it works as intended, and we can store and retrieve values from it. However, it may be difficult to locate the source of an error if our tests require both the controller logic, the model class and the persistent store to be developed before we can test the controller. To test the controller logic separately from the functionality we have yet to implement, we will therefore create a test stub to fake the functionality of LoginService before we have a working implementation of it. The idea is that we substitute our fake implementation with a real one later once we have developed it, using techniques such as dependency injection (not part of this lab).

You should already have project dependencies set up for you in Eclipse, but if not, add the stubbing library Mockito and, if you wish, the JUnit Assertion class from Part 1. In your ValuesController update the methods to properly make use of the LoginService.

Use Mockito to create a stub of LoginServicee , and make sure that your stub acts consistently with respect to storing and retrieving values. An invocation of a method to store values should result in those values being retrieved later, for instance. See the Mockito tutorial on Vogella for inspiration or the Spring tutorial.

After you have verified that all controller tests pass with your stubs, create tests for, and a real implementation of, the LoginService interface. One way to do this is by simply serializing the objects and writing them to a file using BufferedWriter, and reading them back with a BufferedReader. Make sure to write your tests first, and the implementation later, TDD style. Finally, add tests that verify that your controllers work with the new implementation (small-scale integration tests).

Part 3, Testability

In this final part, you will inspect the effect of high coupling, as in direct dependencies between concrete classes, on your ability to test software components in isolation. Here, you will use the Java-based strategy game FreeCol, which is also featured in the parallel software engineering course TDDB84 (Design Patterns). Free cool needs Java 1.8. If necessary, you can change the JRE or JDK version in Eclipse as follows: select Java Build Path -> Libraries, select JRE System Library, click Edit and choose whichever JRE or JDK you like.

We will use the JaCoCo code coverage engine in the EclEmma Eclipse plugin to provide information about the extent to which we exercise all statements or branches of the code examples. Start by forking the project from Gitlab. As you open the project in Eclipse (by copying the sources to your Eclipse workspace and creating a new Java project with the same name as the project), you will see that there are a number of unit tests in the project. Run the unit tests with EclEmma to obtain coverage results for FreeCol. See example screenshot below:

You will notice that some parts are not completely tested, and it may happen that some tests hang during testing (We have seen cases when the game client cannot connect to the test server implementation). Note the coverage of the test code as well: have all statements in the test methods been executed? If not, what does that tell us? See screenshot on the next page:

What is also interesting, is to consider how you would go about creating unit-level tests to verify functionality in the game engine, say in the net.sf.freecol.common.model.Colony class, where some methods are not tested completely, or at all. Note the dependencies in these classes, and draw a dependency diagram, as in, a UML class diagram (you may use the ObjectAid Eclipse plugin to help you). The diagram should show what classes are needed in order to create and use objects of type Colony, for the purpose of testing Colony methods. Using your general knowledge of abstraction and object-oriented design from other courses, suggest a design change that would simplify testing Colony methods separately, without requiring (many) other concrete objects to be created as well. Take inspiration from part 2 where you did exactly this. Also, describe why it is a problem, in concrete terms and in this situation, to have high coupling between concrete classes. What are the effects on your test code? On your code coverage? Could there be issues in understanding the reason behind failing tests?

Reporting your results

See the general instructions for successfully completing the lab assignment. Note that you will have to upload the results on Gitlab before demonstrating to your assistant. Provide answers to the questions above, and describe the bugs in the last exercise as precisely as you can.

• From part 1, you will need to push your tests cases to Gitlab, provide screenshots that demonstrate that your tests pass, and you shall push your revisions of the Triangle application to make it behave correctly.
• From part 2, you will need to provide your test cases of ValuesController along with your stubs, your implementation of the storage system, and the integrated test cases where you use your storage implementation. Provide a screenshot of your successful test cases.
• From part 3, you will need to provide a class diagram of class Colony and related classes, along with answers to the questions in part 3.

Page responsible: Lena Buffoni
Last updated: 2019-09-09