DIT085
Contents
Testing and Verification @ Chalmers / GU - Edition 2015
Contact
Lecturer
- Office: Halmstad University, E 305 (on Fridays at Chalmers: Jupiter, 477)
- Telephone 035 16 71 22 (on Fridays at Chalmers: (031 772) 60 77)
- Email: m.r.mousavi@hh.se
Instructors
Eugene Groshev
- Email: groshev@student.chalmers.se
Björn Olsson
- Email: olssonbv@gmail.com
Schedule
Lectures: Friday 10:15-12:00, Lindholmen Campus, Patricia, Alfons Hall
Supervised Labs: Fridays 13:00-16:45, Lindholmen Campus, Patricia, Mållgan Hall
Objectives
- Knowledge and understanding:
- understand the basic terminology of testing;
- name and describe basic concepts on testing, as well as different testing techniques and approaches
- describe the connection between software development phases and kinds of testing
- Skills and abilities:
- write models in at least one formal specification language;
- construct appropriate and meaningful test cases, and interpret and explain (to stakeholders) the results of the application of such test cases ( using appropriate tools) to practical examples
- plan and produce appropriate documentation for testing
- apply different testing techniques on realistic examples.
- Judgement and approach:
- compare different tools and techniques for testing software, and plan their use in appropriate contexts,
- compare and judge alternatives to testing, such as model checking and runtime verification
- identify and hypothesize about sources of program failures, and reflect on how to better verify the correctness of such programs.
Assessment
Assessment is performed in terms of the deliverable of the practical project (software + report) and a written examination.
- To obtain a Pass (Godkänd: G) mark for the course, the student must obtain a Pass mark for both the project report and the written examination.
- To obtain a Pass with Distinction (Väl godkänd : VG) mark for the course, the student must obtain a Pass with Distinction mark for both the project report and the written examination.
- Otherwise, the student will fail the course and obtain a Fail mark (Underkänd: U).
In order to pass the project with distinction, you need to either have pass with distinction in 2 out of the 3 mandatory parts, or 1 pass with distinction for the mandatory part and a pass for the optional part. In order to pass the project, you need to pass all 3 mandatory parts of the project.
Assessment Guide 2015
Final Examination 2015, Final Examination 2015 with Solutions
Slides and Study Material
Slides
| Lecture | Handouts / Slides | Other Material |
|---|---|---|
| Lecture 1: Terminology and Functional Testing
January 23, 2015 |
Handouts Slides | Chapters 1 and 4 of Ammann and Offutt |
| Lecture 2: Functional Testing
Guest Lecture by Micael Andersson, Volvo (Test-Driven Development and jUnit) January 30, 2015 |
Handouts Slides | Chapters 6 and 7 of Jorgensen
M. Grochtmann and K. Grimm, Classification trees for partition testing |
| Lecture 3: Guest Lecture by Micael Andersson, Volvo (Mocking and Test-Driven Development )
February 6, 2015 |
TDD Examples | |
| Lecture 4: Coverage Criteria
February 13, 2015 |
Chapter 2 Ammann and Offutt
Chapters 9 and 10 of Jorgensen | |
| Lecture 5: Model Checking
February 20, 2015 |
||
| Lecture 6: UI Testing
Guest Lecture by Emil Alégroth, Chalmers February 27, 2015 |
VGT Cheat Sheet (Examples, Exercises) | |
| Lecture 7: Slicing and Debugging
March 6, 2015 |
Handouts Slides |
Chapters 5, 6, and 13 of Zeller M. Wiser, Program Slicing |
| Lecture 8: Reviewing Model Examination
March 13, 2015 |
Model Examination Solutions |
Main Textbook
- P. Ammann and J. Offutt. Introduction to Software Testing. Cambridge, 2008.
Recommended Reading Material
- P.C. Jorgensen. Software Testing: A Craftsman’s Approach. Auerbach Publications, 3rd edition, 2008.
- L. Aceto, A. Ingolfsdottir, K.G. Larsen, and J. Srba. Reactive Systems: Modelling, Specification and Verification, Cambridge University Press, 2010.
- A. Zeller. Why Programs Fail? Morgan Kaufmann, 2nd edition, 2009.
- Sujoy Acharya. Test-driven development with Mockito.
- Marcin Grzejszczak. Instant Mockito.
- F. Vaandrager, A First Introduction to Uppaal. In J. Tretmans, editor. Quasimodo Handbook. To appear.
- M. Wiser, Program Slicing, Proc. of ICSE'81, pp. 439-449, ACM, 1981.
Project Description
The project is about Test-Driven Development of a WhatsApp-like server- and client-side application, which we call WhatsUpGU. The final implementation will comprise:
- A connection-based (TCP-IP-based) server implemented in Java, allowing for multi-threaded processing of multiple requests from clients, the server
has an XML interface for adding, editing, and fetching messages, and
- An Android client, able to connect to the server on a TCP IP connection using the XML-based interface protocol and able to deal with communication faults.
From the testing and verification perspective, the project comprises the application of the following techniques:
- test-driven development,
- unit testing using jUnit,
- gathering coverage metrics using EclEmma (or similar tools),
- integration testing, including developing stubs using Mockito (or similar tools),
- model checking using Uppaal, and
- UI testing using the Sikuli tool.
This is a group project that is to be carried out in groups of 4. You need to have formed your groups in GUL and emailed your group structure to the lecturer and both instructors by Thursday January 27 at 17:00; please put '[DIT085] Group Registration' in the subject line of your email.
The deliverables are to be delivered on the GUL system and discussed orally in the presence of all team members with one of the two instructors. The deliverables comprise a report and the implementation code: the report should document the major steps in each phase and code snippets (few concise examples) of how they are implemented and possibly screen shots of the results. Extensive pieces of program code should not be included in the report. The most important factors in judging the reports are: their logical structure and sufficiently clear explanation of the steps (all figures and code snippets should be accompanied with clear descriptions).
The deadlines are to be respected and each phase is to be delivered both on GUL and discussed before the deadline. You may get an extension of one week for at most two phases; for that you need to send an email before the deadline to the instructors. Sending an email is sufficient for receiving the extension and you need not wait for a response to your email.
As a general principle, when you find an ambiguity in the requirements, make a reasonable assumption and document it clearly in your report.
Frequently Asked Questions
| Practical | Deadline | Notes |
|---|---|---|
| Phase 1: TDD of a Unit | February 13, 2015 23:59 | |
| Phase 2: Integration (Testing) of the Server | February 27, 2015 23:59 | |
| Phase 3: Specification and Model Checking | March 06, 2015 23:59 | |
| Phase 4: UI Testing (Optional, see the rules for pass with distinction) | March 13, 2015 23:59 |
Acknowledgment
This course is based on the material produced for several earlier editions given at TU Eindhoven, the Netherlands. Part of the material has been produced (or is based on the produced material) by Judi Romijn and Tim Willemse.
