DIT085 Ed 2016

Testing and Verification @ Chalmers / GU - Edition 2016

Contact

Lecturer

Mohammad Mousavi

• Office: Halmstad University, E 305 (on Fridays at Chalmers: Jupiter, 477)
• Telephone 035 16 71 22 (on Fridays at Chalmers: (031 772) 60 77)
• Email: m.r.mousavi@hh.se (Please note: I do not access the GU / Chalmers email account.)

Instructors

Sali Masri

• Email: guselmsa@student.gu.se

Omar Abu Nabah

• Email: gusabunom@student.gu.se

Schedule

Lectures: Friday 10:15-12:00, Lindholmen Campus, Patricia, Alfons Hall

Supervised Labs: Thursdays at 08:15-10:00 and Fridays 13:00-16:45, Lindholmen Campus, Patricia, Mållgan Hall

Objectives

• Knowledge and understanding:
  • understand the basic terminology of testing;
  • name and describe basic concepts on testing, as well as different testing techniques and approaches
  • describe the connection between software development phases and kinds of testing

• Skills and abilities:
  • write models in at least one formal specification language;
  • construct appropriate and meaningful test cases, and interpret and explain (to stakeholders) the results of the application of such test cases ( using appropriate tools) to practical examples
  • plan and produce appropriate documentation for testing
  • apply different testing techniques on realistic examples.

• Judgement and approach:
  • compare different tools and techniques for testing software, and plan their use in appropriate contexts,
  • compare and judge alternatives to testing, such as model checking and runtime verification
  • identify and hypothesize about sources of program failures, and reflect on how to better verify the correctness of such programs.

Assessment

Assessment is performed in terms of the deliverable of the practical project (software + report) and a written examination.

• To obtain a Pass (Godkänd: G) mark for the course, the student must obtain a Pass mark for both the project report and the written examination.

• To obtain a Pass with Distinction (Väl godkänd : VG) mark for the course, the student must obtain a Pass with Distinction mark for both the project report and the written examination.

• Otherwise, the student will fail the course and obtain a Fail mark (Underkänd: U).

In order to pass the project with distinction, you need to either have pass with distinction in 2 out of the 3 mandatory parts, or 1 pass with distinction for the mandatory part and a pass for the optional part. In order to pass the project, you need to pass all 3 mandatory parts of the project.

Slides and Study Material

Slides

Lecture	Handouts / Slides	Other Material
Lecture 1: Terminology and Functional Testing January 22, 2016	Lecture 1 Slides Lecture 1 Handouts
Lecture 2: Functional Testing Guest Lecture by Micael Andersson, Combitech (Test-Driven Development and jUnit) January 29, 2016	Lecture 2 Slides Lecture 2 Handouts Guest Lecture 1 Slides Test Driven Dev. Examples	Chapters 6 and 7 of Jorgensen M. Grochtmann and K. Grimm, Classification trees for partition testing
Lecture 3: Guest Lecture by Micael Andersson, Combitech (Mocking and Test-Driven Development ) February 5, 2016	Guest Lecture 2 Slides Test Driven Dev. Examples - Part 2 Test Mocking Examples
Lecture 4: Coverage Criteria February 12, 2016	Lecture 3 Slides Lecture 3 Handouts	Chapter 2 Ammann and Offutt Chapters 9 and 10 of Jorgensen
Lecture 5: Model Checking February 19, 2016	Lecture 4 Slides Lecture 4 Handouts	F. Vaandrager, A First Introduction to Uppaal Gossip and Jobshop Examples USB Channel Example
Lecture 6: UI Testing Guest Lecture February 26, 2016	Lecture 5 Slides (PowerPoint)	VGT Cheat Sheet (Examples, Exercises)
Lecture 7: Slicing and Debugging March 4, 2016	Lecture 6 Slides Lecture 6 Handouts	Chapters 5, 6, and 13 of Zeller M. Wiser, Program Slicing
Lecture 8: Reviewing Model Examination March 11, 2016	Model Examination Model Examination with Solutions

Main Textbook

• P.C. Jorgensen. Software Testing: A Craftsman’s Approach. Auerbach Publications, 3rd edition, 2008.

Recommended Reading Material

• P. Ammann and J. Offutt. Introduction to Software Testing. Cambridge, 2008.

• A. Zeller. Why Programs Fail? Morgan Kaufmann, 2nd edition, 2009.

• Sujoy Acharya. Test-driven development with Mockito.

• Marcin Grzejszczak. Instant Mockito.

• F. Vaandrager, A First Introduction to Uppaal. In J. Tretmans, editor. Quasimodo Handbook. To appear.

• M. Wiser, Program Slicing, Proc. of ICSE'81, pp. 439-449, ACM, 1981.

Practical Assignment

The practical assignment (called "project" henceforth) is about Test-Driven Development of a communication protocol between two processor boards for an autonomous vehicle. One processor board, henceforth called Arduino, is connected to sensors and periodically sends through a USB channel to the other processor, called Odroid, 3 values concerning wheel torque, distance measured by the ultrasonic sensor and distance measured by the infrared sensor. The other processor is connected to the servo and periodically sends through a USB channel to the first processor speed and steering angle.

We develop the protocol at the Arduino side as a Java class in a test-driven fashion and integrate it with a mocked USB channel and Odroid. Subsequently, we build and test a simple command-line interface that sends and receive messages that makes call to the send interfaces on either sides and periodically reads the received values on the other side. Finally, we make a model of the protocol and model check its correctness.

From the testing and verification perspective, the project comprises the application of the following techniques:

• test-driven development,
• unit testing using jUnit,
• gathering coverage metrics using EclEmma (or similar tools),
• integration testing, including developing stubs using Mockito (or similar tools),
• model checking using Uppaal, and
• UI testing using the Sikuli tool.

This is a group project that is to be carried out in groups of 4. You need to have formed your groups in GUL and emailed your group structure to the lecturer and both instructors by Thursday January 29 at 17:00; please put '[DIT085] Group Registration' in the subject line of your email.

The deliverables are to be delivered on the GUL system and discussed orally in the presence of all team members with one of the two instructors. The deliverables comprise a report and the implementation code: the report should document the major steps in each phase and code snippets (few concise examples) of how they are implemented and possibly screen shots of the results. Extensive pieces of program code should not be included in the report. The most important factors in judging the reports are: their logical structure and sufficiently clear explanation of the steps (all figures and code snippets should be accompanied with clear descriptions).

The deadlines are to be respected and each phase is to be delivered both on GUL and discussed before the deadline. You may get an extension of one week for at most two phases; for that you need to send an email before the deadline to the instructors. Sending an email is sufficient for receiving the extension and you need not wait for a response to your email.

As a general principle, when you find an ambiguity in the requirements, make a reasonable assumption and document it clearly in your report.

After you deliver each phase, each and every member of the group should send an email with an estimate of each member's participation in that phase (in percents) and the parts of the deliverable each member has contributed to.

Practical	Deadline	Notes
Phase 1: TDD of a Unit	February 5, 2016 23:59
Phase 2: Integration Testing	February 19, 2016 23:59
Phase 3: UI Testing and Model Checking	March 05, 2016 23:59

The results of the deliverables will be announced through an anonymized Google Spreadsheet.

Acknowledgment

This course is based on the material produced for several earlier editions given at TU Eindhoven, the Netherlands. Part of the material has been produced (or is based on the produced material) by Judi Romijn and Tim Willemse.