Difference between revisions of "DIT085 Ed 2016"

From CERES
Jump to: navigation, search
(Created page with "= Testing and Verification @ Chalmers / GU - Edition 2015 = ==Contact == ===Lecturer=== Mohammad Mousavi * Office: Halmstad University, E 305 (on Fridays at Chalmer...")
 
 
(40 intermediate revisions by one user not shown)
Line 1: Line 1:
−
= Testing and Verification @ Chalmers / GU - Edition 2015 =  
+
= Testing and Verification @ Chalmers / GU - Edition 2016 =  
  
  
Line 44: Line 44:
 
** compare and judge alternatives to testing, such as model checking and runtime verification
 
** compare and judge alternatives to testing, such as model checking and runtime verification
 
** identify and hypothesize about sources of program failures, and reflect on how to better verify the correctness of such programs.
 
** identify and hypothesize about sources of program failures, and reflect on how to better verify the correctness of such programs.
 +
  
 
==Assessment==
 
==Assessment==
Line 59: Line 60:
 
In order to pass the project, you need to pass all 3 mandatory parts of the project.  
 
In order to pass the project, you need to pass all 3 mandatory parts of the project.  
  
−
=== [[media:DIT085_final_examination_march_2015.pdf| Final Examination 2015]],  [[media:DIT085_final_examination_march_2015_with_solutions.pdf| Final Examination 2015 with Solutions]] ===
+
=== [[media:DIT085_assessment_guide_2016.pdf| Assessment Guide 2016]] ===
  
 +
=== [[media:DIT085_final_examination_march_2015.pdf| Final Examination at Chalmers/GU, March 2015]],  [[media:DIT085_final_examination_march_2015_with_solutions.pdf| Final Examination at Chalmers / GU, March 2015 with Solutions]] ===
 +
 +
 +
=== [[media:DIT085_final_examination_april_2015.pdf| Final Examination at Chalmers/GU, April 2015]],  [[media:DIT085_final_examination_april_2015_with_solutions.pdf| Final Examination at Chalmers / GU, April 2015 with Solutions]] ===
 +
 +
 +
=== [[media:DIT085_final_examination_march_2016.pdf| Final Examination at Chalmers/GU, March 2016]],  [[media:DIT085_final_examination_march_2016_with_solutions.pdf| Final Examination at Chalmers / GU, March 2016 with Solutions]] ===
  
  
Line 72: Line 80:
 
|-
 
|-
 
|  Lecture 1: Terminology and Functional Testing   
 
|  Lecture 1: Terminology and Functional Testing   
−
January 22, 2015
+
January 22, 2016
−
||        ||  
+
||         
 +
[[media:DIT085_2016_lecture1_slides.pdf | Lecture 1 Slides]]
 +
 
 +
[[media:DIT085_2016_lecture1_handouts.pdf | Lecture 1 Handouts]]
 +
||  
 
|-
 
|-
 
|  Lecture 2: Functional Testing  
 
|  Lecture 2: Functional Testing  
 
Guest Lecture by Micael Andersson, Combitech (Test-Driven Development and jUnit)
 
Guest Lecture by Micael Andersson, Combitech (Test-Driven Development and jUnit)
−
January 29, 2015
+
January 29, 2016
 
||  
 
||  
 +
[[media:DIT085_2016_lecture2_slides.pdf | Lecture 2 Slides]]
 +
 +
[[media:DIT085_2016_lecture2_handouts.pdf | Lecture 2 Handouts]]
 +
 +
[[media:DIT085_2016_guest_lecture1_slides.pdf | Guest Lecture 1 Slides]]
 +
 +
[[media:DIT085_2016_tdd_examples1.zip | Test Driven Dev. Examples]]
 +
 
||  Chapters 6 and 7 of Jorgensen
 
||  Chapters 6 and 7 of Jorgensen
 
[http://dx.doi.org/10.1002%2Fstvr.4370030203 M. Grochtmann and K. Grimm, Classification trees for partition testing]
 
[http://dx.doi.org/10.1002%2Fstvr.4370030203 M. Grochtmann and K. Grimm, Classification trees for partition testing]
 
|-
 
|-
 
|  Lecture 3: Guest Lecture by Micael Andersson, Combitech (Mocking and Test-Driven Development )
 
|  Lecture 3: Guest Lecture by Micael Andersson, Combitech (Mocking and Test-Driven Development )
−
February 5, 2015
+
February 5, 2016
 
||   
 
||   
 +
 +
[[media:DIT085_2016_guest_lecture2_slides.pdf | Guest Lecture 2 Slides]]
 +
 +
[[media:DIT085_2016_tdd_examples2.zip | Test Driven Dev. Examples - Part 2]]
 +
 +
[[media:DIT085_2016_mocking_examples.zip | Test Mocking Examples]]
 +
 
||
 
||
 
|-
 
|-
 
|  Lecture 4: Coverage Criteria   
 
|  Lecture 4: Coverage Criteria   
−
February 13, 2015
+
February 12, 2016
 
||   
 
||   
−
Path Testing:  [[media:DIT085_lecture3_handouts_2014.pdf| Handouts]] [[media:DIT085_lecture3_slides_2015.pdf| Slides]]  
+
[[media:DIT085_2016_lecture3_slides.pdf | Lecture 3 Slides]]
 +
 
 +
[[media:DIT085_2016_lecture3_handouts.pdf | Lecture 3 Handouts]]
  
−
Data Flow Testing:  [[media:DIT085_lecture4_handouts_2014.pdf| Handouts]]  [[media:DIT085_lecture4_slides_2015.pdf| Slides]]
 
 
  || Chapter 2 Ammann and Offutt
 
  || Chapter 2 Ammann and Offutt
 
Chapters 9 and 10 of Jorgensen
 
Chapters 9 and 10 of Jorgensen
 
|-
 
|-
 
|  Lecture 5: Model Checking   
 
|  Lecture 5: Model Checking   
−
February 20, 2015
+
February 19, 2016
 
||     
 
||     
−
[[media:DIT085_lecture5_handouts_2015.pdf| Handouts]] [[media:DIT085_lecture5_slides_2015.pdf| Slides]]  
+
 
 +
[[media:DIT085_2016_lecture4_slides.pdf | Lecture 4 Slides]]
 +
 
 +
[[media:DIT085_2016_lecture4_handouts.pdf | Lecture 4 Handouts]]
 +
 
 +
 
 
||   
 
||   
 
[[media:intro_uppaal_vaandrager.pdf| F. Vaandrager, A First Introduction to Uppaal]]  
 
[[media:intro_uppaal_vaandrager.pdf| F. Vaandrager, A First Introduction to Uppaal]]  
Line 105: Line 138:
 
[http://www.mbsd.cs.ru.nl/publications/papers/fvaan/handbookuppaal/ Gossip and Jobshop Examples]
 
[http://www.mbsd.cs.ru.nl/publications/papers/fvaan/handbookuppaal/ Gossip and Jobshop Examples]
  
−
[[media:simple_server_2015.zip| Simple Server Example]]  
+
[[media:DIT085_2016_USBChannel.zip | USB Channel Example]]
  
 
|-
 
|-
 +
 +
 
|  Lecture 6: UI Testing   
 
|  Lecture 6: UI Testing   
  
−
Guest Lecture by Emil Alégroth, Chalmers
+
Guest Lecture  
  
−
February 27, 2015
+
February 26, 2016
 
||     
 
||     
  
−
[[media:DIT085_vgt_slides_2015.pptx|Visual GUI Testing (Slides in pptx format)]]  
+
[[media:DIT085_2016_lecture5_slides.pptx | Lecture 5 Slides (PowerPoint)]]
  
 
||  
 
||  
Line 122: Line 157:
 
|-
 
|-
 
|  Lecture 7: Slicing and Debugging   
 
|  Lecture 7: Slicing and Debugging   
−
March 6, 2015
+
March 4, 2016
−
||  [[media:DIT085_lecture7_handouts_2015.pdf| Handouts]] [[media:DIT085_lecture7_slides_2015.pdf|Slides]]  
+
||   
 +
 
 +
[[media:DIT085_2016_lecture6_slides.pdf | Lecture 6 Slides]]
 +
 
 +
[[media:DIT085_2016_lecture6_handouts.pdf | Lecture 6 Handouts]]
 +
 
 
||
 
||
 
Chapters 5, 6, and 13 of Zeller
 
Chapters 5, 6, and 13 of Zeller
Line 129: Line 169:
 
|-
 
|-
 
|  Lecture 8: Reviewing Model Examination   
 
|  Lecture 8: Reviewing Model Examination   
−
March 13, 2015
+
March 11, 2016
−
||  [[media:DIT085_model_examination_2015.pdf| Model Examination]] [[media:DIT085_model_examination_solutions_2015.pdf| Solutions]]  ||  
+
||   
 +
 
 +
[[media:DIT085_2016_model_exam.pdf | Model Examination]]
 +
 
 +
 
 +
[[media:DIT085_2016_model_exam_solutions.pdf | Model Examination with Solutions]]
 +
 
 +
 
 +
  ||  
 
|-
 
|-
 
|}
 
|}
Line 137: Line 185:
 
===Main Textbook===
 
===Main Textbook===
  
−
* P. Ammann and J. Offutt. Introduction to Software Testing. Cambridge, 2008.
 
−
 
−
===Recommended Reading Material===
 
  
 
* P.C. Jorgensen. Software Testing: A Craftsman’s Approach. Auerbach Publications, 3rd edition, 2008.
 
* P.C. Jorgensen. Software Testing: A Craftsman’s Approach. Auerbach Publications, 3rd edition, 2008.
  
−
* L. Aceto, A. Ingolfsdottir, K.G. Larsen, and J. Srba. [http://www.cambridge.org/us/academic/subjects/computer-science/programming-languages-and-applied-logic/reactive-systems-modelling-specification-and-verification Reactive Systems: Modelling, Specification and Verification], Cambridge University Press, 2010.
+
===Recommended Reading Material===
 +
 
 +
* P. Ammann and J. Offutt. Introduction to Software Testing. Cambridge, 2008.
  
 
* A. Zeller. Why Programs Fail? Morgan Kaufmann, 2nd edition, 2009.
 
* A. Zeller. Why Programs Fail? Morgan Kaufmann, 2nd edition, 2009.
Line 155: Line 202:
 
* M. Wiser, Program Slicing, Proc. of ICSE'81, pp.  439-449, ACM, 1981.
 
* M. Wiser, Program Slicing, Proc. of ICSE'81, pp.  439-449, ACM, 1981.
  
−
== Project Description ==
 
  
−
The project is about Test-Driven Development of a [https://www.whatsapp.com/ WhatsApp]-like server- and client-side application, which we call WhatsUpGU.
+
== Practical Assignment ==
−
The final implementation will comprise:
+
 
 +
The practical assignment (called "project" henceforth) is about Test-Driven Development of a communication protocol between two processor boards for 
 +
an autonomous vehicle.
 +
One processor board, henceforth called Arduino, is connected to sensors and periodically sends through a USB channel to the other processor, called  Odroid, 3 values concerning wheel torque, distance measured by the ultrasonic sensor and distance measured by the infrared sensor.  
 +
The other processor is connected to the servo and periodically sends through a USB channel to the first processor speed and steering angle.
 +
 
 +
We develop the protocol at the Arduino side as a Java class in a test-driven fashion and integrate it with a mocked  USB channel and Odroid.
 +
Subsequently, we build and test a simple command-line interface that sends and receive messages that makes call to the send interfaces on either sides and
 +
periodically reads the received values on the other side.  
 +
Finally, we make a model of the protocol and model check its correctness.
  
−
* A connection-based (TCP-IP-based) server implemented  in Java, allowing for multi-threaded processing of multiple requests from clients, the server
 
−
has an XML interface for adding, editing, and fetching messages, and
 
−
* An Android client, able to connect to the server on a TCP IP connection using the XML-based interface protocol and able to deal with communication faults.
 
  
 
From the testing and verification perspective, the project comprises the application of the following techniques:  
 
From the testing and verification perspective, the project comprises the application of the following techniques:  
Line 174: Line 226:
  
 
This is a group project that is to be carried out in '''groups of 4'''.   
 
This is a group project that is to be carried out in '''groups of 4'''.   
−
You need to have formed your groups in GUL and emailed your group structure to the lecturer and both instructors by '''Thursday January 27 at 17:00''';
+
You need to have formed your groups in GUL and emailed your group structure to the lecturer and both instructors by '''Thursday January 29 at 17:00''';
 
please put '[DIT085] Group Registration' in the subject line of your email.
 
please put '[DIT085] Group Registration' in the subject line of your email.
  
Line 189: Line 241:
  
 
As a general principle, when you find an ambiguity in the requirements, make a reasonable assumption and '''document it clearly''' in your report.  
 
As a general principle, when you find an ambiguity in the requirements, make a reasonable assumption and '''document it clearly''' in your report.  
−
 
−
 
−
===[[DIT085 FAQ 2015|Frequently Asked Questions]]===
 
  
 +
After you deliver each phase, each and every member of the group should send an email with an estimate of each member's participation in that phase (in percents) and
 +
the parts of the deliverable each member has contributed to.
  
  
Line 199: Line 250:
 
! Practical !! Deadline !! Notes
 
! Practical !! Deadline !! Notes
 
|-
 
|-
−
|  [[DIT085 Ed 2015 Practical Phase 1|Phase 1: TDD of a Unit]]    ||  February 13, 2015 23:59 ||  
+
|  [[DIT085 Ed 2016 Practical Phase 1|Phase 1: TDD of a Unit]]    ||  February 5, 2016 23:59 ||  
 
|-
 
|-
−
|  [[DIT085 Ed 2015 Practical Phase 2|Phase 2: Integration (Testing) of the Server]]    || February 27, 2015 23:59 ||  
+
|  [[DIT085 Ed 2016 Practical Phase 2|Phase 2: Integration Testing]]    || February 19, 2016 23:59 ||  
 
|-
 
|-
−
|  [[Phase 3: Specification and Model Checking]]        || March 06, 2015 23:59 ||
+
|  [[DIT085 Ed 2016 Practical Phase 3|Phase 3: UI Testing and Model Checking]]         || March 05, 2016 23:59 ||  
−
|-
+
−
|  [[DIT085 Ed 2015 Practical Phase 4|Phase 4: UI Testing]] (Optional, see the rules for pass with distinction)    || March 13, 2015 23:59 ||  
+
 
|-
 
|-
 
|}
 
|}
 +
 +
The results of the deliverables will be announced through an [https://docs.google.com/a/student.gu.se/spreadsheets/d/1L8Pe6VKQi8YCZ498xDJcz2bCAozRJzSVvbfQcl-2FKA/edit?usp=sharing  anonymized Google Spreadsheet].
  
  

Latest revision as of 18:26, 11 May 2016

Testing and Verification @ Chalmers / GU - Edition 2016

Contact

Lecturer

Mohammad Mousavi

  • Office: Halmstad University, E 305 (on Fridays at Chalmers: Jupiter, 477)
  • Telephone 035 16 71 22 (on Fridays at Chalmers: (031 772) 60 77)
  • Email: m.r.mousavi@hh.se (Please note: I do not access the GU / Chalmers email account.)

Instructors

Sali Masri

Omar Abu Nabah

Schedule

Lectures: Friday 10:15-12:00, Lindholmen Campus, Patricia, Alfons Hall

Supervised Labs: Thursdays at 08:15-10:00 and Fridays 13:00-16:45, Lindholmen Campus, Patricia, Mållgan Hall

Objectives

  • Knowledge and understanding:
    • understand the basic terminology of testing;
    • name and describe basic concepts on testing, as well as different testing techniques and approaches
    • describe the connection between software development phases and kinds of testing
  • Skills and abilities:
    • write models in at least one formal specification language;
    • construct appropriate and meaningful test cases, and interpret and explain (to stakeholders) the results of the application of such test cases ( using appropriate tools) to practical examples
    • plan and produce appropriate documentation for testing
    • apply different testing techniques on realistic examples.
  • Judgement and approach:
    • compare different tools and techniques for testing software, and plan their use in appropriate contexts,
    • compare and judge alternatives to testing, such as model checking and runtime verification
    • identify and hypothesize about sources of program failures, and reflect on how to better verify the correctness of such programs.


Assessment

Assessment is performed in terms of the deliverable of the practical project (software + report) and a written examination.

  • To obtain a Pass (Godkänd: G) mark for the course, the student must obtain a Pass mark for both the project report and the written examination.
  • To obtain a Pass with Distinction (Väl godkänd : VG) mark for the course, the student must obtain a Pass with Distinction mark for both the project report and the written examination.
  • Otherwise, the student will fail the course and obtain a Fail mark (Underkänd: U).

In order to pass the project with distinction, you need to either have pass with distinction in 2 out of the 3 mandatory parts, or 1 pass with distinction for the mandatory part and a pass for the optional part. In order to pass the project, you need to pass all 3 mandatory parts of the project.

Assessment Guide 2016

Final Examination at Chalmers/GU, March 2015, Final Examination at Chalmers / GU, March 2015 with Solutions

Final Examination at Chalmers/GU, April 2015, Final Examination at Chalmers / GU, April 2015 with Solutions

Final Examination at Chalmers/GU, March 2016, Final Examination at Chalmers / GU, March 2016 with Solutions

Slides and Study Material

Slides

Lecture Handouts / Slides Other Material
Lecture 1: Terminology and Functional Testing

January 22, 2016

Lecture 1 Slides

Lecture 1 Handouts

Lecture 2: Functional Testing

Guest Lecture by Micael Andersson, Combitech (Test-Driven Development and jUnit) January 29, 2016

Lecture 2 Slides

Lecture 2 Handouts

Guest Lecture 1 Slides

Test Driven Dev. Examples

Chapters 6 and 7 of Jorgensen

M. Grochtmann and K. Grimm, Classification trees for partition testing

Lecture 3: Guest Lecture by Micael Andersson, Combitech (Mocking and Test-Driven Development )

February 5, 2016

Guest Lecture 2 Slides

Test Driven Dev. Examples - Part 2

Test Mocking Examples

Lecture 4: Coverage Criteria

February 12, 2016

Lecture 3 Slides

Lecture 3 Handouts

Chapter 2 Ammann and Offutt

Chapters 9 and 10 of Jorgensen

Lecture 5: Model Checking

February 19, 2016

Lecture 4 Slides

Lecture 4 Handouts


F. Vaandrager, A First Introduction to Uppaal

Gossip and Jobshop Examples

USB Channel Example

Lecture 6: UI Testing

Guest Lecture

February 26, 2016

Lecture 5 Slides (PowerPoint)

VGT Cheat Sheet (Examples, Exercises)

Lecture 7: Slicing and Debugging

March 4, 2016

Lecture 6 Slides

Lecture 6 Handouts

Chapters 5, 6, and 13 of Zeller M. Wiser, Program Slicing

Lecture 8: Reviewing Model Examination

March 11, 2016

Model Examination


Model Examination with Solutions



Main Textbook

  • P.C. Jorgensen. Software Testing: A Craftsman’s Approach. Auerbach Publications, 3rd edition, 2008.

Recommended Reading Material

  • P. Ammann and J. Offutt. Introduction to Software Testing. Cambridge, 2008.
  • A. Zeller. Why Programs Fail? Morgan Kaufmann, 2nd edition, 2009.
  • M. Wiser, Program Slicing, Proc. of ICSE'81, pp. 439-449, ACM, 1981.


Practical Assignment

The practical assignment (called "project" henceforth) is about Test-Driven Development of a communication protocol between two processor boards for an autonomous vehicle. One processor board, henceforth called Arduino, is connected to sensors and periodically sends through a USB channel to the other processor, called Odroid, 3 values concerning wheel torque, distance measured by the ultrasonic sensor and distance measured by the infrared sensor. The other processor is connected to the servo and periodically sends through a USB channel to the first processor speed and steering angle.

We develop the protocol at the Arduino side as a Java class in a test-driven fashion and integrate it with a mocked USB channel and Odroid. Subsequently, we build and test a simple command-line interface that sends and receive messages that makes call to the send interfaces on either sides and periodically reads the received values on the other side. Finally, we make a model of the protocol and model check its correctness.


From the testing and verification perspective, the project comprises the application of the following techniques:

  • test-driven development,
  • unit testing using jUnit,
  • gathering coverage metrics using EclEmma (or similar tools),
  • integration testing, including developing stubs using Mockito (or similar tools),
  • model checking using Uppaal, and
  • UI testing using the Sikuli tool.

This is a group project that is to be carried out in groups of 4. You need to have formed your groups in GUL and emailed your group structure to the lecturer and both instructors by Thursday January 29 at 17:00; please put '[DIT085] Group Registration' in the subject line of your email.

The deliverables are to be delivered on the GUL system and discussed orally in the presence of all team members with one of the two instructors. The deliverables comprise a report and the implementation code: the report should document the major steps in each phase and code snippets (few concise examples) of how they are implemented and possibly screen shots of the results. Extensive pieces of program code should not be included in the report. The most important factors in judging the reports are: their logical structure and sufficiently clear explanation of the steps (all figures and code snippets should be accompanied with clear descriptions).

The deadlines are to be respected and each phase is to be delivered both on GUL and discussed before the deadline. You may get an extension of one week for at most two phases; for that you need to send an email before the deadline to the instructors. Sending an email is sufficient for receiving the extension and you need not wait for a response to your email.


As a general principle, when you find an ambiguity in the requirements, make a reasonable assumption and document it clearly in your report.

After you deliver each phase, each and every member of the group should send an email with an estimate of each member's participation in that phase (in percents) and the parts of the deliverable each member has contributed to.


Practical Deadline Notes
Phase 1: TDD of a Unit  February 5, 2016 23:59
Phase 2: Integration Testing February 19, 2016 23:59
Phase 3: UI Testing and Model Checking March 05, 2016 23:59

The results of the deliverables will be announced through an anonymized Google Spreadsheet.


Acknowledgment

This course is based on the material produced for several earlier editions given at TU Eindhoven, the Netherlands. Part of the material has been produced (or is based on the produced material) by Judi Romijn and Tim Willemse.


Back to Home

Retrieved from "https://wiki.hh.se/ceres/index.php?title=DIT085_Ed_2016&oldid=5902"