Latest revision as of 08:49, 24 May 2017

6th Halmstad Summer School on Testing, HSST 2016

HSST 2016 Poster.

June 13-16, 2016 - Halmstad University, Sweden

HSST 2016 Group Photo.

Introduction

Testing and debugging account for a major part of software development cost and effort, yet the current practice of software testing is often insufficiently structured and disciplined. There have been various attempts in the past decades to bring more rigour and structure into this field, resulting in several industrial-strength processes, techniques and tools for different levels of testing. The 5th Halmstad Summer School on Testing provides an overview of the state of the art in testing, including theory, industrial cases, tools and hands-on tutorials by internationally-renowned researchers.

Tutorials

Automatic Software Verification with the Infer Static Analyzer, Dino Distefano, Queen Mary, University of London and Facebook, UK

For organisations like Facebook, high quality software is important. However, the pace of change and increasing complexity of modern code makes it difficult to produce error-free software. Available tools are often lacking in helping programmers develop more reliable and secure applications. Formal verification is a technique able to detect software errors statically, before a product is actually shipped. In this lecture I will introduce some of the theoretical foundations of the Infer Static Analyzer and its practical application on mobile code development.

Testing and Verification Methods for Many-Core Concurrency, Alastair F. Donaldson, Imperial College, UK

Many-core programming, using CPUs with lots of cores, or using GPUs, is becoming popular as a method for achieving high performance at low energy cost. The reliability of many-core software is affected by several issues at present: the uncertain and evolving nature of many-core architectures, the (un)reliability of many-core compilers, and the difficulty of writing correct many-core applications code. With colleagues in the Multicore Programming Group at Imperial, and with external collaborators, I have been involved in research to improve many-core software reliability through (a) static analysis techniques to help find bugs in, and establish correctness properties of, many-core source code, (b) automatic fuzz testing to identify defects in many-core compilers, and (c) studying the memory consistency models of many-core devices empirically, and of many-core programming languages formally, to establish a solid foundation on which to exploit many-core concurrency in a rigorous manner. I will present an overview of this work during the tutorial, demonstrating live the kinds of defects and issues that make many-core programming hard, showing how our research contributions can help, and discussing a number of open problems that current techniques do not address.

Is Mutation Analysis Ready for Prime Time?, Jeff Offutt, George Mason University, USA

Mutation analysis makes small syntactic changes to software artifacts (such as a method or class), usually to help testers design high quality test data. Mutation analysis has been around for decades, and is widely recognized as a very powerful test design criterion, but its complexity and expense has inhibited practical adoption.

This tutorial will introduce the basics of mutation with both theory and examples. I will then discuss the problems that prevent it from being used outside of the research community, again addressing both theory and practicality. We will then look at current research projects, some of which shows great promise in reducing the cost of mutation to practical levels. These include work in minimal mutation, equivalent mutant detection, and automatic test data generation.

We will conclude with a discussion of some non-testing uses of mutation analysis, including automatic program repair and mutation-based generation of test inputs based on input grammars such as XML schemas. You will finish the tutorial with an understanding, in context, of outstanding research problems that, if solved, can allow mutation to be ready for prime time use.

YouTube Film Part I

YouTube Film Part II

Fault Model-Based Testing from State-Oriented Models. Alexandre Petrenko, Computer Research Institute of Montreal (CRIM), Canada

There is a dictum of Dijkstra that testing can be used to show the presence of bugs, but never to show their absence, but yet, can testing at least guarantee that no bugs of a predefined type are left? Addressing this question, the tutorial provides an overview of research in testing from state-oriented models, including various types of finite state machines and transition systems, where fault assumptions are formalized in fault models and tests are generated to guarantee that any bug that fits a chosen fault model is detected in any implementation of a given specification. We use examples to explain how faults can be modelled and how tests exhaustive for a given fault model can be generated.

The Role of Testing and Tools for Innovation, Per Runeson, Lund University, Sweden

Innovation is a buzzword on the lips of both industrialists and academics. OECD defines four kinds of innovation; product, process, marketing and organizational. Software plays a significant role in product innovation. For example, in the automotive domain, software controlled functions are the basis for 90% of the product innovation. However, software plays a key role also when it comes to how products are designed and produced (process innovation), how it is delivered to the market (marketing innovation) and how different organizations interplay e.g. in ecosystems (organizational innovation). And with software comes testing.

In a survey about innovation concepts, we observed that testers in industry seemed NOT to consider themselves as contributing to innovation. On the contrary in a case study of the community of the open source tool Jenkins, we identified several key aspects related to software testing and tools. The access of “free” tools and features is a main driver to engage in the open innovation. However, in some cases, the flexibility and short turnaround speed for changes in the tools is a major advantage for industry. Specifically, we studied an effort to introduce a test framework to support testing of the tool, and found that the key challenges is access to test resources which leads to the call for frameworks for test automation.

This tutorial will introduce the basic concepts of innovation, and specifically draw on our studies of software testing aspects of open innovation. Industry participants will get insights on how to invest in a community to get best value out from the open community. Researchers will get guidance in what is needed to get their open source tools spread and utilized in a wider audience.

Fault Tree Analysis, Marielle Stoelinga, University of Twente, The Netherlands

Fault tree analysis (FTA) is a very prominent method to analyze the dependability of safety and economically critical assets, like power plants, data centers and web shops. FTA comprises a wide variety of methods and techniques and is being deployed by many companies, like NASA, ESA, Honeywell, Airbus, Toyota, etc.

This tutorial addresses how fault trees can be used to model and analyze the dependability of complex systems. I will give an introduction to the most common quantitative and qualitative analysis techniques, their practical relevance and deployment in practical risk assessments. Moreover, I will illustrate how the use of formal methods help to the modeling and analytical power of FTA. I will end by discussing fault tree extensions, as well as current developments in fault tree research.

Program

June 13 (9:00-17:00)

09:00-09:15 Arrival and Registration

09:15-09:30 Welcome, opening remarks (Mohammad Mousavi)

09:30-10:30 Is Mutation Analysis Ready for Prime Time? Part I (Jeff Offutt) Slides YouTube Film Part I

10:30-11:00 Coffee Break

11:00-12:30 Is Mutation Analysis Ready for Prime Time? Part II (Jeff Offutt) Slides YouTube Film Part II

12:30-14:00 Lunch Break

14:00-15:30 Testing and Verification Methods for Many-Core Concurrency, Part I (Alastair F. Donaldson) Slides - Part I

15:30-16:00 Coffee Break

16:00-17:00 Testing and Verification Methods for Many-Core Concurrency, Part II (Alastair F. Donaldson) Slides - Part II

June 14 (9:30-17:00)

09:30-10:30 Fault Model-Based Testing from State-Oriented Models, Part I (Alexandre Petrenko) Slides

10:30-11:00 Coffee Break

11:00-12:30 Fault Model-Based Testing from State-Oriented Models, Part II (Alexandre Petrenko) Slides

12:30-14:00 Lunch Break

14:00-15:40 Ph.D Symposium, Session I

14:00-14:20 Lars Luthmann, (De-)Compositional Input/Output Conformance Testing based on Modal Interface Automata abstract

14:20-14:40 Ahmad Salim Al-Sibahi, Automated Test Generation for Transformations using Symbolic Execution abstract

14:40-15:00 Rui Wang, Application of Model-based Testing and Runtime Verification for Distributed Systems Protocols abstract

15:00-15:20 Hugo Araújo, Gustavo Carvalho, NAT2TEST: Generating Test Cases from Natural Language Requirements based on CSP abstract

15:20-15:40 Sebastian Kunze, Symbolic Characterisation of Commonalities in Testing Software Product Lines abstract

15:40-16:00 Coffee Break

16:00-17:40 Ph.D. Presentations, Session II

16:00-16:20 Iago Abal, EBA: Effect-Based Analysis of C Programs abstract

16:20-16:40 Stefan Klikovits,, Automatic Test Case Generation for Procedural Languages abstract

16:40-17:00 2 Sara Abbaspour Asadollah, Bugs and Debugging of Concurrent and Multicore Software abstract

17:00-17:20 Ali Parsai, Customizable Mutation Testing for Industrial Environments abstract

17:20-17:40 Rainer Niedermayr, Evaluation and Improvement of Test Suites abstract

June 15 (9:30-12:30)

09:30-10:30 Automatic Software Verification with the Infer Static Analyzer, Part I (Dino Distefano) Slides

10:30-11:00 Coffee Break

11:00-12:30 Automatic Software Verification with the Infer Static Analyzer, Part I (Dino Distefano) Slides

12:30-14:00 Lunch Break

14:00-17:00 Social Event

18:30-22:00 Social Dinner

June 16 (9:30-17:00)

09:30-10:30 Fault Tree Analysis, Part I (Marielle Stoelinga) Slides

10:30-11:00 Coffee Break

11:00-12:30 Fault Tree Analysis, Part II (Marielle Stoelinga) Slides

12:30-14:00 Lunch Break

14:00-15:30 The Role of Testing and Tools for Innovation, Part I (Per Runeson) Slides

15:30-16:00 Coffee Break

16:00-17:00 The Role of Testing and Tools for Innovation, Part II (Per Runeson) Slides

Registration

The registration deadline is April 15April 30, 2016. (Extended)

To apply to the summer school, please proceed to the registration page: http://bit.ly/HSST2016. If you have any dietary requirements, or would like to attend only certain days, please indicate it in the registration form. After submitting the registration form, please use the PayPal page for payment. Upon concluding the payment process, you will receive a confirmation.

The registration fee is 3000 SEK (approx. 325 EUR, ) and covers lunches, coffee breaks, and the study material but does not include the social event and dinner. The ticket to the social event and dinner (excl. drinks) costs 500 SEK and can be included upon registration. The registration fees include Value Added Tax and is hence, VAT deductible for governmental organizations.

Ph.D. Student Presentations

We have 6 time slots for Ph.D. presentations, where each student has 10 minutes to present her/his research project (and possibly results) and receive feedback from our experts. We solicit abstracts of 2 pages in the EasyChair Style in order to make a selection. The abstract should contain a clear overview of the problem description, approach, (existing results, if any,) and future milestone.

Abstract submissions can be made already via the EasyChair Site for HSST 2016. The deadline for submissions is May 15, 2016.

Venue

The workshop will be held on the campus of Halmstad University in Halmstad, Sweden. Halmstad is a popular holiday destination located on the Swedish west coast. Just a few minutes by bicycle or bus takes you from campus to city centre, sandy beaches or forested Galgberget Hill.

Directions to/in Campus

Directions for getting to campus can be found at http://www.hh.se/english/abouttheuniversity/visitus.307_en.html

The campus map can be found here. The summer school will be located at the O 124 lecture hall, which is located at the ground floor of the O building (O Huset). Coffee breaks will be held at the same building. Lunches will be served in the ground floor of the Spiro restaurant in the G building.

Directions to Halmstad

Trains take you directly to Göteborg in 75 minutes, to the Malmö-Copenhagen area in about 2 hours and to Stockholm in 4.5 hours. There are also daily flights from Halmstad Airport to Stockholm.

If you are flying internationally it is generally easiest to fly into Copenhagen (CPH) airport (also known as Kastrup). The best thing about flying into CPH is that you just buy a train ticket when you arrive at the airport and simply take a train from the airport directly to Halmstad. The train leaves from the airport itself approximately once an hour on weekdays. We recommend that you check the time-table at the Swedish Railways site and allow one hour from touchdown to getting to the train station (just outside customs). (It seems that you can take an earlier or a later train on the same day regardless of the exact train you booked, but obviously you will lose your seat reservation if you have made one.) To get to your hotel, you can combine a taxi booking with your train ticket at the Swedish Railways site and a driver will wait for you with your name mentioned on a board once you arrive in Halmstad.

In Halmstad, everything is either in walking distance or a short taxi ride away. Usually there are taxis at the station. If there are none there is a phone that connects directly to the local taxi company. For the eventuality that the phone is not working, it is good to have a cell phone handy. The number for the taxi company is written on the phone.

Note that CPH is in Denmark (and not in Sweden). So, if you need visas for European countries, make sure you get one that works for both.

If for some reason you cannot or do not want to use CPH, the next best international airport is in Gothenburg (GOT), locally known as Landvetter. The tricky thing about using that airport is that you would first have to take a 45 minute shuttle from the airport to the Gothenburg train station, and then take the train to Halmstad. That is one transfer and one wait. You can buy a combined shuttle and train ticket from the the Swedish Railways site.

Accommodation

We have made a pre-booking for a group of rooms at Hotel Continental. The negotiated price is 825 SEK for a single room, 1025 SEK for a double room and 591 SEK for a shared double room. All prices include breakfast.

If you would like to use this offer, please send an email to the hotel reservation and mention the booking code "Summer School 2016".

Here are some other suggestions for the accommodation, with an indication of their price range, (obtained from booking.com) and their distance to the summer school venue

• Scandic Hallandia (~160-200 EUR / night, 2km)
• Hotel Amadeus (~100-120 EUR / night, 2.5km)
• First Hotel Martenson (~130-150 EUR / night, 2km)
• Quality Hotel Halmstad (~80-100 EUR / night, 3 km)
• STF Halmstads Hostel Kaptenshamn (~80-100 EUR / night, 2km)

An annotated Google Map with suggestion for restaurants can be found here.

HSST 2016 Session Photo.

History

This is the sixth edition of the summer school. Information about the previous editions is provided below.

• HSST 2015 (June 8- June 11, 2015),
• HSST 2014 (June 9- June 12, 2014),
• HSST 2013 (June 3- June 5, 2013),
• Second Summer School on Accurate Programming (May 30 - June 1, 2012)

Organizers

Please do not hesitate to contact us if you have any questions or enquiries:

• Stella Erlandsson (Local Organization, stella.erlandsson@hh.se)
• Veronica Gaspes (Organization Chair, veronica.gaspes@hh.se)
• Mohammad Mousavi (Program Co-Chair, m.r.mousavi@hh.se)
• Walid Taha (Program Co-Chair, walid.taha@hh.se)