Random Testing of Simulation Languages
| Title | Random Testing of Simulation Languages |
|---|---|
| Summary | Incorrect simulations are worthless. Develop the technology to catch bad simulators. |
| Keywords | |
| TimeFrame | |
| References | |
| Prerequisites | |
| Author | |
| Supervisor | Adam Duracz, Walid Taha |
| Level | Master |
| Status | Open |
Description
Testing is the dominant method for quality control in software development. Generator-driven tests decrease the effort of implementing test cases by providing a vast and varied amount of data to challenge the system under test. Such tests enable Property-Based Testing, where the specification of the system (properties) is separated from the construction of test data (generators). Generators of simple data types have been studied extensively and can be implemented in a way that mirrors the structure of the type.
The focus of this project is to develop a generator of Acumen models (values of the Acumen AST type). The task is complicated by the fact that important properties of valid, interesting Acumen models are not captured by the AST type. Such models are programs that are executable, have finite trajectories throughout execution, whose behavior is diverse and that are representative of known models written by human modelers.
Research questions
- How can a generator of programs of a hybrid programming language (combining differential equations with discrete state changes) be implemented so that models are valid by construction?
- How can an implementation be designed so that it performs well enough to be useful in practical testing?
- How should such a generator be implemented so that it is easily maintainable and extensible?
Expected outcomes
- A description of how a generator can be constructed to meet the criteria outlined in the research questions.
- An implementation of a generator based on this description.
- An analysis of test coverage achieved by using the generator to test several components of Acumen, including a parser, an interpreter, a compiler and a source-to-source transformation.
Prerequisites
- Course: Advanced Object-Oriented Programming.
- Some familiarity with Scala or another functional programming language, as well as probability theory, will be very useful.
References
- Waild Taha, Paul Brauner, Corky Cartwright, Veronica Gaspes, Aaron Ames, “Core Acumen Tutorial”, 2011.
- Koen Claessen and John Hughes, "QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs", In Proc. of International Conference on Functional Programming (ICFP), 2000.
- Michał H. Pałka, “Testing an Optimising Compiler by Generating Random Lambda Terms”, Licentiate Thesis, 2011.
