Test-driven Agile Simulation
The implementation of complex software systems usually consists of a series of development and testing phases. Starting from requirements definition the development process is divided into a number of design, specification, programming and testing steps. According to the increasing complexity of software systems model-driven approaches are gaining in popularity. In particular, graphical representations like UML (Unified Modeling Language) diagrams of special system features allow to deal with the complexity and also enable advanced analysis and validation capabilities.
The approach of Test-driven Agile Simulation combines model-based simulation and testing techniques to achieve an improved overall quality for the development process. It intends to provide a cheap and agile technique to validate the involved specification models in several iteration steps during early engineering phases. This approach allows detecting of modeling errors or inconsistencies in the system specification and in the used test model as early as possible by simulating a given system and running test cases at the model level.
System models are described by class, composite structure, state, and activity diagrams and can be extended by quantitative aspects (e.g., deterministic or stochastic execution times, routing probabilities) by using the MARTE (Modeling and Analysis of Real-Time and Embedded Systems) profile for UML. Such a system model is transformed and compiled into C++ code and executed by means of the network simulator OMNeT++, results are finally fed back on the UML system model level. From the perspective of simulation, the advantage compared with other proprietary simulators is that simulation models can be developed in an open format. From the perspective of systems engineering, the advantage is that system models can be executed early in a simulation environment. A further advantage is that with the system model quantitative assessments become possible as well.
Independent from this, usage models can be developed as a basis for testing. Here state, activity and sequence diagrams are being used, which describe selected usages of the system or portions of the system. Usage models are no system models and just represent possible inputs and expected outputs. Usage models can be extended by Markov profiles to describe usage frequencies. From such a model it is possible to derive test cases by various strategies. The derived test case sequences plus the specification of the relevant test configurations serve then as a test model in accordance with the UML Testing Profile (UTP). Such a test model can also be transformed to the simulation code and executed on the OMNeT++ level together with the relevant portions of the system model. From the perspective of testing, the advantage is that test cases can be executed early in a simulation environment and that both system and test model can be validated and iterated. From the perspective of simulation, the advantage is that a systematic approach is available to validate simulation models.
The resulting implementation code, being generated from such a system model, is on a higher quality level and the test cases approved on the simulation level can also be used for testing of the implemented system. This approach can be integrated with various development models such as V model or agile development.
We provide the integrated, Eclipse-based tool environment SimTAny, which combines originally heterogeneous tools to provide support for designated phases of the Test-driven Agile Simulation approach. Among others, SimTAny implies the integration of a UML modeling tool (e.g. Papyrus UML), a simulation framework (e.g. OMNeT++), a model-to-text transformation framework Acceleo, and a tool R-Project for statistical calculations. The service-oriented architecture of SimTAny aims to simplify the integration of heterogeneous tools in a distributed development process or in a distributed tool environment.
One of the basic components of SimTAny is the transformation module, which has been developed at our department. The module provides the transformation of UML-based models to executable simulation code. Thereby, the supported modeling methodology relies on several OMG standards like UML2, MARTE, SysML, and UTP. In its current realization SimTAny supports the transformation to discrete event simulations executable on the simulation framework OMNeT++. Nevertheless, our transformation module is designed to be extendable for other simulation engines as well.