Evaluating the Usability of Model Transformations Testing Approach (MTTA)

Abstract

Model Transformation (MT) is a key component in Model Driven Development (MDD). Model transformation is used to transform source model into a target model, improve the model quality and also introduce the design pattern and refactoring. Model transformation are not free from bugs similar to other software development artifacts and it needs to be verified. Code Generators (CG) are a type of model transformation that automatically generate code from software models. To verify a CG using Model Transformation Testing Approach (MTTA) effectively, MTTA requires the users to manually generate test model and develop assertions. Since both tasks are performed manually, a usability study is conducted to gauge the effects of the manual tasks towards the usability of MTTA from three perspective: Learnability, effectiveness and efficiency. The aim of this paper is to identify the usability problems of MTTA related to its learnability, efficiency and effectiveness. Usability test technique is used in this study and questionnaire was used to collect a quantitative data from the participants. A pilot test was conducted with two participants and later eight participants were recruited for the real test. The result shows that MTTA is learnable and effective but inefficient. We conclude with a discussion on the reason why MTTA is inefficient.