Official documentation for the VM* family of model manipulation languages.
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
vmtl:evaluation [2015/08/23 14:48] rvac |
vmtl:evaluation [2015/08/23 15:22] rvac [Statistical Analysis] |
||
---|---|---|---|
Line 25: | Line 25: | ||
The experimental data is presented as a comma-separated values (CSV) file consisting of the following columns: | The experimental data is presented as a comma-separated values (CSV) file consisting of the following columns: | ||
+ | ^Column^Definition^Values^ | ||
+ | | Experiment | experiment identifier | 1, 2 | | ||
+ | |Id | participant identifier, unique within an experiment | positive integers | | ||
+ | | UML | participant's self-assessed UML knowledge | 1, 2, 3, 4, 5 | | ||
+ | | OCL | participant's self-assessed OCL knowledge | 1, 2, 3, 4, 5 | | ||
+ | | MT | participant's self-assessed model transformation knowledge | 1, 2, 3, 4, 5 | | ||
+ | | Programming | participant's self-assessed programming knowledge | 1, 2, 3, 4, 5 | | ||
+ | | Order | the order in which a transformation language was presented to the participant | 1, 2, 3 | | ||
+ | | Language | a transformation language | Epsilon, Henshin, VMTL | | ||
+ | | Score | number of correct comprehension question answers | 0, 1, 2, 3 | | ||
+ | | Time | time in seconds required to complete the comprehension task | positive integers | | ||
+ | | Difficulty | participant's subjective evaluation of the difficulty of completing the comprehension task | 1 -- 5 | | ||
+ | | Effort | participant's subjective evaluation of the effort required to complete the comprehension task | 1 -- 5 | | ||
==== Statistical Analysis ==== | ==== Statistical Analysis ==== | ||
+ | The replication package contains three statistical analysis scripts written in the [[https://www.r-project.org/|R programming language for statistical computing]]. The individual scripts contain code for analyzing comprehension results, cognitive load ratings, and task completion times, respectively. | ||
+ | |||
+ | The scripts have a similar structure. They include descriptive statistics computations (mean, standard deviation), data visualizations (box plots, bar plots), and hypothesis tests (Fisher's exact test, ANOVA, the Wilcoxon signed rank test). |