vmtl:evaluation
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
|
vmtl:evaluation [2015/08/23 14:48] rvac |
vmtl:evaluation [2015/08/23 15:50] rvac |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| + | ~~NOCACHE~~ | ||
| + | |||
| ====== Empirical Evaluation ====== | ====== Empirical Evaluation ====== | ||
| Line 5: | Line 7: | ||
| ===== Replication Package ===== | ===== Replication Package ===== | ||
| - | The replication package can be downloaded as a ZIP archive [[|here]]. | + | The replication package can be downloaded as a ZIP archive [[http://www2.compute.dtu.dk/~rvac/files/experiments/Replication%20Package%20-%20An%20Empirical%20Assessment%20of%20Model%20Transformation%20Language%20Learnability.zip|here]]. |
| It consists of the experimental material used for each experiment, the collected and encoded data, and statistical analysis scripts for analyzing the data. The experimental material and data are published under the [[http://creativecommons.org/licenses/by/4.0/|Creative Commons Attribution 4.0 International License]], and the statistical analysis scripts are published under the [[http://opensource.org/licenses/MIT|MIT License]]. | It consists of the experimental material used for each experiment, the collected and encoded data, and statistical analysis scripts for analyzing the data. The experimental material and data are published under the [[http://creativecommons.org/licenses/by/4.0/|Creative Commons Attribution 4.0 International License]], and the statistical analysis scripts are published under the [[http://opensource.org/licenses/MIT|MIT License]]. | ||
| Line 25: | Line 27: | ||
| The experimental data is presented as a comma-separated values (CSV) file consisting of the following columns: | The experimental data is presented as a comma-separated values (CSV) file consisting of the following columns: | ||
| + | ^Column^Definition^Values^ | ||
| + | | Experiment | experiment identifier | 1, 2 | | ||
| + | |Id | participant identifier, unique within an experiment | positive integers | | ||
| + | | UML | participant's self-assessed UML knowledge | 1, 2, 3, 4, 5 | | ||
| + | | OCL | participant's self-assessed OCL knowledge | 1, 2, 3, 4, 5 | | ||
| + | | MT | participant's self-assessed model transformation knowledge | 1, 2, 3, 4, 5 | | ||
| + | | Programming | participant's self-assessed programming knowledge | 1, 2, 3, 4, 5 | | ||
| + | | Order | the order in which a transformation language was presented to the participant | 1, 2, 3 | | ||
| + | | Language | a transformation language | Epsilon, Henshin, VMTL | | ||
| + | | Score | number of correct comprehension question answers | 0, 1, 2, 3 | | ||
| + | | Time | time in seconds required to complete the comprehension task | positive integers | | ||
| + | | Difficulty | participant's subjective evaluation of the difficulty of completing the comprehension task | 1 -- 5 | | ||
| + | | Effort | participant's subjective evaluation of the effort required to complete the comprehension task | 1 -- 5 | | ||
| ==== Statistical Analysis ==== | ==== Statistical Analysis ==== | ||
| + | The replication package contains three statistical analysis scripts written in the [[https://www.r-project.org/|R programming language for statistical computing]]. The individual scripts contain code for analyzing comprehension results, cognitive load ratings, and task completion times, respectively. | ||
| + | |||
| + | The scripts have a similar structure. They include descriptive statistics computations (mean, standard deviation), data visualizations (box plots, bar plots), and hypothesis tests (Fisher's exact test, ANOVA, the Wilcoxon signed rank test). | ||
vmtl/evaluation.txt · Last modified: 2016/02/12 13:40 by rvac
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 3.0 Unported
