An exploratory study regarding the ease-of-use, comprehensibility, and usefulness of the Empirical Standards Checklists




Cupryk, Cassandra

Journal Title

Journal ISSN

Volume Title



Context/Background: Novice researchers have stated that being provided with guidelines for reviewing empirical research papers would be helpful. The Empirical Standards Checklist Generator is a tool that can generate a variety of Empirical Standards Checklists. An Empirical Standards Checklist contains the core criteria that can be used to review Software Engineering empirical papers. Moreover, my exploratory study aims to determine whether novice researchers could benefit from using the Empirical Standards Checklists to help them review Software Engineering empirical papers. Objective: To investigate whether novice researchers perceive the Empirical Standards Checklists as easy to understand, easy to use, and useful for reviewing empirical papers. Methods: Seven participants completed a survey to evaluate the Empirical Standards Checklists and then participated in a group discussion. During the survey, the participants used the appropriate Empirical Standard Checklists to review a qualitative survey paper and a repository mining paper. They then highlighted the items from the Empirical Standards Checklists that were difficult to comprehend. The participants also answered survey questions that exposed their perceptions of the Empirical Standards Checklists' comprehensibility, ease-of-use, and usefulness. Results: The majority of the participants had positive perceptions of the Empirical Standards Checklists' comprehensibility, ease-of-use, and usefulness. Conclusion: This exploratory study demonstrates that the Empirical Standards Checklist is a promising new tool for reviewing Software Engineering empirical research papers.



empirical standards checklist, empirical standards checklists, usefulness, ease-of-use, comprehensibility, perceptions, survey, group discussion, exploratory study