Towards more Inclusive Software: A Large Scale Analysis of Inclusiveness from User Feedback

Date

2023-11-23

Authors

Arony, Nowshin Nawar

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

In an era of rapidly evolving software usage, addressing the diverse needs of users from around the world has emerged as a critical challenge. Diverse users bring forth diverse requirements, encompassing factors such as human values, ethnicity, culture, educational background, technical expertise, preferences, personality traits, emotional states, and mental and physical considerations. Among the various aspects, inclusiveness, representing a core human value, is often unknowingly neglected during software development, leading to user dissatisfaction. Online platforms, such as forums and social media, offer users a space to express their opinions regarding a software. As a result, in recent times, software companies have recognized these platforms as a source of user feedback. Therefore, in this study, I leverage user feedback from three popular online sources: Reddit, Google Play Store, and Twitter (now known as X) to explore the inclusiveness related concerns from end users. I collected user feedback from the three sources for 50 of the most popular apps in the world. The 50 apps are selected from 5 types of software: business, entertainment, financial, e-commerce, and social media. I employed a Socio-Technical Grounded Theory approach and manually analyzed 23,107 posts across the three sources. Through this process, I identified 1,211 inclusiveness related posts. The research resulted in the development of a taxonomy for inclusiveness comprising 6 major categories: Fairness, Technology, Privacy, Demography, Usability, and Other Human Values. Along with that, I investigated the process of automatically identifying inclusiveness and non-inclusiveness related posts using 5 popular deep learning-based models. Upon experimenting with five deep learning models, I found that GPT-2 performed best on Reddit, achieving an F1-score of 0.838, BERT on the Google Play Store with an F1-score of 0.849, and BART on Twitter with an F1-score of 0.930. My research provides a detailed view of inclusiveness-related user feedback, enabling software practitioners to gain a more holistic understanding of such user concerns. The insights from this thesis can guide software organizations to increase awareness and address the inclusiveness aspects relevant to their product from an end-user perspective. I further provided implications and suggestions that can be used to bridge the gap between user values and software so that software can truly resonate with the varied and evolving needs of diverse users.

Description

Keywords

user feedback, inclusive software, human aspects, socio-technical grounded theory

Citation