Tag Archives: hcad

“It’s Like the Value System in the Loop”: Domain Experts’ Values Expectations for NLP Automation

Status

Dilruba Showkat and Eric P. S. Baumer. 2022. “It’s Like the Value System in the Loop”: Domain Experts’ Values Expectations for NLP Automation. In Proceedings of the ACM Conference on Designing Interactive Systems (DIS), 100–122.

Abstract

The rise of automated text processing systems has led to the development of tools designed for a wide variety of application domains. These technologies are often developed to support non-technical users such as domain experts and are often developed in isolation of the tools primary user. While such developments are exciting, less attention has been paid to domain experts’ expectations about the values embedded in these automated systems. As a step toward addressing that gap, we examined values expectations of journalists and legal experts. Both these domains involve extensive text processing and place high importance on values in professional practice. We engaged participants from two non-profit organizations in two separate co-speculation design workshops centered around several speculative automated text processing systems. This study makes three interrelated contributions. First, we provide a detailed investigation of domain experts’ values expectations around future NLP systems. Second, the speculative design fiction concepts, which we specifically crafted for these investigative journalists and legal experts, illuminated a series of tensions around the technical implementation details of automation. Third, our findings highlight the utility of design fiction in eliciting not-to-design implications, not only about automated NLP but also about technology more broadly. Overall, our study findings provide groundwork for the inclusion of domain experts values whose expertise lies outside of the field of computing into the design of automated NLP systems.

DOI

Where Do Stories Come From? Examining the Exploration Process in Investigative Data Journalism

Status

Dilruba Showkat and Eric P. S. Baumer. 2021. Where Do Stories Come From? Examining the Exploration Process in Investigative Data Journalism. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2: 390:1-390:31.

Abstract

Investigative data journalists work with a variety of data sources to tell a story. Though prior work has indicated that there is a close relationship between journalists’ data work practices and that of data scientists. However, these relationships and data work practices are not empirically examined, and understanding them is crucial to inform the design of tools that are used by different groups of people including data scientists and data journalists. Thus, to bridge this gap, we studied investigative reporters’ data work practices with one non-profit investigative newsroom. Our study design includes two activities: 1) semi-structured interviews with journalists, and 2) a sketching activity allowing journalists to depict examples of their work practices. By analyzing these data and synthesizing them across related prior work, we propose the major phases in the data-driven investigative journalism story idea generation process. Our study findings show that the journalists employ a collection of multiple, iterative, cyclic processes to identify journalistically “interesting” story ideas. These processes both significantly resemble and show subtle nuanced differences with data science work practices identified in prior research. We further verified our proposal through a member check with key informants. This work offers three primary contributions. First, it provides a close glimpse into the main phases of investigative journalists’ data-driven story idea generation technique. Second, it complements prior work studying formal data science practices by examining data-driven investigative journalists, whose primary expertise lies outside computing. Third, it identifies particular points in the data exploration processes that would benefit from design interventions and suggests future research directions.

DOI

A Review on Strategies for Data Collection, Reflection, and Communication in Eating Disorder Apps

Status

Anjali Devakumar, Jay Modh, Bahador Saket, Eric P. S. Baumer, and Munmun De Choudhury. 2021. A Review on Strategies for Data Collection, Reflection, and Communication in Eating Disorder Apps. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI), 1–19.

Abstract

Eating disorders (EDs) constitute a mental illness with the highest mortality. Today, mobile health apps provide promising means to ED patients for managing their condition. Apps enable users to monitor their eating habits, thoughts, and feelings, and offer analytic insights for behavior change. However, not only have scholars critiqued the clinical validity of these apps, their underlying design principles are not well understood. Through a review of 34 ED apps, we uncovered 11 different data types ED apps collect, and 9 strategies they employ to support collection and refection. Drawing upon personal health informatics and visualization frameworks, we found that most apps did not adhere to best practices on what and how data should be collected from and reflected to users, or how data-driven insights should be communicated. Our review offers suggestions for improving the design of ED apps such that they can be useful and meaningful in ED recovery.

DOI

Misfires, Missed Data, Misaligned Treatment: Disconnects in Collaborative Treatment of Eating Disorders

Status

Lauren C. Taylor, Kelsie Belan, Munmun De Choudhury, and Eric P. S. Baumer. 2021. Misfires, Missed Data, Misaligned Treatment: Disconnects in Collaborative Treatment of Eating Disorders. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1: 31:1-31:28.

Abstract

Technology bears important relationships to our health and wellness and has been utilized over the past two decades as an aid to support both self-management goals as well as collaboration among treatment teams. However, when chronic illnesses such as eating disorders (ED) are managed outside of institutionalized care settings, designing effective technology to support collaboration in treatment necessitates that we understand the relationships between patients, clinicians, and support networks. We conducted in-depth, semi-structured, interviews with 9 ED patients and 10 clinicians to understand the ED journey through the lens of collaborative efforts, technology use, and potential detriments. Based on our analysis of these 19 interviews, we present novel findings on various underlying disconnects within the collaborative ED treatment process – disconnects among clinicians, between treatment foci, among preferences in tracking, within support networks, and in patients’ own identities. Our findings highlight how these various disconnects are concomitant with and gaps can stem from a lack of collaboration between different stakeholders in the ED journey. We also identify methods of facilitating collaboration in these disconnects through technological mediators.

DOI

Evaluating Design Fiction: The Right Tool for the Job

Status

Eric P. S. Baumer, Mark Blythe, and Theresa J. Tanenbaum. (2020). Evaluating Design Fiction: The Right Tool for the Job. in ACM Conference on Designing Interactive Systems (DIS). (Eindhoven, the Netherlands; held virtually). [24.0% acceptance rate; Honorable Mention Award]

Abstract

Design fiction has become so widely adopted that it regularly appears in contexts ranging from CEO speeches to dedicated tracks at academic conferences. However, evaluating this kind of work is difficult; it is not clear what good or bad design fiction is or what the judgment criteria should be. In this paper we assert that design fiction is a heterogeneous set of methods, and practices, able to produce a diversity of scholarly and design contributions. We argue locating these diverse practices under the single header of “design fiction” has resulted in epistemological confusion over the appropriate method of evaluation. We identify different traditions within the HCI literature—critical design; narratology and literary theory; studio-based design “crits”; user studies; scenarios and persona development; and thought experiments—to articulate a typology of evaluative frames. There is often a mismatch between the standards to which design fiction is held and the knowledge that speculative methods seek to produce. We argue that evaluating a given instance of design fiction requires us to properly select the right epistemological tool for the job.

DOI | pdf

Topicalizer: Reframing Core Concepts in Machine Learning Visualization by Co-designing for Interpretivist Scholarship

Status

Eric P. S. Baumer, Drew Siedel, Lena McDonnell, Jiayun Zhong, Patricia Sittikul, and Micki McGee. (2020). Topicalizer: Reframing Core Concepts in Machine Learning Visualization by Co-designing for Interpretivist Scholarship. Human-Computer Interaction, Special Issue on Unifying Human Computer Interaction and Artificial Intelligence 35(5-6): 452-480.

Abstract

Computational algorithms can provide novel, compelling functionality for interactive systems. However, designing such systems for users whose expertise lies outside computer science poses novel and complex challenges. This paper focuses specifically on the domain of designing interactive topic modeling visualizations to support interpretivist scholars. It describes a co-design process that involved working directly with two such scholars across two different corpora. The resultant visualization has both several similarities and key differences with other topic modeling visualizations, illustrated here using both the final design and discarded prototypes. The paper’s core contribution is an attention to how our emphasis on interpretation played out, both in the design process and in the final visualization design. The paper concludes by discussing the kinds of issues and tensions that emerged in the course of this work, as well as the ways that these issues and tensions can apply to much broader contexts of designing interactive algorithmic systems.

DOI | pdf

Who is the “Human” in Human-Centered Machine Learning: The Case of Predicting Mental Health from Social Media

Status

Stevie Chancellor, Eric P. S. Baumer, and Munmun De Choudhury. (2019). Who is the “Human” in Human-Centered Machine Learning: The Case of Predicting Mental Health from Social Media. Proceedings of the ACM Human-Computer Interaction 2, CSCW. [Honorable Mention Award]

Abstract

“Human-centered machine learning” (HCML) combines human insights and domain expertise with data-driven predictions to answer societal questions. This area’s inherent interdisciplinarity causes tensions in the obligations researchers have to the humans whose data they use. This paper studies how scientific papers represent human research subjects in HCML. Using mental health status prediction on social media as a case study, we conduct thematic discourse analysis on 55 papers to examine these representations. We identify five discourses that weave a complex narrative of who the human subject is in this research: Disorder/Patient, Social Media, Scientific, Data/Machine Learning, and Person. We show how these five discourses create paradoxical subject and object representations of the human, which may inadvertently risk dehumanization. We also discuss the tensions and impacts of interdisciplinary research; the risks of this work to scientific rigor, online communities, and mental health; and guidelines for stronger HCML research in this nascent area.

DOI

Speaking on Behalf of: Representation, Authority, and Delegation in Computational Text Analysis

Status

Eric P. S. Baumer and Micki McGee. (2019). Speaking on Behalf of: Representation, Authority, and Delegation in Computational Text Analysis. in Proceedings of the AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES). (Honolulu, HI).

Abstract

Computational tools can often facilitate human work by rapidly summarizing large amounts of data, especially text. Doing so delegates to such models some measure of author- ity to speak on behalf of those people whose data are being analyzed. This paper considers the consequences of such delegation. It draws on sociological accounts of representation and translation to examine one particular case: the application of topic modeling to blogs written by parents of children on the autism spectrum. In doing so, the paper illustrates the kinds of statements that topic models, and other computational techniques, can make on behalf of people. It also articulates some of the potential consequences of such statements. The paper concludes by offering several suggestions about how to address potential harms that can occur when computational models speak on behalf of someone.

DOI

Toward Human-Centered Algorithm Design

Status

Eric P. S. Baumer. (2017). Toward Human-Centered Algorithm Design. Big Data & Society, 4(2).

Abstract

As algorithms pervade numerous facets of daily life, they are incorporated into systems for increasingly diverse purposes. These systems’ results are often interpreted differently by the designers who created them than by the lay persons who interact with them. This paper offers a proposal for human-centered algorithm design, which incorporates human and social interpretations into the design process for algorithmically based systems. It articulates three specific strategies for doing so: theoretical, participatory, and speculative. Drawing on the author’s work designing and deploying multiple related systems, the paper provides a detailed example of using a theoretical approach. It also discusses findings pertinent to participatory and speculative design approaches. The paper addresses both strengths and challenges for each strategy in helping to center the process of designing algorithmically based systems around humans.

DOI