Evaluating Design Fiction: The Right Tool for the Job

Status

Eric P. S. Baumer, Mark Blythe, and Theresa J. Tanenbaum. (2020). Evaluating Design Fiction: The Right Tool for the Job. in ACM Conference on Designing Interactive Systems (DIS). (Eindhoven, the Netherlands; held virtually). [24.0% acceptance rate; Honorable Mention Award]

Abstract

Design fiction has become so widely adopted that it regularly appears in contexts ranging from CEO speeches to dedicated tracks at academic conferences. However, evaluating this kind of work is difficult; it is not clear what good or bad design fiction is or what the judgment criteria should be. In this paper we assert that design fiction is a heterogeneous set of methods, and practices, able to produce a diversity of scholarly and design contributions. We argue locating these diverse practices under the single header of “design fiction” has resulted in epistemological confusion over the appropriate method of evaluation. We identify different traditions within the HCI literature—critical design; narratology and literary theory; studio-based design “crits”; user studies; scenarios and persona development; and thought experiments—to articulate a typology of evaluative frames. There is often a mismatch between the standards to which design fiction is held and the knowledge that speculative methods seek to produce. We argue that evaluating a given instance of design fiction requires us to properly select the right epistemological tool for the job.

pdf

Methods for Generating Typologies of Non/use

Status

Devansh Saxena, Patrick Skeba, Shion Guha, and Eric P. S. Baumer. (2020). Methods for Generating Typologies of Non/use. Proceedings of the ACM on Human-Computer Interaction 3, CSCW.

Abstract

Prior studies of technology non-use demonstrate the need for approaches that go beyond a simple binary distinction between users and non-users. This paper proposes a set of two different methods by which researchers can identify types of non/use relevant to the particular sociotechnical settings they are studying. These methods are demonstrated by applying them to survey data about Facebook non/use. The results demonstrate that the different methods proposed here identify fairly comparable types of non/use. They also illustrate how the two methods make different trade offs between the granularity of the resulting typology and the total sample size. The paper also demonstrates how the different typologies resulting from these methods can be used in predictive modeling, allowing for the two methods to corroborate or disconfirm results from one another. The discussion considers implications and applications of these methods, both for research on technology non/use and for studying social computing more broadly.

DOI