Eric P. S. Baumer, Mark Blythe, and Theresa J. Tanenbaum. (2020). Evaluating Design Fiction: The Right Tool for the Job. in ACM Conference on Designing Interactive Systems (DIS). (Eindhoven, the Netherlands; held virtually). [24.0% acceptance rate; Honorable Mention Award]
Abstract
Design fiction has become so widely adopted that it regularly appears in contexts ranging from CEO speeches to dedicated tracks at academic conferences. However, evaluating this kind of work is difficult; it is not clear what good or bad design fiction is or what the judgment criteria should be. In this paper we assert that design fiction is a heterogeneous set of methods, and practices, able to produce a diversity of scholarly and design contributions. We argue locating these diverse practices under the single header of “design fiction” has resulted in epistemological confusion over the appropriate method of evaluation. We identify different traditions within the HCI literature—critical design; narratology and literary theory; studio-based design “crits”; user studies; scenarios and persona development; and thought experiments—to articulate a typology of evaluative frames. There is often a mismatch between the standards to which design fiction is held and the knowledge that speculative methods seek to produce. We argue that evaluating a given instance of design fiction requires us to properly select the right epistemological tool for the job.