Topicalizer: Reframing Core Concepts in Machine Learning Visualization by Co-designing for Interpretivist Scholarship

Status

Eric P. S. Baumer, Drew Siedel, Lena McDonnell, Jiayun Zhong, Patricia Sittikul, and Micki McGee. (2020). Topicalizer: Reframing Core Concepts in Machine Learning Visualization by Co-designing for Interpretivist Scholarship. Human-Computer Interaction, Special Issue on Unifying Human Computer Interaction and Artificial Intelligence 35(5-6): 452-480.

Abstract

Computational algorithms can provide novel, compelling functionality for interactive systems. However, designing such systems for users whose expertise lies outside computer science poses novel and complex challenges. This paper focuses specifically on the domain of designing interactive topic modeling visualizations to support interpretivist scholars. It describes a co-design process that involved working directly with two such scholars across two different corpora. The resultant visualization has both several similarities and key differences with other topic modeling visualizations, illustrated here using both the final design and discarded prototypes. The paper’s core contribution is an attention to how our emphasis on interpretation played out, both in the design process and in the final visualization design. The paper concludes by discussing the kinds of issues and tensions that emerged in the course of this work, as well as the ways that these issues and tensions can apply to much broader contexts of designing interactive algorithmic systems.

DOI | pdf

Speaking on Behalf of: Representation, Authority, and Delegation in Computational Text Analysis

Status

Eric P. S. Baumer and Micki McGee. (2019). Speaking on Behalf of: Representation, Authority, and Delegation in Computational Text Analysis. in Proceedings of the AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES). (Honolulu, HI).

Abstract

Computational tools can often facilitate human work by rapidly summarizing large amounts of data, especially text. Doing so delegates to such models some measure of author- ity to speak on behalf of those people whose data are being analyzed. This paper considers the consequences of such delegation. It draws on sociological accounts of representation and translation to examine one particular case: the application of topic modeling to blogs written by parents of children on the autism spectrum. In doing so, the paper illustrates the kinds of statements that topic models, and other computational techniques, can make on behalf of people. It also articulates some of the potential consequences of such statements. The paper concludes by offering several suggestions about how to address potential harms that can occur when computational models speak on behalf of someone.

DOI

Interpretive Impacts of Text Visualization: Mitigating Political Framing Effects

Status

Eric P. S. Baumer, Jaime Snyder, and Gery Gay. (2018). Interpretive Impacts of Text Visualization: Mitigating Political Framing Effects. ACM Transactions on Human-Computer Interaction (ToCHI), 25(4), 20:1–20:26.

Abstract

Information visualizations are often evaluated as a tool in terms of their ability to support performance of a specific task. This article argues that value can be gained by instead evaluating visualizations from a communicative perspective. Specifically, it explores how text visualization can influence the impacts that framing has on the perception of political issues. Using data from a controlled laboratory study, the results presented here demonstrate that exposure to a text visualization can mitigate framing effects. Furthermore, it also shows a transfer effect, where participants who saw the visualization remained uninfluenced by framing in subsequent texts, even when the visualization was absent. These results carry implications for the methods used to evaluate information visualization systems, for understanding the cognitive and interpretive mechanisms by which framing effects occur, and for exploring the design space of interactive text visualization.

DOI

Comparing Grounded Theory and Topic Modeling: Extreme Divergence or Unlikely Convergence?

Status

Eric P. S. Baumer, David Mimno, Shion Guha, Emiy Quan, and Geri Gay. (2017). Comparing Grounded Theory and Topic Modeling: Extreme Divergence or Unlikely Convergence? Journal of the Association for Information Science and Technology (JASIST), 68(6): 1397–1410.

Abstract

Researchers in information science and related areas have developed various methods for analyzing textual data, such as survey responses. This article describes the application of analysis methods from two distinct fields, one method from interpretive social science and one method from statistical machine learning, to the same survey data. The results show that the two analyses produce some similar and some complementary insights about the phenomenon of interest, in this case, nonuse of social media. We compare both the processes of conducting these analyses and the results they produce to derive insights about each method’s unique advantages and drawbacks, as well as the broader roles that these methods play in the respective fields where they are often used. These insights allow us to make more informed decisions about the tradeoffs in choosing different methods for analyzing textual data. Further- more, this comparison suggests ways that such methods might be combined in novel and compelling ways.

DOI

Machine Learning and Grounded Theory Method: Convergence, Divergence, and Combination

Status

Michael Muller, Shion Guha, Eric P. S. Baumer, David Mimno, and N. Sadat Shami (2016). Machine Learning and Grounded Theory Method: Convergence, Divergence, and Combination. in Proceedings of the ACM Conference on Supporting Group Work (GROUP). (Sanibel Island, FL).

Abstract

Grounded Theory Method (GTM) and Machine Learning (ML) are often considered to be quite different. In this note, we explore unexpected convergences between these methods. We propose new research directions that can further clarify the relationships between these methods, and that can use those relationships to strengthen our ability to describe our phenomena and develop stronger hybrid theories.

DOI

Testing and Comparing Computational Approaches for Identifying the Language of Framing in Political News

Status

Eric P. S. Baumer, Elisha Elovic, Ying Qin, Francesca Polletta, & Geri K. Gay. (2015). Testing and Comparing Computational Approaches for Identifying the Language of Framing in Political News. in Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics – Human Language Technologies (NAACL – HLT) (pp. 1472–1482). Denver, CO.

Abstract

The subconscious influence of framing on perceptions of political issues is well-document in political science and communication research. A related line of work suggests that drawing attention to framing may help reduce such framing effects by enabling frame reflection, critical examination of the framing underlying an issue. However, definite guidance on how to identify framing does not exist. This paper presents a technique for identifying frame-invoking language. The paper first describes a human subjects pilot study that explores how individuals identify framing and informs the design of our technique. The paper then describes our data collection and annotation approach. Results show that the best performing classifiers achieve performance comparable to that of human annotators, and they indicate which aspects of language most pertain to framing. Both technical and theoretical implications are discussed.

pdf (ACL)

Data Set – This includes all 75 annotated articles, as well as descriptions of the format and instructions on use.

Broadening Exposure, Questioning Opinions, and Reading Patterns with Reflext: a Computational Support for Frame Reflection

Status

Eric P. S. Baumer, Claire Cipriani, Mitchell Davis, Gary He, Jaclyn Jeffrey-Wilensky, James Kang, Jinjoo Lee, Justin Zupnick, and Geri K. Gay. (2014). Broadening Exposure, Questioning Opinions, and Reading Patterns with Reflext: a Computational Support for Frame Reflection. Journal of Information Technology and Politics, 11(1), 45-63.

ABSTRACT

Vast amounts of political coverage are generated daily online. Some tools have been developed to help keep track of what is being said, but fewer efforts focus on how things are being said, i.e., how issues are framed. This article presents a study of Reflext, an interactive visualization tool that leverages computational linguistic analysis to support reflection on the framing of political issues. This system was deployed in a field study, during which the tool was used by regular readers of political news coverage during the 2012 U.S. election campaign. The results describe the tool’s support for a variety of activities related to frame reflection, how users integrated tool use with their existing reading practices, and broader issues in how participants interpreted the computational analysis and visualization. These findings contribute to our understanding of how algorithmically based interactive systems might mediate both the practical experiences of and situated interpretation of framing in political content.

DOI | pdf