People of ACM - D. Fox Harrell
March 27, 2018
What prompted you to found the ICE Lab at MIT?
My career is a quest to use the computer as a medium to spur the imagination and galvanize people to critically reflect upon society. Computing systems such as interactive stories, video games, and virtual reality systems are powerful media for building worlds and representing ourselves that can challenge and expand our concepts, worldviews, and values. The ICE Lab provides a research venue through which to pursue and grow this vision. We use artificial intelligence and related computational approaches to engender reflection on social phenomena ranging from social ills such as sexism and racism to global conflict. These systems model such phenomena to create richer experiences for users, help educate diverse learners, and conduct social scientific research studies to assess if, and how, these projects help people become more critically aware of their societies.
The ICE Lab also researches how what we call “virtual identity systems” (technologies with which users represent themselves ranging from avatars to social media profiles) can best serve positive user needs and values. At their best, virtual identities can help us to imagine ourselves in empowering new ways and help us to understand each others’ experiences. At their worst, virtual identity systems can reproduce discrimination, biases, and stereotypes with devastating impacts on users ranging from bullying to threats and acts of violence, as many female video game players and social media users know all too well. Hence, we also strive to use computational approaches not only to express, but to analyze and assess virtual identity technologies to enable the best of what they have to offer and avoid their worst impacts.
Gaming and social media applications are now especially prominent not only in the lives of young people, but people across demographics of age, race/ethnicity, and gender. How might an ICE-developed project address crucial social issues of our time across demographic groups?
Our work supporting people in better understanding social identity phenomena serves these ends. Examples include "Mimesis," an online game that models social and psychological impacts of a subtle form of racism, and "MazeStar," a computer science learning platform (it enables students to create, document, and share games they create and to play our game called "Mazzy" that teaches coding concepts) used in Boston Public Schools to help students from groups underrepresented in STEM learn computer science and see themselves as computer scientists. I also collaborated on "The Enemy," a virtual reality project directed by the war photographer Karim Ben Khelifa. "The Enemy" seeks to counteract the cycle of dehumanizing “the other” as a necessary condition for war by allowing users to experience interviews with real combatants on both sides of global conflicts (e.g., in Gaza, East Congo, and El Salvador) via a VR headset.
As the Human-Computer Interaction Producer on "The Enemy," my aim was to enhance users’ narrative engagement and provide more customized experiences to users based on the categories they fall into, by categorizing users based on measuring proxies for their nervousness and bias and eliciting other information about their backgrounds and relationships to the conflicts. This allows for surprising and interesting narrative events to occur, such as the end users seeing themselves in a virtual mirror as the combatant with which they were measured as being least comfortable. The idea is that you could even be the enemy if you were raised as your enemy was.
Based on your research in cognitive science, what do you think are keys to creating games, VR experiences, and computer-based works that foster user engagement and serve the needs of diverse users?
The ICE Lab produces platforms enabling developers to dynamically change aspects of stories such as metaphors used, cultural perspectives conveyed, and emotional tone evoked. Cognitive science constructs—in particular models of how humans creatively generate metaphors, blend concepts, and categorize things in the world—have been useful toward this end. I have implemented computational models of such phenomena to enable systems to generate new content on the fly and change how narratives are told based upon user input.
One of your most recent projects, Chimeria: Grayscale, seeks to raise awareness of sexual harassment in the workplace. What have you learned from player engagement with Grayscale?
The game explicitly addresses many workplace issues that frequent news headlines (since before the #metoo era, but even more so in recent times): workplace attire, inappropriate language, hiring decisions, etc. "Grayscale" is built atop a patent pending approach to dynamically modeling identity categorization implemented in a platform that is played in a web browser. The interface looks like a familiar email client, all in gray tones except for the color the player selected that is used to highlight messages. Players take on the role of a newly hired human resources manager in a toxic workplace. While reading and selecting responses to emails, players navigate the tensions in the game's narrative setting. There is no way to simply win, rather the player must navigate ethical and career tensions. Ultimately, it's about helping the user to understand the structure of how both overt and subtler forms of sexism operate.
Based on our user studies, participants were shown to have engaged in critical reflection. They contemplated issues ranging from “normal behavior” to their preexisting ideas about social roles. We designed the system to convey a particular social science model of sexism that includes more overt acts as well as subtler forms that may sometimes be dismissed as less harmful, but are oppressive nonetheless. We were pleased that the system sparked reflection and that this reflection can lead to behavior change, and we hope to social change as well. In this manner "Grayscale" could catalyze awareness when people encounter these scenarios in the real world and even spark people to act as advocates and allies.
We think that storytelling is an important way to address nuanced, sensitive issues in a way that respects the complexity of the topic while artfully drawing people in. In this way, "Grayscale" can be seen as an example of a type of narrative system that could be applied to many issues in art, entertainment, education, and training.
D. Fox Harrell is Professor of Digital Media and Artificial Intelligence, and Director of the Imagination, Computation and Expression (ICE) Laboratory at the Massachusetts Institute of Technology (in CSAIL and Comparative Media Studies). His research includes developing new forms of computational narrative, gaming, virtual reality, social media, and related digital media based in computer science, cognitive science, and digital media arts. Harrell has also worked as an interactive television producer and game designer. His most recent book is Phantasmal Media: An Approach to Imagination, Computation and Expression.
His work has been recognized with major research grants from the National Science Foundation (NSF). These projects include “Toward Using Virtual Identities in Computer Science for Broadening Participation” and “CAREER: Computing for Advanced Identity Representation,” for which Harrell received an NSF Career Award. He was the Human-Computer Interaction Producer on a virtual reality work called The Enemy, which won the Rose d’Or Award.