in(A)n(i)mate

in(A)n(I)mate is an interactive AI-driven system that invites participants to speak with objects. The piece showcases an innovative use of GPT’s multimodal feature, through its ability to recognize objects in an image and generate responses in their style. Participants place an object of their choice in front of a black box and engage in a conversation with the object by pressing buttons on the box, mostly unaware that GPT is involved. in(A)n(I)mate provokes a discussion about human relationships with inanimate matter, and considers the role of non-human agents in mediating and animating objects.

​

The world of inanimate objects is one we have limited access to. We can describe objects by their appearance or function, but we do not know what is hidden beneath these aspects. Heidegger argued that our interaction with objects is usually seen through their practical use. Objects are “ready-to-hand” as tools that serve a purpose, integrated within our tasks, and thus almost transparent in themselves. A hammer is usually seen only through the framework of hammering a nail. However, objects can reappear and gain attention when our practical engagement with them is disrupted. This transition occurs when a tool breaks. Then, Heidegger claims, objects become “present-at-hand,” and the attention shifts from their integration into our activity to their status as separate objects that require care and repair.[7]. 

in(A)n(I)mate offers another kind of disruption through which objects become separate from their use and are contemplated as present in the world. We invite participants to engage in conversations with objects. If, for instance, they wish to speak with a hammer, they can present it to the in(A)n(I)mate system and ask it questions. in(A)n(I)mate, using OpenAI’s LLM, GPT-4-turbo, and its multi-modal features, will facilitate the conversation by recognizing the hammer and generating responses in its style. 

 

in(A)n(I)mate is designed as a black box equipped with two buttons. Inside the box is a Raspberry Pi 4 microcomputer, which hosts OpenAI’s speech-to-text model and GPT-4-Turbo, both connected via the API. The box utilizes a wireless internet connection and is interfaced with a webcam, microphone, and speaker. Upon pressing a button, a spoken message instructs the participant to present an object to the box and press a second button. This button captures a photo and sends it to GPT along with this prompt: "What is the most conspicuous object in this image? (Include only the object description in the response, not a full sentence.)" GPT announces the recognized object and participants can then initiate a conversation with the object. Each time the button is pressed, the box emits a sound, and the participant can pose a question. The system captures words and transcribes them into text, which is transmitted to GPT in real-time along with this prompt: "Respond in the style of (the recognized object). Keep your response short and phrase the response as if spoken from a first-person perspective." The generated response is converted back into speech and played aloud through the speaker. 

Participants can converse with the object for as long as they wish. Typical conversations lasted ten to thirty minutes. These conversations were somewhat slow due to GPT’s processing lag. However, this did not significantly alter the flow. To start a new conversation, participants can choose a new object and restart the process. To convey that each object has a distinct being, the system shifts to a different voice style, randomly cycling between six styles provided by the Speech-to-Text model. There is no deliberate reasoning behind attaching a specific voice to an object. If a participant decides to start a new conversation with the same object, the object’s voice style will still change. 

The black box design intentionally underscores its opaque nature. Although participants appear to engage in a dialogue with objects, the interaction is mediated by the components concealed within the box. As conversations lengthened and deepened, participants became immersed in the conversation, rarely questioning or addressing the box. This aesthetic communicates the inherent complexities and misunderstandings often associated with AI systems. 

Interactions with the system were offered at the University of California, Davis during March 2024. The system was placed in a busy space where many students and faculty pass by. The system recorded the conversation logs of about 60 conversations with various objects selected by the participants. The dialogues mediated by in(A)n(I)mate are generated by GPT. When reading the system’s logs, this fact may seem straightforward (Fig. 2-5). However, in real-time interactions with the system, words are spoken aloud and participants may not even realize that GPT is involved. In many cases, participants seemed to suspend their disbelief, gazing at the object while speaking with it, and immersing themselves in conversations despite knowing that these objects could not communicate in human language. Many conversations tended to be humorous, emotional, or argumentative.
 

in(A)n(I)mate is part of a larger research in which we examine how AI agents can seamlessly integrate into everyday life, modify human behavior, impact social relationships, and invite us to think otherwise.