CORTEXA
← Browse
arxivcs.CRcs.HC2026-06-27

Beyond Her: Safety Dynamics in Role-play AI Companions

Zehang Deng, Zhaoyang Xie, Changzhou Han, Hiran Thabrew, Wanlun Ma, Yue Huang, Jason, Xue, Sheng Wen, Tianqing Zhu, Yang Xiang

The film 'Her' pictured a future of love between humans and AI. That future has quietly emerged in the form of Role-play AI Companions (RACs), where emotionally responsive interactions blur the boundary between tool use and relational engagement. However, the safety implications remain poorly understood, as user experiences evolve over time through safety dynamics, spanning both emotional and risk behavioral dynamics, that can gradually shift interactions toward risk. In this paper, we investigate safety dynamics in RAC usage through a two-part mixed-methods study (Study I \& II). (1) Study I consists of semi-structured interviews (N = 16) to identify the key factors shaping these dynamics. We find that users' internalizing problems, the role personality adopted by the RAC, and risk interaction patterns jointly shape safety dynamics. Building on these insights, (2) Study II conducts a 14-day Ecological Momentary Assessment (N = 102) to examine how safety dynamics unfold in real-world usage. We identify distinct user profiles based on internalizing problems and show that interactions with RACs can produce short-term emotional relief while masking longer-term deterioration. Furthermore, vulnerable users exhibit more unstable risk behavioral patterns over time, making risk emergence less predictable and harder to mitigate with static safeguards. Our findings highlight the importance of modeling safety as a dynamic process rather than a static property. We conclude with three-layer design implications for next-generation AI companions, advocating for adaptive safeguards that can respond to evolving emotional and behavioral signals.

View free PDFSource page

Related papers

arxivcs.CRcs.HC2026-07-08

Modeling Failure Dynamics in Time-Constrained Authentication Systems: Evidence of a Success Cliff in USSD Workflows

Aklile Seyoum Mamo, Amanuel Kebede, Anny Christelle Irakoze, Jema Ndibwile

Time-constrained interactive systems such as USSD (Unstructured Supplementary Service Data)-based financial services operate under strict session limits and sequential user interaction. While stronger authentication mechanisms improve security, they also increase interaction comp…

View free PDFSource page
arxivcs.MAcs.AIcs.CRcs.HC2026-07-13

An Explainable Agentic System for Detection of Conversational Scams with Summary-Based Memory

Ahmed Omar Salim Adnan, Yogananda Manjunath, Shivanjali Khare

Following the rapid progress of generative Artificial Intelligence, there is a growing threat posed by conversational scams. These scams often span over multiple weeks or months, gradually build trust and request for money or sensitive information. Existing scam-detection systems…

View free PDFSource page
arxivcs.CRcs.ETcs.HCq-bio.NC2026-07-11

Threat Vectors and the State of the Art in Defense Methods for Security in Neurotechnology

Bryce-Allen Bagley, Nathaniel Rose, Quintus Kilbourn, Matthew Canham

Brain-computer interfaces (BCIs) are a class of diverse hardware modalities, associated software, and connected devices which are widely used in a variety of fields, including neurosurgery, biomedical data analysis, and neuroimaging. Recent years have seen rapid advancements in B…

View free PDFSource page
arxivcs.CRcs.HCcs.SE2026-07-16

Setup Complete, Now You Are Compromised: Weaponizing Setup Instructions Against AI Coding Agents

Aadesh Bagmar, Pushkar Saraf

AI coding agents set up projects by reading documentation and installing the dependencies it lists, without verifying their names, sources, or known vulnerabilities. By editing only a README, requirements file, or Makefile, an attacker can redirect the agent to an untrusted regis…

View free PDFSource page
arxivcs.HCcs.CRcs.CY2026-07-10

Privacy Detective: A Narrative Game that Cultivates Student Developers' Privacy Awareness by Harnessing Legal Documents

Shao-Yu Chu, Jennifer Forsyth, Xu Wang, Haojian Jin

Developers' choices about what data a system collects, how it is used and shared, and what defaults govern user choices directly shape users' privacy experiences. Yet, developers often make problematic privacy-related design decisions without realizing the potential consequences.…

View free PDFSource page