CORTEXA
← Browse
arxivcs.HCcs.AIcs.SE2026-06-29Cited by 0

To Tab or Not to Tab: Measuring Critical Engagement in AI Code Completion Tools Using Behavioral Signals and Attention Checks

Jessica Hutchison, Ian Tyler Applebaum, Kenneth Angelikas, Kush Rakesh Patel, Phuoc Nguyen, Antonio Lazaro, Nicholas Rucinski, Rahad Arman Nabid, Stephen MacNeil

AI code completion tools, such as Github Copilot, provide students with code suggestions to help them write programs. However, recent qualitative studies suggest that students fail to critically evaluate these suggestions. We present Clover, a code completion tool that logs students' interactions with code suggestions and additionally offers attention checks to probe reflective engagement during programming tasks. We also develop a taxonomy of behavioral interaction metrics for AI-assisted programming, informed by literature. We analyzed relationships between interaction patterns, engagement with attention checks, and task performance. We observed that higher rates of tab accept were associated with lower attention check performance, while increased dwell time was associated with higher attention check performance. We conclude by discussing how programming process data and attention checks might support reflective engagement in AI-assisted programming.

View free PDFSource page

Related papers

arxivcs.SEcs.AI2026-07-31

From Code Review to Code Critique: Intent, Drift, and Spotlight for AI-Generated Diffs at Scale

Chandra Maddila, Mashrur Rashik, Euna Mehnaz Khan, Smriti Jha, James Saindon, Nachi Nagappan, et al.

AI coding agents are generating code at volumes that exceed the capacity of traditional peer review. At the same time, existing AI code review tools over-index on low-value suggestions such as style and best practices while under-indexing on the concerns human reviewers prioritiz…

View free PDFSource page
arxivcs.HCcs.AI2026-07-23

AI-Integrated Scientific Inquiry: A Practice-Centered Vision for Science Education

Arne Bewersdorff, Matias Rojas, Xiaoming Zhai

Artificial intelligence (AI) has become part of scientific inquiry. Scientists use AI to observe and measure phenomena, to identify patterns in data, and to build models. As AI moves into scientific inquiry, it gains relevance for science education: students should learn how AI i…

View free PDFSource page
arxivcs.SEcs.AI2026-07-30

Integrating AI into Requirements Quality Learning in Software Engineering Education: A TPACK-Guided Empirical Study

Hansika Ekanayake Mudiyanselage, Rohan Jai Dharmaraj, Malik Abdul Sami, Zheying Zhang

The rapid adoption of generative Artificial Intelligence (AI) in software engineering (SE) practice creates a need for pedagogically grounded approaches to AI integration in SE education, especially in conceptually intensive subjects such as requirements engineering (RE). This st…

View free PDFSource page
arxivcs.SEcs.AIcs.LG2026-07-30

To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing

Amir M. Ebrahimi, Mohammed Mehedi Hasan, Aaditya Bhatia, Gopi Krishnan Rajbahadur, Ahmed E. Hassan

Large language models increasingly write and repair production code, yet evidence is mounting that their test-passing patches leave codebases harder to maintain. We identify one concrete source: deletion avoidance, the systematic tendency to retain code that an intended edit requ…

View free PDFSource page
arxivcs.AIcs.MAcs.SE2026-07-31

Beyond Component Testing: Validating Agentic AI Systems

Fabio Orazio Mirto, Luca D'Agati, Giuseppe Tricomi, Stefano Silvestri, Francesco Longo, Antonio Puliafito, et al.

Agentic AI systems act through multi-step trajectories that combine planning, tool use, memory, interaction, and adaptation. This behavior stretches validation practice beyond component testing and one-shot input--output evaluation, because acceptable system behavior now depends…

View free PDFSource page
arxivcs.HCcs.AI2026-07-24

Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education

Jennie Ren, Jordan H. McDowell, Kyrie Zhixuan Zhou

Generative AI is reshaping programming education, yet educators often infer students' AI-supported learning from classroom observations alone. This experience report presents a trio-ethnography involving two computing educators with different teaching philosophies and one undergr…

View free PDFSource page