Learners' awareness of AI as originator of an academic text during revision: An exploratory study

In AI-assisted writing, learners revise texts originally drafted not by themselves but by a generative AI. We assume that if learners are aware that a text is authored by AI, it can impact their perceptions and revision behaviors, for instance, as compared to the common situation of revising peer-written passages in collaborative writing. This awareness can facilitate a more appropriate understanding of the characteristics of the generated content and support corresponding revision goals. The aim of the present study was to investigate whether and how learners address the originator of the text during the initial stage of revising an academic text purportedly written by either an AI or a peer. Moreover, we analyzed what kind of awareness of the originator’s characteristics learners express during revision. Participants were asked to think aloud while detecting revision occasions in a text labeled as AI-generated (n = 4, experimental condition) or as peer-written (n = 4, control condition). To investigate whether and how learners addressed their purported ‘co-author’, we analyzed the think-aloud protocols regarding instances where learners referenced an AI or a peer. We examined the content of these references to assess learners’ awareness of their co-author’s human-(like) or technical characteristics. The results indicate that learners referred to their assumed ‘co-author’ less frequently when revising a purportedly AI-generated text than when revising a purportedly peer-written text. This invites reflection on the differences between AI-assisted writing and collaborative writing with peers, and particularly on whether learners perceive AI as a writing partner and AI-assisted writing as a collaborative process. Furthermore, participants revising a purportedly AI-generated text addressed anthropomorphic features of AI, while neglecting to address its technical limitations, such as the risk of generating hallucinations. This underscores that learners may lack the AI literacy required for successful AI-assisted writing and thus need support to develop it.