Investigating AI Imitative Capabilities: A Comparative Genre Analysis of Reddit and ChatGPT-4 Spouse Appreciation Posts

Authors

  • Erika Kristine Estanislao Arcenal De La Salle University, University of the Philippines-Diliman
  • Licca Pauleen Valdez Capistrano De La Salle University
  • Marielle Jessie Dela Peña De Guzman De La Salle University, University of the Philippines-Manila
  • Micaela Isabel Molina Forrosuelo De La Salle University
  • Janeson Mirabuna Miranda Department of Humanities and Social Sciences, De La Salle University

DOI:

https://doi.org/10.17576/3L-2025-3101-04

Abstract

ChatGPT-4 has been a helping tool for its users in various fields and contexts. Among these, a recurring topic of inquiry is AI’s ability to mimic human language, particularly appreciative sentiments. In this study, we explore the current capabilities of ChatGPT-4 at mimicking human appreciatory language by comparing its prompt-based generated texts to the appreciative posts of married individuals on Reddit, specifically subreddit r/Marriage. Using Swales' moves analysis as the working framework, we determined and then labelled moves found in the Reddit posts as we examined them as a genre and compared them with AI-generated posts. Findings revealed that ChatGPT-4 imitated 11 out of the 13 moves present in the Reddit posts. Furthermore, five categories of obligatory moves were established to better distinguish between the texts. ChatGPT-4 was observed to have more clearly obligatory moves and was more consistent in following a certain sequence than texts written by a human being. Additionally, ChatGPT-4 generated texts had a higher average word count and gave an impersonal impression due to its lack of the following moves: Description of Spouse, Description of Original Poster, and Relationship Background. With these findings, we recommend further linguistic studies on Reddit posts and other non-academic social media forms as a genre and the continuous use of genre analysis to circumscribe, more precisely, the AI imitative capabilities of human natural language and its implications to future applications. Keywords: artificial intelligence; ChatGPT-4; imitative capabilities; Reddit; moves analysis

Author Biographies

Erika Kristine Estanislao Arcenal, De La Salle University, University of the Philippines-Diliman

Erika Kristine Estanislao Arcenal is a graduate of De La Salle University Senior High School and is currently an undergraduate student at the University of the Philippines Diliman under the BA Film program. Her research interests greatly lie in the avenue of the Arts, primarily filmmaking, creative writing, and performance arts. 

Licca Pauleen Valdez Capistrano, De La Salle University

Licca Pauleen Valdez Capistrano is a De La Salle University Senior High School graduate and is pursuing an undergraduate degree in BS Entrepreneurship at De La Salle University. She is deeply passionate about the humanities and social sciences, specifically literature, psychology, and sociology. Lately, however, she has also discovered a newfound interest in business and innovation.

Marielle Jessie Dela Peña De Guzman, De La Salle University, University of the Philippines-Manila

Marielle Jessie Dela Peña De Guzman is a graduate of De La Salle University Senior High School and is an undergraduate student of the University of the Philippines Manila under the BA Behavioral Sciences program. Her interest lies in the study of psychology and music and the arts.

Micaela Isabel Molina Forrosuelo, De La Salle University

Micaela Isabel Molina Forrosuelo is a graduate of De La Salle University Senior High School and is currently taking up an undergraduate degree under the BS Political Science program at De La Salle University. Her interest lies in the broad field of the social sciences. However, she is also deeply intrigued by languages, a fascination sparked by her love for writing and copyreading, seamlessly blending her interests in both language and society.

Janeson Mirabuna Miranda, Department of Humanities and Social Sciences, De La Salle University

Janeson Mirabuna Miranda is a full-time De La Salle University Senior High School faculty member, handling Practical Research courses under the Department of Humanities and Social Sciences. He is also the managing editor of Sinaya, a pioneering scholarly journal for senior high school teachers and students. His research interests include the language of media, politics, and pop culture, critical pedagogy, and educational linguistics. 

References

Anderson, J., & Rainie, L. (2018, December 10). 3. Improvements ahead: How humans and AI might evolve together in the next decade. Pew Research Center. https://www.pewresearch.org/internet/2018/12/10/improvements-ahead-how-humans-and-ai-might-evolve-together-in-the-next-decade/

Ariyaratne, S., Iyengar, K. P., Nischal, N., Babu, N. C., & Botchu, R. (2023). A comparison of ChatGPT-generated articles with human-written articles. Skeletal Radiology, 52(9), 1755–1758. https://doi.org/10.1007/s00256-023-04340-5

Bawarshi, A. S., & Reiff, M. J. (2010). Genre: An introduction to history, theory, research, and pedagogy. Parlor Press ; WAC Clearinghouse. https://wac.colostate.edu/docs/books/bawarshi_reiff/genre.pdf

Biber, D., Connor, U., & Upton, T. (2007). Discourse on the move: Using corpus analysis to describe discourse structure. Language, 85(3), 694-696. https://www.jstor.org/stable/40492906

Brin, D., Sorin, V., Vaid, A., Soroush, A., Glicksberg, B. S., Charney, A. W., Nadkarni, G., & Klang, E. (2023). Comparing ChatGPT and GPT-4 performance in USMLE soft skill assessments. Scientific Reports, 13(1). https://doi.org/10.1038/s41598-023-43436-9

Casal, J. E., & Kessler, M. (2023). Rhetorical move-step analysis. Conducting Genre-Based Research in Applied Linguistics, 82–104. https://doi.org/10.4324/9781003300847-7

Dudley-Evans, T., (2000). Genre analysis: a key to a theory of ESP?. Ibérica, Revista de la Asociación Europea de Lenguas para Fines Específicos, (2), 3-11.

Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., . . . Wright, R. (2023). Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642

El-Zohiery, A. (2019). Create, campaign, and click: A genre analysis of the interactive features used in hashtag campaigns on Twitter. Cairo Studies in English, 2019(2), 223–257. https://doi.org/10.21608/cse.2019.66675

Field, H. (2023, November 6). Microsoft-backed OpenAI announces GPT-4 Turbo, its most powerful AI yet. CNBC. https://www.cnbc.com/2023/11/06/openai-announces-more-powerful-gpt-4-turbo-and-cuts-prices.html

Goff, P. (2023). ChatGPT can’t think – consciousness is something entirely different to today’s AI. The Conversation. https://theconversation.com/chatgpt-cant-think-consciousness-is-something-entirely-different-to-todays-ai-204823

Haleem, A., Javaid, M., & Singh, R. P. (2022). An era of CHATGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 2(4). https://doi.org/10.1016/j.tbench.2023.100089

Hassani, H., & Silva, E. S. (2023). The role of ChatGPT in data science: how AI-assisted conversational interfaces are revolutionizing the field. Big Data and Cognitive Computing, 7(2), 62. https://doi.org/10.3390/bdcc7020062

Herbold, S., Hautli-Janisz, A., Heuer, U., Kikteva, Z., & Trautsch, A. (2023). A large-scale comparison of human-written versus ChatGPT-generated essays. Scientific Reports, 13(1). https://doi.org/10.1038/s41598-023-45644-9

Hutson, J., & Schnellmann, A. (2023). The poetry of prompts: The collaborative role of generative artificial intelligence in the creation of poetry and the anxiety of machine influence. Global Journal of Computer Science and Technology, 1–14. https://doi.org/10.34257/gjcstdvol23is1pg1

Johansson, I. (2023). A Tale of Two Texts, a Robot, and Authorship : A Comparison Between a Human-Written and a ChatGPT-Generated Text. DIVA. https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1767661&dswid=3136

Kalla, D., Smith, N., Samaah, F., & Kuraku, S. (2023). Study and analysis of chat gpt and its impact on different fields of study. International Journal of Innovative Science and Research Technology, 8(3), IJISRT23MAR956. https://ssrn.com/abstract=4402499

Kelk, I. (2023, November 26). How ChatGPT fools us into thinking we’re having a conversation. Short Attention. https://kelk.ai/blog/how-chatgpt-fools-us

Krueger, K. L., & Forest, A. L. (2020). Communicating commitment: A relationship-protection account of dyadic displays on social media. Personality & Social Psychology Bulletin, 46(7), 1059–1073. https://doi.org/10.1177/0146167219893998

Lenhart, A. (2015, October 1). Chapter 4: Social media and romantic relationships. Pew Research Center. https://www.pewresearch.org/internet/2015/10/01/social-media-and-romantic-relationships/

Malik, A. (2023, November 6). OpenAI’s ChatGPT now has 100 million weekly active users. TechCrunch. https://techcrunch.com/2023/11/06/openais-chatgpt-now-has-100-million-weekly-active-users/?guccounter=1

McCoy, R. T., Smolensky, P., Linzen, T., Gao, J., & Celikyilmaz, A. (2021). How much do language models copy from their training data? evaluating linguistic novelty in text generation using Raven. Transactions of the Association for Computational Linguistics, 11, 652–670. https://doi.org/https://doi.org/10.48550/arxiv.2111.09509

Nicholes, J. (2017). Measuring ownership of creative versus academic writing: Implications for interdisciplinary praxis. NAWE. https://www.nawe.co.uk/DB/wip-editions/articles/measuring-ownership-of-creative-versus-academic-writing-implications-for-interdisciplinary-praxis.html

OpenAI. (2023, March 14). GPT-4. https://openai.com/index/gpt-4/

OpenAI. (2024). How can I access GPT-4? | OpenAI Help Center. https://help.openai.com/en/articles/7102672-how-aksld;jlncan-i-access-gpt-4

Ötügen, R., Takkaç, M., & Yağiz, O. (2021). ÖAİ’de tür analizi: Aşama analizi modelleri ve üstsöylem taksonomileri üzerine bir inceleme. E- Kafkas Eğitim Araştırmaları Dergisi, 8(1), 67–85. https://doi.org/10.30900/kafkasegt.877595

Proferes, N., Jones, N., Gilbert, S., Fiesler, C., & Zimmer, M. (2021). Studying Reddit: A systematic overview of disciplines, approaches, methods, and ethics. Social Media + Society, 7(2), 205630512110190. https://doi.org/10.1177/20563051211019004

Ray, P. P. (2023). ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-physical Systems, 3, 121–154. https://doi.org/10.1016/j.iotcps.2023.04.003

Reddit. (2022, December 8). Revealing this year’s Reddit recap, where we highlight how Redditors kept it real in 2022. https://www.redditinc.com/blog/reddit-recap-2022-global

Reddit. (2023a). Reddiquette. https://support.reddithelp.com/hc/en-us/articles/205926439-Reddiquette

Reddit. (2023b). Top Communities on Reddit - Page 4. https://www.reddit.com/best/communities/4/#t5_2reak

Signorelli, J. (2017). Of Pumpkin Spice Lattes, Hamplanets, And Fatspeak: The Venting Genre As Support And Subversion On Reddit’s R/Fatpeoplestories. Journal of Language Aggression and Conflict, 10(1), 64–84. https://doi.org/http://hdl.handle.net/20.500.13093/etd:1543

Skalicky, S. (2013). Was this analysis helpful? A genre analysis of the amazon.com discourse community and its “most helpful” product reviews. Discourse, Context & Media, 2(2), 84–93. https://doi.org/10.1016/j.dcm.2013.04.001

Su, J., & Yang, W. (2023). Unlocking the power of ChatGPT: A framework for applying generative AI in education. ECNU Review of Education, 6(3), 355–366. https://doi.org/10.1177/20965311231168423

Swales, J. M. (1990). Genre analysis: English in academic and research settings (1st ed.). Cambridge University Press.

Van Ouytsel, J., Van Gool, E., Walrave, M., Ponnet, K., & Peeters, E. (2016). Exploring the role of social networking sites within adolescent romantic relationships and dating experiences. Computers in Human Behavior, 55, 76–86. https://doi.org/10.1016/j.chb.2015.08.042

Vogels, E. A., & Anderson, M. (2020, May 8). Dating and relationships in the digital age. Pew Research Center. https://www.pewresearch.org/internet/2020/05/08/dating-and-relationships-in-the-digital-age/

White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D. C. (2023). A prompt pattern catalog to enhance prompt engineering with ChatGPT. Cornell University. https://doi.org/10.48550/arxiv.2302.11382

Xia, S. A. (2020). Genre analysis in the Digital Era: Developments and challenges. ESP Today, 8(1), 141–159. https://doi.org/10.18485/esptoday.2020.8.1.7

Xu, Y., Liu, X., Cao, X., Huang, C., Liu, E., Qian, S., Liu, X., Wu, Y., Dong, F., Qiu, C., Qiu, J., Hua, K., Su, W., Wu, J., Xu, H., Han, Y., Fu, C., Yin, Z., Liu, M., ... Zhang, J. (2021). Artificial intelligence: A powerful paradigm for scientific research. The Innovation, 2(4), 100179. https://doi.org/10.1016/j.xinn.2021.100179

Zaitsu, W., & Jin, M. (2023). Distinguishing ChatGPT(-3.5, -4)-generated and human-written papers through Japanese stylometric analysis. PloS One, 18(8), e0288453. https://doi.org/10.1371/journal.pone.0288453

Downloads

Additional Files

Published

2025-03-26

Issue

Vol. 31 No. 1 (2025): March 2025

Section

Articles

License

If the article is accepted for publication, copyright of this article will be vested to UKM Press, National University of Malaysia (UKM).


All articles published in this journal are licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license, unless otherwise stated.