Does It Really Work? Perception of Reliability of ChatGPT in Daily Use
DOI:
https://doi.org/10.13136/isr.v14i10S.731Abstract
How do individuals discriminate between what is human-made and what is produced by Artificial Intelligence (AI)? Despite OpenAI’s mission to ensure that AI benefits humanity, their cutting-edge technology, namely ChatGPT, an AI that aims to reproduce natural human language, raises several questions about its widespread use.
This contribution aims to answer the following Research Questions: RQ1 - Are users with no specific knowledge in the field of AI able to distinguish between text produced by ChatGPT or similar language models and text produced by humans? RQ2 - Is there a significant correlation between attribution of text to AI (or human) and specific opinions and attitudes?
This exploratory survey does not intend to generalise the results but to identify possible opinions and attitudes that might have influenced how the participants responded. One hundred people participated in the experiment, which consisted of a survey on their knowledge and perception of ChatGPT and a two-shot Turing Test. They were asked to read various short paragraphs and try to recognise which were written by humans and which were generated by AI.
The results showed that the group analysed experienced severe difficulties in recognising whether a sentence was written by an AI or a human being, that certain perceptual biases interfere with the attribution of a trivially false text, and that the attribution error can be reduced through experience and learning. Although in need of further investigation, these findings can help lay the groundwork for the effects of the interaction between humans and AIs from a social science and computer science perspective.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Fiorenza Beluzzi, Viviana Condorelli, Giovanni Giuffrida
![Creative Commons License](http://i.creativecommons.org/l/by/3.0/88x31.png)
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
(APC) Article and submissions processing charges
ISR does not ask for articles and submissions processing charges APC
Authors who publish in this journal agree to the following points:
- Authors retain the rights to their work and give to the journal the right of first publication of the work, simultaneously licensed under a Creative Commons License. This attribution allows others to share the work, indicating the authorship and initial publication in this journal.
- The authors may enter into other agreements with non-exclusive license to distribute the published version of the work (eg. deposit it in an institutional archive or publish it in a monograph), provided to indicate that the document was first published in this journal.
- Authors can distribute their work online (eg. on their website) only after the article is published (See The Effect of Open Access).