From Helping Hand to Stumbling Block: The ChatGPT Paradox in Competency Experiment - Duk Gyoo Kim; Ahram Moon (Center for Economic Studies and ifo Institute (CESifo), 2024)
We ran a controlled laboratory experiment to examine whether ChatGPT's aid can increase the participants' performance in three different—reading and writing, mathematical problem-solving, and computational thinking—tasks. We find that the math score significantly decreases with ChatGPT's assistance. This result is mainly because the low-ability subjects couldn't discern the hallucinated answers with the correct ones, and it contests the general idea that ChatGPT can complement the workers with less expertise.
Keywords: C91; J24; O33; D83; 330; laboratory experiment; ChatGPT; labor productivity