Please use this identifier to cite or link to this item: https://hdl.handle.net/10419/289437 
Authors: 
Brodeur, Abel
Mikola, Derek
Cook, Nikolai
Brailey, Thomas
Briggs, Ryan
de Gendre, Alexandra
Dupraz, Yannick
Fiala, Lenka
Gabani, Jacopo
Gauriot, Romain
Haddad, Joanne
McWay, Ryan
Levin, Joel
Johannesson, Magnus
Metson, Lennard
Kinge, Jonas Minet
Tian, Wenjie
Wochner, Timo
Mishra, Sumit
Richardson, Joseph
Etingin-Frati, Giulian
Gugushvili, Alexi
Procházka, Jakub
Mohnen, Myra
Möller, Jakob
Montambeault, Rosalie
Montpetit, Sébastien
Collins, Jason
Ellingsrud, Sigmund
Kustov, Alexander
Morin, Louis-Philippe
Morris, Todd
Fleisje, Erlend
Fatemi-Pour, Elaheh
Moser, Scott
Woerman, Matt
Ölkers, Tim
Miguel, Edward
Motoki, Fabio
Kjelsrud, Anders
Muehlenbachs, Lucija
Musulan, Andreea
Czymara, Christian
Habibnia, Hooman
Coppock, Alexander
Tanrisever, Idil
Musumeci, Marco
Rivers, Nicholas
Oliver i Vert, Miquel
Oral, Emre
Abarca, Alejandro
Oswald, Christian
Ousman, Ali
Wroński, Marcin
Häffner, Sonja
Özak, Ömer
Toma, Filip-Mihai
Brown, Myriam
Marino Fages, Diego
Pandey, Shubham
Wang, Joseph Taoyi
Erhart, Andrea
Pavlov, Alexandre
Rodrigues, Bruno
Roelofs, William
Hall, Jonathan D.
Roemer, Tobias
Rogeberg, Ole
Rose, Julian
Fréchet, Nadjim
Totarelli, Maddalena
Roskos-Ewoldsen, Andrew
Hu, Shiang-Hung
Fortier-Chouinard, Alexandre
Rosmer, Paul
Galipeau, Thomas
Nüß, Patrick
Akhtar, Ahwaz
Kotsadam, Andreas
Sabada, Barbara
Lima, Goncalo
Brun, Martín
Saberian, Soodeh
Tran, Van-Anh
Salamanca, Nicolas
Sator, Georg
Rahman, Andaleeb
Hammar, Olle
Scates, Daniel
Yazbeck, Myra
Couillard, Ben
Schlüter, Elmar
Sells, Cameron
Bruns, Stephan
Sen, Sharmi
Korpershoek, Jori
Nchare, Karim
Sethi, Ritika
Qiu, Kangyu
Ramirez Lizardi, Eduardo Alberto
Federice, Alexa
Huang, Yu-Shiuan
Shcherbiak, Anna
Sogaolu, Moyosore
Soosalu, Matt
Buliskeria, Nino
Crechet, Jonathan
Yang, Gustav Chung
Sørensen, Erik Ø.
Kordt, Amund Hanson
Connolly, Marie
Sovani, Manali
Spencer, Noah
Staubli, Stefan
Krashinsky, Lewis
Huebener, Mathias
Stans, Renske
Phan, Linh
Stewart, Anya
Stips, Felix
Fenig, Guidon
Feld, Jan
Crippa, Lorenzo
Yap, Luther
Albrecht, Sabina
Penheiro, Romeo
Stockley, Kieran
Strobel, Stephenson
Struby, Ethan
Calef, Andrea
Huber, Christoph
Borbáth, Endre
Tang, John
Truong, Hung
Tsoy, Nikita
Firouzjaeiangalougah, Mojtaba
Tuzcuoglu, Kerem
Ubfal, Diego
Andlib, Zubaria
Daarstad, Haley
Villalobos, Laura
Jananji, Raphaël
Jetter, Michael
Elminejad, Ali
Lalayev, Nurlan
Caviglia-Harris, Jill
Walterskirchen, Julian
Yassin, Kareman
Ash, Thomas
Ye, Hao
Yoon, Jin Young
Yurris, Chris
Zahra, Tahreen
Liu, Dan
Cameron, Alistair
Zaneva, Mirela
Neubauer, Florian
Li, Peng
Engel, Julia Francesca
Huynh, Kim P.
Park, RyuGyung
Zayat, Aline
Duncan, Jamie
Zhang, Jonathan
Hashimoto, Barry
Zhao, Ziwei
Campa, Pamela
Zhong Yaolang
Bacher, Etienne
Wong, Jared
Bachler, Sebastian
Bacon, Félix
Dao, Danh Chi
Irsova, Zuzana
Bagues, Manuel
Hartley, Jonathan S.
Balogh, Timea
Pérez Martel, Eva
Kundu, Suranjana
Batmanov, Alisher
Fries, Tilman
Laufer, Jill
Nguyen, Quan M. P.
Barschkett, Mara
Basdil, B. Kaan
Baxa, Jaromír
Hausladen, Carina I.
Becker, Sascha
Isler, Ozan
Dao, Dong
Cordeau, Hugo
Beeder, Monica
Beland, Louis-Philippe
Bello, Abdel Hamid
Petrovičová, Tereza
Nguyen, Tuan
Markovits, Daniel Benenson
Weng, Wei-Chien
Benjamin, Grant
Bergeron, Thomas
Lee-Whiting, Blake
Fortier, Reid
Campos-Rodríguez, Santiago
Jakobsson, Niklas
Blimpo, Moussa P.
Wattal, Vasudha
Binetti, Marco
Bonander, Carl
Havránek, Tomáš
Nguyen-Tien, Viet
Bonneau, Joseph
Schmandt, Marco David
Cantone, Giulio Giacomo
Lenz, Gabriel
Carpena, Fenella
Carter, Perry
Ankel-Peters, Jörg
Castañeda Dower, Paul
Rea, Carson H.
Odermatt, Angela
Yang, Thomas Tao
Prettyman, Alexa
Holzknecht, Armando
Castek, Ondrej
de Linde, Astrid
Gallegos, Sebastián
De Melo, Lucas
Deer, Lachlan
De Vera, Micole
Dimitrova, Velichka
Kattan, Lamis
Dollbaum, Jan Fabian
Tastan, Ipek
Li, Tongzhe
Mierisch, Fabian
Vilhuber, Lars
Webb, Matthew D.
Dollbaum, Jan Matti
Kjellsson, Gustav
Donnelly, Michael
Huynh, Luu Duc Toan
He, Harry
Dumbalska, Tsvetomira
Gangji, Areez
Cui, Jeanne
Gao, Xiaoying
Garnache, Cloé
Gáspár, Attila
Niazi, Ali
Jayalath, Tharaka A.
Gavrilova, Evelina
Weber, Bryan
Acharya, Mahesh
Ghosh, Arijit
Lin, Yuchen
Putri, Aqila
Tatić, Dejan
Gibney, Garreth
Gibson, Grant
Dreber, Anna
Godager, Geir
Nikolaishvili, Giorgi
Goff, Leonard
White, Kimberly
Langlois, Audrée
Gong, Da
González, Javier
Weisser, Reinhard
Tatlow, Benjamin
Gretton, Jeremy
Griffa, Cristina
Grigoryeva, Idaliya
Quandt, Julian
Listo, Ariel
Hepplewhite, Matthew
Nordstrom, Ardyn
Grøtting, Maja
Guntermann, Eric
Chauca Strand, Gabriella
Guo, Jiaqi
Wright, Taylor
Campbell, Douglas
Nguyen, Loan Quynh Thi
John, Jenny
Herrera-Rodriguez, Mario
Heuer, Felix
Duong, Kiet Tuan
Heyes, Anthony
Westheide, Christian
Woodard, Ritchie
Topstad Borgen, Nicolai
Ho, Anson T. Y.
Nabin, Munirul
Holmes, Jonathan
Lu, Xuewen
Forshaw, Rachel Joy
Juan, Felipe
Kadriu, Valon
Karim, Sunny
Kelly, Edmund
Reiremo, Adam
Topstad Borgen, Solveig
Dang, Duy Khanh Hoang
Eissa, Yasmine
Tchuisseu Seuyong, Féraud
Chen, Shi
Pelli, Martino
Khushboo, Tazia
Sun, Pu
Kim, Jin
Lukmanova, Elvina
Luscombe, Alex
Borowsky, Jonathan
Lusher, Lester R.
Lyu, Ke
Ma, Hai
Andersen, Synøve Nygaard
Olson, Matt
Mäder, Nicolas
Chzhen, Asya
Adjisse, Sossou Simplice
Arrora, Falak
Makate, Clifton
Frith, Michael James
Malmberg, Alice
Duprey, Thibaut
Maitra, Adit
Hoces de la Guardia, Fernando
Thériault, Rémi
Mandas, Marco
Øien, Henning
Marcus, Jan
Margaryan, Shushanik
Chung, Jong
Brini, Elisa
Márk, Lili
Martignano, Andres
Winter, Jacob
Renée, Laëtitia
Marsh, Abigail
Thivierge, Vincent
Masetto, Isabella
McCanny, Anthony
Hsu, Yu-Hsiang Dexter
Dworschak, Christoph
Leibing, Andreas
McManus, Emma
Year of Publication: 
2024
Series/Report no.: 
I4R Discussion Paper Series No. 107
Publisher: 
Institute for Replication (I4R), s.l.
Abstract: 
This study pushes our understanding of research reliability by reproducing and replicating claims from 110 papers in leading economic and political science journals. The analysis involves computational reproducibility checks and robustness assessments. It reveals several patterns. First, we uncover a high rate of fully computationally reproducible results (over 85%). Second, excluding minor issues like missing packages or broken pathways, we uncover coding errors for about 25% of studies, with some studies containing multiple errors. Third, we test the robustness of the results to 5,511 re-analyses. We find a robustness reproducibility of about 70%. Robustness reproducibility rates are relatively higher for re-analyses that introduce new data and lower for re-analyses that change the sample or the definition of the dependent variable. Fourth, 52% of re-analysis effect size estimates are smaller than the original published estimates and the average statistical significance of a re-analysis is 77% of the original. Lastly, we rely on six teams of researchers working independently to answer eight additional research questions on the determinants of robustness reproducibility. Most teams find a negative relationship between replicators' experience and reproducibility, while finding no relationship between reproducibility and the provision of intermediate or even raw data combined with the necessary cleaning codes.
Subjects: 
Reproduction
Replication
Research Transparency
Open Science
Economics
Political Science
JEL: 
B41
C10
C81
Document Type: 
Working Paper

Files in This Item:
File
Size





Items in EconStor are protected by copyright, with all rights reserved, unless otherwise indicated.