fivethirtyeight article about the reproducibility of published results
Main points:
- Having scientists bet of whether they think a study is reproducible has shown success at finding which studies have results that are less likely to be reproduced
- “If you believe the result will be replicated, you buy the contract, which increases the price. If you don’t believe in a study, then you can short-sell it.”
- “The prediction market correctly called nearly three-quarters (71 percent) of the attempted replications”
- Possible solution: “For example, a journal might decide to use markets to vet studies before publication. If the market says, ‘Yeah, that’s cute, but that’s probably bogus,’ then that’s probably not something you should publish. Since it’s too costly and complicated to replicate every study, another approach might be to enter 100 studies into a prediction market and then select 10 at random to replicate. Presumably, the threat of a replication would create an incentive for researchers to be more careful.”