» Submit your entry for the MZES-GESIS Pre-Registration Challenge  [submissions closed]

Pre-registered studies define hypotheses and research strategies before results are known, thereby maintaining the distinction between postdiction and prediction, which is appreciated conceptually but rarely respected in practice (Nosek et al. 2017). To promote innovative ideas for social science studies using best open science practices, the MZES-GESIS Pre-Registration Challenge rewards the most promising pre-analysis plan with up to 2 x 5 minutes (i.e., 5 minutes in 2 waves) survey time free of charge in the probability-based GESIS Panel or with €2,000 in financial support for data collection.

Pre-registered research aims at avoiding overconfidence in post-hoc explanations of research findings and circumventing researchers’ temptation to use degrees of freedom to generate novel, surprising, and clean results with given data. Hence, the MZES-GESIS Pre-Registration Challenge helps raise awareness for pre-registered analysis plans as an institutional innovation that may help overcome the inflation of false-positive results in published social science research.

Timeline

Details on the submission process

How should my extended abstract submission look like?

Your extended abstract should explain the substantive contribution you plan to make with your research. It should provide the theoretical rationale for your hypotheses, your research design, and your approach to data analysis.
A detailed pre-registration plan is not necessary at the point of initial submission.
When submitting your extended abstract, you will be asked to indicate which rewards (survey data collection GESIS Panel or financial support for data collection) you would like to receive in case your study plan is selected as the best.

How should my full paper submission look like?

Among all submitted abstracts, we will invite three (teams of) authors to submit a full paper submission. These authors will have the opportunity to present their papers (before data collection) at the conference.

The MZES-GESIS Pre-Registration challenge follows the workflow of a registered report (see steps above). Your submission will be a regular full paper, complete with introduction, theory, hypotheses and a methods section, all drafted before data collection. Participants may consult the Nature Human Behavior Submission Guidelines for guidance on which elements should be included in their paper.

A detailed pre-registered analysis plan must specify how each hypothesis will be tested. OSF offers standardized forms to compose pre-registered analysis plans and we encourage participants to use OSF’s or other templates (e.g., as recommended by BITSS here) when drafting the plan and including or referencing the plan in the submitted paper. For experimental designs, a priori power analyses are mandatory. When referring to prior research in estimating potential effect sizes, keep in mind that effect sizes in published literatures tend to overestimate them (see Anderson, Kelley, Maxwell 2017 for strategies to handle this).

How do I submit?

You can submit your extended abstract here [submissions closed].

Details on drafting the paper and the pre-registration plan

Which typical mistakes should I avoid in my pre-registration plan?

Our goal for the conference will be to work together to improve each other’s research. However, typical mistakes in pre-registration plans are listed here and here (mind the focus here on experimental designs in psychology).

I am still unsure what my pre-registration should look like. Where can I find examples?

Participants can find examples of registered reports (pre-registered studies that were reviewed before data collection) here and best-practice examples of pre-registrations here. At EGAP, you can find pre-registration plans with a focus on the social sciences.

Details on the awards for the best pre-registration plan

Do I have to choose between financial support for data collection or fielding survey questions with the GESIS Panel?

Yes.

Is publication guaranteed?

No. We are in the process of negotiating the publication of special issues related to the conference in well-known social scientific journals but no final agreements have been reached at this point. If the special issue will be published as planned, the winner of the pre-registration challenge is invited to publish the winning study in the special issue, but each invited submission to the special issue will undergo separate peer review.

Do I have to publish the study in the special issue?

No. However, during the submission process, we ask you to indicate whether you plan to publish in the special issue or elsewhere.

What is the GESIS Panel?

The GESIS Panel offers the social science community a unique opportunity to collect survey data within a probability-based mixed-mode access panel. The GESIS Panel data are representative of the German-speaking population aged between 18 and 70 years (at the time of recruitment) and permanently resident in Germany. The GESIS Panel comprises about 4,700 panelists (November 2017).

Are there limitations to the topic, design, or methods of the planned study to be sponsored?

No, your planned study must be empirical and on a social science topic, but otherwise no restrictions exist. It can be experimental or observational, cross-sectional or longitudinal, as long as your approach fits your research question. Please note that the GESIS Panel only allows experimental designs with a maximum of four experimental conditions.

Participants interested in fielding their questions in the GESIS Panel are advised to familiarize themselves with the Regular Proposal Submission procedure and the GESIS Panel Survey Style Guide prior to submitting their extended abstract.

How will the best pre-registration plan be selected?

The best paper will be selected by the organizers of the MZES-GESIS Pre-Registration challenge. The criteria specified in the Nature Human Behavior reviewer guidelines (Stage 1 Submissions, p. 4) will guide the evaluation of submissions.

Open Science practices and other obligations

Can I modify the pre-registration plan after the conference and before data collection?

Yes, we encourage you to use the feedback you will receive at the conference and elsewhere to improve the initial draft of your pre-registration plan.

Can I modify the pre-registration plan after data collection and before publication?

Beyond adding results and discussion sections, the substantive content (in particular theory, hypotheses, and strategy for analysis) must remain unchanged and all findings on the hypotheses specified in the pre-registered analysis plan must be reported. However, additional exploratory analyses may be conducted and reported as such. This allows readers to distinguish confirmatory from exploratory findings.

Which obligations do I have?

The MZES-GESIS Pre-Registration Challenge’s goal is to showcase and promote best practices for transparent and reproducible research that is theory-driven and allows readers to separate predictive and postdictive findings. Hence, the winner of the pre-registration challenge pledges to abide by the principles of open science research and to adhere to the guidelines in this document. Among other practices, open science research includes making one’s data and reproduction material available in an open repository. The authors will report findings on all hypotheses specified in the pre-analysis plan.

By submitting their abstract, the authors also agree to credit MZES and GESIS as supporting institutions of their research in the published article.

Please contact Alexander Wuttke with any inquiries you may have regarding the OSSC19 MZES-GESIS Pre-Registration Challenge.

» Submit your entry for the MZES-GESIS Pre-Registration Challenge  [submissions closed]