Pre-specifying the details of a research design before data collection and analysis is central to the RRs publishing format, pre-analysis plans (PAPs), and study pre-registration. What distinguishes the RR format from the other two is that it also incorporates peer-review of the pre-specified research design at Stage 1. Following data collection and analysis, Stage 2 of peer review then ensures that research projects were implemented in alignment with their pre-specified research design, and the results of all pre-specified analyses are clearly reported.

Following acceptance based on pre-results review authors are free to first submit the full-length paper to other journals. This will not compromise the Stage 2 review of the paper if it is subsequently submitted to the JDE.

Note that the Stage 1 proposal or the Stage 2 full paper must not be submitted to another journal while the respective manuscript is under review at the JDE. The Editorial Board therefore asks that corresponding authors notify the JDE if they decide to submit the full paper to another journal. Should the paper be rejected by the other journal, the JDE will observe the pre-results acceptance for up to five years after it was granted.

If the full paper is accepted at another journal, the corresponding author should notify the JDE, and a brief note on the outcome will be published alongside the proposal’s summary on the JDE RRs page on the BITSS website (e.g., “The proposal <title> by <authors> was accepted by the Journal of Development Economics based on pre-results review basis. The full paper based on this proposal was eventually accepted and published as <title> <citation>,<DOI>). The JDE Editorial Board also asks that working papers and published papers acknowledge that the article underwent peer review and was accepted based on pre-results basis at the JDE, citing the proposal summary on the JDE RRs page on the BITSS website.

The JDE does not require, but strongly encourages authors to pre-register their Stage 1 Proposals before the start of data collection. Authors can use the information from their Stage 1 Proposal submission to easily pre-register their study.

For more information, please consult the ‘Pre-registration Resources’ section in the JDE RR Author Guidelines.

The RR format does not restrict authors’ ability to conduct post hoc, exploratory analyses and report them in the full-length article submitted at Stage 2. However, such analyses must be clearly labeled as exploratory and distinguished from the confirmatory analyses that were specified prior to data collection and analyses.

Stage 1 Proposals will not be published as stand-alone items in the JDE following acceptance based on pre-results review. BITSS will post the title and abstract of Stage 1 Proposals that were accepted based on pre-results review in the “Prospective Articles” page on the BITSS website, unless authors choose not to have them posted.

Note that authors are required to submit the accepted Stage 1 Proposal as supporting material to their Stage 2 submissions. Stage 1 Proposals will be included in the appendix of the published article.

Yes. Authors should communicate such concerns to the Editors and BITSS, and point out the details that need to be masked.

During Stage 1 peer review and before data is collected, reviewers will assess whether authors have pre-specified sufficient data quality checks for accuracy, consistency, bias, and completeness. At Stage 2, reviewers will verify that pre-specified data quality checks have been met, and that data is of sufficient quality to test the pre-specified hypotheses.

In addition to pre-specified data quality checks, as part of the Stage 1 RR submission, authors are advised to consider potential challenges that may arise during implementation (e.g., attrition, non-compliance with treatment) and offer strategies to address them.

No, this format is not limited to any particular study design, but may be particularly well suited for RCTs. Any study design may be eligible if the data are yet to be collected/accessed and analyzed at the time of submission. This also includes quasi-experimental and observational studies (e.g., Neumark (2001), Burlig (2018), and the 2016 Election Research Pre-acceptance Competition).

As of January 1, 2019, submissions where follow-up data have been collected (outside of pilot and baseline data) are no longer eligible for this track.

Work should be submitted at a minimum of three months, but ideally six months to a year, before data collection of the key outcome data. All participants, authors, referees and editors, must be blind to the key outcome variables at each stage of revision of the Stage 1 submission. This is useful for two reasons: (a) to maximize the possibility that feedback can be used in shaping final data collection, and (b) to ensure enough time for an R&R round and re-submission before the final data come in.

(Note: it is typically quite useful to have baseline data in hand when submitting the Stage 1 Proposal for pre-results review: baseline data provides useful information on the nature of measures, including their means and variances, which can help to understand the context and carry out meaningful statistical power calculations.)

No, however authors may include pilot data in their Stage 1 submissions to establish proof of concept, effect size estimations, and/or feasibility of proposed methods.

In this approach, researchers use a training sample to run preliminary analyses that inform the Stage 1 Proposal and then refrain from analyzing the rest of the sample. After the Proposal is accepted, researchers use the remainder of the sample (testing sample) to test the pre-specified hypotheses, which constitute the main results of the published study.

Yes, this type of study design is eligible. However, at the time of the Stage 1 submission, authors will need to verifiably demonstrate they have not seen the testing part of the sample before the submission date (e.g., a download receipt plus a signed statement affirming this was the first download of the data).

PAPs and/or pre-registrations can serve as a helpful base for your Stage 1 submission. Stage 1 Proposals go a step further and require that authors outline the importance the research question in terms of its contribution or value added to the development economics literature, as in a regular full-length submission (see criterion 1 of the Stage 1 review criteria here). Stage 1 Proposals are expected to mirror full-length conventional articles, with the exception of the ‘Results’ and ‘Discussion’ sections. Authors may refer to the optional Stage 1 RR Submission Template for more guidance.

Stage 1 Proposal submissions should follow the same structure and formatting rules as standard JDE article submissions, however need to provide enough details to allow for the project to be evaluated in the light of Stage 1 assessment criteria (learn more in the JDE RR Author Guidelines). For items of the Stage 1 Proposal that are not commonly found in JDE papers (e.g. ex-ante plans for data processing, variations from intended sample size, pilot data, etc.), authors can include brief summaries in the body of the paper, and include the rest as an appendix to the Stage 1 Proposal submission.

In studies where parts of an analysis depend on the results of another, authors may consider constructing a decision tree as part of their Stage 1 Proposal (e.g., “If A is observed, then we will adopt analysis A1 but if B is observed then we will adopt analysis B1”). Alternatively, a single primary outcome (or a narrow set of outcomes) can be pre-specified at Stage 1, and the remainder of the analyses may be conducted as exploratory.

Yes, however exceptions may be granted on a case-to-case basis. The JDE will not publish the resulting full-length article unless it has successfully obtained IRB approval.

Publications in this track will look like any other article published in the JDE, with two exceptions: i) the main text will contain a footnote stating that the paper was accepted based on pre-results review, and ii) the accepted Stage 1 Proposal will be included in the supplementary online appendix.

Here are some additional tips to consider:

• Authors must report deviations from the pre-specified research design. These can be addressed throughout the body of the Stage 2 full-length article, or in a specific “deviations from the from pre-specified research design” section of the paper, along with details on the context. If particularly lengthy, they may be included in the appendix.
• Authors are expected to report the results of all hypotheses from the Stage 1 Proposal in the Stage 2 Article, unless a hypothesis is proven to be logically flawed or unfounded. In such cases, it can be included in the appendix with a description and/or justification.
• Analyses that were not part of the Stage 1 submission should be appropriately labeled (e.g. as exploratory, additional, post-hoc, etc.), and included in a separate section.

Once a Stage 1 Proposal is accepted based on a pre-results review, authors will receive an official notice from the Editors.

Note that due to functionality limitations of our manuscript handling platform, this will be paired with an auto-generated “revise and resubmit” notification, though this is not the actual decision. To submit a full paper for Stage 2 review, authors should  submit it by responding to the “revise and resubmit” notification, and choosing the article type from Registered Report Stage 1: Proposal to Registered Report Stage 2: Full Length Article.

No. The results of all hypotheses pre-specified at Stage 1 must be included in the full manuscript submitted at Stage 2. In instances where a pre-specified hypothesis is subsequently shown to be logically flawed or unfounded, authors may include it in an Appendix (if particularly lengthy) or as a footnote, rather than in the ‘Results’ section.

Yes, as long as such analyses are clearly caveated (e.g. reported in a separate section) from the analyses based on the Proposal that was accepted by the JDE at Stage 1.

This may be relevant in instances where the author(s) submit a Stage 1 Proposal based on a pre-analysis plan (PAP) for a project in collaboration with an external partner (e.g., government, funding agency) that asks for particular empirical analyses (e.g., estimates over sub-sample A, given primary outcome(s), etc.), however underwent revision during Stage 1 of peer review. If partners require the original PAP as is, then those analyses can be presented in the final paper, but in addition to the analyses accepted by the JDE at Stage 1.

Yes. Such articles will be considered in the standard review process.

During Stage 1, submissions will be either 1) rejected; 2) returned to authors for the opportunity to revise and resubmit; or 3) accepted based on pre-results review. Pre-results acceptance constitutes a commitment by the JDE to later publish the resulting full paper regardless of the nature of its empirical findings, subject to a successful Stage 2 review.

Information about the duration of peer review for full-length papers in the standard peer review track at the JDE is available here. Given the novelty of the RR track, the JDE does not have information on the duration of the two-stage peer review process yet, however the Editorial Board will make an effort to streamline the process as much as possible.

Once offered acceptance based on pre-results review, authors can include the article as an upcoming publication in their research portfolios with the label: “Accepted at the Journal of Development Economics based on pre-results review”, and the URL of the proposal summary in the “Prospective Articles” page on the BITSS website.

We do not currently have an expiration date for this pilot. However, the JDE will honor its commitment to publish papers that successfully underwent Stage 1 review, regardless of whether RRs are retained as a publishing model at the time of publication.