View the "Introducing Dual-Anonymous Review for SOFIA Proposals" webinar recording here.
In Cycle 9, SOFIA proposals' review will be carried out under the Dual Anonymous Review (DAR) framework. When proposals are evaluated under DAR, not only are proposers unaware of the identity of the members on the review panel, but the reviewers do not have explicit knowledge of the identities of the proposing team during the scientific evaluation of the proposal. This requires that the scientific section of the proposal, as written by the proposer, be anonymized.
The DAR framework for proposal evaluation was pioneered by Hubble Space Telescope, and has now been adopted by several panels and committees within the NASA Science Mission Directory. It has been proven to mitigate possible unconscious biases which may affect proposal evaluation. The primary intent of dual-anonymous peer review is to eliminate "the team" as a topic during the scientific evaluation of a proposal, not to make it absolutely impossible to guess who might be on that team.
This NASA webpage presents the main principles behind DAR, and example of how to effectively anonymize a proposal. The recording of the SMD Town Hall on DAR can be watched here.
What does DAR mean in practice for SOFIA proposers:
For each proposal, the proposer will need to prepare two different pdf files: the proposal body ("Science PDF Attachment"), which must be anonymized by the proposer, and a separate “Team PDF Attachment” file including the biographical sketches. For Legacy Proposals, the “Team PDF Attachment” should also contains the implementation plan and budget. Each of these files should be uploaded by the proposer in USPOT. This is different from earlier cycles, where proposers need to upload a single pdf in USPOT.
To be considered compliant, the proposal body ("Science PDF Attachment") must comply with the following anonymizing guidelines:
- Exclude names and affiliations of the proposing team, including in figures and references to personal websites.
- Do not claim ownership of past work, e.g., "my previously funded work..." or "Our prior analysis demonstrates that...". Third person neutral wording should be used.
- Do not include the names of the personnel associated with the proposal or their organizational affiliations. This includes but is not limited to, page headers, footers, diagrams, figures, or watermarks.
- When citing references, use third person neutral wording. This especially applies to self-referencing. For example, replace phrases like "as we have shown in our previous work..." with "as previously shown in Author et al., ..."
- If proprietary information is included in the proposal, this may reveal (or strongly imply) the investigators on the proposal. In these instances, proposers must use language such "obtained in private communication" or "from private consultation" when referring to such potentially identifying work.
The fair and unbiased evaluation of proposals is of paramount importance to NASA and the SOFIA project. Proposals that are clearly judged to significantly not adhere to the anonymous format will therefore be returned without review. We do, however, understand that dual-anonymous peer review represents a significant shift in the writing of proposals, and as such there may be occasional slips in writing anonymized proposals.
In general, a good-faith effort at anonymizing the proposal is required and will be considered sufficient. The requirement is that the proposal be written in the anonymous format, not that the proposer or team must not be possible to guess by a person familiar with the field.
In addition, on August 3, 8-9 am PDT, the SOFIA Science Center hosted a webinar on Introducing Dual-Anonymous Review for SOFIA proposals, open to the astronomy community. The objective was to present the motivations and guidelines for the implementation of dual-anonymous reviews for SOFIA proposals, and to respond to possible questions and inquiries from users. With presentations from Lou Strolger (STSCI) and B-G Andersson (SOFIA/USRA). View the webinar recording here.
Frequently Asked Questions
Q: I am planning to show in my proposal data which is proprietary. Would that be considered identifying?
A: Proposers must use language such "obtained in private communication" or "from private consultation". Please consider that even if your group could be identified in principle from the presented data presented, such proprietary information does not identify who is leading the proposal. In addition, panelists are not necessarily aware of past proposals.
Q: My proposal is a resubmission of a proposal from an earlier cycle. Is it ok to reuse my former proposal as is, or would that be considered identifying?
A: The proposal from an earlier cycle must be separated into two distinct files: the "Science PDF Attachment" and the "Team PDF Attachment". The "Science PDF Attachment" must comply with the anonymizing guidelines stated above to be considered for review, and should contain no explicit mention of it being a renewal of an existing proposal.
Q: Is the goal to scrub any identifying information from the proposal completely?
A: No, this would be ideal but is not always achievable. The goal is to have a process where reviewers will not focus on the proposers, but on the science. A good-faith attempt to anonymize is acceptable, but blatant disregard will lead to proposal rejection.
Q: Can I put my own papers in the reference list?
A: Yes, but try not to “overload” the reference list with your own papers unless there is a specific reason to do so.
Q: Does the DAR framework apply to the parallel calls for the German community issued by the German SOFIA Institute (DSI; University of Stuttgart)?
A: No, it only applies to the calls issued on behalf of NASA by the Universities Space Research Association (USRA).
Q: Should I fill the 'Related Proposers' and 'Status of Observations' fields in USPOT? That information would identify me as the proposer.
A: The TAC connot see the content of these boxes when they are doing their science review, so they would not identify you to the TAC members. See this example of a proposal in 'TAC-view'. Please fill those boxes truthfully, as they may be used in later stages of the Time Allocation process.