3 Comments
The confidential nature of the NIH peer review process requires that any information pertaining to an application or proposal cannot be disclosed, transmitted, or discussed with any other individual. Therefore, technologies where confidential information might be viewed by others are prohibited, including locally hosted AI technologies where uploaded information could be shared across multiple individuals.
For more information, see our recent blog post, guide notice, and full FAQ set.
Does this policy apply uniformly to all online queries relating to reviewing? For example, suppose an application proposes to study BTK’s role in cancer. Can a reviewer post the following query to Google: “What role does Bruton’s tyrosine kinase play in oncogenic signaling”? How about the following query to PubMed: “BTK AND oncogenesis”? If the answer is yes to the above two examples, it would seem a bit arbitrary to ban posting the same queries to Perplexity or another AI.
No, the policy does not apply to all online queries. A standard web search, or search of online literature databases or other tools, differ from using generative AI in that web and database searches don’t use inputs for potential fodder for future outputs. Search engines simply match inputs to existing resources so do not involve confidential information leaving the reviewer’s control.
There are at least two points to this complex discussion.
First, if you examine my suggested query, there is almost no possible way that asking this query would leak any private information. Other than perhaps that one was reviewing a grant related to BTK and cancer, if the query could be connected to the name of a specific user. However, that particular risk has nothing to do with the type of query engine (e.g., one that uses AI or not). The danger of privacy leak only occurs if the queries are more complex and contain private information.
Second, many search engines do store records of queries. For example, Google keeps records of all queries — that can themselves be queried. It is not just a risk that the question might be used to train the algorithm, but also the greater risk that the question is archived and might become available to see by an evildoer. PubMed is probably safe, assuming NCBI does not archive records of queries — but I would imagine it does, if for no other reason than troubleshooting. Such records could potentially be leaked or hacked.
I fail to see anything _unique_ to AI. And I am not sure there is a Boolean distinction between “standard” and “AI” web searches. For example, after reading the Wikipedia page on Artificial Intelligence, I am not sure that a Google search or a PubMed search cannot be considered AI. As far as I know, the Google algorithm is not publicly available, so the issue is difficult to litigate or come to clear conclusions.
Ethics are very hard to instrumentalize into Booleans. One must consider each case based on all factors associated with the case.