Showing posts with label review process. Show all posts
Showing posts with label review process. Show all posts

Sunday, June 28, 2009

Grant system good at ruling out bad things?

Grant System Leads Cancer Researchers to Play It Safe, by Gina Kolata, The New York Times, June 27, 2009. [Page 1][Page 2][Page 3][FriendFeed entry].

Excerpts from Page 1:
Yet the fight against cancer is going slower than most had hoped, with only small changes in the death rate in the almost 40 years since it [the "war on cancer" initiated by President Nixon in 1971] began.
One major impediment, scientists agree, is the grant system itself. It has become a sort of jobs program, a way to keep research laboratories going year after year with the understanding that the focus will be on small projects unlikely to take significant steps toward curing cancer.
.....
Even top federal cancer officials say the system needs to be changed.
“We have a system that works over all pretty well, and is very good at ruling out bad things — we don’t fund bad research,” said Dr. Raynard S. Kington, acting director of the National Institutes of Health, which includes the cancer institute. “But given that, we also recognize that the system probably provides disincentives to funding really transformative research.”
Excerpt from Page 2:
“They said I don’t have preliminary results,” she said. “Of course I don’t. I need the grant money to get them.”
Excerpt from Page 3:
Some experienced scientists have found a way to offset the problem somewhat. They do chancy experiments by siphoning money from their grants.
Comment: The focus of the article is on the grant funding system for cancer research in the USA. The author, a well-known science journalist, is pessimistic about the success that the current funding system has had in yielding research outputs that have led to any substantial decrease in cancer mortality rates. However, other than briefly mentioning overall cancer mortality rates, she does not attempt to analyze current approaches to cancer control.

In Canada, age-standardized mortality rates, for all cancers and all age groups, have decreased from 248/100,000 in 1984 to 212/100,000 in 2004 (about 15%) for males. In contrast, the corresponding mortality rates for Canadian females were 152/100,000 in 1984 and 147/100,000 in 2004 (a decrease of only about 3%). A detailed analysis is beyond the scope of this brief commentary, but a major reason is that age-standardized mortality rates for respiratory cancers have been higher in males and have been decreasing, while they have been lower in females, and have been increasing.

It has been estimated that, in the USA, "reductions in lung cancer, resulting from reductions in tobacco smoking over the last half century, account for about 40% of the decrease in overall male cancer death rates" (Tobacco Control 2006; 15: 345-347; doi:10.1136/tc.2006.017749). Strong evidence that tobacco smoking and lung cancer rates are related has been available for more than 50 years, since the research work of Richard Doll and Austin Bradford Hill.

We now know a great deal about success stories and best practices for effective, evidence-based tobacco control programs. (See, for example, Success stories and lessons learnt, Tobacco Free Initiative (TFI), World Health Organization).

So, does research play a crucial role in cancer control? Of course it does.

Can it take a very long time for research outputs to have a substantial impact on cancer control? Unfortunately, it can.

Do we have good ways to identify, in advance, areas of transformative research? Unfortunately, no. It can even take a long time to demonstrate that certain research has, indeed, been transformative.

So, what to do? My answer: investment in research is much like investment of venture capital. Only a very small minority of investments yield a big payoff, but one can predict much more easily which investments are likely do badly than which ones are likely to do well.

Monday, December 29, 2008

Criteria for review of pre-applications to CIRM

Pre-Application Review Process for Basic Biology RFA, CIRM, Agenda Item # 4, ICOC Teleconference Meeting, December 23, 2008 [PDF]. Excerpts:
The ICOC, at its most recent meeting, considered a proposal from CIRM staff to introduce a pre-application review process, on a trial basis, for use in the upcoming Basic Biology I & II and Disease Team RFAs.
.....
Proposal: Criteria for Review of Pre-Applications

Pre-application review will focus on the essential aspects of the scientific proposal, without requiring Principal Investigators (PIs) to finalize or describe detailed research plans, budgets, collaborations or personnel. This review will consider a subset of the standard GWG criteria, tailored to the level of information we will seek in the preapplication:
1. Impact and Significance. Whether and to what extent the proposed research: addresses an important problem; significantly moves the field forward, either scientifically or medically; moves the research closer to therapy; and changes the thinking or experimental or medical practice in the field.

2. Innovation. Whether and to what extent the research approach is original, breaks new ground, and brings novel ideas, technologies or strategies to bear on an important problem.

3. Feasibility. Whether and to what extent the aims of the research can be reasonably achieved in the specified timeframe of the award.

4. Responsiveness to RFA. Whether and to what extent the proposed research project or activity adequately and appropriately addresses the goals and objectives presented in the RFA.
CIRM science staff and outside scientific specialists will apply these standard criteria. As with the criteria for GWG review of full applications, science officers will prepare guidelines that describe how these criteria apply to the concept for a specific RFA. These guidelines will be published as part of the RFA, so that PIs can address them in their preapplications.
Note that these criteria will be applied to the Disease Team RFAs.

See also: Stem cell board fails to muster quorum, skips cash request, John M Simpson, Consumer Watchdog, December 24, 2008. Excerpt:
The stem cell agency's staff had to report on the criteria they'll use in the pre-review process. They did and it's outlined here.
For a relevant previous post, see: CIRM/CSCC Joint Announcement: Disease Teams Awards, Cancer Stem Cell News, October 23, 2008.