Readers, please note: In the interest of providing information on this subject, we are posting the raw transcripts from the FCRC CRA-W workshop. This is an unedited transcript, but still should provide you with background information of use. The edited transcripts will be completed by spring 2000.


Panel: Getting Funded
5/1/99
9:00 a.m. - 10:30 a.m.

Caroline Wardle: ... I'm going to give you just some basics of things to think about with funding that are good for all agencies, but then there will be some specific to NSF, and then Jim and Frederica will talk to you about the differences with the other agencies and each of us, if we have time, will just mention something probably at the end about a favorite program of ours or something we'd like you to know about a new initiative that is starting at one of the agencies.

So, let's talk about general things to think about with proposal preparation. By the way, I'm the Deputy Division Director at the Division of Experimental and Integrative Activities at NSF. This is in the direct root of computer and information science and engineering. So, let's look at NSF. Some very basic things about proposal preparation. The important thing to note about NSF is that it uses peer review. That's extremely important for you to know how your proposals are going to be evaluated. Peer review merely means that we bring in -- well, that your proposals will be reviewed by at least three technical peers. They will be drawn from universities, from government labs, from industrial research labs, too. There's two ways of doing it. There's panel review where we bring a group of experts to Washington and we give them 10, perhaps 12, proposals and they will evaluate them all on one day or on two days. So they will be comparing them. There is mail review which we use sometimes, particularly if we have a very specialized topic. So we would like to get some additional reviews but it's so specialized so we literally send it out by mail and then the reviewer will mail it back to us and we add these to the other reviews. So it's very important I come back to that point that it's peer review at NSF.

So these look very kind of reasonable but it's very important. Read the program announcement. There is a program announcement for everything. If there are particular things in the program announcement, make sure that you address those points. Very often in NSF announcement you'll have the general description, then you'll say, in addition to the regular NSF review you will also be evaluated on, and it might give you six bullet points. Make sure you address each one of those bullet points. One, for example, might be if you're in a group proposal it might say, you must have an evaluation in your proposal. If you miss out an evaluation plan, I guarantee the reviewers will jump on that immediately and it will pull down your proposal. So you don't want to have any red flags, you don't want to have things that will pull your proposal down.

Submission deadlines are absolute. At NSF if it says a deadline is 5 p.m. by Monday, then you must have your proposal in by 5 p.m. We do advise that you send it overnight express because if it gets delayed through their fault, it can be tracked down and then we probably would allow it to come in but otherwise we won't even see it. It's turned back at the central processing unit. So it's a really firm deadline at NSF when it says deadline. But where a target date is not. The target date, which we don't use many of now, says that, well, if you get it in at this point then we'll guarantee to get it reviewed in a very timely manner and it will get reviewed along with everybody else. If you come pass the target date, it will get very delayed. It won't get in until the next review cycle.

NSF is trying to get feedback to you within six months of your submitting your proposal. Therefore, it's pretty well moved to panel review because it's much faster. So most of ours are deadlines.

This is really important. The last two. PIs, you should talk with your funding agency. We like to talk to you. Come and talk to us about where our priorities are. We can give you advice on how to write your proposal. Do come and talk to us if you don't know exactly who, but you know somebody, ask them, who should I talk to, send them an e-mail, a little paragraph of your research area and they will point you in the right direction. And if you are new to writing proposals, as many of you will be, get an experienced professor at your institution. Ask them if they would read through your proposal and critique it for you. And don't give it to them two days before you have to submit. Give it to them two weeks before. That will give you time to think about the comments and perhaps make some modifications to your proposal.

Now, this one I'm only going to put up for two minutes because, again, it's very basis but when I'm a reviewer I will say, what are you going to do, how are you going to do it, why is it important, is it feasible, what is your unique contribution? You know, why are you the best person to be doing this and, very important, what are others doing in this area? Many program directors have reported reviewers coming and saying, you know, this looked as though it was a good proposal, but they didn't give me any indication that they knew what was going on in this area. There was no literature search, there was no description of related work. That's very important.

Let me give you some proposal presentation hints. Again, you might think these are obvious but I can assure you having read many proposals, they're not. Present your ideas clearly and succinctly. Don't forget you're scientists, we're scientists, the reviewers are scientists. They are used to reading the scientific style. Nothing irritates me more than a proposal that has grammatical errors and spelling mistakes in it. Please, if you tend to make spelling mistakes, put it through a spell checker and ask a friend or a colleague to read it for you.

So provide adequate technical explanation. You will have, in your reviewers you will have some reviewers who are experts in your area. You will have other reviewers that are experts in related areas and have a broad general knowledge but may not know every little thing about your area. Make sure, just as you would in a scientific paper, that you provide some explanation as to what you are doing.

Budgetary matters. All universities have research offices, either an office or a person. These people know the various ins and outs of the agencies and they're all different. They will help you put the budget together. Do feel free to use those. They will be very helpful. And this last comment I've seen many times, excessive budgets really irritate reviewers. You do not want to irritate reviewers. I'll give you an example. I have seen proposals that ask for equipment and they come in asking for a big graphics work engine and six work stations. And you look through the proposal and you find one professor and two graduate students. So you think, how could these three people use six work stations. And there's no justification in the proposal for that. So that's very irritating. That's what we call padding the budget. So it comes out straight away but you don't want to irritate the reviewers. Come in with a good, tight budget. Justify each part of the budget. Justify release time or justify summer money, justify the equipment, and that will come across as a very nice budget and proposal. And this is very important. You're selling to the reviewers. In NSF, we're peer review driven. If you don't get past the first hurdle of reviewers, you are not going to get funded on that proposal. So remember you are not sending it to your colleagues who perhaps know your area very well, you are writing your proposal for the reviewers.

I thought you'd just like to see some statistics with NSF to give you an idea of how competitive it is. We had just over 1,700 proposals last year of which just over 500 were awarded. So our funding rate, what we used to call success rate, is just the number of awards divided by the number of proposals times 100, that's 30%. So we fund 30%. That's gone up a little bit. The NSF averages the same and it used to be about 28% so it's going up. The average annual budget, as you can see, is just over $100,000 and that time -- you know, most proposals now last between two or three years so you're getting roughly $100,000 every year for two to three years.

So another way, of course, though of looking at this funding success rate of 30% means that your failure rate is 70%. So what, I think to point out, it's very important, do not give up if you do not make it the first time. It's unusual for somebody to be funded the first time. You really have to learn how to write proposals. As you can see, if 70% of the proposals are turned down -- we use the word, declined -- I mean, then it's a really competitive situation. There are budgetary limitations that you have no control over. I mean this year my program has received a budget cut. I couldn't control that and nor can you but it means I will not be able to make as many awards as I had hoped. We try to make as many as we can but there are budgetary controls. So very often when, you know, we say I am going to fund you but I can't fund you at the level at which you ask and I will try to negotiate perhaps a slight reduction.

NSF priorities exert influence. Each year we will have a different priority, different themes at NSF so this is important for you to know about. You'll find out about those if you contact your program director. And this is very important. Look at those reviews. They'll give you good advice as to how to improve your proposal, rewrite it, and resubmit. I have to tell you the story of one institution that was coming and it came in four times, four years in succession. It got turned down three years. It came back. They improved the proposal every year and it won $1 million in the fourth year. They didn't give up. They kept coming back. So we do encourage you, don't give up. Always try to come back.

So I have just one more thing that I want to say. I just want to tell you about something that I'm doing. I am now working planning for a new initiative under the Presidential initiative IT Squared, the information technology that we're hoping that NSF will get a lot more money next year and I have been asked by *Ruj Navishi* to put together plans for programs for women and minorities in information technology. You may have been reading in lots of journals and newspapers, there's a shortage of information technology workers which should make you feel very happy. There are lots of jobs out there, but we're very interested in getting more women into the field. Some of you may not know that the number of women, for example, just receiving Bachelor's degrees, which is where we get our big cohort, has been dropping steadily since 1986. In 1986 we had over 15,000 Bachelor degrees in Computer Science going to women. There are now less than 7,000 and it's still dropping. So we're looking at this situation and women are not going into computer science. So we need to fund some research studies to try to find out why this is happening.

So what we're going to be doing is run a virtual workshop, we're going to run electronic discussion groups on the web. I expect that these will probably get going about July and August, so please keep an eye open. We'll post it on the NSF web and we've put it on the Sister's network, for those of you on that. We'll alert you to that. Please come in. We'd welcome your suggestions and ideas of studies that we can do -- can fund people to do to try to find out why women are opting out of computer science and how we can get them back again.

So I would now like to hand over to Frederica Darema -- oh, we'll have to wait for our technology. So Jim Cassatt will come in and he'll be talking about NIH. And we'll have all the presentations first and then we'll have the question and answer for the remainder of the session.

James Cassatt: Thank you very much, Caroline. Everything Caroline said about NSF, please keep in the back of your mind because most of it including things like proposal preparation, what to do if you fail, reading reviewers comments, not taking things personally, etc., etc., also applies to NIH proposals so I need not repeat that. I am told this is the first time that someone from the National Institutes of Health has actually come before you and so this is going to be an experiment.

The National Institutes of Health is a very confusing organization because it isn't a single organization. There is something like 20 different institutes contained within the National Institutes of Health. Each has its own separate budget and, therefore, the things like success rate, average duration of award depends very much on the institute. Each has its own particular mission and initiatives. So it becomes very complicated. There are a number of the -- most of the institutes are what we call categorical institutes. That is they are focused on some disease such as the National Cancer Institute, the National Heart, Lung, and Blood Institute, the National Eye Institute, etc., etc., etc. You can guess what their missions are related to.

I am from the National Institute of General Medical Sciences and our mission is to fund basic biomedical research. Sometimes we call ourselves the NSF of NIH. When we talk about our mission, that is to fund biomedical research, you should take that in the broadest possible sense. And, in fact, a lot of what we fund at the National Institute of General Medical Sciences has implications for direct medical benefit 20 years away. And we take a good deal of pride in that.

Well, who am I? My name was introduced. I'm the Director of the Division of Cell Biology and Biophysics at the National Institute of General Medical Sciences. There are two things you really need to get from this slide. One is my name, which I didn't put up there. It's Cassatt, C-A-S-S-A-T-T. And the other is my e-mail address, my phone number, and most importantly, our web address. This is the web address for NIH and the web address for NIGMS. Everything I'm going to tell you can be pulled off the web, including my phone number, etc. Web is wonderful. I used to take stacks and stacks of stuff. Now I don't have to take anything because you can pull up anything you want.

Okay. Why do we want you? Biology is changing and it's changing very, very rapidly. You hear buzz words like post genomic. Within two or three years the sequence of the entire human geno will be known. You hear words like post reductionist but since I've been in science 25 or 30 years the approach has been to pull things apart and understand how the details of each component part of something -- and this generally what we're talking in biology is organism all the way down to the cell and down from the cell down to the component parts of the biomolecules. Every group that we pulled together in the last two years has said you've got to start putting things back together. We're talking about new ways of doing experiments. Biology, you did experiments, you got some results, you analyzed the results. There is new chip technology that is being introduced very rapidly so that you can do one experiment and find out what is happening to every single gene within an organism. So information is enormous in biology. Bios don't know how to handle that information. This is a problem in computer science. It's just wide open for computer scientists. We've had, again, any number of panels when Harold *Varmus*, our director, comes and speaks he talks about the need to bring people into biology who are not biologists. People like physicists, chemists, mathematicians, computer scientists. We desperately need this kind of new expertise to be applied to biology to continue the advances that we've made.

Now why should you want us? The NIGMS budget is over a billion dollars. The NIH budget is probably, what $12 billion to $13 billion? The success rate for NIGMS grantees is somewhere between 35% and 38%. Contrast that to 30% you heard on the last slide. Our average size of a grant is around $220,000 total cost. Our average duration of a grant is four years and for new investigators, of which I hope there are many in the audience, we generally get five years worth of support. Just vary all over the place but this gets you started. Caroline was talking about someone needing six work stations. We're commonly handing out pentium farms. It's amazing what you can do with $50,000. You get a lot of dual processor pentiums and you've got a little mini computer -- mini super computer sitting in your laboratory.

So we actually have things to offer. Now, if I turned you on, you want to know how to apply and what happens. NIH requires a form, a 398 form. It's actually kits about this thick. If you want to see what it's like, just go to your sponsored program's office, we send out gazillions of these things all over the country. They will have the kit. Or you can go to our NIH web site and you can download a copy in PDF format. We have not quite gotten to the point of you being able to download this in nice form that you'd fill in by computer and print it. No. We're not in the 20th Century yet.

What you will find, though, when you go and look at the 398 kit, there's a lot of forms that you fill out with budget and everything else. But the most important part is the research plan. This is very similar to what you would submit to NSF and it's very highly structured. Not following this, you do this at your risk. It requires a statement of your specific gains in one page and this should answer it because here is what I'm going to do for the next four or five years. NIH study sections are used to seeing hypotheses. We're trying to get them out of that mode because much of the science that we're seeing is not hypothesis driven but they're used to seeing this. So, again, you should state your hypothesis.

The next section is what is called backgrounders significance. It should answer the question, where does this fit into the overall picture? Why am I doing this?

The next section is preliminary results of progress report. A progress report, if you'd had a grant and you're applying for a competitive renewal, but the preliminary results are, again, things that people are used to looking at. Clearly, if you're a new investigator you're not going to have as many preliminary results as someone who is seasoned. Nonetheless, it should answer the question, here are the things that I have done to show that my approach is feasible.

And finally the method section. Here is how I am going to do this. And this has to be done in 25 pages or less. You can send in some appendix material. Again, everything that Caroline said about how to prepare a proposal falls in -- is applicable here. NIH actually has regulations on font size. Just because they let you use a certain font size doesn't mean you should use it. Picture yourself as a reviewer who is going to review this thing at 10:00 at night after he has put all his kids to bed and he's dead tired, small fonts stretched to the margins, even if allowed, do not make for pleasant reading and do not make reviewers happy.

With respect to peer review, again, we use peer review exclusively. All of our review would be, I guess, the equivalent of NSF panels. When something comes in, the application will receive two assignments. Assignment One is assigned to a study section for review. These study sections are based on scientific areas. There are about a hundred of these study sections within NIH and the study sections will rate your proposal and rank it within the study section and we use those rankings as the primary parameter in making a funding decision. If your grant does well in peer review, basically, if it does well enough it will be funded. If it doesn't do well, it will not be funded. We have all sorts of programs, etc. Ninety-five percent of our applications that come in are investigator initiated and don't respond to any of our program announcements. So peer review is the single most important factor whether your grant is going to get funded. Exactly what Caroline has said.

Secondly, it is assigned to an institute for funding and each of the institutes has its own particular mission. So if you're doing some sort of biological modeling related to cancer, it's going to go to the Cancer Institute. These two decisions are independent. Review is done by different organizations than does the funding.

Now, if I'm done my job you're all saying biology is really exciting and even if I haven't done my job you can look in the job wanted sections of science and you'll see every company is looking for bioinformatisys, computer scientists, etc., who want to get into biology and you find that these are a rare breed. But if I've done my job you're sitting here saying, well, I've never taken a biology course in my life. What do I do? Well, we actually are worried about this and it's going to be approached at various stages in ones career. We have collaborative mechanisms. For example, you're computer scientists, you're university, you're faculty positioned and, hey, this turns me on. We have ways of supporting you if you can collaborate with someone who already has an NIH grant. We have something called mentored K awards. Again, you're computer scientists, I really want to do some biology. Let me find a mentor who has an NIH grant, presumably. You can apply for this -- it's a K25 award and this is brand new. It's just been announced and I'm getting phone calls now about this.

Suppose you're faculty member, you've just gotten tenure. Congratulations. And now you can actually do what you want to do instead of trying to worry about trying to do safe projects to ensure publication and biology turns you on and you really want to collaborate with a biologist as part of your first sabbatical. We have so-called senior fellowships, what we call F33's. Usually the sort of thing that happens is the university will pay half your salary, we can pick up the other half. Again, this would be -- like any other fellowship application, it's written in conjunction with the person you're going to do your project with, you have to have a project that has a recommendation, and so on. These go to study sections and reviewed, etc.

Well, let's say you've just completed your Ph.D. in computer science and you'd like to go to work with someone in biology. We're happy to support you. We recognize the fact that your proposal may be a little naive. We have to have a fair amount of fellowship money and, again, these are reviewed by peers. The mechanism is straightforward and we're on the lookout for people with backgrounds other than biology. I was very pleased. I signed a bunch of 15 fellowships very recently and only 3 of those were from biology majors.

And finally, NIGMS, we don't fund pre-doctoral fellowships with some minor exceptions. All of our pre-doctoral training money is sent to institutions. So, again, you've just gotten a Bachelor's degree or you know someone who's got a Bachelor's degree in computer science and wants to do biology, there are training programs that we support and other support that will take someone who has no biology background whatsoever but has interesting credentials and turn them into computational biologists. And we have a program, for example, in molecular biophysics and many of these are very successful in taking people with no biology backgrounds and training them in the areas to make real contributions in biology, taking advantage of their quantitative backgrounds. Boston University has a program funded by NSF, actually, in bioinformatics. And the emphasis there is to take either biologists who know no computer stuff, no computer science, or take computer scientists, mathematicians who know no biology and bring them to a common point where, again, they're able to make contributions.

So that's all I want to say. I think I will turn it over to Frederica.

Caroline Wardle: All right. We're told that they are coming for -- to put the technology so that Frederica can use her lap top presentation. Are they coming? So let's just wait a couple of minutes. Does anyone have a question just to use up a few minutes?

(Inaudible question)

Caroline Wardle: The web address slides for NIH? Oh, yes. Before it disappears.

(Discussion off the microphone, pause.)

Caroline Wardle: For those people who don't have the handouts, we'll put the handouts on the CRAW web so you can find it when you get back. Any other questions?

(Inaudible question)

Caroline Wardle: The program announcements for NSF are all on the web now. So you can go to NSF and look up computer science if that's where you're interested. Is that the same with NIH, Jim?

Jim Cassatt: The NIH has a formal mechanism for announcing everything. The NIH Guide to Grants and Confex. Anything that's official that NIH wants to announce appears there including program announcement. This is available on the NIH web site and is searchable.

Frederica Darema: DARPA has a broad area agency announcements and they're all posted on the DARPA web page and on the CPD. The easiest way, I would say, is to periodically peruse the DARPA web pages --

??: We can't hear you.

Frederica Darema: Oh. So the DARPA announcements are done -- are posted on the DARPA web pages. They are the broad area agency announcements, BAA's, and they're also officially posted on the CBD but I would suggest the easiest way is to periodically peruse the DARPA web pages in the different directories to find out what their new program announcements are there. Also, DARPA has small business innovative research grants or contracts and I will talk a little bit more about them in my presentation. And also they are announced on the DARPA web pages under the SBI, our office there. So I'll talk a little bit more.

Caroline Wardle: When we get into the full question-answering session, there is a microphone, certainly the one in the middle there. If you can get to one, that would help. Otherwise, I'll remind the speakers to try to repeat the question so that people at the back can hear. Just one more question. It looks as though we're almost there. Any more questions? Very good. All right. I'm going to hand this over to Frederica now. Are you going to go from there, Frederica?

Frederica Darema: Yes.

Caroline Wardle: All right.

Frederica Darema: Good morning. So would like to present to you sort of a prospective of submitting proposals to DARPA, the Defense Advanced Project Research Agency and getting funding for it is the title of the sectioning but most important is what I have in the parenthesis underneath insuring that your ideas get funded. And that's more and beyond submitting proposals and writing good proposals and I will talk a little bit about some of these items. So, certainly, I will give you a perspective from DARPA but also since I -- as Caroline said -- I am officially with NSF. I was for two years on leave from NSF to DARPA and so I will tell you a little bit about some differences between DARPA and NSF opportunities and proposals and how the awards are made. As I said before, I will talk about some more items which pertain in a broader sense of getting your ideas funded.

So DARPA -- so NSF is sort of the agency that has a tradition of programs. The program content that in a sense is driven primarily by the scientific community input and the NSF programs are sustained so they are scoped for a long time, a number of years, and DARPA the programs that are announced are much more agile in their scope. The program manager that comes to DARPA is expected to have a vision for some of the technical directions in their own particular area and also not only have a vision but have also a specific technical idea of how to go about in accomplishing that vision. So having that sort of vision by a program manager, one creates a briefing. The briefing is very much like, you know, you guys writing a proposal. You really have to convince your upper management that it's worth for them putting several tens of millions of dollars into your pocket so you can dispense it to people that you feel that will accomplish this vision. So you have, like you're doing a program in a proposal, you have to demonstrate that there is a need for this work.

Certainly DARPA is the Defense Advanced Projects Research Agency so the primary mission is to develop technology that will help the DOD, the Department of Defense, however, there is no -- in a sense -- barrier between the developing technology that supports the Department of Defense and broader impact. And we have a lot of examples. For example, the Internet and the * by technology. In other words, spearheaded by DARPA many years ago. The *real* side of technology started by DARPA, again, many years ago. And when I say many years ago, it's about 20 or so years ago. So you coming back to a specific program. The program manager, this is what the people that manage this program say, the distinction with NSF they are called program directors. The program manager at DARPA is expected, also, to develop a scope because the idea is not to find the broader sort of needs of the research community, DARPA, in a sense, does not have an approach that we are funding sort of a specific research area. You fund a specific research area but with a given focals.

The other item that a program manager has also to put in the program formulation is what are the milestones for this program? What are the key technologies that will be developed in intermediate steps of the program and to accomplish the final task? And also, you have to also, in a sense, based on these milestones the deliverable and the scope of the program put together what is the required budget to accomplish this task. So the programs at DARPA in some ways are more like, in a sense, a bigger sort of word of what you guys put in an individual proposal. And as Caroline said, you've got to sell your proposals to the program managers and to the reviewers. The program manager at DARPA needs to sell their program, their ideas to their office director and to the DARPA director and deputy director. And once the budget is allocated, then a program announcement, the equivocal of a program announcement, is made. It is called a BAA, Broad Technical Area, that means, Agency, DARPA agency announcements.

So, typically, the programs at DARPA last three to five years, some a few years beyond that but there is this knowledge of agility at DARPA that you don't support programs forever. In other words, you go in there, you make a fundamental change, a quantum leap in the technical scope of the field, and then you move on to address another area. And, of course, you don't sort of like with networking started almost 20 years ago, DARPA has sustained networking research for a number of years. But the *falcos*, the sort of key application or the tack is changing over the years. And that is sort of the right thing because you have to address different things when you enter a field, when you create a field versus when the field matures.

So consequently, also, typically awards are made for three to five years. They are made to academia and to industry, and that's a distinction from NSF which typically and the majority is made to academia. Sometimes in DARPA that sort of depends also on the different directors that DARPA has had. Sometimes the programs are formulated in the first phase two to three years to demonstrate some of the feasibility because sometimes the area that is being fostered is not something that has been addressed by the research community before. So before you sort of allocate a humongous budget, you want to demonstrate whether some of these ideas will pan out at all or have a possibility of panning out. If that sort of becomes promising, then there is another second phase. When BAA is being -- when a program is being approved, a BAA is issued and then proposals come in and then someone awarded. And then there is a possibility the subsequent year and the year after to have additional BAA's to fund work to fill in sort of gaps of where the program manager feels the first round of proposal hasn't addressed appropriately or adequately.

So as I said before, the *mechanisms* are these broad area agency announcements and they are posted on the DARPA web pages and on the CBD because this is the formal process that the government announces opportunities. Sometimes, or actually not sometimes, I would say quite frequently there is a briefing to the community. I mean this briefing is done prior to posting the BAA because the BAA is a procurement so there is nothing that we can say to the prospective PIs after the BAA is issued because you don't want to give an advantage to one PI versus others.

Also, besides the BAA's, there are specific SBIR program. Innovative research programs for small businesses. So these are also announced in conjunction with the BAAs. The proposal deadlines. Typically the BAAs are announced in the beginning of the fiscal year because people know the money they are going to get and have available for that year and after the BAA is being announced, typically 60 days that the proposals are due. Sometimes a little bit more but 60 days is the minimum required.

The proposal sizes, they vary. I mean they are typically from 200K but most of the proposals are higher than that. Up to more than a million dollars per year and these numbers are not public so I'm not going to say more about specifics of those.

So the proposal review is done by panels. There are no *ad hocative* views like NSF and the panels constitute from members of the other program managers and other agencies, industry representatives, and people from the academia, and predominantly people that come *honorly* from the university to serve in the agencies, although reviewers can be used from the academic community. Proposals are rated in more than five categories but the gist is there are proposals that are rated must fund. They are selectable in the sense the research that is proposed is very high quality and they can be funded if funds are available and then there is the category of reject. And there are a few more like reject with a little bit of honor and things like that, but the -- is the same.

So the proposals are notified by mail on the outcome of their review and the proposals may receive a briefing summarizing. We don't give -- DARPA does not give individual sort of reviews and like NSF, for example, where you do send -- the PIs are entitled to see the actual reviews for the proposal. And the proposal success rate is 10% to 20% so now you have the *, the smaller the awards the higher the success rate and the higher the number of proposals that are being awarded.

So some differences from what you have heard from previous presentations both for NSF and NIHs funding, the funded proposals from DARPA are actual contracts. The PIs are contracted to do a specific amount of work which, of course, they have proposed in the proposal. The projects have specific deliverables. They have milestones, research accomplishments, and the technology deliverables are actually payable *. In other words, every quarter you submit what your milestones accomplishments are and then the agents will send the following chunk of money to your institution. So if you don't do the work, in other words, you don't get funded.

As I said before, the products of the research are of value to the DOD but, of course, as we know in many instances there is much broader value to the society at large.

The projects are reviewed periodically. That's a difference between NSF where you are given actually a grant. When you've gotten your chunk of money or your award from NSF, unless you do something very bad, you know, you're not going to lose your grant. Whereas, at DARPA the projects are reviewed periodically. That's good in some ways. You should consider it as positive because it has an opportunity for the program manager that has a broader view of what happens also in other research projects to help some of the shaping of your projects. So it's not negative. But, in a sense, the reviews sometimes, the PIs, I know, they have to provide the quarterly reports and it's a burden but there are good and bad sort of aspects to everything.

So the projects may be terminated. There are no guarantees that because you got a project sort of contracted that that reserves the right to terminate the project. That's can be due to unsatisfactory performance. It can be done due to budget cuts. And it could also be done due to redirection of DARPAs needs.

And the final thing which is sort of the majority of the project, projects are funded to their completion and at the end there has to be a report on the deliverables. And sometimes the deliverables are actually technology products. Not products as they are sold by the company but products that can be used by DOD.

So I said, as I was talking, some of the differences between NSF, here are some more. As I said, I eluded a little in the beginning most of the programs at NSF are community driven. NSF has workshops and invites the community to tell the programming directors at NSF and the NSF -- the various directives, the vision directives, and the director directives. What are the exciting research area for NSF to fund? At DARPA, as I said before, it's mostly generated by the vision that individual *PMs* have come to DARPA have. And as I said, NSF funds research areas at large so technology database research, compiler research, any robotix and so on. At DARPA, you know, a lot of this research is also funded but has a specific * to develop a specific technology. It's application driven. This is not bad. Applied research or application driven research does not mean it doesn't have basic research content. I want to make that very clear. And we have seen it from the impact that DARPA has made. And as I said before, DARPA has contracts, NSF has grants, and DARPA has specific directions versus free thinking, the deliverables, as I said, products in the sense -- in the limited sense of used to DOE and NSF most of the products are publications.

Also, the DARPA PI is typically selected from people that have demonstrated, in a sense, research capabilities so it is very hard for someone that has just finished their Ph.D. to make a proposal to DARPA and get an award. I mean certainly it can be done but that's not typically the case. And also DARPA, unlike NSF, has a lot of mechanisms and programs to staging that have helped beyond researchers to build up their reputations so they have career awards, *power* awards and so on. DARPA does not have those.

So some thoughts of writing successful proposals, a key ingredient, first of all your own novel, great, innovative ideas. So the rest of the bullets here are reports you've heard from the other people. You have to define the scope of your proposals. It cannot be very unspecific, I'm going to cure cancer but not say how you're going to do it. In the executive summary, I mean DARPA has an executive summary and if anybody of the proposal, you have to say what you're trying to solve, why this particular project that you're going to do, why is the time now to do it, what is the quantum leap that you're going to achieve. As also the other thing the speakers said, why this is important with respect to research that is being done already? What is the *differentiating*. Why your method is better. Certainly you have to substantiate why this is responsive to this particular BAA because as I said, the assorted stations at DARPA are done by the BAAs. There's a very small number of proposals that are accepted, sort of an unsolicited basis but this is a very, very small amount of the funding that is --

SIDE B

Frederica Darema: -- there proposals are about three quarters to an inch thick and you receive -- a reviewer receives a thousand of those, you know. People spend a lot of time. Personally, when I receive proposals I spend a lot of time because I am very respectful of the time that people put, but don't make it more. You don't gain more by padding it with words.

So some other things that I want to talk is besides writing good proposals, there are other, you know, sort of auxiliary mechanism but very important in getting your work funded. Getting known to the PM, go and talk to the PMs. Go and talk to them about your ideas. You can do that, as I said before, before the BAA comes out. You cannot talk to them afterwards. Keep in mind, the Program Manager is your friend. I mean, we in the agencies really want to help you to do outstanding research because this is how we get our brownie points. We put out a program and ten years later this great research has been done and they attribute it back to this problem that started. So the PMs are very eager to help each one and every PI to succeed. Keep that in mind.

So the other thing, get involved with writing proposals. Even in Graduate school when your advisor writes a proposal, volunteer to help. Ask other peers to, in a sense, help you -- to give you feedback on what you're writing. These are all things that were addressed before. And consider your feedback very seriously so you can try next time. And if you're junior academic, try to hook up with more seniors but also have in mind that you should establish yourself very quickly on your own.

Some other thing, don't assume that people understand first time what you want to say. In some ways when you tell people something, either in a presentation or in a proposal, tell them what you are going to say, say what you're going to say, and end by saying what you said. Okay. And the more novel the idea is, the more this kind of approach is necessary. Present your ideas to multiple forums so you can get feedback and document your ideas. My favorite I've seen is *verbal ball and script**. All right. The words fly, the writings remain. You want to be able to have people give the idea back to you. Not forget about it.

Other things. Be on top of your field. Be aware. Important trends now are in getting funded is that multi-disciplinary research. Jim mentioned that in a wonderful example, you know, with the efforts that NIH does but this happens with other fields. So be plugged into that. Become familiar with some other areas because there is a leverage (Inaudible) in the boundaries of the research areas. Again, you know, other things is network. Know other PIs. Get mentors and I think there will be presentations or there were presentations and some may talk more about it.

Some other things. Start publishing. Be active in the broader community so you can become known in a broader community because some of the reviewers, the ones that review the proposals, it helps always when they've heard about your name before. Right? And have seen your work before. Go to conferences and things like that. And I want to put a plug in people's involvement in the agencies. All right. First of all, when you are called to review -- to pick up (Inaudible) please accept it. The best way we can support proposals and make cases for it is to have a lot of people provide reviews and that is good for the people who write good proposals and also for the people whose proposals are not successful and they need to get feedback on that. So don't shun this because (Inaudible) your civic duty, participate in workshops that are used to create (Inaudible). That's more at NSF but -- there are other opportunities, even at DARPA where they have meetings for the briefings, for the new program briefings.

Eventually in your careers, I think you should have in mind that it is a good thing to come to the agency and be a program manager, a program director, because getting your ideas from that is making a case that this is your area is very important and can have a broad impact and a big impact in the future. And a very good mechanism to do that is to create programs. In fact, when I was (Inaudible) I asked many people, come and take over my program because otherwise it was not going to be another BAA.

So see that it is an opportunity and also that you don't leave your research but you have an opportunity to make a broader impact in your area. And having said that, I want to put a plug for a program that they announced coming back from DARPA, the next generation software, it pushes for some nice ideas, new ideas, and several technology areas performance engineering of hardware and software, new compiler technology, new obligation composition technology, going to the NSF web pages. I'm not going to give you any more (Inaudible) of mine because you can find the one at the (Inaudible) as Jim said. We need no business cards any more. Not stacks of papers to bring with us.

(Inaudible comment)

Frederica Darema: I've given up on those, too.

Caroline Wardle: Thanks, Frederica. Is this one on? Can you hear me? Good. Questions now. We have questions and answers. Feel free to ask us anything. You can direct it to a particular agency or we'll just ask for any volunteers to answer it. Anybody have any questions, anything to do with getting funding?

*Della Firander*: Hi. My name is Della Firander. I have three questions. Could you please compare the cancellation policy on NIH and NSF projects. You said that DARPA has a mechanism for canceling projects. The second one is, how can I meet project managers at conferences? How do I know who is who? And the third one is about feedback. Could you compare the feedback that's given to the projects.

Caroline Wardle: Okay. Let me quickly answer for NSF cancellation. Frederica mentioned we give you a grant and it might be anything from one and three years. You have an annual report to do and as long as you seem to be making progress, we keep continue -- we will continue the funding. If we think you are not making progress, we can and come do a site visit. We rarely do this for individual grants. We certainly do it for the larger group grants and sometimes, it doesn't happen very often because people usually do do the work that they say they will, but occasionally we have gone in and the site visitors have recommended that the funding be terminated and we have done so. So we just stop any continual funding on that project. But where it really hurts you is that when you come in with another proposal, it is usually known in the field, as well as by NSF, that you did not do good work on the previous funded project and so it would be extreme -- you know, your credibility has been hurt for future projects. But we are not draconian as DARPA is. We do give you the grants and give you a little leeway.

Conferences. The best thing is to get in touch with your program directoral manager before a conference is coming up. Find out who they are and whatever agency and just say, oh, by the way I'm going to be at so and so conference. Will you be there. And then you'll find out directly that way. That's the best way. Make contact with them before hand or if you're not sure, just contact somebody you know at NSF and, as I said, ask them who is the person you should speak to.

NSF sends you all reviews that are done by external reviewers. We just remove the name of the reviewer because they are confidential but you will be sent them and it usually takes a couple of months after the review took place.

Jim Cassatt: If I could answer the same questions. In terms of -- you get your money, you actually apply for your grant every year even if awarded for five years but the review is done by staff and is fairly perfunctory. As long as you're making good progress, nothing happens. As far as meeting program managers at meetings, you better invite us. We're not going to come to computer meetings, frankly. We do go to a number of meetings more related to the biological sciences. The third question was review. When you submit an NIH grant you will get back in -- you will get two things back rather quickly -- one thing you get back fairly quickly. That is the grant is reviewed by a study section. You will get back rather quickly after that study section meeting, a postcard which tells you the score, and then there will be a number there that you can call, usually my number or somebody else's number. You say, I got this score. What does it mean in terms of funding? And people will say, I can't make any guarantees but. Forget it. On the borderline, hey, it looks really good.

With respect to feedback, about six to eight weeks after the review you will get a summary statement. The summary statement contains the verbatim comments of the reviewers, sometimes slightly altered to protect confidentiality, and sometimes you will have someone else's dissection for whom English is not his native tongue and the person who writes these things will change it slightly. There will be a summary paragraph which summarizes everything. This is why the proposal did this. Reviewer One is very enthusiastic but let's face it, in discussion Reviewer Two's comments were taken more seriously. Unlike NSF and, perhaps, even DARPA in the back will be a roster of the people who are on that study section. NIH believes in openness. You have a right to know who was on this dissection who reviewed your proposal. You have a right to know by the Freedom of Information of anyone on the outside who has seen your proposal. You can request that information under the Freedom of Information Act. You have a right to know the composition of the study section prior to the review.

Caroline Wardle: Do you want to add anything?

Frederica Darema: The only thing I want to add is because I talked about the termination of grants and the criteria, is in meeting the PM is looking at DARPA web pages and see where the PMs that are supporting programs that you are interested in in your technical area, give them a call, send e-mail, and ask them if you can go and meet them in Washington or you can ask them if they will be visiting your area, your university, your conference or something like that. People are willing to meet with individual PMs.

Caroline Wardle: If you have a number of people in your university or say from surrounding universities and you to say, look, I'd like to get together 9 or 10 people, or even 5 to 6 people, could you come and spend an afternoon with us. That's something you can ask, too. If we can fit it into our schedule we will also do that. More questions?

??: Yes. This is a question for James Cassatt. I was wondering if you could clarify the priority scoring and the percentile rankings and also, too, the funding levels because I think you gave 35% to 38% for GMS but I think the funding at the 20th percentile in terms of proposal ranks.

Jim Cassatt: Okay. What happens at a study section is grants are reviewed, reviewers present -- they're assigned to usually two or three reviewers to present detailed reviews or discussion at the study section. The study section, each member then, is asked to vote a priority score from 1 to 5, 1 being the best, 5 being the worst. The scores are averaged, multiplied by 100. We give you a score that is from 100 to 500, then these are percentiled within the study section such as the 1st percentile is the best, the 100th percentile is the worst. The success rate, which is what I gave which is different from a percentile cut-off, the success rate is -- for NIGMS, that's our institute and that's different from all of NIH, our success rate has been better than most every other institute for the last ten years, is between 30% and 35%. This does not translate into a percentile cut-off. And those of you who are more mathematically and alpharythmically literate than I am, you can probably figure out why. Our pay line, this is the percentile cut-off to which we go by vote, is somewhere around -- right now it's about 27, 28. As we get towards close to the pay line we call this a gray zone and then that area we, as program managers, program directors, everyone has their term for these heads of -- leeway into what we do with those proposals.

Caroline Wardle: I think I would just like to add one more thing that I don't think I mentioned with NSF. I believe Jim, and correct me if I'm wrong, is different between NSF and NIH. The program managers at NSF have a little more leeway than I believe in the funding recommendations at NIH. Our reviewers do a similar thing and they put proposals in three categories. Highly recommended, recommended, not recommended. But we do not have to fund them in rank order. We typically don't ask them to rank order but merely to put them in those categories. Usually we find that the highly recommended are outstanding but we can actually fund some that might be regarded as lower, they're in the recommended category. And very often we will do it for a number of reasons that is within the mission of NSF. One, we've talked with that PI. We happen to like what they're doing, they're relatively junior, haven't had a lot of experience in writing proposals and we really like to help young and new investigators. So we will often recommend funding for somebody that didn't get into the very highest category. We also have a responsibility to fund a geographical distribution and also large universities and small universities so we try to have a mix. So we will very often go out of order and recommend funding because we think there are some special reasons why we should fund that person. Am I correct in thinking that you were really bound to the rank order at NIH?

Jim Cassatt: No. No. We can actually do anything we want to. Practically speaking, it requires strong justification to not fund someone in the "highly ranked" category and you can translate that into numbers if you want but certainly we superimpose on top of the ranking our own programtic criteria such as we really have a strong push to fund new investigators. We look very carefully at how much other support this person has. How does this -- what we call a program relevance, in other words, is this person doing the same thing everyone else is doing but with a different twist or is this something really new and exciting which may not have scored quite so well because it is new and exciting. You can picture all these things that we do. Study sections actually also look at these things as well and so while we can do a lot of stuff, if you were to say, what do we do, probably effects probably fewer than 10% of the grasp, just on a numerical basis. But we do take these things into account.

Caroline Wardle: Very good.

Sharon *Prof*: Hi, I'm Sharon Prof. I'm from Compact Systems Research Center. I have two questions. The easier one first. Is a new investigator necessarily a young investigator?

Jim Cassatt: No.

Sharon Prof: What about somebody who has come out of --

James Cassatt: No.

Caroline Wardle: I regret saying that. I should have young, I shouldn't have said new.

Sharon Prof: Okay. So somebody who hasn't gotten grants before.

Caroline Wardle: Somebody who has just recently gotten their Ph.D. or been in industry and coming into the university, yes, from an industrial background.

Sharon Prof: Yeah.

Jim Cassatt: New is what you want it to be.

Sharon Prof: And the second question. I have a rumor that I'd like you to confirm or deny. I heard that it is sort of frowned upon for more senior researchers to be going after the smaller NSF grants, that the senior researchers are more expected to be going after the DARPA grants, and people sort of look at it funny if they're basically content to go after the smaller grants and have just a couple of students. What do you think about that?

Caroline Wardle: Why don't you answer that, Frederica.

Frederica Darema: Yeah. I think this is an unfounded rumor. I think the amount of the award is commensurate to the idea and it is possible a senior PI or a person who has a lot of years of experience to think of something very novel that only requires a small amount of money to demonstrate the concept. It is absolutely delightful if that person submits a proposal like that. Nobody would frown against it. There are no -- I mean, this is absolutely a false rumor.

Jim Cassatt: I will just make a comment. The National Academy has just announced their new electees for 1999. It did my heart good to see Joe *Felzenstein* listed. Most of you don't know Joe Felzenstein, probably none of you do, but he's a population biologist, very much into computers, and he runs a very small operation of about $70,000 a year of external funding.

Caroline Wardle: Are there more questions? Feel free to ask very basic questions. We'll answer anything we can.

??: My name is *. I'm professor at * University. My question is, is the NSF presently funding robotic research?

Caroline Wardle: I'm sorry. Does it fund robotics?

??: Research. Yes.

Caroline Wardle: Yes. Yes, in fact it funds it in the direct * and the size directorate. It's the IIS division. There is also some robotics work, I think, in the engineering directorate.

Ann Condin: Hi. I'm Ann Condin from the University of Wisconsin. I have a question about submitting proposals to different institutions or different funding agencies at the same time on similar ideas. How does that work? I mean, if you want to be sure to get funding on something, can you submit it to two agencies?

Jim Cassatt: NIH doesn't care but we -- our grant things are changing but there used to be a place called other support. All we want to know is know about it. NSF can comment. It's idiosyncratic to which part of NSF you're from, I think.

Caroline Wardle: No. NSF has a rule that if you have submitted the same proposal to another agency, you must list all proposals at the end. In fact, it has a slightly different -- an answer to a different question is you cannot submit the same proposal to two different groups within NSF. That is not allowed. But you can change a word in a proposal and then they will --

Jim Cassatt: I do know in the biological division, Mary Clutter said, no, we will not accept duplicates to NIH.

Frederica Darema: This is true also for NSF. You cannot submit exactly the same proposal you have submitted to another agency and as Caroline said, you have to disclose all your pending proposals. In other words, proposals that have been submitted and at the time that the program ends or the program director is going to decide to make an award, they will -- at least I would and I'm sure most people do that, they will ask the PI what is the status on this other awards because we don't want to fund duplicate work. I mean this is a problem, frankly, because the money is not enough to fund all the good work and this should not be aggravated by funding duplicate work.

Jim Cassatt: Actually, it's illegal to accept federal money for two projects -- from two sources for the same project. In fact, if you were to get away with it and they would audit, you actually have to give money back.

Frederica Darema: That's also true. Right.

Caroline Wardle: I have often had for some of the larger grants. PIs have a pending proposal in with DARPA and they've announced it and said it's complimentary so if they review well in my panel and I contact the DARPA program office and I say, you have a proposal from so and so, I've got one too, and then we discuss it. We find out how it is doing there because it's important to me. We have to make a decision whether my -- the work in my proposal is dependent on the work in the DARPA proposal. So if I want to make an award but DARPA doesn't, then mine won't fly without the DARPA funding. So there are lots of reasons that the agencies do work together. We talk to the program directors in the other agencies to make sure that it makes sense and everything fits.

Sarah *Graham*: I'm Sarah Graham from John Hawkins University. I was wondering how often from these proposals you ever saw research directions change along the way and, if so, what happened with the funding?

Jim Cassatt: At what point are you talking about in terms of research direction?

Caroline Wardle: Do you mean when you received a grant and then want to change --

Sarah Graham: I mean after you receive a grant and perhaps in the three years something happens, they follow another lead instead of the initial primary goal.

Jim Cassatt: At NIH, say we give you a five-year grant and let's say in year two you want to have a significant change in what you're doing, you actually need permission from us. Now we're very understanding. It's a matter of degree. I mean you have to start with something that was way down here that you were going to do. If something takes off, fine. Great.

Caroline Wardle: Same with NSF. As Frederica pointed out, we like lots of curiosity driven research so if you've got some idea that's come out of the side of your research that you think is much more interesting, talk to your program director and they will advise you as to whether that's fine. As long as they say, yes, then it's fine. So you must talk to your program director. But it's certainly possible.

Jim Cassatt: One little nuance on that though is at least at NIH every four years, five years you have to come back in for competitive renewal. There are a couple of things reviewers will look at. How much progress did you make during your previous grant period, as Caroline said. They also want to know, hey, you put down specific things, one, two, three, four, five. Which ones did you accomplish. And if you decided to go off on a different track which was really exciting, say so and explain why you didn't do these things but you really went off in this direction.

Jennifer Schopf: Hi. I'm Jennifer Schopf from Northwestern and I had a question about -- I know Caroline mentioned a new call that was based for women and minorities in computer science. Ca you address how those grants are reviewed as opposed to the general grants? I've actually heard that women shouldn't apply for some of those because they're viewed as special grants and you're not equal.

Caroline Wardle: Actually, I think we have some senior women in the audience here who have received such grants and could probably answer that better than I. You're referring to like existing programs such as the POWRE, which is the Professional Opportunities for Women in Research and Education. And prior to that there was visiting professorships for women and research planning grants for women. So there is a very small number, by the way, for women and it really is institution dependent. I have heard women in major research universities say it is not regarded. And, of course, they vehemently disagree it should be and we can give you some ammunition that will help you educate like the department chair. We can give you the funding rates and you'll find out it's very tight. I don't think it's a 30% funding rate, I think it's way down. So it's very competitive. But I've had other -- I've had women at MIT say it was wonderful having the POWRE grant because I could spend it on anything I wanted. And she was on group grants where there was a lot of money but she didn't have a grant that she could spend for anything she wanted whether it was supporting a graduate student or buying equipment. So she found it wonderful. So it really does depend. And I don't know if we have any about those senior -- do you get one of the women's grants? You might want to answer that. It's better to go to the source, I think. There's some very well known women like Nancy Levison, people like that who had visiting professorships for women.

??: Yeah. I haven't gotten one of those but let me say that -- let me say that I think from the PIs point of view, you know, money is money and if money helps you further your research career, that is a good thing. The thing that I think might be interesting, and maybe Janie can answer this in the tenure session, is does it make a difference to your tenure committee if the source of your funding has been specialty grants, a grant that for women and minority versus a grant that's for all technical people and a certain program or something like that. I think that's where it might make a difference but in terms of the PI, if you can get more resources to do your research, I would think go for it.

Caroline Wardle: I would say take the money and run. Yes. Very often the programs for women are to do something that you cannot get funded under a regular research program. They're different. And I would use those as a jumpstart to get you -- and then absolutely aim for the regular research programs. So they are usually different. Ann, do you want to speak on some of this?

Ann: I just want to speak to that because I got one of the visiting professorships for women grant and I used it to augment sabbatical funds and visit the University of Washington, and I thought that the University of Washington really took it seriously that I had this and were really impressed that I was able to get it and the fact that it was a women's program didn't seem an issue at all. And it made a big difference because I had my own office. I would have had to share an office if I didn't have that grant. I was bringing in money to the University of Washington. I was supporting a student and so just my access to resources quite apart from the funding money of the grant itself completely changed as a result of getting this grant and I was able to work with the top professors in the country. At the University of Washington I was able to sit in on the course that Filgreen taught and problems that the interface of biology and computer science and it really was highly regarded at the University of Washington to have this grant. So I put in a plug for these grants.

Pat Teller: Actually, I was going to ask a question but I'll address that also. I'm from a minority institution, University of Texas at El Paso. My name is Pet Teller and we have been very successful with a minority grant which is to develop a model to bring undergraduates into research and this is directed at minorities and women. But even on our own institution, that kind of grant is looked at with less respect than another grant. And that's a shame.

Caroline Wardle: But I think it's changing. Universities now are becoming much more aware that they have to put more effort and focus on teaching. There's been a lot of pressure not meeting from parents but from legislature so we've really started to see a change happening and I think there's a lot of concern about education and I think bringing women and minorities in is also a part of that, a broader issue of diversity. So it is changing. It just changes slowly. But I definitely -- I have people coming in now that they're bringing their presidents along with them who are backing them for all of this and universities are starting to change the reward system. That's always the question I asked. You know, have you changed the reward system? Are you giving tenure to people who are superb teachers and have done some creative things? Are you promoting them? That's the key. It is starting to change.

Pat Teller: And I do agree with you on that because our Provost just recently sent a memo indicating that the tenure decision process was being changed. So there is hope. But my question is on unsolicited proposals. I've heard actually different opinions with respect to NSF as to whether it is a good idea or not to submit an unsolicited proposal. So I was wondering if you could address that.

Caroline Wardle: At NSF, and what I have seen personally, is a fairly small number of solicited programs because -- I mean proposals. Our programs do have particular areas. I'll ask Frederica to comment. I think you're going to find they're very discipline dependent. I have seen big, very big, infrastructure grants coming unsolicited. We also, by the way, do have special projects. Where it's been used in my directorate is if you want to do something that doesn't fit under the existing programs, then it becomes a special project. But my advice would be always talk to the program officer that's closest to the area you're interested in and ask their advice. Frederica, do you have in your experience --

Frederica Darema: Yeah. I want to corroborate that. That is certainly, if you submit and solicit a proposal you should speak maybe -- more than one program manager or director to whom to send to and typically people are very, very good in trying to find who are the sort of people that can sponsor that proposal. But also it helps if you have done some of the homework yourselves by contacting the individual program managers and make them aware so when one of them sort of takes the ball, takes your proposals and brings it together, there is a little bit of awareness from their part about these proposals. So, yes, we are doing a lot of effort in sort of addressing the proposals that are in between the different sort of programs and --

Caroline Wardle: See the one thing that you have to remember is that budgets are usually pre-allocated to certain programs so if an unsolicited program comes in that doesn't fit a particular program, then there is always the question is -- if it is very highly reviewed, does anybody have any budget left that they could fund something that came in unsolicited? But it is done but do talk to people as Frederica said.

Frederica Darema: Also, proposals can be held for a while. Certainly NSF has sort of made a target, a measurable target under the GPRA, the Government Performance Reform Act, to review proposals and make decisions within six months but I'm sure PIs prefer that their proposals is held a little bit more so they can eventually get an award rather than the proposal being rejected. So, as I said, a lot of effort can be done and are done, and maybe if you have ideas of how this can be improved we will welcome your input. By the way, if anybody has ideas that -- for any subject, please send us e-mail. We welcome your input.

Caroline Wardle: Next question.

Kathy *Fizler*: Hi. I'm Kathy Fizler from Rice University. I've heard that if you apply for an NSF grant, it is much more difficult to get money to support post doctoral salaries as opposed to graduate student salaries. Would you comment on the kinds of resources that your respective agencies are more or less willing to get funding for?

Caroline Wardle: All right. Should we let Jim have the start of this since he's been quiet for a while?

Jim Cassatt: No problem. There are -- we actually have more problems with graduate student salaries than we have with post doctoral salaries and there's actually something called reasonableness of cost. That is, we actually have dollar limits on what we will pay for the total educational cost of a graduate student on a grant. What was happening was we were seeing graduate student salaries of $16,000 quite reasonable. On top of that, $20,000 tuition, which gives you $36,000 versus a first year post doctorate for $28,000. People look at this and the GSA said, that's not reasonable and they were right so I think we actually have limits of what we will pay for the total cost of education of a graduate student. But really what a study section looks at when they look at the budget is how many FTEs are required to do this work and we certainly have no problems funding graduate students, post doctorates, and technicians.

Caroline Wardle: Frederica, do you want to say something? You want to address the NSF?

Frederica Darema: From DARPAs point of view, certainly (Inaudible) contracts is the work to be done, there is no sort of preference or negativism towards either post docs or graduate students. It is sort of considerate part of the proposal whether it has a good management plan of what the wait is with respect to the individual contributors to the project. So there is no -- because DARPA is not -- DARPAs mission is not to support graduate students. It's a strong mission in DARPA to support graduate students as it is at NSF. But certainly DARPA recognizes the need of fostering the new researchers of the future and, therefore, there is an effort to -- the graduate students are supporting DARPA.

Caroline Wardle: At NSF there is a lot of support for graduate students on the research grants. It is included as part of the budget and that's why we fund the graduate students. And that's very important to NSF. Post docs. Well, you have hit a hot button here. Post docs are very controversial, probably external community as well as inside. The opinions are split in the size direction. Now we're talking about computer science. If you talk about chemistry, mathematics, post docs are a normal course of your career path there in academia. In computer science they have never been a normal part in the same way. Very controversial. Many program directors believe there should be no post docs in computer science and some think they should be. Now we do have a program, we have a small program which does fund post docs in computational science and post docs in experimental systems and they are very specific and it's -- to take a post doc who has been trained in some computer science area and to retrain them in those areas, in computational science or experimental science. There's been lots of discussion over the all the years I've been at NSF. Should we have a post doc program? Should we not? It's still controversial just as it was nine years ago. So I think I would agree with your statement that if you put in a research proposal, funding to support a graduate student, that will be viewed as just part of a normal research proposal. A post doc would have much more difficulty unless it was very, very strongly justified. And it might depend on the program director. The program director's priorities might be not to support post docs. They would rather support another junior faculty member that's got on a tenure track line. So I think my question there is, talk to the program director again and see if they would support a well reviewed proposal with a post doc and follow their advice.

??: Could you extend that question to addressing supporting programming, support programmers or --

Caroline Wardle: That's harder at NSF depending on the projects. That's more of infrastructure so we have a lot of big equipment infrastructure programs where you can buy large equipment. Typically a department will come in or a big research group and we will allow technical staff on most of those which means the systems level or the network engineers but, typically, not the programming. Not the straight user and application programming. It's just that we have so -- you know, we're trying to leverage the money to the best possible way so we want to support the research rather than the applications developments later. Frederica.

Jim Cassatt: NIH, again, doesn't care. The problem is -- I mean we have a number of projects with the primary emphasis not computer science but do computational something or other. And the program may be in the central part of that project and NIH will support it. The problem comes in the cost to programmers. They are a lot more expensive than biologists and the study sections look at this and start scratching their heads a lot.

Frederica Darema: So the emphasis, in a sense, reflects the missions of the agencies. At NSF the mission is generating research so, therefore, as Caroline said, is looked upon that if you want to develop a piece of software you should use your graduate students. At DARPA the view is different because you are supposed to develop products and certainly that is not looked as negative.

Caroline Wardle: I think we've just got time for one more question and one more minute left.

Susan *Haler*: I'm Susan Haler. I'm from the University of Wisconsin Parkside. A quick comment and then a quick question. I had the good fortune of reviewing for a career and for power, successively, one fall and one spring, and the POWRE proposals were much more competitive than the career proposals. Much more difficult. I would encourage you, any of you that have a POWRE to find out what the funding rate was whenever you had it because --

Caroline Wardle: Does everybody know these acronyms? The career is the five year program for new investigators. You have to be within, I think, four years of receiving your Ph.D. It's for men and women. That's an NSF wide program and *size* puts in lots of money into that. POWRE is the program for women and there's a variety. It could be junior, senior women on that one.

Frederica Darema: Right. And that is reflected, in a sense, in the higher quality of proposals because when you have more senior people in the pool, certainly there is --

Susan Haler: Yes. That partly is it.

Caroline Wardle: But you had one.

Susan Haler: I did have one question. I am from, as a small number of people here are, from a primarily teaching institution where research is not the primary emphasis and outside of EHR, that sort of educational wing of NSF, are there any funding opportunities at all that are realistic to apply for? Particular things like RUI? I've heard stories about research at undergraduate institutions. They have a program that it's more or less just a way, a vehicle for projects to add undergraduates onto their larger projects.

Caroline Wardle: Oh, no, RUI, Research in Undergraduate Institutions, is a program whereby you apply for faculty research, individual faculty research. You can apply for equipment but you have to be from the PUI, a Primarily Undergraduate Institute, and there's rules on that. These funny acronyms. I think it's a while since I did a RUI award but you have to fill in a special form that says you are a PUI and that means that you're -- like the liberal arts colleges, the comprehensive -- the Bachelor's and Master's degrees and small doctoral institutions are sometimes allowed to apply. If you don't' give more than 16 -- if the university does not give more than 16 doctorate degrees per year, they're PUI. And so this is a way to let people from smaller institutions come in with research proposals. I'm not answering research rather than education. And so you have to have an extra section in your proposal that is not in a regular proposal which is how is the work you're going to do effect the teaching at your university? So it's looking for you to take your research and bring it into the curriculum. So the RUI program is, again, it's set aside. We have to put a certain amount of money to people who come into the RUI program.

As mentioned, DHR is the educational directorate. That is a huge directorate with lots and lots of money that funds primarily educational proposals. Some of them are research and education but most of them are education applications. But each directorate has a little bit of education they do and the -- we have one program called educational infrastructure which is a program that funds about $400,000 a year over three years to fund innovative educational activities. We jointly run a program with the engineering directorate, CRDC, and that's a combined research and educational program. This is an area, by the way, that we will also entertain the unsolicited proposals and we call them special projects. So we've had interesting things come in to do with education. They clearly don't fit any research program but they've come into special projects and we have funded some of those.

Again, the best thing to do is to ask the program director that you know, anybody you know, how would I send in an educational proposal. And depending on what you're suggesting we might suggest one of the educational directorate ones. We might say, that sounds interesting. Send us in a proposal and we'll deal with it as special projects.

Jim Cassatt: NIH has an area grant program specifically for people of smaller institutions. The grants are three years, $75,000 for three years; that's $25,000 a year but it is limited to smaller institutions for research at those institutions.

Frederica Darema: For DARPA, of course, it does not have this kind of specific programs but the equivalent of that and out of which of that are these SBIRs and since I didn't have time to talk about it during the presentation -- the SBIRs give like 99 -- up to 99 -- for the first year and if the project is successful and it's reviewed after the first year, then you get up to $375,000 for the subsequent two years. That's per year, the $375,000. So that's a very good way of allowing small businesses that want to do some research also to have this capability and I think that's very important for the economy because this business has eventually become maybe the --

Caroline Wardle: I think we're out of time. Thank you so much for coming and please enjoy the rest of the workshops.

Two brief announcements. Our break coincides with only part of the FCRC breaks so if you want coffee, the next 15 minutes of your time. Also, the three rooms for the three different next panels. This is one -- the room over there that we had a panel in last time is one ...