Readers, please note: In the interest of providing information on this subject, we are posting the raw transcripts from the FCRC CRA-W workshop. This is an unedited transcript, but still should provide you with background information of use. The edited transcripts will be completed by spring 2000.


Margaret Simmons: Good morning. I want to welcome you all to this panel on models of industrial and national laboratory research. I_m Margaret Simmons from the National Partnership for Advanced Computational Infrastructure, which is a giant mouthful. For those of you who don_t know, it_s one of the two high performance computing centers funded by the National Science Foundation in the US. It_s headquartered at the San Diego Super Computer Center. Our panelists this morning, in addition to me, are Julia Hirschberg from AT&T Research Labs and she_s head of the human computer interface research department at AT&T Labs. She works in spoken language interfaces and the relationship of intonation and discourse. She received her Ph.D. in computer science from the University of Pennsylvania working in natural language processing. From _85 to _95, she worked at AT&T Bell Laboratories in speech synthesis in the study of intonation and discourse and then I guess the laboratories began to split up in all different kinds of ways, so now she is at AT&T research labs. She also has a Ph.D. in history from the University of Michigan with a specialty in 16th century Mexican social history, which I think is absolutely fascinating.

Our next panelist down the row is Milena Mihail, who is currently at Georgia Tech. She has a BA in electrical engineering from Polytechnic University in Athens, Greece. And her Ph.D. is in computer science at Harvard University. She has spent 10 years as a member of the technical staff and a manager at Bell Communications, so she_s part of the telecommunications industry, and just recently she has moved here to Atlanta to be on the faculty at Georgia Tech.

Our third panelist is Mary Zosel from Lawrence Livermore National Laboratory. She spent her entire career at Lawrence Livermore since she got her Ph.D. in computer science at the University of Washington. She says that with the exception of spending a few years dealing with the endless problems related to mass storage and tapes, she has spent most of her time in the field of programming tools and environments. Involved first with all aspects of development of tools for the Livermore Cray systems and most recently she_s been working on ASCII simulation development project. ASCII is the department of energy defense program_s development program at both, at all three national laboratories, *San Diego*, Los Alamos and Livermore.

And as I said, I_m Margaret Simmons. Someone asked the question in the last panel about researchers without Ph.D._s and I_m one of those who worked at Los Alamos National Laboratory for 25 years before taking off to do a few different things. I went to Washington for a few years and then on to San Diego and I agree with the answer that was given that, while not having a Ph.D. makes a difference early on in your career at least at a national laboratory, the difference disappears once you get to working on research projects and on teams. However, that said, my other feeling is that in some sense, it_s a union card and once you get it, you are considered for all kinds of, especially as a woman, once you get it, you are considered for all kinds of things that they don_t think of you for because you don_t have your union card. So that makes it, that does make a difference.

I have a couple of slides here. Talking more about the panel today, we set ourselves two goals for this panel. One of them is to communicate to you how research gets done in both national and industrial labs and another one is, at the end of the panel, we hope that you, as new researchers will be able to ask the right questions during your job interviews at these labs, if this is your interest, so that you can judge whether the environment fits your aspirations and your temperament, because I think that_s very important. So we_ve posed this as a series of questions that I hope you will keep in mind and sort of hold our feet to the fire to be sure we_ve answered for you by the end of the panel. These are questions that we thought you might have about how research gets done in the national labs and in industrial labs. How do people actually do their work? My feeling of that is, what do you do at 9:00 in the morning when you come to work? Are you working as individual researchers or do you get to work in teams? That_s a cultural thing within the laboratories, by and large. If you are in teams, how are the teams chosen? How is the team leader chosen? Can you, as a researcher, as a junior researcher, become a leader easily or is it sort of, you have to earn the right to be the leader? Do people work in small or large groups? Are the projects these giant projects with casts of thousands, or are there one and two people projects? How does research get started? Is there a process within the lab for requesting funding or for getting a project approved, and what kind of research gets the most support? Do you have to be in the right area in order to get supported from the place?

Let_s say you really do decide you want to do this kind of research, what are the questions that you might want to ask at an interview to illicit the kind of information that will help you decide whether this is something you really want to do. And this is such questions as, where does the funding come from? Is it my responsibility as a researcher or is there this general source off somewhere? When I first started at Los Alamos, all of the research funding in the research group went into a pot and we were encouraged to pursue questions of interest to us that had relevance to what the laboratory was interested in, but we were not told specific projects on which to work. That_s changed now. Now it is, there is much more accountability in research, at least within the computer research group at Los Alamos. The next question you might want to ask is what_s the level of interaction with Universities? Is that encouraged? Or are you discouraged from forming collaborations with University researchers. For some national laboratories and for some industrial laboratories as a matter of fact, what_s the sensitivity and openness of the research? Will you be constrained from publishing for national security reasons, or from proprietary reasons? Will the results be classified or company proprietary? Again, is publication encouraged, is it required, or is it frowned on? And different places really have different feelings about that. If the research is not open, and someone asked the question in the early one this morning, that they are going to have to take a two year job where they are not going to be allowed to publish, then what opportunities would you have to develop professionally? How would you keep up with what_s going on and keep your name in the open?

Finally, I think you might want to know, what_s the atmosphere? Is it an academic type of atmosphere, are you encouraged to keep up to date, or is it much more deliverable? The last question that I think might be worth asking is under what circumstances might your research be subject to cancellation? Do you get a guarantee of a certain length of time, is it at the whim of the person for whom you work, or does it come from on high, or is it bottom up driven? So, those questions, I_m putting those questions before you and then I_m going to ask each of the panel members to give you a little bit of their take on what this is like from their individual perspectives. And our first speaker will be Julia Hirschberg.

Julia Hirschberg: So my name is Julia Hirschberg, I_m, I_ve been a manager at AT&T, well, what was Bell Labs, now we_re AT&T Labs, this is a constant problem for all of us who were at Bell Labs or AT&T, is sort of explaining ourselves. For about five years, I suppose, and before that I was just normal, happy researcher. I think I_ve finally just as a career perspective, I think I_ve finally figured out how to be a manager and also get my work done, because I consider my real work to be my research, and that_s a happy solution. I_d just like to say a few words about AT&T Labs-Research, which was our official name for research in AT&T Labs. That took several years of working out. It was established after the split up of AT&T into AT&T, Lucent and NCR. And I think AT&T Labs was sort of established over a two plus year period. There are about 2,200 people in the labs as a whole, and there are approximately 450 people doing research and we have locations in New Jersey and California. I have someone who works for me, I_m based in New Jersey and she is based in California, so we_re a human computer interface research group, including research on computer supported collaborative work, and so for us, it_s a real practical challenge of just how to keep in touch. I don_t know how many of you may face that sort of situation, but with two body problems, that_s more and more a possibility. And I think that_s really a challenge that people who are in that situation have to face very directly, because it_s very easy to get out of touch with your management and with your collaborators if you are in a remote location.

These are some of the areas that we focus on at AT&T Labs and Research. My background is primarily in AI and in speech, but we also have a very distinguished math department, which David Johnson is in, and we work on network management and other aspects, computer science, most areas, electronic commerce of course. All sorts of, we_re very heavily now, as I think probably a lot of labs are, into Internet related and web related things. And also broad band and wireless technology.

So, to try to start to answer some of the questions that Margaret and Mary Zosel posed in our, yeah. So Margaret, why were yours not out of focus and mine are? Okay, is it all right if I stand over here? Can you see that? I_ll try to stand back here. So, our research environment, it_s very difficult to try to analyze something that you live in. So if you have other questions, I hope, other than focusing, you_ll just ask me as I_m standing here. But essentially, we_re organized into a set of research departments that may contain actually, I said eight to 20, but there are some that have three people, and there are other that have 25 people, but I think sort of the eight to 15 is the standard size of group, and then those groups are organized into larger labs that maybe, I don_t know, maybe 60 people, something like that, sometimes up to 100. The research departments and the labs are organized roughly by discipline and so for example, in our laboratory, we have mostly AI focused people. In other labs, we have mostly speech people, or mostly math oriented people.

Research is entirely preemptively funded. And this is something of course, everybody_s been alluding to this, but if you_re looking for a job, this is what you want to find out, where does the money come from. For us, the money just comes. We don_t have to go out to business units and find it. It_s possible, if you have larger projects in mind that are going to require millions of dollars and contractors or something like that, and a few people do. I always find them somewhat daunting, but there are some people who really think at a larger level, then it is possible to get extra funding, but I would say the more funding your project needs, the more you manage to get, the more sense of responsibility, obviously, there is to actually produce something that was what you planned to produce. That you negotiated with the people who are giving you this additional funding. But I would say the typical researcher AT&T Labs and Research just gets funding and doesn_t have to worry about where the money comes from.

Essentially I_d say I_ve been in academics, as Margaret said, I was history professor at Smith College for about eight years before I sort of saw the light and decided to go into computer science, and then I sort of accidentally got a Ph.D. in computer science because I love to go to school. So I_ve seen academics, but probably at a sort of small college level, and I_ve seen industrial research. And the main difference that I found when I came to Bell Labs was that things were much more hierarchical than I had been used to in academics. In academic departments, there was a sense that we were all equal colleagues. When I got to Bell Labs, it was very clear that your department head had more power than you and didn_t_ have to consult you about hiring decisions. In my book, good department heads do that, but there is more of a sense that there_s a hierarchy, and that was a shock to me. It took me about 10 years to get used that, I think.

Anyhow, as far as the normal things that you do in life, it seems to me that life at our, in our research lab is much like academics, except that you don_t have to teach classes and you don_t have to have office hours for your students. And that for me was a big plus. But you go, you have, you arrange colloquia, you invite visitors, you have reading groups, and you basically sit in your office or sit in your colleague_s offices and do research. You_re also very much encouraged to publish, to attend conferences, to become famous in your field. Sometimes, it_s not as appreciated by your management as you think it should be, so you have to educate them about what the appropriate, you know, why it_s so important that you got this invited talk at this particular conference. And I would emphasize that educating your upper management is a big part of what you need to do, because they may not be in your field, I imagine this is equally true in academics. So you have to be the one to explain to them why what you are doing is really important.

We also have summer programs, and in fact this is probably our main, this is when we come closest to what I remember of academic life. So we bring in every summer, what seems to me like hundreds and hundreds of summer students. In our lab, we have 60 people in our lab, and I think we_re bringing in 24 summer students. We also bring in academic visitors. And by the way, how many of you are graduate students? Oh, great. So I would encourage you, a number, I think IBM also does this, *Bellcore* used to do this, and I don_t know, maybe they still do, but they_re great opportunities, particularly for women and minorities to participate in summer programs that can be really excellent spring boards to either a future job or thesis topic or a whole research career from the undergraduate level up through the graduate level. You have to know to apply for these things in the fall, because they fill up pretty quickly and they_re very competitive, but I encourage you to apply to any of these places, to think about applying for working for a summer in an industrial research lab and seeing what it_s like. I think most people have a lot of fun.

Our summer programs also include, as I said, academic researchers, and this is the time when you often, a lot of people have collaborations with people at Universities, sometimes people at other labs and the summer time is when you really have the freedom to bring those people in and get a lot of work done. So summer for us, in academics, summer was always the easy time. You know, you don_t have much of a schedule, you get your research done at home, whatever, you go on vacation, but for us, at an industrial research lab, because summer is the time we have access to students, that_s our really busy time. So everybody_s totally frantic during the summer time and the labs fills up with all sorts of extra people. It_s very exciting and we get a lot of work done.

Another characteristic of our research environment is that the managers, gee I think almost entirely, have come up through the research ranks. And this has it_s good and bad aspects. They may be frustrated because they can_t get their own work done, that_s a minus. They, at least you can talk to them. Even if they_re not in your field, you have the feeling that since they_re researchers and they_ve been through this, and presumably they_ve been highly regarded researchers, they can understand what you_re talking about and they will appreciate the full measure of your achievement. The down side of it is that they_re not chosen to be managers because they_re particularly good with people necessarily. That can be a nice bonus, but quite often, we have a lot of managers who are terrible with people, so that_s, I mean there are pluses and minuses, but that_s true.

A reward structure is also something you would want to ask about if you were looking for a job. For us, I think it_s kind of a standard academic sort of things are rewarded, papers, presentations, scientific recognition, also for us, and I_m interested, this is probably true of other industrial research labs, one of the things that AT&T sees as a great value in research is the creation of intellectual property. This is so in those high level negotiations with their competitors, they can say we have 20,000 patents, you have 20,000 patents, we can just agree to cross-license our patents, and I won_t pay you and you won_t pay me. So we_re very much encouraged to generate patents. And this is something for me that was totally different from academic life. When I thought getting a patent was just something an engineer did, but everybody is encouraged to get patents.

Also, one thing that_s changed for me I think since I_ve been at Bell Labs, AT&T Labs, which was since 1985, it is more the case that if you make contributions to the company by way of consulting, by way of transferring some piece of technology, which for you may be 10 years out of date, but to them may still be very valuable, you do get rewarded. That_s something, in my experience that didn_t used to happen so much in the past, but is more and more true. But it_s not, in any sense, like that dominates our reward structure, it_s just that it_s possible for people who make those kinds of contributions now to be rewarded. So we have a sort of two tier, two way reward structure. Normal academic research is very highly rewarded, but you have to be really good. And that, I think, is also more true. You have to be really, really good in your academic achievement, but also contributions to the company.

Styles of research, I was trying to figure this out, I think we have three. That_s sort of an easy answer. One is the traditional, old Bell Labs, AT&T Labs style, which is individual research. You sit in your office, you have ideas, you do experiments, whatever it is you do in your particular discipline, you get great results, you publish them, you_re asked to present papers, etc. We also have a lot of bottom up collaborations. A couple of people get together, they_re interested in something, they start talking, they draw in somebody else, it_s completely sort of bottom up, but it turns in to something you might call a project, but it_s only a project insofar as the people who are involved in it want to continue doing the research. So it_s really not led by anybody in particular, except by sort of common agreement. We are starting to have larger projects where somebody will get an idea, and then, as I was saying, will get a chance to get some funding, some, a larger amount of funding, but you sort of have to say, well, here_s the time line of what I plan to do with it and here are the people involved. You actually have to demonstrate that you_ve given the organization of it a little thought. It_s still, as I understand it, pretty disorganized in terms of the way I would think most sort of project research goes. It_s very informal, and if you screw up or if you don_t meet your deadlines, nobody does anything to you. Who knows, that may change, but at least in my experience, our large projects are like nothing else that people say in our development organization would consider to be projects. Which I like, but I_m sure other people probably find a little disorganized.

One question that you really should ask, and it_s really, you_ll find if you ask it, I think most people that you are interviewing with would have a hard time answering. And they_ll give you one answer and then they_ll sort of start thinking about it, and you may have a very interesting and profitable discussion, is how do people succeed at the place that you_re thinking about getting a job at. In my experience at the lab I am now in and at Bell Labs, a lot of times, people succeed largely by self-promotion. That is, you do some work, and it may be truly excellent, and somebody else does some work and that may be truly excellent. The person who succeeds the best will be the person who goes around talking up their research and explaining, usually to influential people, department heads, whoever it is that is part of the review structure, why what they_re doing is so wonderful and why they_re so wonderful. I personally have trouble doing that. I was not brought up to do that. It seems to me a little selfish, but in fact, that_s the way it goes. That_s the way a lot of achievement comes to pass. A better way, if you find this a little distasteful, or, is to have a department head who does this for you. And it need not be your own department head. But if you have somebody in upper management, in fact, the more people who are in your immediate management who think highly of you, the better you will succeed. And so making those kinds of contacts is extremely important. And external recognition is also traditionally a more acceptable way I think for many of us to do this, but getting recognized on the outside is probably a more indirect method of getting internal recognition, because then you have to not only get the external recognition, but you have to make sure that people internally know about it. And it can be remarkably difficult to do this. So that_s another way that things work, at our place at least.

And another way to do that is business collaboration. And as I say, things are changing a bit and now it is more possible, I think, to get rewarded for the contributions that you actually make to business. It_s often difficult for people in research to evaluate these, and so that_s a question I haven_t answered, because often somebody will say, well, I went to 50 meetings and I was a part of this team, and I was a part of that team, and it may be that that was a really valuable contribution, it may just mean that you went to 50 meetings. So I think researchers still have a hard time evaluating the extent to which other researchers who make a contribution to the business have actually done so.

Okay. Those are some answers to the questions proposed.

Margaret Simmons: Thanks, Julia. Our next speaker...

Milena Mihail: Do I have to come, or can I just speak from here? I don_t have transparencies.

Margaret Simmons: You can speak from, right from there, if you use the microphone. That_s all.

Milena Mihail: Okay. So, I_m going to say much less. All right. So, I_m Milena Mihail. Let me say, in two sentences, what my trajectory has been. I have a, I_m not American. I_m Greek, and I have a Bachelor_s in electrical engineering and the reason I went to electrical engineering, I had no idea about the field, I was just good in math in high school and it was the most competitive school to go, so I went. And I came to the US for graduate work and I chose computer science because I thought it was intellectually challenging. It was, the thing to do, of course still is the thing to do, but it was the thing to do about 15 or some 20 years ago when I started this. And I got my Ph.D. from Harvard, and I have been for many years at Bell Communications Research. So essentially I_ve worked in the industry most of the time. My field is mainly theory, so I_m an applied mathematician by training, but of course being in the telecommunications industry, I_ve done some networking, quite a bit of networking. And I just moved to the academia a few months ago. I moved from the New York/New Jersey area here to Atlanta, and I_m in Georgia Tech right now. I moved to resolve the two body problem. My husband is here and we have an eight-month-old baby. So, okay. So this is roughly who I am.

The person who introduced me in computer science, and I guess most of us at some point or the other can point at one person who they consider their mentor, he_s Christos *Papadimitrio*, and for those of you who have, you know, he has written so many books and is *automata* theory, an algorithms book, and okay. So he has told me repeatedly, when you want to speak, especially when you_re not very familiar with who your audience is going to be, here is one good model, try to convey one idea. Not zero, not too many, because people, but just try to say one thing. So pick one thing that you think is important to convey and just try to, you know, make sure that people at least understand one.

Okay, so this is what I want to say, and I think it is a *saddle* point, and it took me several years to figure out that it is the case, at least in the industry, that whatever the model people describe you, and there was a wonderful description about what is the model for example, at AT&T Research Labs, which used to be Bell Labs, and you might here more descriptions, and you_ll hear more descriptions in this workshop, and it_s wonderful. Okay, here is a fact that sometimes people will not tell you, that this is, it is not a static model, it is a dynamic model, it is a shifting model. And this you should always be aware of. So this is what I am trying to say, that what research in the industry or in national labs used to be 20 years ago, 10 years ago, what it is now and what it will be, I_m sure, 10 years from now, is not the same thing. And this is especially relevant for people who are starting their careers to know. So as you plan your career, as you say you know, what is it that I_m going to do, as you go to your research, as you go to your job interview, right, and you_re going to meet all these managers and future colleagues, for each one of the questions that you_re going to ask, so you_re going to ask all these wonderful questions that have been mentioned here, so what is it, where does the funding come from? What work do we do? How is the work decided? How is the work rewarded? How do people advance their careers in this place? So for each one of these questions, make sure you ask what was the case five years ago, what do you expect the case is going to be three years from now, what do you expect the case is going to be six years from now? It will be very important for you to know the answers to these questions. So when you meet who_s going to be your future manager or your future colleagues, make sure you ask them, by the way, what were you doing five years ago, and what do you think you will be doing five years from now? It will be very relevant for you.

And precisely because this is a very shifting model, and this is, it didn_t used to be a very competitive environment, the industrial environment. It used to be the case that you would get a job at the labs and plan to retire on this. It is not the case any more, right? So you make sure that you always understand how the model is shifting as you are advancing your careers. Make sure you compare the model in the lab that you live and work in to labs nearby, to, okay, so if you work at AT&T Research Labs, I would make sure that you know what Bell Labs is doing, what IBM is doing, what Microsoft is doing, and I think basically that_s, if I wanted to convey one thing, it_s this one. That this is a very shifting model, it is a very dynamic model, and for each one of the points, each one of the questions that you want answered and for the plan that you want to make, make sure you understand the derivative. How things change and how things are going to change as your careers advance. So that_s the point I wanted to make.

Margaret Simmons: And that brings up a questions, actually, that I_d like to ask you right now, is do you feel that the academic model has not changed over the last, you_re now in academia, is that model a much more slowly changing model?

Milena Mihail: I think it_s a much more slowly changing model. And you know, by definition, the academia, so when you enter, you enter as an assistant professor, I mean at some level, but you know, within five or seven years, you are tenured, right? So once you are tenured in the academia, you know, you might, the danger is that you might stay in your room and in the same position for many, many years, but tenure is a very, very important thing. There is no such thing as tenure in the industry. And that makes a big difference. So by definition, the academic model is more robust.

Julia Hirschberg: But don_t you think that might actually change? I mean tenure is not necessarily here forever. Look at the British universities, where they_ve actually gotten rid of it.

Milena Mihail: No, no. In fact, in Georgia Tech nowadays, they have a process they call a post-tenure review, and it happens every five years. But I think, you know, I think we are a very long way from seeing tenure go away. No matter what your career is, you have to be at the, what is the expression, the tip of your toes. But I think much more so in the industry than, and especially coming from Bell Labs, so when there was the, in _85, right, it was a very, very big change at Bell Labs, and again when there was the split, there was a very big change. I was at Bell Communications Research, and when I started, what was it, 12 years ago, it was, you were just rewarded for writing wonderful academic thinkers. And within 10 years, I saw so many changes. And it has been the case at IBM. You feel that research oscillates together with stock value. And nowadays, Microsoft is a paradise for research. I mean pure research, mathematics, *communidorics*, but I_m not sure what_s going to happen, I think Microsoft will do very well, but...

Julia Hirschberg: Yeah, but it_s not clear whether Microsoft will continue to be interested in having a large research organization. You don_t know.

Milena Mihail: Yeah, you don_t know.

Julia Hirschberg: Actually, when Microsoft very publicly became interested in developing a top-flight research organization, that was great for all the industrial research labs, because then they could say to their upper management, Microsoft thinks research is really important. And I think that was really one of the biggest things that happened for us in justifying our worth, was simply that a competitor that people are very worried about, thinks it_s very important and is willing to put millions and millions of dollars into it.

Margaret Simmons: Okay, our last panelist this morning is Mary Zosel from Lawrence Livermore National Lab. Mary, do you want to sit there, or do you want...

Mary Zosel No, I_m going to sit.

Margaret Simmons: She_s over here in the corner.

Mary Zosel I did not bring foils. One of the things that was part of our instructions was we were supposed to convey enthusiasm for working in the national lab or industrial settings. And I do want to do that. I think it_s the best of all worlds. And while we_re not in a computing science laboratory per se, we_re in a physics laboratory, it_s still got enormous challenges on the computing end of the thing. And we_ve got great resources to work with. And that some of the things that you might have to really be scrounging for in other organizations that you might get in a large industrial or national laboratory setting that you have.

Some of the things that I wanted to hit in terms of the points we were asked to talk about was what are the formal and informal mechanisms of research in the laboratories. At the DOE laboratories, and there are several DOE people here, including Donna Crawford who_s directly involved in the ASCII program that I am, so feel free to find Donna and ask her similar questions. The DOE laboratories have something called lab directed research and development. And this says that for all of the programmatic mission related things that they get funding for, wherever they are, the laboratory is allowed to rake off a certain amount of percentage off of the top of that money and spend that for something that the laboratory thinks will enhance their future mission, and their future ability to do their work. So it_s not here and now kind of thing, it_s future kind of thing. And the lab, at our laboratory, I don_t know if it_s the same at the other laboratories, that_s broken into three pieces. One small piece goes out and allows everyone at the laboratory to, and it_s very competitive and hard to get...

(end of side one)

... that you have among the 10 to 15 best ideas of the year for this particular piece of research. That_s a very small, typically a one year, or one to two years, one person kind of a project. There_s the next level of formal research funding that_s the exploratory research, where your organization says it_s in the best interest of our organizations if we_re looking for something related to the futures of the wonderful mathematical solver, or a new set of storage techniques or something. And so they will work within their organization to get research proposals. These are then weighed against the other organizations that are coming up with proposals and the quality of the organizational proposals are the ones that get the most money. There will be some money coming into your organization, but it_s the overall quality. And then there_s things that, the highest level of the management thinks it_s strategically important that we get into this particular area. And they carve off a substantial amount of money for big projects. And it_s been encouraging the last few years to see that the computing in a physics lab is coming more to the forefront in the strategic initiative kinds of things that the laboratory where they_re saying it_s really important that we have good computing for all of the laboratory, not just those folks that are doing the few big physics programs.

That_s formal research, and there_s also the formal research proposals that go to other government agencies and things like that. There_s a lot of research that_s done right within the programmatic requirements, both in the computing science side and in support of the physics side. And so those aren_t even the formal research, but the problems are so big in the ASCII area where we work, that almost everything you touch is research in some flavor just because it_s big problems that people are trying to solve. A typical team, probably, unless it_s your one person project, even for two or three person projects, somebody_s going to be the project leader. And someone, research has been changing. The point has been made. There_s a lot more accountability going now and someone is making a project report, someone is writing the proposals, someone is saying what deliverables are happening, and even if it_s in a two or three person project, there_s someone who_s been designated as a lead. And there_s annual cycles of funding to decide whether there are more people or less people going into this particular area. Projects will come in all sizes, all the way from the one to many people, in fact it_s a hierarchy of projects usually, and you might think, well, my project is just three or four people, but if you think in the bigger picture, as Donna would tell you, well your project_s actually several hundred people when you add up all the people across, in our case, the three laboratories that are working together, and start adding up the big picture, it starts looking pretty big.

But we heard this morning that it was important to try to put your research in perspective of pieces. What steps can you take. And that_s something when we started in, as far as research things, being told that we had to write project plans and things like that, but you say, this is research. I can_t write a plan for my research. Well, people are doing that now. And even if the plan is that I_m going to have a paper or a report by the end of the third quarter of this year, it_s a plan. And it_s got a deliverable. And people are getting better, even the academic collaborations we have are now being asked to try to plan out their research cycle for people. And people are getting better at even writing plans for research. And I think it will help, it_ll help bring the research along in terms of that.

The environment that we work in. Those things, all those questions you should ask about, I think it_s the best of the environment. There are seminars, there are lots and lots of university interactions in our organizations. Those are growing all the time because we realize that it_s important to keep the universities engaged in the high performance computing where we are. And to do that, we have to have a lot more university interactions. There are summer students around, people do go to conferences, people have good equipment to work with. So as far as the research, you can teach if you want to teach. There are opportunities to teach. And so there_s all kinds of the things that you would expect to see in an academic, but it is a different flavor than an academic organization. I already mentioned that people, it_s been changing. And Jean said she was working a lot harder in academia. I_ve done a little teaching and I know that that takes a lot of time. But I think good people everywhere are working harder, in all jobs, because that_s the, when we did the messages down from the government were do more with less. Those people who are good are working harder everywhere, and so I don_t think that_s just in academia that that_s changed. I think that_s been one of the things that_s been going with the passage of time. And I already mentioned that there are more expectations about planning and deliverables and things like that.

The research projects go all the way from a little tiny project. I thought I would describe two or three projects to you. And I wanted to say something about how they, what succeeds and what doesn_t, but an sample single person project at our laboratory would be a project that Karen *Hallerbach* had a couple, two or three years ago where she was taking one of the large mechanics code that has also been used in car crash kinds of things, and she was studying what the mechanical stresses were on the bones of your body and what might cause things to break or not. Now there was a sample of a single person doing a project that wasn_t even necessarily directly mission oriented, but it was using the kinds of codes that the laboratory had. There_s two, like a two person project going across the hall from me is where Dan *Shikory* and Mark *DiShayno* are busy working on what on earth do you do when you have these graphics files produced from things that are so big you can_t even store them on all the disks, and how do you possibly pick out the features that people are interested in. And so those two guys are sitting there, sometimes what they have is something going on, they_ve got their offices together and part of it_s in Dan_s screen and part of it_s on Mark_s screen and you_re standing out in the hall looking back and forth. But it_s a very tight, two person collaboration. They_re part of a bigger project, but really, they_re a piece of an interesting collaboration. A more common thing might be a half a dozen people working together, like our multi-grid solver group downstairs. When you start getting into that, you probably have some help of a software engineer associated with it, and you have senior people and junior people, rather than being a couple colleagues working with it. And that_s where you_re even more likely to have the project lead, and all the project plans developed for you.

And even larger than that are the, our code teams now all have computer scientists explicitly called out them, and they have a computer science lead on all their major code teams, and it happens at our laboratory, that all three of the major codes, the computer science lead is actually a Ph.D. Which I find very interesting. And so, this, their main goal is a big physics code, but they_re computer science Ph.D._s playing a good role in helping that that be a quality code solving interesting problems going out. And so there_s a whole range of areas where people get involved in things.

What_s *seats*? This is just a slightly different twist on what Julia was saying on there. When you succeed is, succeeding is getting more funding for your research and the kinds of things you want and getting recognition. Well your managers have that same challenge to go, and what you can do that has splash and can get interesting things that your manager likes to brag about, and your manager likes to drag you in to give a demo for or to talk about, that goes all the way up, and so it_s the matter of getting visibility up through your organization. And when you can make your organization look good, you do succeed and you do get rewarded. And so some people in the past, I_ve heard them saying, well my manager wants to take all the credit for what I do, but if you are working for a good manager, your manager_s giving you the credit, but your work makes your manager look good, too, and then everybody succeeds up the chain in doing that.

I think that_s a sample of questions and I_d be happy to take...

Milena Mihail: If I may come back on this point of success, it_s extremely important. So there are all these, you know, bullet items that we are being evaluated, and again, what I have found is that slightly higher level, here is way to think about success. It_s how much impact you make. So if you stand back one step, ask the following question of yourselves, and this is whether you_re in the academia or the industry. So take your environment. How would it look if you were not there? What difference do you make? I_ve seen this explicitly asked when people are soliciting recommendation letters for tenure cases. So, how would the field look without the work of this person? You know, if you_re in an industrial level, where ever you are, you always take a step back and say, how would my lab look if I were not there? What can I make, to differentiate myself, how can I make a difference? And that is always, I found, a very relevant thing to keep in mind.

Margaret Simmons: Now, are there questions from the audience? And the one thing is, they_ve asked us to please use the microphone so that it appears on tape. Questions?

Q: Sharon Pearl, Contacts * Research Center, I was just wondering, Julia, you talked a lot about how to get rewards. I was wondering what types of rewards they give you for publishing, patents. You know, is it monetary, is it promotions.

Julia Hirschberg: Promotion. Right. Not sure that_s a reward. There are very few at our place, extraordinary awards for anything. Mostly the reward that you get is simply the result of your performance review and then your salary will be, reward will occur commensurate with that. There is a little bit of extraordinary award, like you may get, we have this variety of different performance awards and merit awards, and different components of you salary, which I think is designed to make you very puzzled about what you_re getting for what. But there is a little bit of individual performance award and you may get a couple thousand dollars more if you had a best paper award or something like that. But for the most part, I think our place is not very good in finding ways of rewarding people for specific things that it would like to promote, and we have to get better at doing that. I know other places have, well, at Bell Labs, our *Nopensius* was famous for giving out candy bars and T-shirts, which were so cheap, but you know, it was such a big status symbol if you got, I mean, you would acquire a whole bunch of these same kind of T-shirts or these same books and people would say, _Oh, I have five of those._ And so I think you can reward people in a much more intelligent way without spending that much money. And every once in awhile, we say we_ve got to figure out how to do that better, but no one is really working on that. I don_t know, at your place, do they have good ways of doing this?

Q: Inaudible

Julia Hirschberg: That_s true, we do have a little bit of that. And we do get, what_s bad is, we get $500 when you file a patent, which as a result, means that people file a lot of what I think are kind of worthless patent applications.

Q: Inaudible

Julia Hirschberg: Oh that_s good. That_s excellent.

Q: Inaudible

Milena Mihail: So financial rewards, I think very, very much across the industry, and even across the big labs, so if you negotiate with Microsoft, the stock option that they will give you when you join them, is the thing to negotiate. Also start ups. And also financial rewards, sometimes how your salary looks when you_re in the industry, it_s a step function usually, so it might be roughly the same for many years and then you might get a promotion, or you might get a counter offer from another lab. And then it might go up by, very substantial factor. So I think financial rewards are there, but they don_t happen, you know, not necessarily, you don_t experience them at the annual...

Julia Hirschberg: Or often, I don_t think industry knows how, or we don_t know very well how to reward people for particular things that we would like to encourage. I mean, people get stock options but it_s usually because you_re afraid they_re going to go to another company, or something like that, which doesn_t seem quite fair.

MS?: But academia doesn_t have these kinds of rewards either, so I think that it_s whatever industry does is probably a little bit ahead, at least monetarily, of academia.

Julia Hirschberg: Yeah, I know because of the Computing Research Association surveys our salary structure really changed a lot, at least for people with computer science degrees, so that was a wonderful thing.

MZ?: Financial rewards are always the biggest incentive, but I thought just for recognition things, good work often results in someone being asked to be the person that will lead another team or will suggest the next project, or will take the technical direction in some way, and so there are the other kinds of rewards that happen, too, from good works of just starting to have the recognition as the key person to talk to in this particular area.

Q: So, I_m Lori Freitag from Argonne National Lab, and I just have a question on if you could comment on how the research model might change when you_re working on classified or proprietary type projects, versus more sort of open, externally recognized academic style research.

Mary Zosel Well from the classified side, I_ll speak and I_ll let the others talk about the proprietary side. At our site, there are certainly things that are classified, but the computing aspects of that usually are largely unclassified in terms of a lot of the work they do. And so even our people working on classified projects can talk about large amounts, large areas of their work and can write papers. There are opportunities for them within the classified side, too. There are some classified code conferences and things like that, so when, ** it_s the physicists that get more involved in that particular area where they really don_t have the unclassified side to talk, but there are those opportunities. But in the computing science area, by and large there_s an unclassified aspect to almost everything you_re doing.

Milena Mihail: And I would say it_s the same, it is the same thing when you_re working with proprietary material. In fact, it is a very good, it can, okay, so when you have access to proprietary material or you do proprietary work, there is always going to be a part of your work that you will not be able to publish, but one thing that people do, is if you try to extract whatever information you can get from that proprietary material, and make a model that your industry or company will approve as non-proprietary, then you can be one step ahead from other researchers. This is what I experience, for example in the telecommunications industry, something that is very proprietary is the topology of their network and the traffic that they experience. That_s very proprietary, okay? So we had access to it, but we could never speak about it. Now if you can make a statistical model and you can convince the telephone company that look, it is okay to release this model, it will not reveal what the metropolitan Atlanta telephone network will look like so that MCI can go and put their routers and compete with you, but it will be good if other researchers know about it, because then you_ll get better routing * for example. So if you can make such a statistical model, you can take advantage of the fact that you have access to proprietary or classified materials. There are ways around this.

Julia Hirschberg: Yeah, mostly we just try to patent things, and then it may delay publication a little bit. You have to go through a paper review process, and if it_s deemed that is something that should be patent protected, then you have to go through some patent process, but there are fast ways of filing provisionally now, so that you can publish relatively quickly. So I personally haven_t, but I know there is some data, certainly a lot of data that_s, legally we aren_t allowed to release to anyone because it would be customer data, and a lot of people are doing research on that and managing to publish, I think, fairly easily, the results of their research because it doesn_t reveal anything that is private or proprietary.

Milena Mihail: Yes, one should think about access to proprietary and classified information, not as a restriction, but as a resource and try to take advantage of it.

Margaret Simmons: And actually apropos of the changing model that Milena talked about, at Los Alamos national lab at any rate, in the early days, the people doing classified work were not encouraged to publish. It was considered a waste of your time. But in this more competitive environment, in our new model, they are very much encouraged to do what Mary said, which is extract the non-classified pieces of the research and to publish what they can after they get the classified information out of the work. So it_s different.

Q: Hi, I_m Margaret Wright from Bell Labs. I wanted to add a comment about the rewards, which it just occurred to me as people were talking, there_s also the reward of having your life made easier. And I think, I was in academia and now I_m at Bell Labs, and I think it_s better in the labs in the following sense. If you want a fast new machine, or something expensive, or to go on an expensive trip and you_re doing well, you get to go. I mean, you don_t have to write a grant and apply for money and do this, there_s a support system for the machines in place, and things like that, so they_re not exactly financial rewards, but when I go to visit my friends in academia, I_m sometimes sort of startled by the fact that if they want something, unless they negotiated a very good deal when they got their job, they have to get it themselves. It_s more the individual entrepreneur model. And I think if you_re in an industrial or a national lab, you know, you get a fast line at home, they take care of it for you, and this is not money in your pocket, but it makes your job easier and more pleasant. So that_s another reward. And if you_re doing well, you tend to get more of that. Can I go on this expensive trip? Of course. And is there anything else I can do for you?

Q: I have to bring this back up. Two questions. When, with respect to the rewards, monetary rewards, are you finding it hard to recruit? I mean, it_s certainly true that in the Silicon Valley, the industry is pulling a lot of the very young, bright people because the opportunity of making it in a start up are so high, right now, this very moment, that a lot of the, I mean a lot of graduate students don_t even finish their Ph.D._s and the ones that do have a foot in a start up already, so we are having a harder time hiring.

??: Well, certainly, the laboratory where I am now, which is part of a university, but it is funded by the government, we_re having a terrible time recruiting people because the compensation level is nowhere near what industry can pay, and I don_t know about the other national laboratories or even the industrial...

JH?: We_re having a harder time, and I think mostly, it_s not so much for the first level jobs that we_re having a hard time, but we_re getting people stolen from us, because I recently had an experience with this wonderful person who worked for me. He was offered like almost twice what we were paying him, and we were paying him a lot. And luckily, all you can do is try to convince them that in the long run, this may not be a good career decision, and that can be very difficult, because they think, everybody seems to have one, the need for one start up in their blood. And then they get it out of their system, maybe, because they have a horrible experience. Or they have a wonderful experience and they don_t need to ever be employed again. But I agree, this is, it_s really hard to convince people that they shouldn_t just take this money.

MZ?: We end up, because we_re in the Bay Area, sometimes getting people that apply that expect the Silicon Valley type salary because of our location. And that_s another kind of a problem. When somebody comes in who_s relatively new and quotes that they want a six-figure salary and you go, uh, okay. Well, you know you_re right near Silicon Valley, aren_t you? Doesn_t everyone get paid that here? So recruiting is very hard right now. And one_s pleased to see so many people here and we just hope that, I think the computing science numbers, when they do their statistics, I haven_t seen them lately, but they haven_t been growing the way we need them to grow.

Milena Mihail: I think the field is experiencing the Internet explosion the same was they experienced the PC, you know in the _80s. And all the industrial places I know are having a hard time recruiting. I know the academia is having a very hard time recruiting, and this is also another example of the shifting model that, 10 years ago, we were going through a bottle neck and there were simply no academic jobs. So I can talk about Georgia Tech, they are trying to recruit 10 faculty in the computer science department this year, this is a department of 50 and they_re trying to expand to 60 in one year, and the industrial and systems engineering department is trying to recruit eight. These are massive numbers and it_s happening throughout academia, so I think everybody_s, I just think that the market is much bigger than people to staff it. I think everybody_s...

??: I think the graduate students should listen to that. It_s a great time to find a job right now. The other question I had, particularly for the industrial labs, is how much is your management trying to get government funding for the projects that you_re working on?

JH?: Almost not at all for us. It_s too much of a hassle to deal with all the grant, I mean, we_re not used to grant administration. There is, there are a couple of people, who independently, got a fairly sizable *Darpa* grant, kind of accidentally. And I think what_s bad about that is that they_re sort of using it, I mean now they have half a million dollars a year and they_re fellow members of the technical staff don_t have this money. And it_s very difficult for their management, which is one of my best friends, to deal with the situation where they can go anywhere they want because they have this government money. And so I think it_s a serious problem for us more than an opportunity. That_s the way it_s being viewed at the moment. That could change, of course.

Margaret Simmons: Actually, that is my job at San Diego super computer center, which is going to government agencies and putting together research programs for the researchers at SDSC to be sure that they are properly funded to do the research that they want to do. But that_s a new sort of approach for this laboratory in particular, and I don_t know if that kind of thing will move into other labs or not.

??: PARC traditionally has had a combination of Xerox funded and government funded work and that_s why I was just wondering.

Q: I have a comment about recruiting. My name is Rita Chen. I just got my Ph.D. degree and I applied for the ** and * but, even Julia ** part of the small size or the medium size company because the pay good, pay very well, and also they fast. They interview me and within two weeks, they sent me e-mail, said they want me so bad. So my question really is, can you * your process of recruiting a little bit.

JH?: Actually we have some instances where people, where this process can be very speedy. It depends on who you were applying to. If someone is interested in interviewing a large number of candidates, it_s not going to be as fast as you might want it. My experience with people who get offers very quickly from small companies is those small companies also want a response from them very quickly. And so if you want to keep your options open, and say, interview at other places that may not be able to make a decision so quickly, sometimes you_re forced to make a decision before you have, you know, offers that you might get from those other places. So I think the speed works for you and it also may work against you occasionally. But I agree, it would be nice if these giant bureaucracies worked a little more speedily.

??: Can I throw in something also from an industrial organization, a small one, that it helps us a lot when we know what the people we_re interviewing want and need. So if you_re in ** don_t be shy about telling the people who are interviewing you that that_s the case, because they_ll often be able to respond if they know. And particularly if you can say, _I have this great offer from Company X,_ that gives the person who wants you to come to another place ** to go to their management and say, _She has a great offer from Company X, she must really be wonderful and we need her so much._ So you_re giving leverage to the person who_s interviewing you, if you let them know that.

??: Universities, however, don_t tend to move very fast no matter what your argument is.

Q: I_m *Chris Naneinen*, a Ph.D. student from Oakland University, Michigan. I_m in my final stages of my Ph.D. and I plan to get into the industrial research area. My question is how do you handle this transition from researcher to manager? How do you level up to other people who have just an MBA or how do they test your people skills? Does it come into the picture when you go for the manager position? How does it work out?

Julia Hirschberg: Well, since I mentioned this originally, for us there is no training. You can get training, but there is nothing mandatory. The transition, I_ve found most helpful other people who had rather recently become managers and who will sort of initiate you into the rites of what you have to do. But I think there are many different styles of management at our place, and it largely depends on whether your personal people skills were already there. I mean, I think people have often said that if you want to be promoted, you should start acting like the job that you want to be promoted to. You can also be promoted because innocently you were acting like that and not hoping for a promotion and then they say, _Oh, well there_s someone,_ you know, you happen to be in the right place at the right time. I found it difficult because I think I did care a lot about the sort of people aspects of my position and probably spent an awful lot of time on that initially, and now I think I_m a little bit more ruthless about deciding that I have to also make a lot of time for my own research, or I won_t be a good manager because I_ll quit.

Milena Mihail: But actually, I think this might be something that women have an advantage. I have found, this is very personal, but by nature, women have better interpersonal, researchers, scientists, women scientists, because ** better interpersonal skills, so I think if you are in a research career, industrial or otherwise, the competition to go ahead in management might be smaller than if you are, I don_t know, a lawyer or a doctor, I think that is not a big problem for women, at least for first level managers. Then there is of course, the glass ceiling, which is there very much.

??: I just wanted to comment in there is that sometimes the management positions, if they_re open because there was a palace coup of some sort, that somebody_s decided that out with the whole organization, then who knows what comes about in terms of, it_s they_re getting new blood across the works. And those happen at organizations. But if it_s that there_s a new opening because a new group has been formed or because someone has left or retired or moved on, then things are much more open in terms of that. But there_s both the technical project leader kinds of positions and there_s the people managing kinds of positions. And having done both of them myself, I very much prefer the technical part of the things rather than the people one. I personally got kind of torn apart when we were doing some downsizing and the idea that we might actually have to let go of someone and my bottom performer was a woman who had just been separated from her husband and had two kids. And I says, _I can_t imagine myself walking into her office and telling her she didn_t have a job._ Now, fortunately, we didn_t have that downsizing, but it was tearing me apart inside. And so the woman_s people skills can work on both sides, because you can empathize with the positions on there, and so I found, when I was doing the people things, that I just didn_t have enough time for doing the technical things that I_d want to. And you_d go to a conference and you_d say, _What are you working on?_ _Well, I_m seeing that everybody is doing their safety training,_ and whatever. I found it kind of deadening in the particular way we were doing it at the time. It_s improved again, but the technical side was much more interesting.

Q: My name is *Gerta Camberavo* from Washington University. I_ve been an instructor and * and I_ve never worked actually, and I was always curious about research in national research labs. And at this point what I_m interested about the summer research opportunities in a sense I_m not a fresh Ph.D. or for example, I don_t have already established collaborations, is there a way actually to get this experience and see how things work?

??: Gee, that_s really hard because you probably won_t...

Q: I know it_s hard which is why I_m asking.

JH?: Actually we_ve had an experience, and this was just one individual case, where a person happened to, I think knowing at least somebody, even if it isn_t an established research relationship, is the key, but if you simply know somebody who can figure out who might be in, would be in your area in that lab, and who might be interested in some kind of a summer collaboration. And then you can find out what kind of work they_re doing, again from this contact you have, and you can propose something. We actually had this happen with somebody that no one knew, but she happened to know another consultant, he happened to find out the appropriate area, and she_s got a summer job. And she_s somebody, I guess in your similar position. But the trouble is, of course, there are no established mechanisms as there are for graduate students and undergraduates, programs that you can go within, so that_s tough.

Margaret Simmons: I think the same thing is true at the national laboratories. I don_t know about Livermore, but at Los Alamos, there is an active summer program, but it tends to be organized from the graduate schools and from undergraduate schools, but if you know someone who was willing to have you work with them, or who is interested in working with you, than you can usually put together a summer job or a summer program.

MZ?: But you_re often looking for those, for long term resources and long term payback on the thing, where someone who_s going to be there for a very short time needs to have an important technical contribution to help make on the thing, or have the possibility of being a future employee. And so it_s a tricky position to be in. Although there are positions where we bring people in that they_re just 50% time or something, it_s a working mother or someone trying to do job re-entry but not sure that it_ll work. And maybe bring them in as a junior member of a project and then they work up.

Q: Hi, I_m Tatiana *Spezman* from the University of Maryland, and I am graduating this year and kind of considering different career options. I have a question about, like when you get fresh hired to an industrial lab or a national lab, what is your position in the lab, or what is your position in the group, and how does it changes with time? Most of what I mean, when you get hired to the university and become an assistant professor you are mostly independent and in five years, you are either tenured and have an established position or you go on. When you get to the grad school, when you_re a junior graduate student and you are in the group, often the case that you are supposed to do whatever senior graduate students don_t have time to do and you earn your way up until you have other junior graduate students maybe helping you to do something. So what happens when you get hired to the research lab? Do you again get into the position that you_re supposed to all kind of dirty jobs for more senior people?

Milena Mihail: So before anybody else answer this, I want to say something preemptively, okay? So that the procedures in the industry are much more ** than what they are anywhere else, and that we should always remember. I mean, there are always rules, but you should always remember that figuring out in the industry what is the next thing to do, what is the smartest thing to do, what do you do when you go to your office at 9:00 in the morning, is always a more tricky question. It is not straightforward.

JH?: It_s much more difficult to decide sort of where you are in the pecking order. We_ve fairly recently introduced some more structure in that, so before, everybody was just normal. Either you were a manger or you were a happy, normal person. And really the only differentiating factor was you could become a distinguished member of technical staff, but nobody paid much, except perhaps some people who took that too seriously, but now, because there was this crying need to find out, well where am I with respect to other people, and am I better than ...

(end of tape)