November 2012 Vol. 24/No.5
By Betsy Bizot
When people in the computing field talk about numbers in computing – numbers of degrees granted, students enrolled, faculty, dollars in salary or research expenditures – they often refer to the annual CRA Taulbee Survey. But Taulbee is not the only source of information on computing. How do Taulbee results compare to some of the other available information?
CRA is proud of the Taulbee survey, which has been conducted for more than 40 years and is currently sent to more than 260 PhD-granting academic units (departments and schools or colleges of computing) of computer science, computer engineering, and information in North America. It collects information about students, faculty, salaries, and research expenditures. Taulbee is an excellent source of information for many purposes and the only real source of information for some purposes. Even for information covered in other ways, Taulbee results are generally available 9 months or more earlier and so provide a leading indicator of more comprehensive results. However, Taulbee has its limitations, particularly when discussing bachelor’s degrees, because Taulbee surveys only PhD-granting departments and many bachelor’s degrees in computing are granted by non-PhD departments.
NSF’s numbers are compiled by the National Center for Science and Engineering Statistics from multiple sources. NCSES has a wealth of information at http://www.nsf.gov/statistics/ Their results cover all disciplines of science, mathematics, and engineering. In addition to reports and associated standard data tables, NCSES also offers the WebCASPAR database at https://webcaspar.nsf.gov/ which can be used to create custom data tables.
The comparisons in this article use data from two national sources. The Integrated Postsecondary Education Data System (IPEDS) is managed by the National Center for Education Statistics for the Department of Education; it gathers comprehensive information from all US postsecondary institutions. The Survey of Earned Doctorates (SED) is completed by individuals, not institutions; it is sent annually by NSF to all individuals completing doctorates in the United States. SED results broken out by detailed field include the category of “computer sciences,” which encompasses computer science, information science, and some specialized areas such as artificial intelligence and computer graphics.
How well do Taulbee and NSF numbers agree?
Figure 1 compares Taulbee, IPEDS, and SED numbers of PhDs granted. As expected, the three sources track quite closely. The numbers reflect “computer science” from IPEDS, “computer sciences” from SED, and the degrees granted by US Computer Science and US Information programs from Taulbee (which may include some computer engineering degrees granted by combined Computer Science and Engineering or Electrical Engineering and Computer Science departments). Note that in 2008, Taulbee began including information PhDs as well as computer science and computer engineering; before that, information programs and degrees were not included.
Figure 2 compares Taulbee and IPEDS for bachelor’s degrees granted. In addition, on the right-hand axis, it shows the percent of total US CS bachelor’s degrees accounted for by Taulbee. During 1994 to 2010, Taulbee included only a quarter to a third of total bachelor’s degrees. However, Taulbee parallels the more comprehensive IPEDS numbers in general trends (peak in 2004, valley in 2009, turnaround beginning in 2010).
Figure 3 compares Taulbee and SED on the percentage of new PhDs taking employment in industry vs. academia. The Taulbee numbers in this figure represent the same total number of employed PhDs as in the annual Taulbee reports, but the percentages are calculated differently in two ways to be comparable to results available from the SED. First, postdoctorates are not counted as employed (SED counts them as continuing study), and second, the percentages of new PhDs going to industry and to academia are out of those reporting domestic employment, not out of all PhDs in that year. The Taulbee and SED percentages parallel quite closely; this is a nice check on the accuracy of the employment the departments report to Taulbee compared to employment reported by the new PhDs themselves in the SED. Not shown on the figure, Taulbee reports higher numbers of new PhDs in each employment type because the SED includes only US computer sciences while the Taulbee adds Canadian and US CE PhDs, but the pattern is clearly unaffected by that difference in scope. In 2005 and particularly in 2010, Taulbee reports a slightly higher percentage to industry than does the SED; this may reflect the fact that Taulbee treats postdoctorates as a subcategory of academia and therefore departments may be counting industry postdocs as industry employment.
Why do Taulbee and NSF numbers disagree?
Taulbee generally agrees well with the more comprehensive NSF results. Differences may come from several causes.
The CRA Taulbee Survey differs in scope and intent from federal efforts such as IPEDS and the Survey of Earned Doctorates. Because of its focus on a single field and the participation of a relatively small number of departments, Taulbee results can be made publicly available as much as a year before comprehensive results through NSF. These comparisons of PhD degrees, bachelor’s degrees, and PhD employment suggest that Taulbee is a reliable leading indicator for PhD information. For bachelor’s degrees, Taulbee results generally mirror the trends of the field as a whole, but include well under half of the degrees. We know from other sources that the PhD-granting departments are statistically different from the non-PhD departments in, for example, the number of women and underrepresented minorities who receive bachelor’s degrees (significantly higher in the non-PhD departments). Therefore, Taulbee PhD results provide a reliable picture of the state of the field, while Taulbee bachelor’s results are useful but should be interpreted with caution.
Thanks to Mark Fiegener, SED Project Officer, Human Resources Statistics Program, NSF, who provided the SED data that was used in Figure 3. Thanks also to Stu Zweben, CRA Survey Committee Chair, who provided valuable feedback on an earlier draft of this article.
1828 L STREET, NW SUITE 800, WASHINGTON, DC 20036 | P: 202-234-2111 | F: 202-667-1066