The Damocles Sword of Academic Publishing:
Sharper Students or Duller Sword in the MIS Field?

Abstract

Introduction

In his Communications of the ACM article describing the demographics of candidates for faculty positions in computer science, Anthony Ralston pointed out that ``the number of publications on the CVs of new Ph.D.s is impressive, even if almost all of them are written jointly with advisors. When I received my Ph.D. (in mathematics) in 1956, I was proud already to have one publication, since then the first publication for most people was the post-degree publication of their dissertation research'' [4]. There are two basic goals of the current analysis: to find out whether the pattern Ralston speaks of is found in candidates for faculty positions in the Management Information Systems (MIS) field, and to examine the publication outlets for students in this field while at the same time comparing the availability of these outlets to the students. Thus, this study was conducted partly to analyze existing trends, and partly as a guide for new students in the MIS field.

Management Information Systems (MIS) as a field is comprised of students and faculty in MIS departments of business schools. It also includes people in interdisciplinary programs across a number of departments (e.g., Computer Science, Business, Geography, Public Administration, Communication, Information Science, and Sociology). Most MIS doctoral students are members of ACM and some Computer Science doctoral students end up working as faculty in the MIS field, either through cooperation with faculty at the Business School, or through employment in the Business School. Also, there seem to be good opportunities for computer science students to enter the MIS field. In the 1997/98 recruiting year, there were about two positions available in MIS at the entry level for each doctoral student that had declared an interest in these positions.

Depending on the specific institution at which a student applies for a faculty position, emphasis will be placed on different elements of their experience. Among the factors that will receive the most weight are: publications, presentations, teaching experience, working papers, reputation of institution, reputation of faculty on dissertation committee, references, the network and contacts of both the applicant and faculty members with whom the applicant works, personality, service to own institution, job responsibilities, and professional experience. It is believed that the volume and quality of a student's publications are among the most important factors in securing an on-campus interview. When this interview has been granted, the experience gathered from doing presentations at conferences may further positively affect a candidate's chances. Publishing is important not just because it enhances the possibility of employment, but also because it increases understanding of the publishing process. After gathering knowledge about this process, the student may decide that a tenure track position is not as desirable as he had earlier believed. The student may then turn his attention to elements of the doctoral program that will enhance the probability of securing a good position in industry or government. Otherwise, the student may continue the process of preparing for a successful career in academia.

A problem with which students are often faced at the beginning of their stay in a doctoral program is the lack of absolute standards. While many programs have rigid requirements of specific courses and comprehensive exams, doctoral students whose goals are to become a faculty member in their field face the problem of understanding the unwritten rules about what attracts recruiters. Because of the constant fluctuation in demand for faculty members in a given field, students often begin their studies without the knowledge needed to increase their possibilities for later employment. By the time students who do not have the benefit of good advice from their fellow students or faculty advisors reach the point where they understand whether publishing is important for their career, it may be too late to catch up. The remainder of this discussion is dedicated to the improvement of that understanding.

Research Methods and Descriptive Information

Data for 147 students was gathered. Of these, 40 were excluded from the study because they had received their Ph.D. or D.B.A. at an earlier point in time; including their publications would have skewed the results. Only students that received, or expected to receive, their doctoral degrees after August 1997 were included. Because of the desire to understand the publication pattern of students at American universities, students with degrees from foreign institutions were also removed from the pool. It is, however, interesting to note that among the 14 international students removed, there were some excellent candidates with very impressive publication records and degrees from renowned universities such as the University of Oxford and the London School of Economics and Political Science. Of the five students that were not available in the spring of 1998, data from 1997 were used for three of them, and two students were excluded from the study. The data used for the study were the resumés of the remaining 91 students who expected to receive their doctoral degrees from U.S. universities sometime after August 1997.

To examine whether rankings of universities and schools has any effect on student productivity, two rankings were used: the Carnegie rankings of universities, which is a ranking system that is focused on the whole university's federal funding and number of doctorates awarded annually, and the recent US News and World Report rankings of business schools. The Carnegie ranking was not expected to correlate with the number and quality of student publications. A `Research University I' ``gives high priority to research''; they award 50 or more doctoral degrees each year, and receive annually $40 million or more in federal support. A `Research university II' also awards 50 or more doctoral degrees each year, and receives between $15.5 million and $40 million in annual federal support. A 'Doctoral University I' awards at least 40 doctoral degrees annually, whereas a 'Doctoral University II' awards at least ten doctoral degrees in three or more disciplines, or 20 or more doctoral degrees in one or more discipline [1]. This information is included to provide a more complete picture of the universities at which these students have received or will receive their degrees. From this information, it can be gathered that most students get their degrees from large research universities. Comparing these data with the number of students within each category [1], the division of students in this sample is fairly representative of the general division of students among the categories, with the exception of 'Doctoral II' institutions, who predictably have a lower doctoral student to other student ratio.
 

Research University I   51 students (56 %)	Research University II  15 students (17 %)
Doctoral University I   22 students (24 %)	Doctoral II  3 students (3 %)

Table 1. Carnegie Rankings of Students' Institutions (pe rcentages rounded)

Because the Carnegie rankings were expected to have little predictive value, the recent US News & World Report ranking of business school specializing in management information science was used [5]. The report ranks what it considers to be the top 30 business schools in the field. This was used to rank the doctoral programs in this study into four tiers. Those ranked one to ten constituted tier one, whereas tier two consisted of the programs ranked eleven to twenty. Tier three consisted of programs ranked twenty-one through thirty, and tier four was reserved for programs or business schools not ranked in US News & World Report. It is interesting to note that 60% of the students belonged to programs not ranked by the report. This, however, should not be taken to mean that these programs are necessarily of lesser quality than the ranked programs. Some students were educated in programs not directly located within the business school, such as the University at Albany, S.U.N.Y. 's interdisciplinary Information Science Ph.D. program, where the students take courses within a number of schools and departments, some of which are highly ranked in the same US News & World Report study [5].
 

Tier I  17 students (19 %)	Tier II  8 students (9 %)
Tier III   11 students (12 %)	Tier IV  55 students (60 %)

Table 2. US News & World Report Ranking of Students' Schools

When examining students' publications, all conference presentations will be referred to as publications, regardless of whether the conference published proceedings, and whether the students' papers were published in those proceedings. Because the data are reported by the students themselves, their accuracy is hard to verify. In some cases the data provided by students were compared against real conference or journal publications to verify accuracy. Some of the data was found not to be up-to-date.  Though one would expect the students looking for a position to update their resumé frequently, several resumés were clearly not updated. A case in point was the student claiming that he expected to receive his doctoral degree in August 1997. His resumé had not been updated six months after that date. Also, one student listed his submitted presentation to ICIS9297 -- a paper that was not found in the proceedings -- more than a six months after the student must have been notified that the paper was not accepted. Students have much to gain by trying to look as professional and accurate as possible; at the same time, the students have much to lose by reporting false data. Because of these factors, the data is expected to be fairly accurate.  Journal and conference presentations that were listed as ``submitted'' were not counted; however, if they were listed as accepted and forthcoming, they were included in the study. Technical reports and unpublished cases were removed from the study along with doctoral student consortium presentations.

Results and Discussion

To examine whether students just publish in journals and conferences outside of the field, or in outlets that are extremely easy to gain acceptance to, the student publications were cross-referenced against a ranking of journals and conferences in the MIS field. Hardgrave and Walstrom [3] ranked the publication outlets in the field based on a wide survey of faculty members in the field. The three rows that start with `ranked' display only the student publications accepted for such outlets. The last row displays the ranked conference publications, but adds the student publications in the Association for Information Systems Americas Conference. This turned out to be a major outlet for student publications, and was not included in the Hardgrave and Walstrom study because ``at the time of [the] study, AIS had yet to have its first meeting'' [3]. The difference between the median and the mean for these statistics indicates that the mean is ``pulled up'' by a few extremely productive students.
 

	Books	Chapters	Journals	Conferences	Ranked Journals	Ranked Conferences	Ranked conferences incl. AIS
Correlation	-.059	-.198	-.283**	-.177	-.096	.011	.054
Significance	.577	.060	.007	.094	.364	.917	.610

** Correlation is significant at the 0 2E01 level (2-tailed).
Table 4. Correlation b etween ranking of schools and publications

To explore whether the number of student publications correlates with the US News and World Report ranking of business schools, the Pearson correlation coefficient was calculated (Table 4). It is interesting to note that the only significant correlation was found between ranking and journals. The higher the ranking (closer to number one), the higher the number of journal publications. This correlation, though, was very low--too low to draw any conclusions. Interestingly enough, this correlation disappears when only considering journals that are ranked in the MIS field. To further examine the publication productivity, Figure 1 shows at what percentile a particular student would be based on the student's number of publications. The `ranked conferences' include AIS Americas Conference publications.

Figure 1. Student Publications

Another interesting item to consider is the number of student publications in ranked and recognized MIS journals. One would expect the student publications to crowd the bottom half of the scale, but this is not the case.  Hardgrave and Walstrom listed 53 ranked journals in their study; the mean ranking of student publications was about 13. With a median of 10 and a standard deviation of 11, it seems quite clear that when students publish in these journals, they publish in those that are highly ranked. It is unknown whether the other journals the students were found to publish in are of similar quality in other fields. Though the answer to that question is hard to find, one student paper was found to have been published in a top journal in the nursing field, Nursing Research.

The papers on which students collaborated had up to six authors, but three was the most common number of authors, followed by two. Among the papers where the student's author-position was found (22 out of 35), it was most common that the student was the second or first author. However, students that are first or second authors of papers are probably more likely to list their publications in correct order, whereas students that are among the peripheral authors are more likely to list all other authors at the end of the reference (usually saying `with' and then listing the other authors, thereby making data interpretation more difficult). Finally, the most common outlets for student conference publications are displayed in Table 5.
 

Conference Name	Ranking in Hardgrave and Walstrom	Number of student publications (of 379)
AIS Americas Conference	Not ranked	75
Annual Decision Sciences Institute	5	62
INFORMS Annual Conference	8	18
Regional Decision Sciences Institute	11	17
ICIS	1	15
IRMA International Conference	9	14
Hawaii International Conference on System Sciences	2	13
Academy of Management Annual Conference	10	8
Sum		222

Table 5. Popular Conferenc es among Student Scholars

It is very interesting to note from Table 5 that although the Regional Decision Sciences Institute is a very popular outlet for student scholars, the relatively new AIS Americas Conference has taken over the lead. Though the AIS conference only started in 1995, it already has more student presentations than any other conference examined. It would seem that the informal nature of the conference, coupled with its reasonably inclusive publication standards, have made it the preferred conference for many doctoral students.

Conclusions

Though the current hiring situation for students in the MIS field is very favorable, with more positions than candidates to fill those positions, the students are not resting on their laurels. Students are preparing themselves for their coming positions as researchers by publishing in a number of outlets. It seems that the ``publish or perish'' attitude has reached the students of our institutions. Many Doctoral students are publishing in well-respected journals and conferences.

It is interesting to note that the students studied in this research did about 70 percent of their publishing outside of the traditional MIS journals and conferences.  This attests to the highly multidisciplinary nature of the field in addition to the wide acceptance of MIS students' studies in other fields.

If we accept Ralston's notion that students are publishing more and earlier than was once expected, the question becomes whether students are getting better at publishing and do more interesting research, or whether the conferences and journals available have lowered the standards necessary to publish. Only more research can clearly answer this question.

References

1

Boyer, Ernest L., A Classification of Institutions of Higher Education 1994. Carnegie Foundation, December 1994

2

``Damocles,'' Microsoft Encarta 98 Encyclopedia. Microsoft Corporation, 1998

3

Hardgrave, Bill C., and Walstrom, Kent A, Forums for MIS Scholars, Communications of the ACM, 40, 11 (November 1997), pp. 119-124.

4

Ralston, Anthony, The Demographics of Candidates for Faculty Position s in Computer Science, Communications of the ACM, 39, 3 (March 1996) pp. 78-84.

5

U.S. News and World Report, Americas Best Graduate Schools 1998 : Plus a Directory of Law, Business, Medical and Engineering Schools, U S News Specialty Marketing, 1998

Biography

Last Modified:
Location: www.acm.org/crossroads/xrds5-2/damocles.html