Empirical evidence on the relationship between GRE scores and post-graduate performance

There is a massive meta-analysis by Kuncel et al (2001) that empirically evaluates the correlations between various aspects of the GRE with multiple post-graduate performance criteria.

Based on hundreds of studies and thousands of participants, GRE shows reasonable correlations with post-graduate GPA (i.e., around observed r = .21 to r=.43). Similar correlations emerged between GRE and faculty ratings of the student.

Correlations of GRE with research productivity and publication citation counts were smaller, but still positive. This is not surprising given that these are more distal outcomes and there are many non-academic reasons why people may or may not pursue an academic career or have publishable results.

The authors thus concluded that GRE was a valid predictor of a wide range of graduate outcomes. They also noted that "subject tests tended to be better predictors than the verbal, quantitative, and analytical tests."

In general, selection decisions are assisted by standardisation, and a big part of academic achievement involves measuring baseline ability. Thus, the GRE combines both standardisation and competence measurement.

Update 2023: A new meta-analysis was published by Feldon et al (2023) that reviews GRE correlates with academic outcomes. They report correlations between GRE and GPA of .24 (total), .20 (verbal), .17 (quantitative), and .21 (analytical).

Response to comments

There have been a few points made in other answers and in comments, which I'll comment on here:

• Conflict of interest: @msw wrote that they "consider the cited paper to be junk because ETS provided all available data". In general, I don't find the results in the meta-analysis surprising. Most tests like the GRE tend to have fairly strong correlations with general cognitive ability. It is well established through thousands of independent studies that IQ scores correlate fairly well with both school grades and job performance (i.e., in the r=.50 range; see Neisser et al 1996 for a field consensus review). The results show correlations less than .50, but that's not surprising given some of the issues around standardisation, practice, range restriction, domain specificity and so on.
• Belief there is no correlation based on personal observation: Note that if the correlation is around .20, that means that 4% of variance has been explained. That leaves a huge amount of variance in performance to still be explained. It would not be surprising to meet many people that did well on GRE and poor in graduate school or vice versa. Thus, it is problematic to rely on personal experience when it comes to evaluating the validity of tests where such correlations are likely to be only modest.
• Small correlations are useful: While a .20 correlation is small, it can still help make selection decisions. In particular, when evaluating the suitability of selection tools, you need to contrast the validity of a given tool with other available tools (e.g., interviews, GPA, references, and so on). I'm not as familiar with results in the graduate selection domain, but certainly in the employee selection domain, which is quite analogous, cognitive ability tests tend to correlate more highly than interviews, references and so on (for a comprehensive meta analysis of employee selection, see Schmidt and Hunter, 1998). That said, the best selection decisions are typically obtained by integrating multiple selection tools. Furthermore, the small correlation also should highlight to individuals who score poorly on the GRE that it is not that predictive, and therefore it shouldn't discourage an individual from pursing post-graduate study.
• Does training invalidate the GRE?: @user8134 wrote "many people significantly improve their GRE scores by taking courses with prepping companies like Kaplan. So it seems GRE test does not measure any intrinsic ability/talent necessary for grad. school." I agree that individual differences in training and preparation for the GRE may influence test scores. That said, if you characterise test scores to be determined by true ability, training, and error variance, then I would expect that true ability would remain the much larger source of explanation in test scores. This is based on general observations about testing for ability based assessment. In general, the degree to which training is an issue would depend on how much the test materials teach to the specific test. Overall, I would assume that this would reduce the potential validity of GRE, but that the GRE would still be useful. Furthermore, training and nuisance factors can be used to do better on many selection instruments. For example, people can be coached on how to frame their CV or how to answer questions in interviews. Such training is potentially a source of error variance, but it doesn't invalidate CVs and interviews completely.
• Why would anyone care about post-graduate GPA? (@JeffE) @JeffE further notes " in PhD programs, the only thing that really matters is the student's research output.". Some post-graduate courses include meaningful graded coursework and others don't. For the courses that do include meaningful coursework, then such coursework provides a more standardised way of measuring post-graduate performance. So the validity of GRE in predicting such outcomes is not surprising. And thus, presumably we could generalise this to being indicative of how people perform in less standardised aspects of post-graduate performance. Of course, there's an inferential leap here, but in general, performance in related domains tend to correlate (e.g., coursework in mathematics with research performance in mathematics); it's not perfect, but it's still a positive correlation. Furthermore, if the validation study includes some post-graduate coursework where everyone gets top marks and such data is mixed with studies where grades are valid measures of performance, this would only serve to attenuate the observed correlation. Thus, this would suggest that the correlation between GRE and post-graduate GPA is higher than reported by the meta-analysis. Also, the meta-analysis does report correlations with research output and they are weaker but still positive. It also reports correlations with supervisor ratings.
• Better alternatives to GRE: None of my comments above are necessarily advocating the use of the GRE. Developing an effective selection and recruitment system whether it be for employment or post-graduate admission is a complex task. That said, most post-graduate selection systems would want to get a reliable and valid measure of academic aptitude. GRE, IQ tests, other ability tests, undergraduate GPA, all have reasonably validity evidence. And in general standardisation and efficiency are important. So, for example, administering your own selection tools takes more time, whereas taking pre-existing measures like GPA and GRE is more efficient.
• Ethics of requiring applicants pay money to complete GRE: Several people are critical of the GRE on the basis that it costs several hundred dollars to complete. I think that this is a perfectly legitimate question, but that the question of predictive validity can be answered separately. Such a fee could potentially discriminate against low income applicants. That said, presumably the fee in comparison to forgone wages associated with completing a post-graduate degree is fairly small.

References

• Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2001). A comprehensive meta-analysis of the predictive validity of the graduate record examinations: implications for graduate student selection and performance. Psychological bulletin, 127(1), 162. PDF
• Neisser, U., Boodoo, G., Bouchard Jr, T. J., Boykin, A. W., Brody, N., Ceci, S. J., ... & Urbina, S. (1996). Intelligence: Knowns and unknowns. American psychologist, 51(2), 77. PDF
• Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological bulletin, 124(2), 262. PDF
• David F. Feldon, Kaylee Litson, Brinleigh Cahoon, Zhang Feng, Andrew Walker & Colby Tofel-Grehl (2023) The Predictive Validity of the GRE Across Graduate Outcomes: A Meta-Analysis of Trends Over Time, The Journal of Higher Education, DOI: https://doi.org/10.1080/00221546.2023.2187177

This is a good question. I have not been a fan of the GRE for 20+ years, although (through whatever luck I had a good-enough number on it myself that it didn't harm me...) many are. Having been on admissions committees and very much involved with graduate programs for 30+ years, I've had ample opportunity to see the (non-) correlation of success in graduate mathematics with GRE subject-test scores. (The other parts are often useful as tests of English fluency, mainly.)

Of course, the world would be a simpler place if GRE subject-test scores really could show talent for higher mathematics. We note that the Educational Testing Service (in NJ, that makes the GRE and other stuff) is a for-profit that has a vested interest in maintaining its products' apparent importance.

The cost is unfortunate, and certainly discriminates against people whose currency doesn't compete well with USD.

The vaguely useful bit of information provided by GRE is that, well, yes, it is the only thing that most applicants will have done, thus, if one insists on "simple comparisons", it is the only thing that allows that. It is clear that comparison of GPAs is even more pointless.

I have known admissions committees that simply ranked applicants by GRE subject test score. There! Done! :)

No, I do not care very much about GRE numbers, but in a way I am glad that some admissions committees do, in a fashion to the way that under-valued stocks are good investment values.

I do think that the elite graduate programs use GRE subject test as a convenient filter, because it selects somewhat for "quickness/cleverness", and they can afford to "lose" some prospects, because they have so many who are "quick/clever".

In the U.S., having a GRE subject test score is also a sign of awareness that people expect you to take it. Thus, it doesn't matter so much what one's score is, but that one _is_aware_ ... even if it is only of "expectations".

But, in summary, no, I see no point in it. But there are economic incentives for ETS to keep making money. And a great number of admissions committees in math have personal predilections that lead them to be fond of (over-) simple numerical quantification, so... there-we-are.

Edit: As @msw observes in a comment, indeed, if GRE measured significant academic achievement cumulative over several years... it would be odd that one could usefully do the "prep" courses ETS provides. :)

Yes, performance on GRE probably is a good indicator of how well a kid can do on a multi-hour, timed, multiple-choice test, etc. Yes, if we make subsequent coursework resemble this (!?) then we give the GRE predictive power. No, I do not recommend making everything multiple-choice! But, amazingly, some people do believe that this could be done, and purportedly save us all a lot of work.

Sure, these things measure something, and produce numbers that can be manipulated. There are people who are inexorably drawn to the possibility of making final decisions in those terms, even when the significance of the numbers is unclear. Meanwhile, reading letters of recommendation and personal statements is obviously not easily quantifiable. Of course!

If it were really the case that "standardized testing" could tell what its promoters like to insinuate, it would be convenient, indeed. But, again, some decades of experience indicate that these tests do not indicate whether or not people can sustain interest over 4+ years, work hard for 4+ years, continue to develop scientific sensibility, and so on. And the latter issues prove to be vastly more important for completion of a PhD. In direct observation of about 700+ grad students, I'd estimate that fewer than 20 dropped out or failed due to lack of intellectual capacity or lack of prior knowledge. Rather, loss of interest in the subject, or personal issues (mental/physical health) dominate. This "sample" of mine includes a very wide range of GRE scores and even GPA.

More anecdotally, several specific examples stick in my mind, of very low percentile on GRE subject test (bottom 10 percent or smaller...) but exceptional achievement in coursework, prelims, and thesis work. These peoples' potential was easily visible in letters of recommendation and personal statement.

In my department, GRE scores are used mostly as calibration for students who have good grades but are from universities/programs that we are not familiar with and whose quality we are therefore not sure about. For students with good credentials from strong programs the GRE is pretty much irrelevant.