College ranking and student sorting: Evidence from China

Zhigang Li a Chao Wang b School of Economics the University of Hong Kong

Abstract

China has over 1,000 universities, admitting around 7 million students yearly. Providing information on the school quality may elevate the sorting of students and improve the quality of human capital. In this study, we exploit publicly available university rankings in China and school-major level admission score data to estimate the sorting effect of information provision on school quality. We find evidence that the public ranking of universities significantly affects the decision of students and the outcome of university-student sorting.

a b

Corresponding author Tel.: +852-9722-3859 E-mail address: zli.economics@gmail.com Corresponding author Tel.: +1-917-9753195 E-mail address: cw1379@nyu.edu

Catalogue
1. 2. Introduction.........................................................................................................................3 Admission System in China................................................................................................4 2.1. 2.2. 2.3. 3. An Introduction to the Admission System in China................................................... 4 Impacts of the Admission System in China ............................................................... 4 Admission System in China and Representative Anglophone Countries .................. 5 Estimation Method ..................................................................................................... 5 Measurement of Students’ Quality............................................................................. 6 Selection of College Ranking Reports ....................................................................... 7 Evaluation Method of the CEUSC ......................................................................... 8 A Transverse Comparison of Ranking Reports ...................................................... 9 Construction of Variables and Regression Models and Samples ............................. 10 Estimation Results and Implications ........................................................................ 10 Robustness Checks ................................................................................................... 13 From the Perspective of Universities and the Government ...................................... 14 From the Perspective of Students ............................................................................. 14 The So-called General Public’s Misinterpretation and the Aftermath ..................... 15 What Are Characteristics of a Meaningful Ranking Report?................................... 15

Estimation Strategies ..........................................................................................................5 3.1. 3.2. 3.3. 3.3.1. 3.3.2. 3.4.

4.

Results and Implications ...................................................................................................10 4.1. 4.2.

5.

Ranking Report Revisit.....................................................................................................14 5.1. 5.2. 5.3. 5.4.

6.

Conclusion ........................................................................................................................15

References ................................................................................................................................18 Appendix ..................................................................................................................................21 Tables .................................................................................................................................. 21 Figures ................................................................................................................................. 35

1. Introduction
This paper is an empirical study on the rank effect in Mainland China. It measures how students’ university choices respond to changes in college ranking reports. In China, universities admit the majority of students purely on the students’ scores in the National College Entrance Examination. Students apply by selecting universities, in their order of preference, with or without the information regarding their examination scores. Consequently, students with high scores in the national exam may not be admitted into top schools if they fail to predict the admission cut-off lines of those universities, which can be seen as an inefficient allocation of valuable human capital. We hope that this estimation of the ranking effect in university applications will facilitate the effective allocation of limited educational resources to students in Mainland China. We also aim to compare the rank effect in Mainland China and the rank effect in representative Anglophone countries. In the United States, an exogenous one-rank improvement leads to a 0.9% increase in applications for that university (Luca & Smith, 2009). Meredith (2004) shows the significant impact that movements of the top 25 American universities and public schools in ranking reports have on their admission outcomes. Due to the completely distinct enrollment system, education system, institutional setting (Lee & Barro, 2001), and the arithmetic of university league tables (Dill & Soo, 2005) between Mainland China and the United States, we cannot show the cross-country rank effect of these two countries in a similar manner. However, the alternative way we devised to measure the rank effect may enhance people’s understanding of it. In addition, this paper examines how university ranking reports in China improve the school-student matching and raises the return of human capital investment in China. This effect has significance because of the important link between the higher education of human capital and a country’s economic performance (World Bank, 2005). We also investigate the accessibility of information on university quality for applicants. An online survey1 was conducted in order to understand students’ attitudes toward college ranking reports in Mainland China (See Table 1). According to the survey, 64.87% of respondents can access college ranking reports through the television, newspapers or the Internet. However, they are often unaware of the names and the sources of these reports. In this paper, we first introduce major characteristics of the admission system in China and examine the estimation problems faced by students. Next, we discuss the selection of ranking reports and construct variables for our estimation equation and regression models. Then, we propose an estimation method to measure the rank effect, and conclude with a list of our findings regarding the rank effect in Mainland China.

1

We tried to introduce and study the rank effect in a more intuitive way: an online survey. The online survey was deliberately designed with subjective, overlapping or ambiguous options, such as “little”, “moderate” and “much”. However, the unquantifiable information and the informal index that we got from the survey are not included in our estimation equations.
3

2. Admission System in China
2.1. An Introduction to the Admission System in China
Provincial-level education bureaus are in charge of the university admissions of local students. The only criterion for admission in a specific province of China is the students’ National College Entrance Examination scores from that year. Student enrollment is divided into one “pre tier” and three general tiers. The top hundred or so universities, which are mostly public universities, generally enroll students in pre- and first-tier admission. The remaining tiers are for enrolment into the other thousands of universities and colleges. Since the tier system is major-specific, one university may belong to two or more tiers. In such cases, each major admits students according to the tier it belongs to. National examination takers may apply to several universities in these 4 tiers. They use a standardized university application preference sheet to select several universities and their preferred majors for each admission tier, in order of propensity. After the national examination results are released, all universities in the “pre tier” admit students simultaneously, followed by the first, second, and finally the third tier universities. For example, major X of University ABC admits students from Province K in the “pre tier” admission. The department in charge of ABC’s major X admission ranks all students from Province K who selected University ABC out of all “pre tier” universities as their first choice by their national examination scores. Students are admitted according to their rank until University ABC’s quota in Province K is full. The score of the last student admitted by University ABC in the “pre tier admission” is that university’s “pre tier” admission cut-off line in Province K for that particular year. Other universities cannot admit students that have been accepted into University ABC. Due to the large number of applicants, a student who cannot be admitted to his or her first-choice university in a tier must wait for university admissions in the next tier. Table 2 illustrates a more detailed multi-tier admission procedure using a total number of 35 applicants. The university admission system is actually much more complex because of the various regional regulations of the National College Entrance Examination. In addition to bonus point problems and miscellaneous preferential policies, which will be briefly discussed in the next section, students in some Chinese provinces can apply for universities after they know their National College Entrance Examination scores, whereas students in most provinces have to apply for universities before they take the national examination, basing their application choices on their “predicted scores”. For this study, we assume that students’ performances in the national exam demonstrate the true quality of their studies, as the matching efficiency and contingencies of students’ performances are beyond the scope of this paper.

2.2. Impacts of the Admission System in China
Even top students can fail to be admitted if they do not wisely choose a university that matches their National College Entrance Examination scores. Although it is impractical for Chinese students to apply for more than one university in the same tier, top students will usually not accept admission to universities in the second tier or lower. Because the National College Entrance Examination in China is held only once a year, some students who fail to secure an admission have to spend another year in
4

middle school waiting to take the next examination, while those who cannot afford another year in middle school are forced to terminate their studies to seek work in a low-paying job, where they will probably stay indefinitely, if not for all of their working life. Moreover, some students receive bonus points from governmental, regional or some specific universities’ preferential policies. Without this advantage, students have an increased risk of failure. Therefore, to some extent the university application process in Mainland China resembles a game. According to China Youth Daily, the percentage of students who receive bonus points varies from one province to another. In 2010, this number ranged from 0.13% to 17.19%. Besides, some provinces do not publish the exact number of students with bonus points. Unfortunately, related data on the population of students with bonus points are currently not available. Neither is the effect of whether the bonus point policy has a marked impact on the rank effect, which is beyond the realm of this study.

2.3. Admission System in China and Representative Anglophone Countries
The examination regulations, admission systems and procedures in the United States and British are completely different from those in China. Firstly, the Scholastic Aptitude Test, a United States counterpart to the National College Entrance Examination, can be taken several times before applying for universities. The score remains valid for several years, and colleges have an open admission policy (Manski & Wise, 1983; Curs & Singell, 2002). Also, the complete admission process in the United States can be treated as a four-stage university application system: 1) students apply and send their SAT scores along with other applications materials to more than one school; 2) admission decisions are released; 3) financial aid decisions from institutions (which is not applicable in China before enrollment); 4) possible enrollment if accepted. In China’s closed and uniform admission system, it is impossible for students to be admitted by several universities. Once they have filled in their university preference sheets, Chinese university applicants have no alternatives.

3. Estimation Strategies
3.1. Estimation Method
� ����,�� = ��1 ����,��−1 + ��2 ����������,�� + ��3,�� ���������� + ��4,�� ���������� + ��5,�� ������ + ��7,�� ���� + ��8,�� ���� + ����,�� + constant

Q s,t : Average provincial rankings of the students admitted by the University s in year t Rank s,t : Rank of University s in year t names : Fixed effect of the University s fromm : Fixed effect of the students' home province m yrt : Fixed effect of the year t wlt : Fixed effect of the students' major stream pc: Fixed effect of the admission tiers es,t : Residuals

We assumed that there is an oversupply of university applicants in China every year
5

relative to the limited quotas of universities, and that people do not care much about the yearly methodology changes of university rankings (Ehrenberg, 2003). To estimate the effect of exogenous variation in university rankings on the quality of the university’s successful applicants, we employed the Arellano-Bond method. This identification strategy is superior to ordinary least squares, fix effect Prais-Winsten regression, and Corcharne-Orcutt regression (AR1) since it splits out any potential correlation between a university’s rank over two consecutive years and sweeps the heterogeneity (Greene, 2002). Five groups of dummies were set to observe the endogenous properties of different universities. The university dummy shows the endogenous properties of a university independent of its ranking. The location dummy controls the regional differences, such as policies in student enrollment, the regional economy gap, to name just a few. The year dummy controls the changes in ranking measurements used for university ranking reports and the fluctuations in examination paper difficulty in different years. The major stream dummy identifies preferences of university applicants from different major fields of study. The estimation equation above 2 is the most generalized form of all models that we selected to test the rank effect in Mainland China. Note that we mainly adopted the average rankings of admitted candidates as the measurement of students’ quality. The main independent variable is the ranking of the university.

3.2. Measurement of Students’ Quality
We used the average provincial rankings of the admitted students rather than admission cut-off lines to measure the quality of students that a university can enroll, for several reasons. Firstly, we measure the quality of students by their average rankings in the national college entrance examination because admission scores in most Chinese provinces are not standardized, raw scores. The difficulty of the examination papers fluctuates yearly. Secondly, provincial ranking is a good measure of admitted students from different areas. The admission scores of the universities vary between all provinces because examination papers differ depending on location, and provinces have different quotas for the number of students that can be admitted. From 2004 to 2009, there were around three to four versions of “national papers” made by nation-level educational agencies and several “regional papers” made by provincial-level educational agencies. Finally, average provincial rankings of students admitted best describe the rank effect out of other possible indicators of students’ quality, such as cut-off lines in different estimation methods. The regression results of other indicators are available on request.
2

Provincial admission quotas in current year and last year are statistically insignificant and are dropped in the estimation equation. One may suggest a dummy for a university’s tuition fee in the estimation equation as van der Klaauw (2002), Griffith (2007) and Curs (2002) have independently studied students’ financial aid and net-tuition elasticity. A common issue faced by a student is whether he or she can afford the tuition fee for a Chinese university. However, most of the universities in China are public schools which are highly regulated and mainly subsidized by the government, and therefore have less flexibility to adjust net tuition. The tuition fee gap across universities in China is minor.
6

In addition, it is necessary to consider the quality of students according to their stream in our study of the rank effect. Students in most provinces specialize in the science or liberal arts stream before they enter senior middle school. The subject examinations students take depends on the stream they chose to study. Because most universities’ research capacities in the science field and the liberal arts field are not balanced, the major stream dummy is indispensable for a general rank of the ranking reports in the regression equation. 3

3.3. Selection of College Ranking Reports
Detailed information on examination papers across Mainland China from 2004-2009 (Some provinces may employ national paper for a certain subject. Others may use different versions of national papers for different subjects. Such minor information is currently unavailable.) Examination subjects are: Chinese, English, Math, and the Science Combo (Physics, Chemistry, and Biology) or the Liberal Arts Combo (History, Geography, and Politics). There are three or four versions of these National papers: [N1] [N2] [N3] [N4]. Several regional examination papers are made by individual provinces or province-level cities. Some provincial-level educational agencies make versions of the Chinese, English, Math, and the Science Combo (Physics, Chemistry, Biology) or the Liberal Arts Combo (History, Geography, Politics)) [R1], while others make papers without the stream combos (Chinese, English, Math, plus one, i.e. “3+X”, or all of the mentioned subjects, including Physics, Chemistry, Biology, History, Geography, Politics) [R2].  National papers [N1] [N2] [N3] [N4] are mainly for the following provinces or province-level cities (Further provincial level information is omitted.) Throughout 2004-2009 Gansu, Guangxi, Guizhou, Hebei, Inner Mongolia, Jilin, Qinghai, Shanxi, Tibet, Xinjiang, Yunnan 2004-2006 Hainan, Ningxia 2004-2005 Anhui, Shaanxi, Sichuan 2004 Jiangxi, Shandong 2004-2007, 2009 Heilongjiang  Regional papers [R1] are mainly for the following provinces or province-level cities. (Further provincial level information is omitted.) Throughout 2004-2009 Beijing, Chongqing, Fujian, Hebei, Hunan, Shanghai, Tianjin, Zhejiang 2005-2009 Jiangsu, Jiangxi, Shandong 2006-2009 Anhui, Guangdong, Shaanxi, Sichuan 2007-2009 Hainan, Ningxia 2004, 2006-2009 Liaoning 2008 Heilongjiang  Regional papers [R2] are mainly for the following provinces or province-level cities. (Further provincial level information is omitted.) Throughout 2004-2009 Guangdong 2004-2007, 2009 Jiangsu 2005 Liaoning
7
3

Four different college ranking reports from four different institutions are adopted separately in our regression. They are "A Report on the Competitiveness Evaluation of Universities and Subjects in China" (CEUSC) by Mr. Shulian Wu, a ranking report from "netbig.com" (netbig), "Competitiveness Ranking of Chinese Universities" (CRCU), and the "Chinese University Evaluation Report" (CUER). All of them categorize ranking measures into input, process, and output, which coincide with the definition of a comprehensive organizational report card theorized by Pascarella (2001) and Gormley (1999).

3.3.1.

Evaluation Method of the CEUSC

"A Report on the Competitiveness Evaluation of Universities and Subjects in China" (Zhongguo Daxue Pingjia) by Mr. Shulian Wu is one of the most popular online Chinese ranking reports; it is the first result when using “University ranking in China” (in Chinese) as a search phrase on Google. According to the publications of the ranking institution, “A Report on the Competitiveness Evaluation of Universities and Subjects in China” has been released on February in most years since 1997. This is a few months before the National College Entrance Examination. Every year, the ranking report follows similar ranking criteria. As shown in Figure 2, a university and its students are scored by detailed D-level evaluation categories. The final rank measures the overall quality of a university based on factors such as its undergraduate and postgraduate training capacity, and research capacity in the science and liberal arts streams. Scores of A-level, B-level and C-level evaluation categories are released in the ranking report. Overall rank (the A-level evaluation category ranking), is explicated, published and highlighted in the ranking report. We can also rank all schools by their scores in B-level and C-level evaluation categories and observe the ranks in a narrowed criterion. Generally, Ak , Bk and Ck are the published ranking components of school k in the report. R1, R2, R3, R4 and R5 with dotted lines in Figure 2, are coefficients for each detailed D-level evaluation category with solid lines. Formulas for published ranking components are:
Ak = �� ���1 ��1�� �� ∑��=1 ��1�� + ��2 ��2�� � , ��1 �� ∑��=1 ��2�� + ��2 = 1, �� = 1,2, … ��

W is the sum of all ranked universities’ scores ��1 and ��2 are the weights of student training and the research standard corresponding to their ratio in the national education input. 4

Factors denoted by ��’s below are also weights that are set by the ranking institution.
B1k = �� ���3
C1k = ���5 ∑�� 1

B2k = �� ���8

The following R coefficient measures the outcome of students after graduation. It assumes that fresh undergraduate students from better universities have higher
4

V1 is the score of an undergraduate student after graduation.

4 ��=1 ��1 ��1�� ∏��=1 ������

�� ��1�� ∏4 ��1�� ��=1

��1�� �� ∑��=1 ��1��

��3�� ��4�� + ��9 �� � , ��8 + ��9 = ��2 , �� = 1,2, … �� ∑�� ��3�� ∑��=1 ��4�� ��=1
+ ��6 ∑��
��=1 ��2��

+ ��4

��2�� � , ��3 �� ∑��=1 ��2��
��2��

+ ��7 ∑��

+ ��4 = ��1 , �� = 1,2, … ��
10 ��=1 ∑��=3 ������

∑10 ������ ��=3

� ∑�� ��1�� ��=1

Factors denoted by an “��” vary from year to year, and correspond to the input and distribution of educational resources. This information is included in the year dummy of the estimation equation.
8

employment rates compared to the social employment rate. Evaluation categories like this are in line with students’ main concerns regarding school applications.
P is the duration of the underrate study program. P=4 �� is the social employment rate r1k is the employment rate of fresh undergraduate students in school k

R1k =

P r1k �� + ��(1 − ��1�� )(�� + .25) + ��(1 − ��)(1 − ��1�� )(�� + .5) + (1 − ��)2 (1 − ��1�� )(�� + 1) ⎛ ∑�� ⎜ ��=1 ���������������� + ∑�� ���������������� ��=1 � � ∑�� �������� ��=1 ⎝ ⎞ ���������������� �� ∑��=1 ���������������� ⎟ � � ∑�� �������� ��=1 ⎠

���������������� is the science stream average admission score 5 in province i ���������������� is the liberal art stream average admission score in province i 1 ���� ����ℎ������ �� �������������� �������������� ������������ ���������������� ���� ���������������� �� �������� = � 0���� ����ℎ������ �� �������� ������ ������������ �������������� ������������ ���������������� ���� ���������������� �� 1 ���� ����ℎ������ �� �������������� �������������� ������ ������������ ���������������� ���� ���������������� �� �������� = � 0���� ����ℎ������ �� �������� ������ ������������ �������������� ������ ������������ ���������������� ���� ���������������� ��

R 2k =

∑�� (�������� + �������� ) ��=1

R 3k

A brief summary of graduate school and research capacity evaluations are shown below for completeness.
V2 is the score of a postgraduate student V3 is the score of a PhD student C3k = ��6�� (∑21 ������ + ��23�� ) ��=16

C2k = [����5�� + (1 − ��)]��2 ��11�� + ��5�� ��3 ��12�� + ��13�� + ��14�� + ��15��

1.1, �� ���������� ����ℎ������ ���� �������������������������� ��������ℎ������ �������������������� ⎧1.05, �� ���������� ����ℎ������ ���� �������������������������� ��������ℎ������ �������������������� ⎪ = 1.00, �� ���������� ����ℎ������ ���� �������������������������� ��������ℎ������ �������������������� ⎨0.6, �� ���������� ����ℎ������ ���� �������������������������� ��������ℎ������ �������������������� ⎪ ⎩1.00, ������ ������������������ ���� �������������������������� ��������ℎ������ ��������������������

3.3.2.

A Transverse Comparison of Ranking Reports

The ranking report from "netbig.com", supported by the “netbig” company, is free online. It has ranked universities since 1999. Detailed scores of different quality-related categories are provided. The final rank is based on the total score of all quality categories, which include the general score, the reputation score, the faculty resource score and the faculty score. The "Competitiveness Ranking of Chinese Universities" is provided by the Research Center for Chinese Science Evaluation at Wuhan University. This ranking report is published online every year during July and August; a few months after the examination. The report ranks universities respectively in both science and liberal arts streams. Unfortunately, only rankings for the top 30 schools are published yearly. The "Chinese University Evaluation Report" (Zhongguo Daxue Pingjia Yanjiu Baogao) is published by "The Chinese University Alumni Association", a private institution. This ranking report has been released every year in December, since 2003.
5

One may argue that students’ admission scores are included in the ranking reports. Our reasoning is that this component measures the quality of students who were enrolled last year, so there is no need to include last year’s dependent variable (the ranking of admitted students) in the estimation equation. However, the regression results display that last year’s dependent variable, as well as university rankings, are still a main concern for university applicants.
9

Public universities, private universities and colleges are ranked separately. According to the results from the survey on university ranking reports showed in Table 1, 16.22% of the respondents have heard of "A Report on the Competitiveness Evaluation of Universities and Subjects in China". It is the most popular of the four reports mentioned above. The regression results from Table 3 also indicate the superiority of "A Report on the Competitiveness Evaluation of Universities and Subjects in China" over the other three ranking reports. The superiority of "A Report on the Competitiveness Evaluation of Universities and Subjects in China" (CEUSC) may be interpreted as follows. It appears that the report is obtainable by most students as it is publicized and promoted by popular media. Apart from that, the internet-based ranking information can be easily found through available search engines, and meets the specific needs and expectations of consumers, which may improve comprehensibility (Dill & Soo, 2005; Pascarella, 2001; Gormley & Weimer, 1999). Although more than half of the online survey respondents cannot remember which university ranking report they referred to, school ranking is one of the main concerns in their university applications and so respondents’ university choices are likely influenced by the CEUSC rankings. Regression results of the other three ranking reports are available, but not displayed in this paper.

3.4. Construction of Variables and Regression Models and Samples
As the main ranking report is released a few months before the examination, the current year’s university rankings have a direct impact on the quality of the choices made by this year’s applicants. We use the current ranking of universities rather than the previous year’s rank as the independent variable. We found that unlike the majority of institutions, universities that enroll students from fewer than 19 provinces in China do not fit our models (See column (6) of Table 6). A possible explanation is that Chinese students applying for these schools may care less about the rankings as they have already applied to other schools. In addition, the enrollment policies of those universities may be different from those of most other universities.

4. Results and Implications
4.1. Estimation Results and Implications
In reference to column (3) of Table 6, moving up one rank in the ranking report of a certain year raises the average ranking of the students admitted by a university by around 19 for that year. This rank effect uses the evaluation criteria of the respective ranking report to measure the overall quality of the universities, including the outcome of students after graduation, which is indicated by one of the R coefficients. In addition, this result was obtained after we ruled out the following factors. Firstly, the educational resources, examination evaluation system and application system all vary between provinces. For example, Chinese students from different provinces use different textbooks and take different examination papers, the difficulty of which fluctuates every year.

10

Another factor is whether students choose universities prior to taking the National College Entrance Examination. Students in some provinces choose universities after they know their examination results, whereas students in other provinces choose universities a few days after taking the examination, without knowing their final scores. As mentioned previously, the rank effect is obtained after we control endogenous properties for different universities, including the change of ranking measurements in university ranking reports, different regional enrollment policies, existing regional economic development gaps, halo effects (Clarke, 2002), and the existing reputations of the universities, which do not necessarily indicate teaching quality (Terenzini & Pascarella, 1994). When we compare the results in column (3), column (4) and column (5) of Table 6, we find that ranking reports are more important in guiding students’ choices of lower-ranked schools instead of the top 19 schools. This discovery is inconsistent with the findings of counterpart studies conducted in the United States and Britain (Bowman & Bastedo, 2008; Monks & Ehrenberg, 1999). Consequently, we prefer to discuss the ranking effect of all schools except for the top 19 schools rather than solely the top 25 schools (a school grouping pattern adopted by Meredith (2004) and Luca (2009)). This method would make our conclusions as general as possible, and bypasses the “group jumping problem”. From 2004-2009, all top 19 universities in the ranking report remained in this category; a similar pattern was showed in the U.S. News and World Report rankings (Bowman & Bastedo, 2008). Hence, the “sheepskin effect” experienced by institutions that move in or out of the top 19 universities, which was mentioned by Meredith (2004) and Monks (1999), is not applicable to the study of university league tables in China. Regressions on multi-grouped universities are displayed in Table 7. The first two columns of Table 7 highlight a critical issue faced when grouping the sample universities by their ranking intervals. If the sample universities within a ranking interval vary every year due to rank ‘jumping’, the results of the rank effect may be spurious and unexpected. Therefore, it is better to regress without having those universities jump ranking intervals every year. The recommended university samples are those marked in columns (2) to (6) in Table 7. Both Table 6 and Table 7 indicate that the rank effect dominates when students choose lower-ranked schools. One reason for this finding is that it is more costly to gather comparative information (Sauder & Lancaster, 2006) about schools with a lower reputation, so students that choose among lower-ranked schools rely on the results of ranking reports. To put it another way, the results of regressions on the top 19 schools are not as good as either the regression results for all schools or those of the lower-ranked schools from the ranking report. Another possible reason is that most students applying for the top 19 schools give more consideration to major-level rankings rather than the general ranking, which aggregate all categorized ranking elements into a single index (Ehrenberg, 2003).

11

Moreover, some top universities like Zhejiang University and Jilin University are forced by the government to merge with lower-ranking local universities. The high ranking or quick rank increases experienced in the short term by the newly merged institution mostly results from the expansion of scale rather than any improvement in academic quality. Furthermore, the average quality of students enrolled in these merged universities is inferior to the average quality before the merger, because a significant part of students are actually enrolled in the “branch schools” of the merged universities i.e. departments run by the low ranking local universities or institutions. The application system in China and the huge imbalance created by the large number of applicants that face limited admission quotas may also contribute to this phenomenon. Students that cannot be admitted to their first choice university (usually a top 19 university) will not have a second chance to be admitted to another top university. Therefore, students may not care about the ranking positions of all top 19 schools because they can only effectively apply for one of them. Hence, tiny ranking changes of the top 19 schools every year will not have an immediate impact on students’ application strategies or on the quality of students admitted into the top 19 schools. A comparison between Table 6, Table 8, Table 9 and Table 10 demonstrates that people who consult ranking reports care more about the overall ranking or exogenous reputation of the university, rather than its rank in a narrower criterion. In Table 6 and Table 8, the quality of students from the both science and liberal arts stream fit the fix effect model and the Arellano-Bond method model, especially when we regress on the ordinal ranking of the universities’ undergraduate schools in Table 8. Because most universities’ research capacities for the science field differ from that of the liberal arts field, the quality of students from different streams should theoretically fit the overall rank in a different pattern. In China, majors related to science and engineering are better developed than majors in the liberal arts field, so universities that are stronger in science and engineering tend to be ranked higher in the general ranking category. Also, separate ranks in purely science or liberal arts research are available in the ranking report. Although it is reasonable to assume that students from different streams tend to use the relevant stream ranking of a university to choose schools, we found that a university’s rank in liberal art research doesn’t significantly affect the choices of liberal arts stream students (See Table 10). The four tables mentioned above shed light on the essence of the rank effect. Van der Klaauw (2002) analyses students’ enrollment decisions by comparing the student’s utility associated with each choice alternative, which can be applied when Chinese students fill in the university preference sheets. The costs that affect utility include living expenses, tuition, loss of vested social relations, the risk of not completing a difficult undergraduate program, and opportunity costs of the alternatives. The benefits are reputation among friends, teachers and relatives, the influence of peer interactions on the college learning process (Pascarella & Terenzini, 1991), potential earnings after graduation (Cannings, Mahseredjian, & Montmarquette, 2002), and positive expectations of a better-than-average post-graduation outcome (Allen & Burgess, 2010). At times, rankings are based on criteria that only have minimal relevance or even have a negative correlation to what we know about the impact of colleges on students and are therefore largely invalid from a research point of view (Pascarella, 2001; Astin, 1999; Terenzini & Pascarella, 1994; Dearden, Ferri, &
12

Meghir, 2002). However, people still keep in mind the positions of universities in the general rankings, and highly ranked universities gain social fame (Jin & Whalley, 2008). University league report users who desire a better reputation are prone to make decisions according to the general rankings, which are a simple aggregate of all ranking elements, and these may potentially have statistical inaccuracies or are based on flawed methodology (Bowden, 2000; Dill & Soo, 2005). Deep down, the paradox mentioned above is not entirely groundless. As university ranks, considered as signaling in the context of asymmetric information, are closely followed and employed by the general public, most successful applicants for top ranking schools are carefully selected from extremely competitive pool of students. This way, we may reason, going adrift with the stream and blindly applying for a higher ranked university cannot be laid on thick. Table 11 and Table 12 indicate that the rank effect coefficient doesn’t change much if we exclude local student enrollments. Slightly changes in the ranking effect and large gaps in local and non-local student enrollment (See Table 5) prove that non-local students rely on public information such as ranking reports when choosing universities outside of their hometown. Furthermore, mobility is not a major concern when Chinese students apply for universities. Similarly, in a comparison using findings in Italy and in the United States by Sartarelli (2011), the relationship between students’ university choices and mobility is significant between regions in countries with large geographical areas but is insignificant in Italy (Sartarelli, 2011; Curs & Singell, 2002). In addition, other factors such as the influence of regional policies, family background, sorting bias for university local applicants and a community environment that is more favorable to local students have limited impact on the ranking effect nationwide.

4.2. Robustness Checks
Using the Arellano-Bond method to check the robustness of the rank effect in China, we find that the ranking results are even more statistically significant when we do regression with last year’s admission scores in the estimation equation (See Table 6, Table 8, Table 11 and Table 12). In this case, the fixed effect model is also acceptable. The Arellano-Bond method illustrates that when predicting students’ choices, the current year’s university ordinal ranking changes are as reliable as students’ rankings and admission scores from the previous year. Furthermore, the equation may be piecewise defined or valid for a certain group of universities, if applicants treat the ranking of top schools and the rank of non-top schools separately. Columns (3) and (5) of Table 6 and Table 8 demonstrate a significant rank effect on lower-ranked schools. Later studies on the rank effect should focus on “sheepskin effects” after the completion of college (Heckman, Layne-Farrar, & Todd, 1995) and the possible nonlinearity of school ranking on students’ choices. When examining the last column of Table 6, Table 8, Table 9, Table 10 and Table 11, we observed that in China, the model is not illustrative enough to describe the ranking effect of universities that enroll students from fewer than 19 provinces.

13

Table 13 aims to check the causal relationship between students’ choices and the ranking results in the report. If a university only recruited local students, the previous year’s admission scores and students’ rankings are more illustrative than the ranking effect. In fact, the non-local students consider public information such as ranking reports as their main guide for applying to universities in other provinces. This behavior coincides with the findings from Table 11 and Table 12.

5. Ranking Report Revisit
College rankings in the United States date back to 1870, when the Bureau of Education rank-ordered universities based on statistical information (Meredith, 2004). There are mainly four parties involved with such reports: universities, government, students and the public. With few laissez-faire policies on the availability of university performance in China, universities and governments share the same interests in the development of teaching quality, and treat the rankings in a similar manner.

5.1. From the Perspective of Universities and the Government
Rankings help inform universities and policymakers of any possible improvements in teaching quality. As the public and funding institutions pay more attention to higher-ranked universities (Jin & Whalley, 2008), schools that consistently ranked highly over a number of years may find themselves in a virtuous cycle, continually receiving more financial resources per capita. However, colleges may feel pressured to manipulate the rankings by improving student selectivity in the short run (Ehrenberg, 2003) rather than investing in the long run teaching quality, due to the inherent pitfall of the ranking result, a single aggregate index.

5.2. From the Perspective of Students
Appropriate ranking reports facilitate the comparison and analysis of universities in a shared way for students, which promote university competition (Department for Education and Skills, 2003) and academic quality. If there is a well-designed, trusted, and public-accessible college league table, it is harder for admission officers from different universities to fool applicants using false advertising. As the principal users of the ranking reports, students are not likely to make application decisions independent of others’ viewpoints. They often attempt to secure admission to the universities with great names, top ranks and good public reputations. A critical misinterpretation of the ranking reports may distort their school applications, and this will be discussed further in the next section. Students cannot be blamed for applying to schools with high overall rankings, because people tend to believe that highly ranked schools provide a better learning experience. However, Terenzini (1994) shows that the graduates who have completed four years of undergraduate study at some colleges have only reached a level of achievement or development approximately equal to freshmen entering other institutions. A well-known example frequently quoted by numerous literatures is that research capacity, one of the key overall ranking elements, may not necessarily indicate a university’s teaching capacity. How can a university full of reputable experts in frontline research contribute to your
14

future? Firstly, these experts may be busy doing hands-on research and spend little effort or time teaching fundamental courses. Also, they will not be of much help in graduate study if your interests are not in their field of research.

5.3. The So-called General Public’s Misinterpretation and the Aftermath
Employers and head-hunters may prefer graduates from highly ranked universities due to their belief that these graduates were outstanding compared to other applicants prior to entering these so-called top schools. No wonder that graduates from the most selective institutions usually earn handsome salaries, a topic that is well studied by Dill (2005). However, the aggregate earning-quality relationship is weak for regular universities in the United States, England and Wales (Heckman, Layne-Farrar, & Todd, 1995; Dearden, Ferri, & Meghir, 2002). In fact, school input-type variables that are ranked in the report do not clearly indicate the probability of employment after obtaining qualifications. Moreover, student selectivity, another ranking element, is based on a small sample of students’ quality and thus may be of little value to the majority of university applicants. In other words, valid university report cards are not emphasized in most of the league tables.

5.4. What Are Characteristics of a Meaningful Ranking Report?
A well-designed radar chart with information that targets major concerns in each vertex may be of great use for both academic- and career-oriented students (See Figure 3). Ranking elements may or may not relevant to students’ future. However, such presentation helps students, as consumers, to choose according to their own circumstances. A good ranking report should never aggregate the overall school quality, academic competiveness, research capacity, teaching capacity, actual knowledge of different majors, employment, earnings, financial resources, etc. into a single-index ranking table, because this collation encourages readers to focus on the ranking results in a narrow sense. The definition of top-ranking universities differs depending on the future career plans of students. Furthermore, the teaching quality feedback and other comments from full-time university students enrolled cannot be neglected. Otherwise, universities may treat student admission as a one shot deal, and this may lead to the moral hazard problem after enrollment. Last but not the least; government can help improve the quality of ranking information available to both students and universities.

6. Conclusion
In this paper, we estimated the ranking effect in China, a counterpart study of the ranking effect in the United States (Luca & Smith, 2009). There are six major findings of this study:

15

Firstly, a university’s rank in ranking reports is a major index used by Chinese students to judge the faculty resources and teaching quality of the university. Considering the vast number of applicants in China every year, it is assumed that there are sufficiently large numbers of students whose scores meet the admission cut-off lines of all universities relative to the limited quotas of universities. Moving a university forward by one rank will lead to an increase in the average rank of admitted students by 17 positions or more, which is approximately 21.0% of the nation-wide student enrollment population. In the United States, an exogenous improvement of one rank leads to a 0.9% increase in applications. Our study demonstrates the fact that university league tables, which compare the performance of different institutions, are a potentially efficient and effective means of providing necessary information to student consumers (Dill & Soo, 2005). Secondly, although a more germane but narrowed ranking of universities is approachable; students mainly care about the overall ranking of universities and their undergraduate schools. It makes sense that ordinal ranking of undergraduate schools better fit the model than the overall rankings, in particular for the ranking of the top 19 schools, because overall rankings include the rankings of graduate schools, a factor which is irrelevant for students who are applying for undergraduate schools. However, the notable regression results in overall rankings and less significant results for narrowed rankings show people’s interests when they refer to ranking reports. Thirdly, applicants are rational when following undergraduate school rankings. In Table 8, the rank effect can illustrate the change of the admission cut-off lines of universities that enroll students from fewer than 19 provinces. Also, science stream students and liberal arts stream students in Mainland China choose universities according to their general ranks, in a similar pattern. Weak regression results in Table 9 and Table 10 indicate people’s bias towards the overall ranking of a university or its undergraduate school. In addition, the rank effect is more evident in the choices of lower-ranked universities, which highlights the relative importance of the cost of information collection for general universities. Besides, it is possible to say that people are more interested in the ranking of different majors when they apply for the top 19 schools. Meanwhile, the merging and reform of universities such as Zhejiang University and Jilin University would greatly affect the regression result. The results of regressions without these merged universities are not attached here. Comparatively, Luca (2009) shows a statistically significant rank effect involving the top 25 universities in the United States. Lastly, it is safe to say that the rank effect of the lower-ranked universities that enroll students from more than 19 provinces is quite similar across enrollment regions. Universities’ ranks are the main factor considered when students apply for universities outside their home provinces. This result is independent of the various differences in educational resources, examination application systems and the evaluation systems of these provinces. Our study suggests that private information on the quality of universities is limited. Hence, providing such information to students may improve the school-student
16

matching procedure and raise the return of human capital investment in China. Our study also measures the return of information provision of ranking reports, if there are no other, more improved indicators. In addition to university rankings, others’ opinions, conformity, and propaganda from various types of sources may affect students’ choices of universities. The bias of our estimates may be unavoidable. It is reasonable to consider public information as one of the components in the rank effect, since the ranking report will include all available nationwide information on these universities, in addition to other private or unquantifiable factors.

17

References
Allen, R., & Burgess, S. (2010). Evaluating the Provision of School Performance Information for School Choice. The Centre for Market and Public Organisation, Department of Economics, University of Bristol, UK. Astin, A. W. (1999). Involvement in Learning revisited: Lessons we have learned. Journal of College Student Development, 40(5), p. 587. Bowden, R. (2000). Fantasy Higher Education: university and college league tables. Quality in Higher Education, 6(1), pp. 41-60. Bowman, N. A., & Bastedo, M. N. (2008). Getting on the Front Page: Organizational Reputation, Status Signals, and the Impact of U.S. News and World Report on Student Decisions. Research in Higher Education, 5(5). Cannings, K., Mahseredjian, S., & Montmarquette, C. (2002). How do young people choose? Economics of Education Review, 21(6), pp. 543-556. Clarke, M. (2002). Some Guidelines for Academic Quality Rankings. Higher Education in Europe, 27(4), pp. 443-459. Curs, B., & Singell, L. D. (2002). An analysis of the application and enrollment processes for in-state and out-of-state students at a large public university. Economics of Education Review, 21(2), pp. 111-124. Dearden, L., Ferri, J., & Meghir, C. (2002). The Effect of School Quality on Educational Attainment and Wages. The Review of Economics and Statistics, 1, pp. 1-20. Department for Education and Skills. (2003). The Future of Higher Education. London: Her Majesty's Stationery Office. Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy:A cross-national analysis of university ranking systems. Higher Education, 49, pp. 495-533. Ehrenberg, R. G. (2003). Method or Madness? Inside the USNWR College Rankings. Wisconsin Center for the Advancement of Postsecondary Education Forum on The Abuse of College Rankings,. Madison. Gormley, W. T., & Weimer, D. L. (1999). Organizational Report Cards. Cambridge, Mass: Harvard University Press. Greene, W. H. (2002). Econometric Analysis (5th ed.). Prentice Hall. Griffith, Amanda, & Rask, K. (2007). The Influence of the U.S. News and World Report Collegiate Rankings on the Matriculation Decision of High-Ability Students: 1995-2004. Economics of Education Review, 26.

Heckman, J., Layne-Farrar, A., & Todd, P. (1995). Does Measured School Quality Really Matter? An Examination of the Earnings-Quality Relationship. National Bureau of Economic Research. Jin, Z., & Whalley, A. (2008). The Power of Attention: Do Rankings Affect the Financial Resources of Public Colleges? National Bureau of Economic Research. Lee, J.-W., & Barro, R. J. (2001). Schooling Quality in a Cross Section of Countries. Economica, 68(271), pp. 465-488. Luca, M., & Smith, J. (2009). Why is First Best? Responses to Information Aggregation in the US News College Rankings. Manski, C. F., & Wise, D. A. (1983). College Choice in America. Cambridge, Massachusetts, United States: Harvard University Press. Meredith, M. (2004). Why do universities compete in the ratings game? An Empirical Analysis of the Effects of the USNWR College Rankings. Research in Higher Education, 45. Monks, J., & Ehrenberg, R. G. (1999). U.S. News and World Reports College Rankings: Why Do They Matter? Change, 31(6), pp. 42-51. Pascarella, E. T. (2001). Identifying excellence in undergraduate education: Are we even close? Change, 33(3), pp. 19-23. Pascarella, E. T., & Terenzini, P. T. (1991). How College Affects Students: Findings and Insights from Twenty Years of Research. San Francisco, California, United States: Jossey-Bass. Sartarelli, M. (2011). Compulsory and College Education: A Non-Exhaustive Survey of Schooling Choices and Related Outcomes. European University Institute mimeo. Sartarelli, M. (2011). Students' Revealed Preferences and College Quality. Evidence from College Choice in Italy. European University Institute. Sauder, M., & Lancaster, R. (2006). Do rankings matter? The effects of U.S. News & World Report rankings on the admissions process of law schools. Law and Society Review, 40(1), pp. 105-134. Terenzini, P. T., & Pascarella, E. T. (1994). Living with myths. Change, 26(1), pp. 28-32. van der Klaauw, W. (2002). Estimating the Effect of Financial Aid Offers on College Enrollment: A Regression-Discontinuity Approach. International Economic Review, 43(4), pp. 1249-1287. World Bank. (2005). Higher education in developing countries: Peril and promise. Wu, S., & Lv, J. (2010). A Report on the Competitiveness Evaluation of Universities and Subjects in China. Science of Science and Management of S.& T., pp. 5-13.

Appendix Tables
Table 1 The Result of the Online Survey
1. Of
6

Which the following university ranking report(s) for Chinese Universities have you heard of?  “A Report on the Competitiveness Evaluation of Universities and Subjects in China”  “Competitiveness Ranking of Chinese Universities”  Ranking report from "netbig.com"  “Chinese University Evaluation Report”  Not sure, I am aware of the ranking report from the Internet.  Not sure, I am aware of the ranking report from newspapers and the TV news.  I’m not interested in the ranking report.  Others
2. What

% of Respondents 16.22% 8.11% 2.70% 0.00% 35.14% 29.73% 2.70% 5.41% % of Respondents 11.73% 21.60% 38.27% 25.93% 2.47%

impact does the ranking of Chinese universities have on your final choice?  No impact  Little impact  Moderate impact  Considerable impact  An enormous impact
3. How

are the following factors relevant to your choice of a university when you are in the senior middle school? (Importance index 1 refers to the most important one of the following factors. 5 refers to the least important one of all.)
Importance index  Location of the University  General rank of the University  Major-level rank of the University  Others’ opinions  Faculty resources and teaching quality of the University 1 2 3
% of Respondents

4

5

Weighted average 7

26% 34% 38% 11% 40%

27% 30% 25% 16% 19%

20% 20% 18% 25% 16%

14% 9% 10% 20% 12%

10% 5% 7% 25% 10%

2.4600 2.1500 2.1700 3.2300 2.2400

6

This survey (in Chinese) is available on http://www.eSurveysPro.com/Survey.aspx?id=d8aca717-df72-4ba6-a569-fa3f0f590b38. The number of respondents totaled 163 by March 18, 2011, 97% of whom are university students in Mainland China. Respondents’ profile is available in Figure 1. 7 The sum of the percentage of respondents multiplied by the corresponding importance index

Table 2 Example of Multi-tier admission procedure in Province K

8

Students’ scores and a summary of their preference sheets
Student ID Total Score

Pre tier
Preferred University SMA Major 1 Major Preferred University 1 2

First tier
SMA

First tier
Major 2 Preferred University 2 SMA Major 1

Second tier
Major Preferred 2 University 1 SMA

Second tier
Major Preferred University 2 2 SMA Major 1 Major 2

22 30 21 12 23 4 3 20 34 31 11 10 1 7 33 14 9 2 24 8 28 16 35 5 25 18 29 15 19 13 6 26 27 17 32

715 663 636 624 608 596 595 569 557 539 528 518 478 469 410 409 408 371 360 282 274 273 244 239 229 215 211 201 196 155 150 109 50 38 29

ABC ABC ABC ABC ABC ABC ABC ABC

× √ × √ × √ √ × × × √ √ ×

Z X X X Z X X X

Z Z X

ABC BCD BCD BCD ABC ABC BCD BCD BCD BCD ABC ABC ABC ABC BCD

× √ √ × × × × × × × × √ × √ × √ × √ √ × × × √ × √ √ √ √

Major 1

Y B X B Y Z B B B X Z Y Y Z B B B

Z B Z X BCD

BCD BCD BCD CDE CDE

× × × √ × × × √ × √ × × √ √ √ × × √ √ √ √ × √ × √ √ √

Major 1

A A A A C CDE CDE C A BCD

√

× √ × √ √ × √ √

C C

A A

X

A

X Y Z Y X

ABC BCD BCD

× × ×

Y X X

CDE BCD B CDE BCD CDE BCD BCD CDE CDE BCD BCD BCD BCD BCD BCD BCD BCD BCD BCD CDE BCD BCD

A A C A C A A C C A A A A A A A A A A C A A

BCD

A

ABC ABC ABC ABC ABC

Z Z X Z Z

BCD CDE

A C

A

X BCD BCD

CDE CDE

C A

A C

X

ABC

× × × ×

Z

X

BCD ABC BCD BCD BCD BCD ABC BCD BCD ABC BCD

X Z B X B X Y X X Z X

B Y B

ABC

√

Y

CDE CDE

√ ×

A C

A

ABC ABC ABC

Z X X

CDE CDE A BCD

√ × √ √

A C A

A

ABC B Y B BCD

√ √

Z B

ABC

×

X

CDE

C

A

Based on the assumption that only 3 universities enroll students in Province K
Admission tier Pre tier Pre tier First tier First tier First tier First tier Second tier Second tier Second tier Univers ity ABC ABC ABC ABC BCD BCD BCD CDE CDE Major X Z Y Z B X A A C Quota in Province K 4 students 2 students 5 students 3 students 1 students 2 students 3 students 2 students 10 students 21 30 22 4 34 11 9 31 24 12 3 1 10 FULL 35 2 19 8 23 FULL 7 33 FULL FULL 28 FULL 26 20 FULL 13 FULL FULL FULL FULL FULL 32 Admitted Students' IDs FULL FULL 5 FULL FULL FULL FULL FULL #N/A FULL FULL FULL FULL FULL FULL FULL FULL #N/A FULL FULL FULL FULL FULL FULL FULL FULL #N/A FULL FULL FULL FULL FULL FULL FULL FULL #N/A FULL FULL FULL FULL FULL FULL FULL FULL #N/A FULL FULL FULL FULL FULL FULL FULL FULL #N/A Major level School level Average scores Average rankings admission admission of students of students cut-off line cut-off line admitted admitted 569 569 615 4 595 155 410 557 244 274 196 29 155 244 274 29 447 443 351 252 14 14 19 22

8

Unsuccessful applicants’ IDs are highlighted. SMA means “subject to assignment” by the department in charge. #N/A means there are still vacancies in the major of the university. Keeping other variables constant, if student 16 has applied to CDE’s major C without SMA and student 6 has applied to CDE’s major C with SMA, student 6 would have been one of the successful applicants. In this case, student 16 would not be enrolled, even though there are still 5 vacancies in CDE’s major C. Vacancies may either be reallocated to other provinces or just be left unfilled.

Table 3 A Comparison of Four Ranking Reports, 2005-2009 Regression on Quality of All Examinees from All Provinces,
China 9 Dependent variable= average rankings of the students admitted

(1)
Ordinal Rank from CEUSC Ordinal Rank from CRCU Ordinal Rank from netbig Ordinal Rank from CUER Last Year's Dependent Var Constant

(2) 38.74*** (10.42) 1.244 (10.33)

(3) 27.51*** (10.63)

(4) 38.01*** (10.64) 2.548 (10.40)

(5) 24.64 (31.72)

(6) 27.98 (32.26) 5.708 (19.21) 19.51 (15.90)

15.21 (9.685)

-3.726 (6.167)

-3.834 (5.981) -3.405 (7.211)

1.642 (11.65)

0.122** (0.0487) 2387.7*** (356.6)

0.0150 (0.0468) 1892.5*** (451.3) 2765

0.0585 (0.0479) 2355.9*** (432.1) 2991

0.0166 (0.0464) 1984.9*** (484.8) 2765 409.4 (813.3) 3882 0.668 -376.4 (1014.9) 3641 0.671 0.665
FE All schools

Obs. R2 Adjusted R
Remarks
2

3115

0.662
Arellano-Bond method All schools Arellano-Bond method All schools Arellano-Bond method All schools Arellano-Bond method All schools FE All schools

Standard errors in parentheses * means significant at 10% level, ** is 5% level, and *** is 1% level. Regression controls apply to year, admission tier, stream, students’ location and school-level fixed effects.

9

The sample universities enroll students from more than 30 provinces in China and are ranked in all four ranking reports. The sample population is optimized after data cleaning.

Table 4 Summary of Ranking Statistics for Schools in the Optimized Ranking Groups (Intervals)

10

University code

10003 10001 10335 10246 10248 10284 10358 10486 10558 10183 10698 10213 10055 10422 10056
Average statistics for top 19 schools after data cleaning

mean 1.00 2.00 3.00 5.40 4.60 5.20 9.80 8.60 10.40 9.80 11.60 13.80 16.60 15.20 16.60 8.90667

sd 0.00 0.00 0.00 0.89 1.34 0.45 2.68 0.89 1.67 1.10 1.67 0.45 1.14 0.45 0.89 0.908

min 1.0 2.0 3.0 4.0 4.0 5.0 7.0 8.0 8.0 9.0 10.0 13.0 15.0 15.0 16.0 8

max 1 2 3 6 7 6 13 10 12 11 14 14 18 16 18 10.0667

N 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

Here, all universities in the top 19 schools and the other schools are grouped according to their best ranking from 2005 to 2009.

10

Summary of Ranking Statistics for Schools in the Optimized Ranking Groups (Intervals) (Cont’d)

10002 10286 10384 10006 10141 10247 10561 10699 10611 10730 10019 10145 10007 10532 10285 10290 10459 10251 10287 10008 10307 10614 10559 10635 10288 10697 10613 10504 10423 10010 10542 10004 10159 10294 10255 10217 10386 10673 10216 10434 10520 10299 10359 10530 10013 10272 10674
Average statistics for all the other schools after data cleaning

22.60 21.20 21.60 22.60 25.40 25.00 26.00 27.20 28.40 30.80 30.80 32.00 33.40 36.60 37.40 42.80 37.80 37.00 40.00 41.60 43.00 47.00 48.00 54.20 48.80 50.60 53.80 57.00 58.00 63.00 62.00 64.80 74.80 67.00 77.40 76.40 75.40 77.80 77.00 77.00 80.20 79.60 83.60 86.80 87.60 87.80 94.20

2.41 1.30 0.89 0.55 1.52 1.00 1.00 1.64 1.14 1.92 1.30 2.12 2.61 4.62 1.52 6.26 3.83 1.00 4.30 2.07 4.06 2.35 3.24 14.20 1.92 4.34 1.79 2.83 1.58 4.42 3.39 6.30 11.05 2.92 5.94 3.21 5.32 5.76 4.18 3.08 5.31 2.41 4.28 3.77 3.65 3.03 3.70

20.0 20.0 21.0 22.0 24.0 24.0 25.0 25.0 27.0 28.0 29.0 29.0 31.0 33.0 35.0 35.0 35.0 36.0 36.0 38.0 39.0 43.0 44.0 44.0 46.0 46.0 51.0 55.0 56.0 58.0 58.0 59.0 63.0 64.0 69.0 71.0 71.0 72.0 73.0 73.0 74.0 77.0 78.0 81.0 84.0 84.0 90.0

26 23 23 23 27 26 27 29 30 33 32 34 37 43 39 52 44 38 45 43 49 49 53 79 51 56 55 62 60 69 66 73 87 71 84 79 83 84 82 81 86 83 87 90 92 90 100

5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

52.617 3.42617 48.8511 56.9149

Table 5 Summary Table of Universities' Enrollment Population in each tier (and each stream).

mean

sd

min max

N

All schools enrolling students in more than 82.47 236.29 1.0 4672 11750 19 provinces

Top 19 schools enrolling students in 84.96 271.67 1.0 4672 3165 more than 19 provinces

All the other schools enrolling students in 81.56 221.84 1.0 4295 8585 more than 19 provinces All schools enrolling students in more than 52.92 58.28 19 provinces w/o local enrollment Top 19 schools enrolling students in 53.42 65.02 more than 19 provinces w/o local enrollment All the other schools enrolling students in 52.74 55.60 more than 19 provinces w/o local enrollment

1.0 1087 11380

1.0 798

3065

1.0 1087 8315

All schools enrolling students in less than 66.14 287.95 1.0 4612 4945 19 provinces All schools enrolling students in less than 20.98 28.61 19 provinces w/o local enrollment

1.0 736

4415

Table 6 Regression on Quality of All Examinees from All Provinces, China
Ranking Report from “A Report on the Competitiveness Evaluation of Universities and Subjects in China”, 2005-2009 11 Dependent variable= average rankings of the students admitted

(1)
Ordinal Rank from CEUSC Last Year's Dependent Var Constant

(2) 19.92** (8.846)

(3) 19.03*** (6.214) -0.106*** (0.0317)

(4) 9.849 (16.17) 0.173*** (0.0477) 845.7*** (164.4)

(5) 17.13** (7.262) -0.103*** (0.0377) 5300.7*** (430.4)

(6) -28.17 (21.65) -0.0249 (0.0593) 9068.1*** (1240.1)

95.92*** (1.859)

559.5*** (88.92)

2433.3*** (246.4)

4055.8*** (280.8)

Obs. R
2

11750 0.185
2

11750 0.734 0.732
FE All schools

7050

1899

5151

2967

Adjusted R
Remarks

0.185
OLS All schools

Arellano-Bond method All schools

Arellano-Bond method Top 19 schools

Arellano-Bond method

Arellano-Bond method

All the other schools

Standard errors in parentheses * means significant at 10% level, ** is 5% level, and *** is 1% level. Regression controls apply to year, admission tier, stream, students’ location and school-level fixed effects.

The sample universities enroll students from more than 19 provinces in China. The sample universities in the last column enroll students from less than 19 provinces in China.

11

Table 7 Grouped Regression and Group Jumping Problem in the Ranking Report, 2005-2009 Regression on 12 Quality of All Examinees from All Provinces, China
Dependent variable= average rankings of the students admitted

(1)
Ordinal Rank from CEUSC Last Year's Dependent Var Constant

(2) 39.35* (23.77) -0.282*** (0.105) 5624.3** (2303.9)

(3) 9.849 (16.17) 0.173*** (0.0477) 845.7*** (164.4)

(4) 15.88 (26.63) 0.176*** (0.0614) 2122.7*** (684.2)

(5) 7.998 (16.54) 0.240*** (0.0580) 3735.0*** (807.1)

(6) 21.00** (10.52) -0.264*** (0.0684) 8173.4*** (947.7)

-0.188 (16.74) 0.0874 (0.117) 6933.9*** (1619.0)

Obs. R
2

695

393

1899

1263

1472

1672

Adjusted R2
Remarks Arellano-Bond method 80-100 schools Arellano-Bond method 80-100 w/o group jumping schools Arellano-Bond method top 19 school w/o group jumping schools Arellano-Bond method 20-34 w/o group jumping schools Arellano-Bond method 35-58 w/o group jumping schools Arellano-Bond method 59-100 w/o group jumping schools

Standard errors in parentheses * means significant at 10% level, ** is 5% level, and *** is 1% level. Regression controls apply to year, admission tier, stream, students’ location and school-level fixed effects.

12

The sample universities enroll students from more than 19 provinces in China.

Table 8 Regression on Quality of All Examinees from All Provinces, China
Ranking Report from “A Report on the Competitiveness Evaluation of Universities and Subjects in China”, 2005-2009 13 Dependent variable=average rankings of the students admitted

(1)
Ordinal Rank of undergraduate school from CEUSC

(2) 4.894 (4.821)

(3) 6.923** (2.718) -0.109*** (0.0317)

(4) 23.25** (10.20) 0.178*** (0.0475) 710.1*** (118.6)

(5) 7.353** (3.119) -0.106*** (0.0377) 5822.6*** (260.6)

(6) 16.64* (9.222) -0.0356 (0.0595) 6792.5*** (644.3)

57.47*** (1.942)

Last Year's Dependent Var Constant

2022.0*** (95.54)

2490.0*** (244.8)

4604.1*** (173.8)

Obs. R
2

11750 0.069
2

11750 0.734 0.732
FE All schools

7050

1899

5151

2967

Adjusted R
Remarks

0.069
OLS All schools

Arellano-Bond method All schools

Arellano-Bond method Top 19 schools

Arellano-Bond method

Arellano-Bond method

All the other schools

Standard errors in parentheses * means significant at 10% level, ** is 5% level, and *** is 1% level. Regression controls apply to year, admission tier, stream, students’ location and school-level fixed effects.

The sample universities enroll students from more than 19 provinces in China. The sample universities in the last column enroll students from less than 19 provinces in China.

13

Table 9 Regression on Quality of Science Stream Examinees Enrolled in Tier 1 from All Provinces, China
Ranking Report from “A Report on the Competitiveness Evaluation of Universities and Subjects in China”, 2005-2009 14 Dependent variable= average rankings of the students admitted

(1)
Ordinal Rank of Science Research from CEUSC

(2) 2.509 (8.824)

(3) 13.34** (6.590) 0.00812 (0.0314)

(4) 11.40 (22.98) 0.105 (0.0654) 1457.7*** (253.5)

(5) 13.44* (7.560) 0.0245 (0.0357) 5413.7*** (431.6)

(6) -34.55 (24.00) -0.286** (0.122)
15028.0***

76.58*** (1.901)

Last Year's Dependent Var Constant

1971.9*** (90.52)

196.2 (247.5)

4414.5*** (299.6)

(2085.6)

Obs. R
2

8115 0.167 0.167
OLS All schools

8115 0.787 0.785
FE All schools

4869

1263

3606

558

Adjusted R2
Remarks

Arellano-Bond method All schools

Arellano-Bond method Top 19 schools

Arellano-Bond method

Arellano-Bond method

All the other schools

Standard errors in parentheses * means significant at 10% level, ** is 5% level, and *** is 1% level. Regression controls apply to year, admission tier, stream, students’ location and school-level fixed effects.

The sample universities enroll students from more than 19 provinces in China. The sample universities in the last column enroll students from less than 19 provinces in China.

14

Table 10 Regression on Quality of Liberal Arts Stream Examinees Enrolled in Tier 1 from All Provinces, China
Ranking Report from “A Report on the Competitiveness Evaluation of Universities and Subjects in China”, 2005-2009 15 Dependent variable= average rankings of the students admitted

(1)
Ordinal Rank of Liberal Arts Research from CEUSC

(2) -7.951 (9.072)

(3) 4.091 (10.65) 0.416*** (0.0201)

(4) 13.50 (17.95) 0.195*** (0.0446) 257.3 (159.7)

(5) 7.066 (13.47) 0.429*** (0.0234) 866.4* (510.9)

(6) -4.396 (13.89) 0.400*** (0.0264) 1418.1** (658.9)

35.04*** (1.671)

Last Year's Dependent Var Constant

466.7*** (59.69)

371.3* (193.8)

796.5** (310.1)

Obs. R
2

3395 0.115
2

3395 0.529 0.521
FE All schools

2037

636

1401

1305

Adjusted R
Remarks

0.114
OLS All schools

Arellano-Bond method All schools

Arellano-Bond method Top 19 schools

Arellano-Bond method

Arellano-Bond method

All the other schools

Standard errors in parentheses * means significant at 10% level, ** is 5% level, and *** is 1% level. Regression controls apply to year, admission tier, stream, students’ location and school-level fixed effects.

The sample universities enroll students from more than 19 provinces in China. The sample universities in the last column enroll students from less than 19 provinces in China.

15

Table 11 Regression on Quality of All Examinees from All Provinces, China (Excluding local enrollment)
Ranking Report from “A Report on the Competitiveness Evaluation of Universities and Subjects in China”, 2005-2009 16 Dependent variable= average rankings of the students admitted

(1)
Ordinal Rank from CEUSC Last Year's Dependent Var Constant

(2) 20.31** (8.732)

(3) 18.77*** (6.408) -0.0751** (0.0300)

(4) 14.59 (16.51) 0.157*** (0.0479) 811.9*** (167.3)

(5) 16.27** (7.495) -0.0685* (0.0354) 4964.3*** (424.2)

(6) -7.051 (21.79) -0.0157 (0.0531) 7645.5*** (1206.1)

94.80*** (1.850)

570.9*** (88.41)

2471.7*** (243.4)

3888.3*** (283.3)

Obs. R
2

11380 0.188
2

11380 0.738 0.736
FE All schools

6828

1839

4989

2649

Adjusted R
Remarks

0.187
OLS All schools

Arellano-Bond method All schools

Arellano-Bond method Top 19 schools

Arellano-Bond method

Arellano-Bond method

All the other schools

Standard errors in parentheses * means significant at 10% level, ** is 5% level, and *** is 1% level. Regression controls apply to year, admission tier, stream, students’ location and school-level fixed effects.

The sample universities enroll students from more than 19 provinces in China. The sample universities in the last column enroll students from less than 19 provinces in China.

16

Table 12 Regression on Quality of All Examinees from All Provinces, China (Excluding local enrollment)
Ranking Report from “A Report on the Competitiveness Evaluation of Universities and Subjects in China”, 2005-2009 17 Dependent variable= average rankings of the students admitted

(1)
Ordinal Rank of undergraduate school from CEUSC

(2) 5.517 (4.758)

(3) 7.423*** (2.797) -0.0776*** (0.0300)

(4) 22.49** (10.41) 0.162*** (0.0477) 834.5*** (130.6)

(5) 7.705** (3.213) -0.0704** (0.0354) 5541.3*** (261.2)

(6) 22.27** (9.190) -0.0188 (0.0530) 6121.2*** (620.7)

56.36*** (1.937)

Last Year's Dependent Var Constant

2035.7*** (95.12)

2528.4*** (241.9)

4459.7*** (175.1)

Obs. R
2

11380 0.069
2

11380 0.738 0.736
FE All schools

6828

1839

4989

2649

Adjusted R
Remarks

0.069
OLS All schools

Arellano-Bond method All schools

Arellano-Bond method Top 19 schools

Arellano-Bond method

Arellano-Bond method

All the other schools

Standard errors in parentheses * means significant at 10% level, ** is 5% level, and *** is 1% level. Regression controls apply to year, admission tier, stream, students’ location and school-level fixed effects.

The sample universities enroll students from more than 19 provinces in China. The sample universities in the last column enroll students from less than 19 provinces in China.

17

Table 13 Grouped Regression and Causal Relation Check of the Ranking Report, 2005-2009
Ranking Report from “A Report on the Competitiveness Evaluation of Universities and Subjects in China”, 2005-2009 18 Dependent variable= average rankings of the students admitted

(1)
Ordinal Rank from CEUSC Last Year's Dependent Var Distance × Ordinal Rank Constant

(2) -0.00421 (12.34) -0.105*** (0.0317) 0.0129* (0.00718)

(3) 9.849 (16.17) 0.173*** (0.0477)

(4) -6.257 (32.25) 0.173*** (0.0477) 0.0101 (0.0179)

(5) 15.88 (26.63) 0.176*** (0.0614)

(6) -2.648 (51.76) 0.176*** (0.0614) 0.0123 (0.0291)

(7) 7.998 (16.54) 0.240*** (0.0580)

(8) 1.932 (35.36) 0.241*** (0.0580) 0.00407 (0.0230)

(9) 21.00** (10.52) -0.264*** (0.0684)

(10) -21.57 (20.67) -0.274*** (0.0685) 0.0274** (0.0114)

19.03*** (6.214) -0.106*** (0.0317)

4055.8*** (280.8)

4047.5*** (280.6)

845.7*** (164.4)

967.5*** (169.5)

2122.7*** (684.2)

2094.3*** (685.9)

3735.0*** (807.1)

3743.3*** (810.0)

8173.4*** (947.7)

8301.6*** (948.6)

Obs. R2 Adjusted R2
Remarks

7050

7050

1899

1899

1263

1263

1472

1472

1672

1672

Arellano-Bond method all schools

Arellano-Bond method all schools

Arellano-Bond method top 19 w/o group jumping schools

Arellano-Bond method top 19 w/o group jumping schools

Arellano-Bond method 20-34 w/o group jumping schools

Arellano-Bond method 20-34 w/o group jumping schools

Arellano-Bond method 35-58 w/o group jumping schools

Arellano-Bond method 35-58 w/o group jumping schools

Arellano-Bond method 59-100 w/o group jumping schools

Arellano-Bond method 59-100 w/o group jumping schools

Standard errors in parentheses * means significant at 10% level, ** is 5% level, and *** is 1% level. Regression controls apply to year, admission tier, stream, students’ location and school-level fixed effects.

18

The sample universities enroll students from more than 19 provinces in China. The sample universities in the last column enroll students from less than 19 provinces in China.

Figures
Figure 1 Respondents’ Profile from the Online Survey Male Female Age 30 Current education level Middle School Undergraduate Postgraduate PhD or above 3.09% 79.63% 15.43% 1.85% 1.23% 50.00% 22.84% 18.52% 7.41% 64.81% 35.19%

Distribution of Respondents’ National Examination Location
Beijing Guangdong Shanghai Tianjin Jiangsu Hubei Sichuan Shaanxi Hebei Shanxi Henan Liaoning Jilin Heilongjiang Inner Mongolia Shandong Anhui Zhejiang Fujian Hunan Guangxi Jiangxi Guizhou Yunnan Hainan

10.26% 7.05% 7.69% 1.92% 5.77% 7.05% 3.21% 3.21% 3.21% 4.49% 4.49% 1.92% 8.33% 1.28% 0.64% 5.77% 7.05% 3.85% 3.85% 5.13% 1.28% 0.64% 0.64% 0.64% 0.64%

Figure 2 Evaluation Structure of "A Report on the Competitiveness Evaluation of Universities and Subjects in China" (Zhongguo Daxue Pingjia) by Mr. Shulian Wu

Undergraduate Training [C1k]

Student Training Capacity [B1k]

Overall Ranking [Ak]

Postgraduate Training [C2k]

Natural Science [C3k] Research Capacity [B2k] Liberal Arts [C3k]

Employment rate [R1k] Last year's admission score [R2k] Undergraduate teaching assessment [R3k] Faculty-student ratio [R4k] Population of undergraduate students who get the degree [D1k] Average academic standard of the faculty [D2k] Bilingual courses [D3k] Frontier teaching [D4k] Special program [D5k] Faculty resources [D6k] Textbooks [D7k] Undergraduate level "Challenge Cup" academic competition result [D8k] Mathematical Modeling competition [D9k] Undergraduate teaching award [D10k] Average academic standard [R5] Population of postgraduate students who get the degree [D11k] Population of PhD students who get the degree [D12k] Number of outstanding papers [D13k] Postgraduate level "Challenge Cup" academic competition result [D14k] Postgraduate teaching award [D15k] Science for military use [R6k] Papers published or referred in SCD, SCI, SSCI, A&HCI etc. [R16k] [R17k] Academic papers in natural science [R18k] Arts and property rights, etc [R19k][R20k][R21k][R22k][R23k] Science for military use [R6k] Papers published or referred in SCD, SCI, SSCI, A&HCI etc. [R16k] [R17k] Arts and property rights, etc. [R19k][R20k][R21k][R22k][R23k]

Figure 3 Demo Radar Chart for Top 5 Universities in 2009

19

Postgraduate Training 0 20 40 60 80 100 120 140 Liberal Arts 160 Natural Science
University code

10003 10001 10335 10248 10284

Undergraduate Training

19

Values in reverse order