Kudos
Kunnskapsdokumenter i offentlig sektor

Evaluering

Evaluation of the national research school scheme

The national research school scheme National research schools or PhD schools have long traditions in other countries, but were first formalized in Norway in 2008 under a national scheme organized by the Research Council of Norway (RCN). The background for the introduction of the scheme was a concern that the strong increase in PhD students had led to reduced completion rate and a higher average age at the time of completion. The purpose of the research schools was to “set a standard for other environments on how to drive PhD students through their research training with good results within estimated time to degree”. The first five research schools under the national scheme were established in 2008. Since then, another ten schools were established in 2012 and seven schools in 2015. The main conclusion of this evaluation is that the national research school scheme has led to more national cooperation between research environments, and that the components of the training program provided to the PhD students is of high quality. So far, it is, however, not possible to see any measurable effects of this on completion degree and completion time, which reflects that the schools have been more concerned about increasing the quality rather than the efficiency of the education. The research schools have contributed to internationalization of the teaching, but we find few measurable effects on students’ (long-term) stays abroad or international co-authorship. We recommend that the RCN considers a more differentiated research school scheme with differentiated funding and evaluation criteria. Main question of the evaluation This evaluation provides answers to five questions about the national research schools’ results and organizing. First, have the research school scheme contributed to fewer students dropping out, i. e. has the completion rate increased? Second, has the scheme led to the students using shorter time to complete their doctoral degrees, i. e. has the completion time been reduced? Third, has the introduction of the scheme led to more national cooperation among stakeholders within a given discipline or thematic area? Fourth, has the scheme contributed to more internationalization among the involved institutions and students? The research schools have been given a great degree of freedom in shaping their organization and activities. The fifth question is whether any organizational features of the research schools are associated with a higher achievement of objectives. In answering these questions, we draw upon several data sources. A survey was sent to the research schools (18 out of 22 schools responded) and to environments that had applied for a grant under the scheme, but whose applications were rejected (23 ‘schools’ responded). Based on name lists from the research schools, we have used data from NIFU’s Register of Doctoral Degrees to compare completion rate and completion time between PhD students at research schools with other PhD students not affiliated with a research school. The name lists were also used to compare the two student groups’ degrees of international co-authorship using data from the national publication database CRIStin (Current Research Information System in Norway). No effect (yet) on completion rate and completion time from the research school scheme In our analysis, we have not been able to document any increase in completion rates following the introduction of the national research school scheme. The completion rate for PhD students at the national research schools are practically identical to the completion rates for all other PhD students in Norway. The overall completion rate for schools operating under the scheme is just 0. 4 percentage points higher than for all other PhD students. The completion rates differ substantially between the research schools, and are higher in thematically oriented schools compared to discipline oriented schools. The results for completion time is almost identical. When comparing the research schools operating under the national scheme with other PhD students in Norway, the difference in time spent on the PhD is practically identical. The difference between the two groups can be counted in days, not months or years. It should be noted that there are methodological issues in identifying the research school students. Moreover, it is too early to estimate the results from the youngest research schools. Hence, our conclusions on completion time and completion rates come with some reservations. Many of the research schools are little oriented towards international activities Many of the respondents in our study were surprised that questions about their international activities were included. They simply did not see this as a main goal for the school. Other schools reported high levels of international activity, and highlighted that one of the key added values of the research school scheme was how it enabled the schools to engage in such activities. Some of the schools have a high degree of international participation, with international lecturers at doctoral courses, their workshops and conferences have participation from abroad etc. The research schools have a larger share of international lecturers as course leaders than PhD courses outside of the scheme, and it seems as the research schools have contributed to more use of international supervisors. Most of these activities seem rather short-term, i. e. we do not find any added value on long-term mobility across borders for PhD students. PhD students outside of the national scheme are just as likely to have stays abroad as students at the research schools. Furthermore, there is, with some noticeable exceptions, no indication that foreign PhD students are more likely to attend courses at the research schools than at other PhD hosting institutions. Many respondents emphasize the development and maintenance of international networks as a very important factor of the research schools and report that the senior researchers have expanded their international networks. Still, from the limited evidence we have on the international activity of the PhD students, this does not seem to translate in similar internationalization of the students’ research and publication activities. We find no evidence that PhD students at the research schools have more international co-authors than other students, neither during their PhD period or in their publications after completing the PhD. In sum, the teaching activities and supervision at the research schools have more international contributions than at other schools, but the general observation we make is that given the available resources at the research schools, the amount of international activities/collaboration involving the PhD students is not particularly high compared to other PhD institutions in Norway. National cooperation has been strengthened in discipline oriented schools Among supervisors and researchers from the partner institutions, there has been an increased contribution to courses, workshops and other academic activities at other partner institutions of the research school. It is not common to have research stays at other partner institutions and few schools reported that participating in a research school had led to increased research cooperation across institutions. It appears that the national cooperation is mainly related to teaching activities. There are still some research schools that have not taken steps to make their doctoral education more uniform or to harmonize the use of ECTS points for courses offered by the research school. Furthermore, cross-institutional supervision at the research schools’ Norwegian partner institutions is just as seldom for PhD students at the research schools as it is for other PhD students. Notably, the discipline-oriented schools have put far more emphasis on strengthening the national cooperation, than what has been the case in the thematic-oriented schools. Research schools are promoting academic quality rather than making the education more efficient Our impression is that the contribution of the research schools is not so much to speed up the educational process, nor to create mechanisms that ensures that fewer students drop out. The scheme is simply not designed as a tool to increase the efficiency of the PhD training. Rather, it seems to be aimed at increasing the academic quality of the doctoral training, but this outcome is not necessarily linked to efficiency. In fact, it could be the opposite. The impression from our survey and from the mid-term evaluations, is that the activities that take place at the schools are of high quality. We have, however, not included any quality indicators related to academic quality in our evaluation. The schools themselves, but also rejected applicants to the national research school scheme, believe that there has been much added value on increased cooperation between research environments, and that the scheme has enabled the schools to offer a better and more coordinated PhD education, a strong course portfolio (often with contributions from international lecturers), and also PhD education in new thematic areas not covered by existing PhD programmes. Despite these, and many other, positive contributions from the scheme, we have observed some elements that the RCN needs to address in order to improve the goal fulfilment on the main outcome indicators studied here, of which academic quality of the doctoral degree is not one. A more differentiated scheme, with differentiated funding and evaluation criteria Our recommendations are mainly related to the funding structure of the scheme, to a more differentiated scheme based on different types of schools and their different needs, and to the introduction of special efforts to improve completion rate and completion time. We recommend that the funding level may differ between types of research schools, where we argue that thematically oriented schools are costlier to operate due to the fact they are creating new courses and teaching activities from scratch. Discipline oriented schools can to a much higher degree be cost effective based on utilizing available personnel, courses, etc. Between these two ‘ideal types’ of organizing research schools, there will of course be schools with certain elements of both. A more fine-tuned funding based on the type of school, may also lead to different evaluation criteria of the two, where international and national cooperation play very different roles. While the thematic schools may need to engage themselves in international cooperation to build up researcher education in a new field (hence, the RCN should use internationalization as a performance indicator), the discipline schools are often far more oriented at national collaboration, trying to unify national resources and teaching within the field (hence, the RCN should use national cooperation as a performance indicator). Many of the discontinued schools have struggled to maintain their activities after the funding from the RCN ended, or have simply been shut down. We therefore recommend that the RCN consider making the funding level and period more flexible, including e. g. a phase-out after the eight years of funding under the scheme stops, enabling the institutions to take over and find other funding. Finally, we believe that the RCN should make efforts to ensure that improving completion rate and completion time are specifically addressed in the calls for school grants, in the proposals and in the contract between the RCN and the research schools. More efforts in the calls, and in the schools, should be made on further qualifying of supervisors, for the participating partners in a school to make systems where the students should have two supervisors representing two different institutions inside the school (nationally or internationally, in order to promote academic and social ties across institutions or countries), and that the supervisors should commit to make formal individual plans with the PhD student on how to improve progress and completion, in line with good experiences from Denmark. More work should also be done at the schools to create a ‘research school identity’ among the students, which is lacking at many schools.