The value of research funding for knowledge creation and dissemination A study of SNSF model articles 2008 pdf Research Grants Humanities and Social Sciences Communications
The value of research funding for knowledge creation and dissemination A study of SNSF model articles 2008 pdf Research Grants Humanities and Social Sciences Communications
RSS feed About the journal Aims and scope Journal Information Indexing Editors Journal policies Editorial process Open access funding and payment Referee instructions Calls for papers Important update Events Publishing Partnerships Advocacy Contact Publish with us For authors Submit manuscript Search Search articles by subject, keyword or author Show results from All journals This journal Search Advanced search Krücken G, Meier F Turning the university into an organizational actor. In: Drori GS, Meyer JW, Hwang H Globalization and organization: world society and organizational change, vol 18. pp. 241–257. You can also search for this author in PubMed   Google Scholar Swiss National Science Foundation, Berne, Switzerland Discover content Journals A-Z Articles by subject Publishing policies Nature portfolio policies Open access Author & Researcher services Reprints & permissions Research data Language editing Scientific editing Nature Masterclasses Nature Research Academies Libraries & institutions Librarian service & tools Librarian portal Open research Recommend to library Advertising & partnerships Advertising Partnerships & Services Media kits Branded content Career development Nature Careers Nature Conferences Nature events Regional websites Nature Japan Nature Korea Nature Middle East Legal & Privacy Use of cookies Manage cookies/Do not sell my data Accessibility statement Terms & Conditions California Privacy Statement © 2021 Springer Nature Limited For doing so, we employ a nearest neighbor propensity score matching. That is, we pair each grant recipient with a non-recipient by choosing the nearest ‘twin’ based on the similarity in the estimated probability of receiving a grant and the average score that the submitted applications received. Note that we select the twin researcher from the sample of unsuccessful applicants so that matching on both, the general propensity to win and the proposal’s evaluation score, allows to match both on an individual as well as on proposal characteristics to find the most comparable individuals. Surkis A, Spore S The relative citation ratio: what is it and why should medical librarians care? J Med Libr Assoc 106:508–513 Oancea A Research governance and the future of research assessment. Palgrave Commun 5:27 We use the lmer package in R and a negative binomial family. The results from the probit estimation on the funding outcome are presented in Table 4 . The table first shows the model for the full sample which provides the propensity score for the estimation of treatment effects on articles and citations to these articles, and on preprints. The second model shows the model for the sub-sample of researchers in the LS used for estimating treatment effects on the RCR. The third model shows the estimation for the full sample, but accounting for pre-sample FCR, and provides the propensity score for the estimation of the treatment effect on the FCR. The fourth model controls for pre-sample altmetrics values and serves the estimation of the treatment effect on future altmetrics scores. Consistent across all specification, the results show that the evaluation score is a key predictor of grant success. The higher the score, the more likely is it that a proposal gets approved. The grant likelihood for male researches is higher than for females as well as for older researchers. The latter result can have various reasons, which are outside the scope of this paper and are being discussed elsewhere Footnote 18 . As expected, past research performance is another strong predictor of grant success where peer-reviewed articles matter more than preprints. In addition to quantity, past research quality increases the probability of a proposal being granted. Interesting in more recent years , quality rather than quantity appears to predict grant success as it is the average number of citations to pre-period publication rather than their number that explains funding success. Some characteristics on the researchers without unique ID can be found in Table S 2 in the Supplementary material. Silberzahn R, Uhlmann EL, Martin DP, Anselmi P, Aust F, Awtrey E, Bahník Š, Bai F, Bannard C, Bonnier E, Carlsson R, Cheung F, Christensen G, Clay R, Craig MA, Rosa AD, Dam L, Evans MH, Cervantes IF, Fong N, Gamez-Djokic M, Glenz A, Gordon-McKeon S, Heaton TJ, Hederos K, Heene M, Mohr AJH, Högden F, Hui K, Johannesson M, Kalodimos J, Kaszubowski E, Kennedy DM, Lei R, Lindsay TA, Liverani S, Madan CR, Molden D, Molleman E, Morey RD, Mulder LB, Nijstad BR, Pope NG, Pope B, Prenoveau JM, Rink F, Robusto E, Roderique H, Sandberg A, Schlüter E, Schönbrodt FD, Sherman MF, Sommer SA, Sotak K, Spain S, Spörlein C, Stafford T, Stefanutti L, Tauber S, Ullrich J, Vianello M, Wagenmakers E-J, Witkowiak M, Yoon S, Nosek BA Many analysts, one data set: making transparent how variations in analytic choices affect results. Adv Methods Pract Psychol Sci 1:337–356 Bornmann L Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics J Informetr 8:895–903 In addition to the closeness on MD, we use elements of exact matching by requiring that selected control researchers belong exactly to the same subject field and to be observed in the same year as the researchers in the treatment group. This allows to account for different publication patterns across disciplines and also for time trends in funding likelihood and in the outcome variables. To better understand those differences in funding effect, we refer to Fig. 2 for the article counts and Fig. 3 for the average number of citations per article. Those figures show the predicted article or citation count depending on the funding group and the age group or the research field. For all those subgroups, SNSF funding in t −1 has a positive effect on the outcome. However the size of this effect differs substantially. The youngest age group seems to benefit considerably from the funding in terms of predicted difference between treatment and control researchers in article count, but also in citation per article . More senior funded researchers perform similarly well compared to researchers with the same characteristics but no funding. It is noteworthy that for older researchers the difference between groups is again higher indicating that funding helps to keep productivity up. We obtain very similar results based on post-estimations with interaction effects in the matched samples from the propensity score matching approach . The results from our analysis based on different estimation methods show that grant-winning researchers publish about one additional peer-reviewed publication more per year in the 3 years following funding than comparable but unsuccessful applicants. Moreover, these publications are also influential as measured by the number of citations that they receive later on. SNSF PF seems to promote timely dissemination as indicated by the higher number of published preprints and researchers’ higher altmetrics scores. The funding impact is particularly high for young researchers as well as for researchers at a very late career stage when funding keeps output levels high. These results add new insights to the international study of funding effects which provided partially ambiguous findings as our review in the next section illustrates. In summary, the results presented in the following stress the important role played by project funding for research outcomes and hence for scientific progress. Institutional funding alone does not appear to facilitate successful research to the same extent as targeted grants which complement institutional core funds. Hausman N University innovation and local economic growth. Rev Econ Stat . st_a_01027 . Rubin DB Assignment to treatment group on the basis of a covariate. J Educ Stat 2:1–26 Schmidt J Das Hochschulsystem der Schweiz: Aufbau, Steuerung und Finanzierung der schweizerischen Hochschulen. Beitr Hochschulforsch 30:114–147 Humanities and Social Sciences Communications ISSN 2662-9992 Novelty is measured by the extent to which a published paper makes first time ever combinations of referenced journals while taking into account the difficulty of making such combinations. Ayoubi C, Pezzoni M, Visentin F The important thing is not to win, it is to take part: what if scientists benefit from participating in research grant competitions? Res Policy 48:84–97 The Sinergia scheme is closely linked to PF, so that we will not differentiate between them in the following. Stephan PE How economics shapes science. Harvard University Press, Cambridge. Anyone you share the following link with will be able to read this content: Battistin E, Rettore E Ineligibles and eligible non-participants as a double comparison group in regression-discontinuity designs. J Econom 142:715–730 Fudickar R, Hottenrott H, Lawson C What’s the price of academic consulting? effects of public and private sector consulting on academic research. Ind Corp Change 27:699–722 View all journals Search My Account Login Explore content About the journal Publish with us Sign up for alerts RSS feed nature humanities and social sciences communications articles article The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants Download PDF Article Open Access Published: 21 September 2021 The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants Rachel Heyard   ORCID: orcid.org/0000-0002-7531-4333 1 & Hanna Hottenrott   ORCID: orcid.org/0000-0002-1584-8106 2 , 3   Humanities and Social Sciences Communications volume  8 , Article number:  217 Cite this article Serghiou S, Ioannidis JPA Altmetric scores, citations, and publication of studies posted as preprints. JAMA 319:402–404 Shown are the averages of the publication an preprint counts and the altmetrics for each year of observation. PI stands for principal investigator. Hutchins BI, Yuan X, Anderson JM, Santangelo GM Relative citation ratio : a new metric that uses citation rates to measure influence at the article level. PLoS Biol 14:1–25 Note that we also tested the robustness of this result to when focusing on PF as treatment and adding the researchers with a funded Sinergia project to the control group, but adjusting with a Sinergia dummy variable. The size of funding as PI and co-PI effects and their confidence intervals were comparable. For all research areas, SNSF funding has a positive effect on article count and number of citations. STEM researchers however benefit most with a percentage change of 23% more articles as funded PI compared to no funding; funded researchers from the LS publish 15% more articles and the SSH researchers 12%. This could reflect that in STEM and LS the extent to which research can be successfully conducted is highly funding-dependent, while this is not necessarily the case in the SSH. Yet regarding the number of citations per article, the SSH researchers benefit most . This suggest that funding may support the quality of research and hence its impact more in the SSH field. Thus, it should be noted that even though SSH researcher publish and are cited less in absolute numbers, we still see a substantial positive effect of SNSF funding on the outcomes. The respective figures for the remaining outcomes can be found in the supplementary material; more specifically Fig. S. 5 for the altmetric score, Fig. S. 4 for the preprint count and Fig. S. 6 for the FCR, in the supplementary material. An alternative approach is to employ pre-sample information of the researcher as a proxy for unobservable characteristics, such as a researcher’s ability or writing talent which impact research output in the sample period. We conducted such linear feedback models as robustness tests and present them in Supplement S. 2.1 . Berg JM, Bhalla N, Bourne PE, Chalfie M, Drubin DG, Fraser JS, Greider CW, Hendricks M, Jones C, Kiley R, King S, Kirschner MW, Krumholz HM, Lehmann R, Leptin M, Pulverer B, Rosenzweig B, Spiro JE, Stebbins M, Strasser C, Swaminathan S, Turner P, Vale RD, VijayRaghavan K, Wolberger C Preprints for the life sciences. Science 352:899–901 Sorry, a shareable link is not currently available for this article. Heyard, R., Hottenrott, H. The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants. Humanit Soc Sci Commun 8, 217 . s41599-021-00891-x Thank you for visiting . You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser . In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript. The comparison of the distribution of the propensity score and the evaluation score before and after matching shows that the nearest neighbor matching procedure was successful in balancing the sample in terms of the grant likelihood and—importantly—also the average scores . This ensures that we are comparing researchers with funding to researchers without funding that have similarly good ideas and are also otherwise comparable in their characteristics predicting a positive application outcome. The balancing of the propensity scores and the evaluation scores in both groups after each matching are shown in Tables 5 and 6 . Note that we draw matches for each grant-winner from the control group with replacement and that hence some observations from researchers in the control group are used several times as ‘twins’. Table S. 5 in the supplementary material shows that across the different matched samples <10% of control researcher–year observations are used only once and about 60% up to 25 times. About 10% of control group researchers are used very frequently, i.e. more than 160 times. Poege F, Harhoff D, Gaessler F, Baruffaldi S Science quality and the value of inventions. Sci Adv 5:eaay7323. Arora A, David P, Gambardella A Reputation and competence in publicly funded science: estimating the effects on research group productivity. Ann Econ Stat 49–50, 163–198. ANZSRC Outcomes paper: Australian and New Zealand Standard Research Classification Review 2019. Ministry of Business, Innovation & Employment. with \ being the outcome variable in the treated group, \ being the counterfactual for i and n T is the sample size . Footnote 9 The results suggest that the funding has a persistent output effect amounting to about one additional article in each of the 3 years following the year of funding. The effect on preprints is already significant in the first year, but also turns out to sustain in later years suggesting that research results from the project are probably circulated via this channel. In contrast to these results, we find for altmetrics that they are significantly higher early on, but not in the medium-run. When looking at citation-based measures as indicators for impact and relevance, we see that the number of citations stays significantly higher in the medium-run, but effect size declines somewhat indicating that researchers publish the most important results earlier after funding. This is also reflected in the results for the average number of citations and the probability to be highly cited. For the FCR, the effect is less persistent as the difference between groups fades after the first year. For the RCR the differences in means is strongest in the first year after the grant and only significant at the 10% level in t    3. Since only a few cases are identified to hold major international grants but no SNSF funding, we do not differentiate between these groups in the following. Note that the data was retrieved from the ERC Funded Projects Database included only grants acquired since 2007. Beaudry C, Allaoui S Impact of public and private research funding on scientific production: the case of nanotechnology Res Policy 41:1589–1606 The SNSF is Switzerland’s main research funding agency. The SNSF is mandated by the Swiss confederation to allocate research funding to eligible researchers at universities, colleges and research organizations. Note that we did not test the interaction for the RCR outcome, as this analysis was done only for the LS field. Finally, by explicitly investigating outputs over several years after funding, our study contributes new insights on the persistency of effects. Since a large share of project funding typically goes into wages of doctoral and post-doctoral researchers which require training and learning on the job, there may be a considerable time lag between the start of the project and the publication of any research results and an underestimation of output effects when considering only immediate outcomes. Digital Science Dimensions available from sions.ai . Accessed Sept 2020, under licence agreement. This study investigates the effect of competitive project funding on researchers’ publication outputs. Using detailed information on applicants at the Swiss National Science Foundation and their proposal evaluations, we employ a case-control design that accounts for individual heterogeneity of researchers and selection into treatment . We estimate the impact of the grant award on a set of output indicators measuring the creation of new research results , its relevance , as well as its accessibility and dissemination as measured by the publication of preprints and by altmetrics. The results show that the funding program facilitates the publication and dissemination of additional research amounting to about one additional article in each of the three years following the funding. The higher citation metrics and altmetrics by funded researchers suggest that impact goes beyond quantity and that funding fosters dissemination and quality. Wang J, Lee Y-N, Walsh JP Funding model and creativity in science: competitive versus block funding and status contingency effects Res Policy 47:1070–1083 While the insights on a positive effect of funding on the number of subsequent scientific articles are in line with previous studies, compared to previous results, the effects that we document here are larger. The reason for that may be related to the fact that the SNSF is the main source of research funding in Switzerland we can therefore identify researchers for the control group who really had no other project grant in the period for which they are considered a control. We also observe co-PIs which may in other studies—due to a focus on PIs or lack of information—be assigned to the control group. Both may lead to an under-estimation of funding effects in previous studies. Moreover, by counting all publications of these researchers, we further take not only articles directly related to the project into account, but also that there are learning spillovers and synergies beyond the project that improve a researcher’s overall research performance. Jacob BA, Lefgren L The impact of research grant funding on scientific productivity. J Public Econ 95:1168–1177 Open Access funding enabled and organized by Projekt DEAL. Blundell R, Griffith R, Windmeijer F , Dynamics and correlated responses in longitudinal count data models. In: Seeber GUH, model articles 2008 pdf Francis BJ, Hatzinger R, Steckel-Berger G , Statistical modelling. Springer New York, New York, pp. 35–42. The results show a similar pattern across all estimation methods indicating an effect size of about one additional article in each of the 3 years following the funding. In addition, we find a similarly sized effect on the number of preprints. The comparison across methods suggests that if accounting for important observable researcher characteristics as well as proposal quality parametric regression results and non-parametric models lead to similar conclusions with regard to publication outputs. Importantly, a significant effect on the number of citations to articles could be observed indicating that funding does not merely translate into more, but only marginally relevant research. Funded research also appears to reach the general public more than other research as indicated by higher average altmetrics in the group of grant-winners. In terms of the RCR and FCR the results indicate that there might be an effect on the funded researchers’ overall visibility in the research community. However, the effects on the RCR are not robust to the estimation method used. In addition to count-type outputs, we estimate the effect of funding on continuous output variables such as the average number of yearly citations per article or the researcher’s average yearly altmetric score. For these output types we estimate linear regression models based on a comparable model specification with regard to F i t −1 , X i t , T t and v i . Munich Data Science Institute, Garching bei München, Germany de la Cuesta B, Imai K Misunderstandings about the regression discontinuity design in the study of close elections. Annu Rev Political Sci 19:375–396 Benavente JM, Crespi G, Figal Garone L, Maffioli A The impact of national research funds: a regression discontinuity approach to the Chilean fondecyt Res Policy 41:1461–1475 Severin A, Martins J, Heyard R, Delavy F, Jorstad A, Egger M Gender and other potential biases in peer review: cross-sectional analysis of 38–250 external peer review reports. BMJ Open 10:e035058 TUM School of Management, Munich, Germany Note that the researchers with missing age were deleted since this is an important control variable; the missing institution type were regrouped into unclassified. Additionally, for the analyses, the funding information will be used with a one year lag and at least one year of observation is lost per researcher. The final sample used for the analyses consists of 72,738 complete observations from 8,282 unique researchers. Fang F, Casadevall A Research funding: the case for a modified lottery. mBio 7:e00422-16. The authors declare no competing interests. Franzoni C, Giuseppe S, Stephan P Changing incentives to publish. Science 333:702–3 Figure 1 represents the evolution of the yearly average number of articles, preprints and the altmetric score per researcher depending on the funding status of the year before . The amount of articles published each year has been rather constant or only slightly increasing, while the preprint count increased substantially over the past years. Recent papers also have a higher altmetric scores than older publications, even though they had less time to raise attention. It is important to note, however, that since we do not account for any researcher characteristics here, the differences between funded and unfunded researchers cannot be interpreted as being the result of funding. Yet, increasing prevalence of preprints and altmetrics suggest that they should be taken into account in funding evaluations. Payne A, Siow A Does federal research funding increase university research output? Adv Econ Anal Policy 3:1018–1018 In addition to the effect in the year after funding , we are interested in the persistency of the effect in the following years up to . It is likely that any output effects occur with a considerable time-lag after funding received. The start-up of the research project including the training of new researchers and the set-up of equipment may take some time before the actual research starts. In principle, we could of course expect the effect to last also longer than three to four years. However, after 4 years, the treatment effect of one project grant may become confounded by one follow-up grants. Tables 5 and 6 show the results for the different outcome variables also for different time horizons. In an alternative estimation approach, we apply a non-parametric technique: The average treatment effect of project funding on scientific outcomes is estimated by an econometric matching estimator which addresses the question of “How much would a funded researcher have published if she had not received the grant?”. This implies comparing the actually observed outcomes to the counterfactual ones to derive an estimate for the funding effect. Given that the counterfactual situation is not observable, it has to be estimated. Hottenrott H, Lawson C Fishing for complementarities: research grants and research productivity Int J Ind Organ 51:1–38 Gerfin M, Lechner M A microeconometric evaluation of the active labour market policy in Switzerland. Econ J 112:854–893 The value of research funding for knowledge creation and dissemination A study of SNSF model articles 2008 pdf Research Grants Humanities and Social Sciences CommunicationsThe value of research funding for knowledge creation and dissemination A study of SNSF model articles 2008 pdf Research Grants Humanities and Social Sciences Communications Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Arora A, Gambardella A The impact of NSF support for basic research in economics . Ann Écon Stat 79–80, 91–117. Reale E Analysis of national public research funding —final report, Technical Report JRC107599, Publications Office of the European Union. Excellent publications in this study were for instance papers in the upper quarter of journals included in the Science Citation Index . Data provided by the SNSF has been used to retrieve a set of researchers of interest. These researchers have applied to the SNSF funding instrument project funding or Sinergia Footnote 10 as main applicant or co-applicant Footnote 11 . The PF scheme is a bottom-up approach as it funds costs of research projects with a topic of the applicant’s own choice. Unfortunately the altmetric cannot be retrieved as a time-dependent variable from Dimensions but only as the altmetric state at the time point of data retrieval . Therefore the altmetric informs us on the cumulative importance an article published at t got until September 2020. Carayol N, Matt M Does research organization influence academic production? laboratory level evidence from a large european university Res Policy 33:1081–1102 The central interest of the study is the effect competitive project funding has on a researcher’s subsequent research outputs. The information on SNSF funding indicates whether a researcher had access to SNSF funding as a PI and/or co-PI in a certain year. We differentiate between PIs and co-PIs to test whether the funding effect differs depending on the role in the project. On average the researchers in our data set are funded by the SNSF for 4.6 years during the observation period; for 3.3 years as PI of a project . In total 20,476 distinct project applications are included in the data. On average a PIs is involved in a total of 3.7 project applications ; in 3.1 submissions as PI, and in 2.3 submissions as co-PI. About 66% of all projects in the data have one sole PI applying for funding, 22% have a PI and a co-PI, 8% a PI and two co-PIs, and 4% are submitted by a PI together with three or more co-PIs. Note that the percentage of successful applications in our data set is 48% over the whole study period . Table 3 summarizes the results of the four linear mixed models for the continuous outcomes: the average yearly number of citations per publication, the yearly average altmetric, the yearly average RCR and the yearly average FCR. Regarding the citation patterns, there is strong evidence that SNSF funding has a positive effect; especially PIs on SNSF projects have their articles cited more frequently . Articles by LS researchers are cited most compared to researchers from other fields. This is also the case for researchers from ETH Domain and older researchers. For altmetrics and citation ratios, we employ a logarithmic scale to account for the fact that their distributions are highly skewed; we can then interpret the coefficients as percentage change. Regarding altmetrics, research funded by the SNSF gets an attention score that is 5.1% higher compared to other researchers. Researchers in LS have by far the highest altmetrics followed by researchers in the SSH. There is no strong evidence for an effect of the funding on the average yearly RCR. This implies that in the short-run research outcomes of SNSF-funded researchers are as often cited as a mixed average of articles funded by the NIH or other important researcher funded world-wide, but also not significantly more than that. Younger researchers and researchers from the ETH Domain have higher RCRs. The results also suggest a positive relation between SNSF funding and a researcher’s FCR. Provided by the Springer Nature SharedIt content-sharing initiative Understanding the role played by competitive research funding is crucial for designing research funding policies that best foster knowledge generation and diffusion. By investigating the impact of project funding on scientific output, its relevance and accessibility, this study contributes to research on the effects of research funding at the level of the individual researcher. Table 1 further shows descriptive statistics for the gender of the researchers, their biological age, as well their field of research and the institution type. These variables capture drivers of researcher outputs and are therefore taken into account in all our analyses. Almost 77% of the researchers are male and about 60% are employed at cantonal universities, 24% at technical universities and about 17% at University of Applied Sciences and University of Teacher Education . The research field and institution type are defined as the area or the type the researcher applies most often to or from. The field of life sciences has the largest proposal share in the data with about 39%. These variables serve as confounders together with the pre-sample information on the outcome variables since they may explain differences in output and therefore need to be accounted for. Note that 1615 researchers in our data did not publish any peer-reviewed papers in the five year pre-sample period. Table S. 1 in the supplementary material shows how the confounding variables vary between the research fields. Neufeld J, Huber N, Wegner A Peer review-based selection decisions in individual research funding, applicants’ publication strategies and performance: the case of the ERC starting grants. Res Eval 22:237–247 Using the matched comparison group, the average effect on the treated can thus be calculated as the mean difference of the matched samples: Jaffe AB Real effects of academic research. Am Econ Rev 79:957–970 Fleming L, Greene H, Li G, Marx M, Yao D Government-funded research increasingly fuels innovation. Science 364:1139–1141 We define P i t as the research output of researcher i in year t and F i t −1 as a binary variable indicating whether this same researcher i had access to SNSF funding in year t −1. Note that this indicator takes the value one for the entire duration of the granted project. The funding information is lagged by one year as an immediate effect of funding on output is unlikely. Note that, we will differentiate between funding as PI and as co-PI . The general empirical model can then be expressed as
Jonkers K, Zacharewicz T Research performance based funding systems: a comparative assessment. Technical Report JRC101043, Publications Office of the European Union. Warren HR, Raison N, Dasgupta P The rise of altmetrics. JAMA 317:131–132 The original data set comprised 11,228 eligible researchers. 10% of the latter could not be identified in the Dimensions database. Among the researchers found using their name, the supplementary information from the SNSF database did not match in 1% of the cases, and we were not sure that we found the correct researcher. For 12% of the researchers found in Dimensions no unique ID could be retrieved. After removing these observations, we observe a total of 8,793 distinct researchers and the final data set is composed of 82,249 researcher-year observations. On average researchers are observed for 9.35 years. The maximum observation length, from 2005 to 2019 is 15 years, and 2,319 researchers are observed over this maximal study period. All the publication data was retrieved in September 2020. Lechner M Identification and estimation of causal effects of multiple treatments under the conditional independence assumption. In: Lechner M, Pfeiffer F Econometric evaluation of labour market policies. Physica-Verlag HD, Heidelberg, pp. 43–58. If granted, a co-applicant is entitled to parts of the funding. Hottenrott H, Thorwarth S Industry funding of university research and scientific productivity. Kyklos 64:534–555 Gläser J, Serrano-Velarde K Changing funding arrangements and the production of scientific knowledge: introduction to the special issue. Minerva 56:1–10 We are grateful to Tobias Phillip for helpful comments on the study design and on previous versions of this manuscript and to Matthias Egger for an additional careful review of the manuscript prior to submission. This work was supported by the SNSF . Konkiel S Altmetrics: diversifying the understanding of influential scholarship. Palgrave Commun 2:16057 If Dimensions found more than one ID for a certain name, we used further information on the researcher available to the SNSF to narrow the ID-options down. This supplementary information was, if present the ORCID, the current and previous research institution, country and birth year. Only researchers with a unique ID could be used in the following. See Table S 2 in the supplementary material for a comparison of the researchers that were found and not found. Tahmooresnejad L, Beaudry C Citation impact of public and private funding on nanotechnology-related publications Int J Technol Manag 79:21–59 While the previous estimation approaches modeled unobserved heterogeneity across individuals, the non-parametric matching approach addresses the selection into the treatment explicitly. It accounts for selection on observable factors which may—if not accounted for—lead to wrongly attributing the funding effect to the selectivity of the grant-awarding process. We model a researcher’s funding success as a function of researcher characteristics. In particular, this includes their previous research track record and the average of all evaluation scores for submitted proposals received by the researcher. In addition, we include age, gender, research field and institution type. We obtain the propensity score to be used in the matching process as described in the section “Non-parametric treatment estimation”. Froumin I, Lisyutkin M Excellence-driven policies and initiatives in the context of bologna process: rationale, design, implementation and outcomes. In: Curej A, Matei L, Pricopie R, Salmi J, Scott P The European higher education area. Springer. Lăzăroiu G What do altmetrics measure? Maybe the broader impact of research on society. Educ Philos Theory 49:309–311 The funding program analyzed in this study is open to all researchers in Switzerland affiliated with institutions eligible to receive SNSF funding. This allows us to study treatment effect heterogeneity over researchers’ life cycle and research field. The results suggest here, that funding is particularly important at earlier career stages where PF facilitates research that would not have been pursued without funding. With regard to treatment effect heterogeneity across fields, we find the highest effect of funding on the article count for STEM researchers and the highest funding effect on citations in SSH. To predict the article count the baseline confounding variables were fixed to Year 2015–19, Male, Evaluation Score Score AB-A, University, LS in the age interaction model and age lower to 45 for the field interaction model. We see a significant positive percentage change of 18% for the youngest age group among PIs and 115% for the most senior researchers compared to no SNSF funding. Additionally, the effect of funding is largest for STEM researchers (23% more articles as PI compared to unfunded researchers. The effect in LS and SSH is less prominent, 15% and 12%, respectively. Tables 5 and 6 show the estimated treatment effects after matching, i.e. the test for the magnitude and significance of mean differences across groups. Note that the number of matched pairs differs depending on the sample used and that log values of output variables were used to account for the impact of skewness of the raw variable distribution in the mean comparison test. The magnitude of the estimated effects is comparable to the ones of the parametric estimation models. Researchers with a successful grant publish on average 1.2 articles and about one additional preprint more in the following year, their articles receive 1.7 citations ) more than articles from the control group. In terms of altmetrics we also see a significant difference in means which is 1.15 points higher in the group of grant receivers. Also, in terms of the FCR and the RCR, there are significant effects on the treatment group. The probability to be among the ‘highly cited researchers’ is 5.5 percentage points higher in the group of funded researchers. This means publications in t    1 are cited at least three times as much as the average in the field. support/solutions/articles/6000233311-how-is-the-altmetric-attention-score-calculated- with ϕ being the vector of parameters. X i t represents a vector with explanatory factors at t including observed characteristics of the researcher and the average quality of the grant applications as reflected in the average evaluation score. Further T t captures the overall time trend, v i is the unobserved individual heterogeneity, and ϵ i t is the error term. In addition to resource-driven effects, there may also be direct dissemination incentives related to public project funding. On the one hand, funding agencies may encourage or even require the dissemination of any results from the funded project. On the other hand, the researchers may have incentives to publish research outcomes to signal project success to the funding agency and win reputation gains valuable for future proposal assessments. Table 2 summarizes the results of both negative binomial mixed models for the count outcomes . The incidence rate ratios inform us on the multiplicative change of the baseline count depending on funding status. The model for the publication count was fitted on the whole data set, while the model for the preprint count is fitted on data since 2010, because the number of preprints was rather small in general before. SNSF funding seems to have a significant positive effect on research productivity, regarding yearly publication counts as well as yearly preprint counts . Footnote 17 An ‘average’ researcher without SNSF funding in t −1 publishes on average 4.64 articles in t . A similar researcher with SNSF funding as PI in t −1 would publish 5.6 articles in t . PIs on an SNSF project publish more. The same is true for male researchers and younger researchers for preprints. Researchers from ETH Domain publish more than the ones from Cantonal Universities. Researchers publish more in recent years. Researchers in the LS publish more peer-reviewed articles compared to other research areas. Regarding preprints, we observe a different picture. Here STEM researchers publish more than researchers in LS. Adams JD, Griliches Z Research productivity in a system of universities. Ann Écon Stat 49–50, 127–162. Mali F, Pustovrh T, Platinovšek R, Kronegger L, Ferligoj A The effects of funding and co-authorship on research performance in a small scientific community. Sci Public Policy 44:486–496 Jaffe AB Building programme evaluation into the design of public research support programmes. Oxf Rev Econ Policy 18:22–34 Graves N, Barnett AG, Clarke P Funding grant proposals for scientific research: retrospective analysis of scores by members of grant review panel. BMJ 343:d4797. For the predictions the baseline confounding variables were fixed to Year 2010–14, Male, Evaluation Score Score AB-A, University, LS in the age interaction model and age lower to 45 for the field interaction model. A significant positive percentage change of 10% for the youngest age group among PIs compared to no SNSF funding can be observed for the average number of citations. The remaining changes in citation number are not significant. Then, the effect of funding is largest for SSH researchers . what are the contents of articles of association