Gender Bias in Peer Review and Research Funding
An Empirical Analysis of Grant Applications in Pakistan
Keywords:
Gender Bias, Research Grants, Funding, Peer Review, Bulk ReviewAbstract
The current study examined research project submissions by discipline. It also reviewed the peer evaluation reports of these projects. All projects were submitted to a funding agency for grant consideration. There was no gender bias in the selection of the Scrutiny Committee. This committee shortlisted 622 projects out of 1,787 submissions. Female representation in both submitted and shortlisted projects was below 20%. Each shortlisted project was evaluated by two subject experts. They provided detailed comments and a final score. Grade A indicated the highest funding priority, B was medium, and C was the lowest. Each project received two grades, resulting in six possible combinations: AA, BB, CC, AB, BC, and AC. Tests such as Cronbach’s alpha and Fleiss’ Kappa showed low internal consistency between peer reviewers’ grades. This suggests that the scores were based on scientific criteria and not awarded randomly. However, further analysis showed a trend. Male applicants received more favorable scores than females. Female applicants were at a relative disadvantage. The study recommends that peer review alone should not determine funding decisions. A follow-up bulk review, like the one done by the Scrutiny Committee, is necessary. This can help minimize gender bias. Detailed comments and grades from peer reviews can guide the bulk review. They help assess the strengths and weaknesses of each project. This ensures that decisions focus on scientific merit and the project's potential for socio-economic impact through local research. While bulk review may not fully remove gender bias, it reduces the chances of gender-based discrimination. Funding decisions would then be based on well-argued scientific evaluations.
References
Abma-Schouten, R., Gijbels, J., Reijmerink, W., & Meijer, I. (2023). Evaluation of research proposals by peer review panels: broader panels for broader assessments? Science and Public Policy, 50(4), 619–632. https://doi.org/10.1093/scipol/scad009
Bravo, G., Grimaldo, F., López-Iñesta, E., Mehmani, B., & Squazzoni, F. (2019). The effect of publishing peer review reports on referee behavior in five scholarly journals. Nature Communications, 10(1), 322.
Brodie, S., Frainer, A., Pennino, M. G., Jiang, S., Kaikkonen, L., Lopez, J., Ortega-Cisneros, K., Peters, C. A., Selim, S. A., & Văidianu, N. (2021). Equity in science: advocating for a triple-blind review system. Trends in Ecology & Evolution, 36(11), 957–959. https://doi.org/10.1016/J.TREE.2021.07.011
Budden, A. E., Tregenza, T., Aarssen, L. W., Koricheva, J., Leimu, R., & Lortie, C. J. (2008). Double-blind review favours increased representation of female authors. Trends in Ecology and Evolution, 23(1), 4–6. https://doi.org/10.1016/j.tree.2007.07.008
Conklin, M., & Singh, S. (2022a). Triple-blind review as a solution to gender bias in academic publishing, a theoretical approach. Studies in Higher Education, 47(12), 2487–2496. https://doi.org/10.1080/03075079.2022.2081681
Conklin, M., & Singh, S. (2022b). Triple-blind review as a solution to gender bias in academic publishing, a theoretical approach. Studies in Higher Education, 47(12), 2487–2496. https://doi.org/10.1080/03075079.2022.2081681
Cox, A. R., & Montgomerie, R. (2019). The cases for and against double-blind Reviews. PeerJ, 2019(4). https://doi.org/10.7717/peerj.6702
Cruz Castro, L., & Sanz Menéndez, L. (2020). Grant allocation disparities from a gender perspective: Literature review. Synthesis report.
Finance, B. 2021-22 document D. of. (2021). Budget 2021-22.
Haffar, S., Bazerbachi, F., & Murad, M. H. (2019). Peer Review Bias: A Critical Review. Mayo Clinic Proceedings, 94(4), 670–676. https://doi.org/10.1016/J.MAYOCP.2018.09.004
Helmer, M., Schottdorf, M., Neef, A., & Battaglia, D. (2017). Gender bias in scholarly peer review. ELife, 6. https://doi.org/10.7554/eLife.21718
Henker, B., Whalen, C. K., & Collins, B. E. (1979). Double-blind and triple-blind assessments of medication and placebo responses in hyperactive children. Journal of Abnormal Child Psychology, 7, 1–13.
Hermansson, K., Jacobsson, C., & Österberg, R. (2021). Gender equality in research funding. A Study Of, 11.
Higgins, A. C., Chubin, D. E., & Hackett, E. J. (1990). Peerless science: Peer review and U.s. science policy. Soc. Forces, 69(2), 634.
Jung Jisoo and Kim, J.-I. and Y. J. W. (2017). A Practical Approach to Constructing Triple-Blind Review Process with Maximal Anonymity and Fairness. In S. Choi Dooho and Guilley (Ed.), Information Security Applications (pp. 198–209). Springer International Publishing.
Kern-Goldberger, A. R., James, R., Berghella, V., & Miller, E. S. (2022). The impact of double-blind peer review on gender bias in scientific publishing: a systematic review. In American Journal of Obstetrics and Gynecology (Vol. 227, Issue 1, pp. 43-50.e4). https://doi.org/10.1016/j.ajog.2022.01.030
Larivière, V., Ni, C., Gingras, Y., Cronin, B., & Sugimoto, C. R. (2013). Bibliometrics: Global gender disparities in science. Nature, 504(7479), 211–213. https://doi.org/10.1038/504211a
Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17.
Lerback, J., & Hanson, B. (2017). Journals invite too few women to referee. Nature, 541(7638), 455–457. https://doi.org/10.1038/541455a
Lerchenmueller, M. J., & Sorenson, O. (2018). The gender gap in early career transitions in the life sciences ☆. Research Policy, February 2017, 1–11. https://doi.org/10.1016/j.respol.2018.02.009
Liévano-Latorre, L. F., da Silva, R. A., Vieira, R. R. S., Resende, F. M., Ribeiro, B. R., Borges, F. J. A., Sales, L., & Loyola, R. (2020). Pervasive gender bias in editorial boards of biodiversity conservation journals. Biological Conservation, 251, 108767. https://doi.org/10.1016/J.BIOCON.2020.108767
Llorens, A., Tzovara, A., Bellier, L., Bhaya-Grossman, I., Bidet-Caulet, A., Chang, W. K., Cross, Z. R., Dominguez-Faus, R., Flinker, A., Fonken, Y., Gorenstein, M. A., Holdgraf, C., Hoy, C. W., Ivanova, M. V., Jimenez, R. T., Jun, S., Kam, J. W. Y., Kidd, C., Marcelle, E., … Dronkers, N. F. (2021). Gender bias in academia: A lifetime problem that needs solutions. Neuron, 109(13), 2047–2074. https://doi.org/10.1016/J.NEURON.2021.06.002
Mayo, N. E., Brophy, J., Goldberg, M. S., Klein, M. B., Miller, S., Platt, R. W., & Ritchie, J. (2006). Peering at peer review revealed high degree of chance associated with funding of grant applications. Journal of Clinical Epidemiology, 59(8), 842–848. https://doi.org/https://doi.org/10.1016/j.jclinepi.2005.12.007
Mills, D. S., Ramos, D., Estelles, M. G., & Hargrave, C. (2006). A triple blind placebo-controlled investigation into the assessment of the effect of Dog Appeasing Pheromone (DAP) on anxiety related behaviour of problem dogs in the veterinary clinic. Applied Animal Behaviour Science, 98(1–2), 114–126.
Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., & Handelsman, J. (2012). Science faculty’s subtle gender biases favor male students. Proceedings of the National Academy of Sciences of the United States of America, 109(41), 16474–16479. https://doi.org/10.1073/pnas.1211286109
Mulligan, A., Hall, L., & Raphael, E. (2013). Peer review in a changing world: An international study measuring the attitudes of researchers. In Journal of the American Society for Information Science and Technology (Vol. 64, Issue 1, pp. 132–161). https://doi.org/10.1002/asi.22798
Nielsen, M., Haun, D., Kärtner, J., & Legare, C. H. (2017). Journal of Experimental Child The persistent sampling bias in developmental psychology : A call to action. Journal of Experimental Child Psychology, 162, 31–38. https://doi.org/10.1016/j.jecp.2017.04.017
Nikpour, M., Shirvani, M. A., Azadbakht, M., Zanjani, R., & Mousavi, E. (2014). The effect of honey gel on abdominal wound healing in cesarean section: a triple blind randomized clinical trial. Oman Medical Journal, 29(4), 255.
Ranga, M., Gupta, N., & Etzkowitz, H. (2012). Gender effects in research funding. Bonn: Deutsche Forschungsgemeinschaft.
Recio-Saucedo, A., Crane, K., Meadmore, K., Fackrell, K., Church, H., Fraser, S., & Blatch-Jones, A. (2022). What works for peer review and decision-making in research funding: a realist synthesis. Research Integrity and Peer Review, 7(1), 2. https://doi.org/10.1186/s41073-022-00120-2
Rodgers, P. (2017). Peer Review: Decisions, decisions. ELife, 6, e32011. https://doi.org/10.7554/eLife.32011
Smith, R. (2006). Peer review: a flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99(4), 178–182.
Snodgrass, R. (2006). Single-versus double-blind reviewing: An analysis of the literature. ACM Sigmod Record, 35(3), 8–21.
Squazzoni, F., Bravo, G., Farjam, M., Marusic, A., Mehmani, B., Willis, M., Birukou, A., Dondio, P., & Grimaldo, F. (2025). Peer review and gender bias: A study on 145 scholarly journals. Science Advances, 7(2), eabd0299. https://doi.org/10.1126/sciadv.abd0299
Tomkins, A., Zhang, M., & Heavlin, W. D. (2017). Reviewer bias in single-versus double-blind peer review. Proceedings of the National Academy of Sciences, 114(48), 12708–12713.
Ucci, M. A., D’Antonio, F., & Berghella, V. (2022). Double- vs single-blind peer review effect on acceptance rates: a systematic review and meta-analysis of randomized trials. American Journal of Obstetrics & Gynecology MFM, 4(4), 100645. https://doi.org/10.1016/J.AJOGMF.2022.100645
Wessely, S. (1998). Peer review of grant applications: what do we know? The Lancet, 352(9124), 301–305. https://doi.org/10.1016/S0140-6736(97)11129-1
