TY - JOUR
T1 - Evaluation of Situational Judgment Tests in student selection in Indonesia and the impact on diversity issues
AU - Soemantri, Diantha
AU - Findyartini, Ardi
AU - Yolanda, Sophie
AU - Morley, Emma
AU - Patterson, Fiona
N1 - Funding Information:
The authors would like to acknowledge the support of Moi Kok Wah and Stuart Martin, both former employees of Work Psychology Group, who provided consultation in SJT item development, item writer training and early parts of data analysis and feedback on the manuscript concept. The authors would also like to acknowledge the academic staff at Faculty of Medicine Universitas Indonesia who contributed in SJT item development, and all medical students and candidates who participated in the study.
Publisher Copyright:
© 2022, The Author(s).
PY - 2022/12
Y1 - 2022/12
N2 - Background: Internationally, medical selection relies heavily on prior academic attainment which has an adverse impact on the diversity of selected students. Since non-academic attributes are also important, this study aims to evaluate the use of a Situational Judgment Test (SJT) for selection and the impact on student diversity relating to gender, ethnicity and socio-economic status. Previous SJT research has almost entirely originated from a Western context and this study focuses on new evidence in a South East Asian context with a different demographic profile. Methods: Thirty faculty members developed 112 SJT scenarios assessing professionalism, communication and self-awareness domains. The scenarios underwent a concordance stage where stakeholder input was sought on the content appropriateness, to define the item scoring key, followed by an initial psychometric evaluation with first and second year medical students (N = 436). Based on these results, 30 scenarios, consisting of 128 nested items, were selected for pilot testing and evaluation regarding diversity issues with two cohorts of applicants in 2017 (N = 446) and 2018 (N = 508). Results: The SJT demonstrated good internal consistency (Cronbach’s alpha of 0.80 and 0.81 respectively). There were significant differences in SJT scores based on gender in both years, where females consistently outperformed males (p =.0001). However, no significant differences were found based on high school origin, parental educational background or ethnicity. Conclusions: This is the first study to evaluate the use of an SJT in Indonesia, which has a unique diversity profile compared to Western countries. Largely, the preliminary results replicate previous studies of the potential diversity benefits of using an SJT as a tool for medical student selection and has the potential to level the playing field regarding socio-economic status and ethnicity. Further studies exploring more variables representing diversity are warranted to confirm the early results in this study.
AB - Background: Internationally, medical selection relies heavily on prior academic attainment which has an adverse impact on the diversity of selected students. Since non-academic attributes are also important, this study aims to evaluate the use of a Situational Judgment Test (SJT) for selection and the impact on student diversity relating to gender, ethnicity and socio-economic status. Previous SJT research has almost entirely originated from a Western context and this study focuses on new evidence in a South East Asian context with a different demographic profile. Methods: Thirty faculty members developed 112 SJT scenarios assessing professionalism, communication and self-awareness domains. The scenarios underwent a concordance stage where stakeholder input was sought on the content appropriateness, to define the item scoring key, followed by an initial psychometric evaluation with first and second year medical students (N = 436). Based on these results, 30 scenarios, consisting of 128 nested items, were selected for pilot testing and evaluation regarding diversity issues with two cohorts of applicants in 2017 (N = 446) and 2018 (N = 508). Results: The SJT demonstrated good internal consistency (Cronbach’s alpha of 0.80 and 0.81 respectively). There were significant differences in SJT scores based on gender in both years, where females consistently outperformed males (p =.0001). However, no significant differences were found based on high school origin, parental educational background or ethnicity. Conclusions: This is the first study to evaluate the use of an SJT in Indonesia, which has a unique diversity profile compared to Western countries. Largely, the preliminary results replicate previous studies of the potential diversity benefits of using an SJT as a tool for medical student selection and has the potential to level the playing field regarding socio-economic status and ethnicity. Further studies exploring more variables representing diversity are warranted to confirm the early results in this study.
KW - Medical students
KW - Non-academic attributes
KW - Selection
KW - Situational judgment test (SJT)
KW - Undergraduate
UR - http://www.scopus.com/inward/record.url?scp=85127462515&partnerID=8YFLogxK
U2 - 10.1186/s12909-022-03247-4
DO - 10.1186/s12909-022-03247-4
M3 - Article
C2 - 35366862
AN - SCOPUS:85127462515
SN - 1472-6920
VL - 22
JO - BMC Medical Education
JF - BMC Medical Education
IS - 1
M1 - 239
ER -