Introduction: For medical students navigating the complex residency application process, online platforms like the Student Doctor Network (SDN) forums have become increasingly popular resources. SDN serves as a public forum where aspiring doctors discuss various aspects of medical education, and more recently, specialty-specific forums have emerged for residency applicants to share their application data. These self-reporting forums are intended to help applicants gauge their competitiveness by reviewing peer-reported information. However, the critical question remains: How accurate is the Student Doctor Network as a source of reliable data for emergency medicine (EM) residency applicants? To investigate this, our study compared self-reported data from SDN forums with objective data from the National Resident Matching Program (NRMP) to assess the reliability of SDN for EM applicants.
Methods: We conducted a retrospective analysis of self-reported data from Doctor of Osteopathic Medicine (DO) and Doctor of Medicine (MD) candidates on SDN emergency medicine forums. The data spanned three residency application cycles: 2014, 2016, and 2018. We then compared this self-reported SDN data to the National Resident Matching Program’s (NRMP) charting outcomes for each of these years, providing a benchmark of objective data against the forum submissions.
Results: Across the three application cycles, a total of 360 emergency medicine residency applicants self-reported their data on the Student Doctor Network. A significant majority, 79%, of these posts occurred during the 2018 cycle, coinciding with the increased use of Google Docs spreadsheets for data collection within the forums. In the initial two years analyzed, the mean United States Medical Licensing Examination (USMLE) scores reported on SDN were comparable to the actual NRMP data. However, for the 2018 application cycle, a noticeable difference emerged. Applicants self-reporting on SDN indicated higher mean USMLE Step 1 scores (234, 95% Confidence Interval [CI], 233-236) and Step 2 scores (250, 95% CI, 248-251) compared to the NRMP data (Step 1: 231; Step 2: 241). Interestingly, reported contiguous residency program ranks on SDN were similar to NRMP data across all years. The proportion of applicants reporting membership in the Alpha Omega Alpha Honor Medical Society on SDN was also consistent with NRMP figures, but only in the most recent year, 2018.
Conclusion: Our findings suggest that while Student Doctor Network Forums provide a platform for residency applicants to share information, there’s a potential for bias. Specifically, self-reported USMLE Step scores on SDN showed a slight upward bias in the most recent application cycle studied when compared to objective NRMP data. The increasing volume of self-reporting on SDN, particularly with the ease of spreadsheet submissions, does not automatically translate to greater accuracy. It remains uncertain whether this growth in self-reported data will enhance the reliability of SDN forums for future emergency medicine applicants. Given the inherent limitations of self-reported data and the observed bias, it is crucial for emergency medicine residency applicants to interpret information from Student Doctor Network forums with caution and to supplement this resource with other, more objective sources during their application process.
