Comparação da confiabilidade entre três escalas para avaliação de habilidades clínicas em laboratório de simulação e três tipos diferentes de avaliadores. Estudo experimental unicêntrico, 2019

Abstract

The use of clinical skills training through simulation into medical school curricula has been a major innovation for education, enabling students to train ethically in a safe environment without risk to patients. The best way to evaluate skills in a simulated environment is not well defined. Few studies have compared teachers and rating scales reliability. Objective: To describe how different clinical skill assessment scales in simulation laboratory and different teachers profiles interfere with assessment reliability. Methodology: 28 students from the third year and 20 students from the sixth year of UNIFENAS-BH Medical School were recruited. Students performed a practical skills test on newborn anthropometry in the simulation laboratory, where they were filmed for further performance assessment using three scales: Training Level Rating Scale (TL), Global Rating Scale (EG); and checklist (CL). Nine evaluators were selected divided into three profiles: three pediatricians with no experience in teaching skills, three pediatricians with experience in teaching skills, and three skill teachers who were not pediatricians. One teacher from each profile used only one of the three proposed scales, so that the three scales were used in each of the three profiles. The total score for each of the scales was calculated, with a possible total of 9 points. Results: when comparing the different scales, regardless of the teacher profile, it was observed only that the isolated grade of the CL was able to discriminate the third and sixth year students. Multivariate analysis showed that TL, isolated score (p = 0.346) or total score (p = 0.407), was the only scale that was not influenced by the teacher profile. The reliability analysis performed according isolated scales score showed that only the TL has good reliability. The addition of anthropometric measurements increased the reliability of scales and teachers. Conclusion: there is a difference in reliability between the scales. TL plus anthropometric measurements was the most reliable. The teacher profile directly affected the reliability of the scales, but the expert teachers showed acceptable reliability when anthropometric measurements were added to the scales. Only TL was able to correctly discriminate the two groups and was not influenced by teacher profile