Soft Skills and Technical Skills in Teacher Evaluations Carried out by Students
DOI:
https://doi.org/10.22201/iisue.20072872e.2024.44.1894Keywords:
teacher attitudes, teacher assessment, textual analysis, social skills, professional knowledge, UruguayAbstract
What is the significance of both soft skills and hard skills in teacher evaluations conducted by students? Through the application of textual analysis methods to a set of teacher ratings and comments, the researchers conducted a qualitative examination and tested quantitative hypotheses regarding the language used in high and low ratings. The findings indicate a significant relationship between the language used in the comments and the ratings. It was observed that references to soft skills serve as better indicators of the ratings compared to references to hard skills. However, qualitative analyses suggest that in the case of teachers rated as excellent, hard skills are not overlooked but rather presumed. In addition to its methodological contribution, this study adds to the discourse on biases in teacher evaluation and the essential qualities to be nurtured in teachers.
Downloads
References
Administración Nacional de Educación Pública (ANEP) (2023), Marco Curricular de la formación de grado de los educadores, Montevideo, ANEP.
Arceo, Eva y Raymundo Campos (2019), “Gender stereotypes: The case of MisProfesores.com in Mexico”, Economics of Education Review, núm. 72, pp. 55-65, DOI: https://doi.org/10.1016/j.econedurev.2019.05.007.
Aristóteles (2005), Ética a Nicómaco, Madrid, Alianza.
Azab, Mohamud, Rada Mihalcea y Jacob Abernethy (2016), “Analysing RateMyProfessors evaluations across institutions, disciplines, and cultures: The tell-tale signs of a good professor”, en E. Spiro y Y.-Y. Ahn (eds.), Social informatics, vol. 10046, pp. 438-453, DOI: https://doi.org/10.1007/978-3-319-47880-7_27.
Bardach, Lisa y Robert Klassen (2020), “Smart teachers, successful students? A systematic review of the literature on teachers’ cognitive abilities and teacher effectiveness”, Educational Research Review, núm. 30, pp. 100312, DOI: https://doi.org/10.1016/j.edurev.2020.100312.
Bardach, Lisa, Robert M. Klassen y Nancy E. Perry (2022), "Teachers’ Psychological Characteristics: Do They Matter for Teacher Effectiveness, Teachers’ Well-being, Retention, and Interpersonal Relations? An Integrative Review", Educational Psychology Review, núm. 34, pp. 259-300, DOI: https://doi.org/10.1007/s10648-021-09614-9.
Body, Lawrence, Natalia Ramos, Olivia Recondo y Manuel Pelegrina (2016), "Desarrollo de la Inteligencia Emocional a través del programa mindfulness para regular emociones (PINEP) en el profesorado", Revista interuniversitaria de formación del profesorado, núm. 87, pp. 47-59.
Burroughs, Nathan, Jacqueline Gardner, Youngjun Lee, Siwen Guo, Israel Touitou, Kimberly Jansen y William Schmidt (2019), "A Review of the Literature on Teacher Effectiveness and Student Outcomes", IEA Research for Education, Teaching for Excellence and Equity, pp. 7-14. Cham, Springer, DOI: https://doi.org/10.1007/978-3-030-16151-4_2.
Boyd, Ryan (2017), “Psychological text analysis in the digital humanities”, en S. Hai-Jew (ed.), Data Analytics in Digital Humanities, Nueva York, Springer International Publishing, pp. 161–189, DOI: https://doi.org/10.1007/978-3-319-54499-1_7.
Boyd, Ryan y Andrew Schwartz (2021), “Natural language analysis and the psychology of verbal behavior: The past, present, and future states of the field”, Journal of Language and Social Psychology, núm. 40, pp. 1–21, DOI: https://doi.org/10.1177/0261927x20967028.
Brown, Michael, Michelle Baillie y Schwandel Fraser (2009), “Rating Ratemyprofessors.com: A Comparison of Online and Official Student Evaluations of Teaching”, College Teaching, núm. 57, pp. 89-92, DOI: https://doi.org/10.3200/CTCH.57.2.89-92.
Bruns, Barbara y Javier Luque (2014), Profesores excelentes: cómo mejorar el aprendizaje en América Latina y el Caribe, Washington, Banco Mundial.
Coladarci, Theodore y Irv Kornfield (2007), “RateMyProfessors com versus formal in-class student evaluations of teaching”, Practical Assessment, Research, and Evaluation, vol. 12, núm. 6, pp. 1-16, DOI: https://doi.org/10.7275/26ke-yz55.
Cornelius-White, Jeffrey (2007), “Learner-Centered Teacher-Student Relationships Are Effective: A Meta-Analysis", Review of Educational Research, núm. 77, pp. 113-143, DOI: https://doi.org/10.3102/003465430298563.
Davison, Elizabeth y Jamie Price (2009), “How do we rate? An evaluation of online student evaluations”, Assessment & Evaluation in Higher Education, num. 34, pp. 51-65, DOI: http://doi.org/10.1080/02602930801895695.
Funder, David y Daniel Ozer (2019), “Evaluating effect size in psychological research: Sense and nonsense”, Advances in Methods and Practices in Psychological Science, num. 2, pp. 156-168, DOI: http://doi.org/10.1177/2515245919847202.
Gregory, Karen (2012), “How undergraduates perceive their professors: a corpus analysis of Rate My Professor”, Journal of Educational Technology Systems, núm. 40, pp. 169-193, DOI: https://doi.org/10.2190/ET.40.2g.
Harris, Douglas y Stacey Rutledge (2010), “Models and Predictors of Teacher Effectiveness: A Comparison of Research About Teaching and Other Occupations”, Teachers College Record, núm. 112, pp. 914-960, DOI: https://doi.org/10.1177/016146811011200304.
Hattie, John (2009), Visible learning, New York, Routledge.
Hemphill, James (2003), “Interpreting the magnitudes of correlation coefficients”, American Psychologist, núm. 58, pp. 78–79, DOI: https://doi.org/10.1037/0003-066X.58.1.78.
Jerez, Oscar, Cesar Orsini y Beatriz Hasbún (2016), “Atributos de una docencia de calidad en la educación superior: una revisión sistemática”, Estudios Pedagógicos, núm. 42, pp. 483-506, DOI: http://doi.org/10.4067/S0718-07052016000400026.
Kaya, Metin y Abdullah Selvitopu (2019), “A meta-analysis of the effects of some factors on teachers’ classroom management skills”, International Journal of Contemporary Educational Research, núm. 6, pp. 409-425, DOI: https://doi.org/10.33200/ijcer.621313.
Kennedy, Brendan, Ashwini Ashokkumar, Ryan Boyd y Morteza Dehghani (2021), “Text analysis for psychology: Methods, principles, and practices”, en M. Dehghani y R. Boyd (Eds.), The Handbook of Language Analysis in Psychology, Nueva York, Guilford, pp. 3-62.
Kim, Lisa, Verena Jörg y Robert Klassen (2019), “A Meta-Analysis of the Effects of Teacher Personality on Teacher Effectiveness and Burnout”, Educational Psychology Review, núm. 31, pp. 163–195, DOI: https://doi.org/10.1007/s10648-018-9458-2.
Kindred, Jeannette y Shaheed Mohammed (2005), ““He Will Crush You like an Academic Ninja!”: Exploring Teacher Ratings on Ratemyprofessors.com”, Journal of Computer-Mediated Communication, núm. 10, pp. 314-322, DOI: https://doi.org/10.1111/j.1083-6101.2005.tb00257.x.
Klassen, Robert y Virginia Tze (2014), “Teachers’ Self-Efficacy, Personality, and Teaching Effectiveness: A Meta-Analysis", Educational research review, núm. 12, pp. 59–76, DOI: https://doi.org/10.1016/j.edurev.2014.06.001.
Kreitzer, Rebbeca y Jennie Sweet-Cushman (2022), “Evaluating Student Evaluations of Teaching: a Review of Measurement and Equity Bias in SETs and Recommendations for Ethical Reform”, Journal of Academic Ethics, núm. 20, pp. 73–84, DOI: https://doi.org/10.1007/s10805-021-09400-w.
Lei, Hao, Yunhuo Cui y Ming Chiu (2017), “The Relationship between Teacher Support and Students’ Academic Emotions: A Meta-Analysis", Frontiers in Psychology, núm. 8, pp. 1-12, DOI: https://doi.org/10.3389/fpsyg.2017.02288.
Loureiro, Silvana, Marina Míguez y Ximena Otegui (2016), "Desempeño docente en la enseñanza universitaria: análisis de las opiniones estudiantiles", Cuadernos de Investigación Educativa, núm 7, pp. 55–67, DOI: https://doi.org/10.18861/cied.2016.7.1.2576.
Manchini, Nigel, Óliver Jiménez y Natalia Ramos (2022, Agosto 8), “Teachers' Soft Skills: a Scoping Review", PsyArxiv, DOI: https://doi.org/10.31234/osf.io/trjub.
Murray Dakota, Clara Boothby, Huimeng Zhao, Vanessa Minik, Nicolás Bérubé, Vincent Larivière y Cassidy Surimoto (2020), “Exploring the personal and professional factors associated with student evaluations of tenure-track faculty”, PLoS ONE, núm. 15, pp. e0233515, DOI: https://doi.org/10.1371/journal.pone.0233515.
Ozili, Peterson (2023), “The acceptable R-square in empirical modelling for social science research", MPRA, <https://mpra.ub.uni-muenchen.de/116496/> [Consulta: marzo de 2023].
Pennebaker, James (2018) “Telling Stories: Analyzing Text to Understand Personality, Social behavior, and Narratives”, Microsoft Research Talks, Washington, <https://www.youtube.com/watch?v=e25YKwSKOo4> [Consulta: marzo de 2023].
Qaiser, Shahzad y Ramsha Ali (2018) “Text Mining: Use of TF-IDF to Examine the Relevance of Words to Documents”, International Journal of Computer Applications, núm. 181, pp. 25-29, DOI: https://doi.org/10.5120/ijca2018917395.
Quin, Daniel (2017), “Longitudinal and Contextual Associations Between Teacher–Student Relationships and Student Engagement: A Systematic Review”, Review of Educational Research, núm. 87, pp. 345-387, DOI: https://doi.org/10.3102/0034654316669434.
R Core Team (2022), R: A language and environment for statistical computing, Viena, R Foundation for Statistical Computing, <https://www.R-project.org/> [Consulta: diciembre de 2022].
Ramírez-Esparza, Nairán, James Pennebaker, Florencia García y Raquel Suría (2007), “La psicología del uso de las palabras: un programa de computadora que analiza textos en español”, Revista Mexicana de Psicología, vol. 24, núm. 1, pp. 85-99, DOI: http://hdl.handle.net/10045/25918.
Reid, Lándern (2010), “The role of perceived race and gender in the evaluation of college teaching on RateMyProfessors.Com”, Journal of Diversity in Higher Education, núm. 3, pp. 137-152, DOI: https://doi.org/10.1037/a0019865.
Roorda, Deborah, Helma Koomen, Jantine Spilt y Frans Oort (2011), “The Influence of Affective Teacher–Student Relationships on Students’ School Engagement and Achievement: A Meta-Analytic Approach”, Review of Educational Research, núm. 81, pp. 493-529, DOI: https://doi.org/10.3102/0034654311421793.
Rosen, Andrew (2018), “Correlations, trends and potential biases among publicly accessible web-based student evaluations of teaching: a large-scale study of RateMyProfessors.com data”, Assessment & Evaluation in Higher Education, núm. 43, pp. 31-44, DOI: https://doi.org/10.1080/02602938.2016.1276155.
RStudio Team (2022), RStudio: Integrated Development for R. Mass, Rstudio, <http://www.rstudio.com>.
Sancar, Raziye, Deniz Atal y Deniz Deryakulu (2021), “A new framework for teachers’ professional development”, Teaching and Teacher Education, vol. 101, pp. 103305, DOI: https://doi.org/10.1016/j.tate.2021.103305.
Scheerens, Jaap, Greetje van der Werf y Hester de Boer (2020), Soft Skills in Education: Putting the evidence in perspective, Nueva York, Springer International Publishing, DOI: https://doi.org/10.1007/978-3-030-54787-5.
Silge, Julia y David Robinson (2016), “tidytext: Text Mining and Analysis Using Tidy Data Principles in R”, Journal of Open Source Software, núm. 1, pp. 37-38, DOI: http://doi.org/10.21105/joss.00037.
Sonntag, Michael, Jonathan Bassett y Timothy Snyder (2009), “An empirical test of the validity of student evaluations of teaching made on RateMyProfessors.com”, Assessment & Evaluation in Higher Education, núm. 34, pp. 499-504, DOI: https://doi.org/10.1080/02602930802079463.
Spilt, Jantine, Helma Koomen y Jochem Thijs (2011), “Teacher Wellbeing: The Importance of Teacher–Student Relationships”, Educational Psychology Review, núm. 23, pp. 457-477, DOI: https://doi.org/10.1007/s10648-011-9170-y.
Stuber, Jennie, Amanda Watson, Adam Carle y Kristin Staggs (2009), “Gender expectations and on-line evaluations of teaching: evidence from RateMyProfessors.com”, Teaching in Higher Education, núm. 14, pp. 387-399, DOI: https://doi.org/10.1080/13562510903050137.
Stadthagen-Gonzalez, Hans, Constance Imbault, Miguel Pérez Sánchez y Marc Brysbaert (2017), “Norms of valence and arousal for 14,031 Spanish words”, Behavior Research Methods, núm. 49, pp. 111-123, DOI: https://doi.org/10.3758/s13428-015-0700-2.
Tausczik, Yla y James Pennebaker (2010), "The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods", Journal of Language and Social Psychology, núm. 29, pp. 24-54, DOI: https://doi.org/10.1177/0261927X09351676.
Thapa, Amrit, Jonathan Cohen, Shawn Guffey y Ann Higgins-D’Alessandro (2013), “A Review of School Climate Research”, Review of Educational Research, núm. 83, pp. 357-385, DOI: https://doi.org/10.3102/0034654313483907.
Timmerman, Thomas (2008), “On the Validity of RateMyProfessors.com”, Journal of Education for Business, núm. 84, pp. 55-61, DOI: https://doi.org/10.3200/JOEB.84.1.55-61.
Vuorinen, Kaisa, Anna Pessi y Lotta Uusitalo (2021), “Nourishing Compassion in Finnish Kindergarten Head Teachers: How Character Strength Training Influences Teachers’ Other-Oriented Behavior”, Early Childhood Education Journal, núm. 49, pp. 163-176, DOI: https://doi.org/10.1007/s10643-020-01058-0.
Wickham, Hadley, Mara Averick, Jenniffer Bryan y Yutani Hiroaki (2019), “Welcome to the tidyverse”, Journal of Open Source Software, núm. 4, pp. 1-4, DOI: https://doi.org/10.21105/joss.01686.
Zhou, Jin y Jun-min Ye (2020), “Sentiment analysis in education research: a review of journal publications”, Interactive Learning Environments, núm. 31, pp. 1–13, DOI: https://doi.org/10.1080/10494820.2020.1826985.
Ziyan Liu, Li Yingnan, Zhu Wenying, He Yuanping y Li Dongbin (2022), “A Meta-Analysis of Teachers’ Job Burnout and Big Five Personality Traits”, Frontiers in Education, núm. 7, pp. e822659, DOI: https://doi.org/10.3389/feduc.2022.822659.
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
