Email this article Criterion Validation of the Scales of Autonomy, Collaboration, Empathy, Problem-solving and Self-confidence of the 3SQ. Soft Skills Self-evaluation Questionnaire Adapted for Lower Secondary School
Abstract
The Soft Skills Self-evaluation Questionnaire (3SQ, 41 items) was validated using the responses of 1216 Italian students attending the fourth and the fifth grade of upper secondary school (Lucisano & du Mérac, 2019a), demonstrating good psychometric properties (du Mérac & De Santis, 2020; du Mérac, Livi, & Lucisano, 2020). The adaptation of the 3SQ (40 items) for 8th-grade students was validated in Rome using the responses of 507 students, confirming the factorial structure, sufficient independence, and a good internal consistency (Cronbach alpha between .75 and .92 and 52.23% of the total variance explained). Here, we present the criterion-related validity of the five scales of the adapted 3SQ: Self-confidence, Autonomy, Problem-solving, Cooperation, Empathy. During a PhD research project (Scippo, 2021), these dimensions were also measured in Rome with other instruments, using a sample of 403 8th grade students. The analysis of the data confirms good reliability of the five scales (between .79 and .92), shows good indices of the confirmatory factor analysis of all the scales (RMSEA = .05, SRMR = .08), and reveals significant correlations (between .38 and .68) between the scales of the adapted 3SQ and the other scales measuring the same dimensions. In conclusion, this concurrent validity check reinforces the validity of the adapted 3SQ construct and, consequently, its usefulness for both research and guidance.
Keywords
Full Text:
PDFReferences
Ackerman, P. L., Beier, M. E., & Bowen, K. R. (2002). What we really know about our abilities and our knowledge. Personality and Individual Differences, 33, 587-605.
https://doi.org/10.1016/S0191-8869(01)00174-X
Anastasi, A. (1982). Psychological testing (5th ed.). New York: Macmillan.
Anderson, A. B., Basilevsky, A., & Hum, D. P. J. (1983). Measurement: Theory and techniques. In P. H. Rossi, J. D. Wright, & A. B. Anderson (Eds.), Handbook of survey research (pp. 231-287). New York: Academic Press.
https://doi.org/10.1016/B978-0-12-598226-9.50013-6
PMCid:PMC2589669
Bailey, K. D. (1982). Methods of social research. New York: The Free Press (trad. it. Metodi della ricerca sociale, a cura di M. Rossi. Bologna: il Mulino, 1995).
Barbaranelli, C. (2003). Analisi dei dati. Tecniche multivariate per la ricerca psicolog-ica e sociale. Milano: LED Edizioni Universitarie.
Barnette, J. J. (2000). Effects of stem and Likert response option reversals on survey internal consistency: If you feel the need, there is a better alternative to using those negatively worded stems. Educational and Psychological Measurement, 60(3), 361-370.
https://doi.org/10.1177/00131640021970592
Bédard, K. (2016). Biais d'auto-évaluation de compétence sociale, perception du soutien social et difficultés d'adaptation psychosociale à l'adolescence. PhD thesis in Psychology, Université du Québec, Montréal.
Bédard, K., Bouffard, T., & Pansu, P. (2014). The risks for adolescents of nega-tively biased self-evaluations of social competence: The mediating role of so-cial support. Journal of Adolescence, 37, 787-798.
https://doi.org/10.1016/j.adolescence.2014.05.004
PMid:25086456
Bonino, S., Lo Coco A., & Tani F. (1998). Empatia. I processi di condivisione delle emozioni. Firenze: Giunti.
Boud, D. (1995). Assessment and learning: Contradictory or complementary? In P. Knight (Ed.), Assessment for learning in higher education (pp. 35-48). Lon-don: Kogan Page.
Boud, D., & Falchikov, N. (1989). Quantitative studies of student self-assessment in higher education: A critical analysis of findings. Higher Education, 18(5), 529-549.
https://doi.org/10.1007/BF00138746
Bouffard, T., Boisvert, M., & Vezeau, C. (2002). The illusion of incompetence and its correlates among elementary school children and their parents. Learning and Individual Differences, 14, 31-46.
https://doi.org/10.1016/j.lindif.2003.07.001
Bouffard, T., Roy, M., & Vezeau, C. (2005). Self-perceptions, temperament, soci-oemotional adjustment and the perceptions of parental support of chroni-cally under achieving children. International Journal of Educational Research, 43, 215-235.
https://doi.org/10.1016/j.ijer.2006.06.003
Bouffard, T., Vezeau, C., Roy, M., & Lengelé, A. (2011). Stability of biases in self-evaluation and relations to well-being among elementary school chil-dren. International Journal of Educational Research, 50, 221-229.
https://doi.org/10.1016/j.ijer.2011.08.003
Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn brain, mind, experience, and school. Washington, DC: National Academy Press.
Brendgen, M., Vitaro, F., Turgeon, L., & Poulin, F. (2002). Assessing aggressive and depressed children's social relations with classmates and friends: A mat-ter of perspective. Journal of Abnormal Child Psychology, 30(6), 609-624.
https://doi.org/10.1023/A:1020863730902
PMid:12481975
Cappellari, L., Castelnovo, P., Checchi, D., & Leonardi, M. (2017). Skilled or edu-cated? Educational reforms, human capital, and earnings. In W. Solomon, K. Pouliakas, G. Russo, & K. Tatsiramos (Eds.), Skill mismatch in labor markets (pp. 173-197). London: Emerald Publishing Limited.
https://doi.org/10.1108/S0147-912120170000045005
Caprara, G. V., Steca, P., Gerbino, M., Paciello, M., & Vecchio, G. (2006). Look-ing for adolescents' well-being: Self-efficacy beliefs as determinants of posi-tive thinking and happiness. Epidemiologia e Psichiatria Sociale, 15(1), 30-43.
https://doi.org/10.1017/S1121189X00002013
PMid:16584101
Cillessen, A. H. N., & Bellmore, A. D. (2004). Social skills and interpersonal per-ception in early and middle childhood. In P. K. Smith & C. H. Hart (Eds.), Blackwell handbook of childhood social development (pp. 355-374). Malden, MA: Blackwell.
https://doi.org/10.1111/b.9780631217534.2004.00024.x
Cimatti, B. (2016). Definition, development, assessment of soft skills and their role for the quality of organisations and enterprises. International Journal for Quality Research, 10(1), 97-130.
Cole, D. A., Martin, J. M., Peeke, L. A., Seroczynski, A. D., & Hoffman, K. (1998). Are cognitive errors of underestimation predictive or reflective of depressive symptoms in children: A longitudinal study. Journal of Abnonnal Psychology, 107, 481-496.
https://doi.org/10.1037/0021-843X.107.3.481
Connolly, J. (1989). Social self-efficacy in adolescence: Relations with self-concept, social adjustment, and mental health. Canadian Journal of Behavioral Sci-ence, 21, 258-269.
https://doi.org/10.1037/h0079809
De Santis, C., & du Mérac, E. R. (2019). Valutare le soft skill a scuola, insegnanti e studenti a confronto. In G. Domenici & V. Biasi (a cura di), Atteggiamento scientifico e formazione dei docenti (pp. 203-207). Milano: FrancoAngeli.
De Santis, C., & Stanzione, I. (2020). Percezione delle soft-skills. Adattamento e try-out del 3SQ - Soft Skills Self-evaluation Questionnaire per studenti della scuola secondaria di primo grado. Giornale Italiano della Ricerca Educativa, 13(25), 63-73.
De Vellis, R. F. (1991). Scale development: Theory and applications. Newbury Park: Sage.
du Mérac, É. R. (2017). Contesti educativi e atteggiamenti di leadership. Indagine sullo sviluppo degli atteggiamenti di leadership democratica in ragazzi di 15-16 anni. Roma: Armando Editore.
du Mérac, É. R., & De Santis, C. (2020). Contesti di apprendimento delle soft skill degli studenti. Italian Journal of Educational Research, 1, Sezione SIRD, 147-155.
du Mérac, É. R., Livi, S., & Lucisano, P. (2020). Teens' Voice 2018/2019. Percezio-ni di sé e della società - Opinioni e consigli per la scuola. Roma: Nuova Cultura.
Engelberg, S. (2015). A developmental perspective on soft skills: Speech at «Soft skills and their role in employability - New perspectives in teaching, assess-ment and certification». Paper presented at the Workshop in Bertinoro, FC, Italy, November 18-19.
Gerber, J. P., Wheeler, L., & Suls, J. (2018). A social comparison theory me-ta-analysis 60+ years on. Psychological Bulletin, 144(2), 177-197.
https://doi.org/10.1037/bul0000127
PMid:29144145
Hoffman, K. B., Cole, D. A., Martin, J. M., Tram, J., & Seroczynski, A. D. (2000). Are the discrepancies between self- and others appraisals of competence pre-dictive of reflective of depressive symptoms in children and adolescents: A longitudinal study, part II. Journal of Abnormal Psychology, 109(4), 651-662.
https://doi.org/10.1037/0021-843X.109.4.651
PMid:11195989
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance struc-ture analysis: Conventional criteria versus new alternatives. Structural Equa-tion Modeling: A Multidisciplinary Journal, 6(1), 1-55.
https://doi.org/10.1080/10705519909540118
Lee, A., Hankin, B. L., & Mermelstein, R. J. (2010). Perceived social competence, negative social interactions, and negative cognitive style predict depressive symptoms during adolescence. Journal of Clinical Child and Adolescent Psy-chology, 39, 603-615.
https://doi.org/10.1080/15374416.2010.501284
PMid:20706914 PMCid:PMC2921900
Livi, S., Cecalupo, A., Scarci, F., & Luongo, S. (2019). La classe scolastica come rife-rimento sociale della persona. Effetti sul bullismo e sul confronto sociale tra gli studenti. In G. Benvenuto, P. Sposetti, & G. Szpunar (a cura di), Tutti i bisogni educativi sono speciali. Riflessioni, ricerche, esperienze didattiche (pp. 97-116). Roma: Nuova Cultura.
Lucisano, P., & du Mérac, É. R. (2015a). Teen's Voice. Aspirazioni, progetti, ideali dei giovani. Roma: Nuova Cultura.
Lucisano, P., & du Mérac, É. R. (2015b). School and scouting: The touchstone. Scuola Democratica, 3(6), 545-568.
Lucisano, P., & du Mérac, É. R. (2016). Teen's Voice 2. Valori e miti dei giovani. 2015-2016. Roma: Nuova Cultura.
Lucisano, P., & du Mérac, É. R. (2019a). Soft Skills Self-evaluation Questionnaire (3SQ), caratteristiche e proprietà psichometriche. In P. Lucisano & A. Notti (a cura di), Convegno internazionale SIRD: Training actions and evaluation processes (pp. 609-622). Lecce: Pensa Multimedia.
Lucisano, P., & du Mérac, É. R. (2019b). Teen's Voice 3. Valori, contesti e lavoro. 2016-2017. Roma: Nuova Cultura.
Marsh, H. W. (1987). The big-fish-little-pond effect on academic self concept. Journal of Educational Psychology, 79(3), 280-295.
https://doi.org/10.1037/0022-0663.79.3.280
McElhaney, K. B., Antonishak, J., & Allen, J. P. (2008). They like me, they like me not: Popularity and adolescents' perceptions of acceptance predicting social functioning over time. Child Development, 79(3), 720-731.
https://doi.org/10.1111/j.1467-8624.2008.01153.x
PMid:18489423 PMCid:PMC3073367
McGrath, E. P., & Repetti, R. L. (2002). A longitudinal study of children's depres-sive symptoms, self-perceptions, and cognitive distorsions about the self. Jounal of Abnormal Psychology, 111, 77-87.
https://doi.org/10.1037/0021-843X.111.1.77
Moretti, G., Biasci V., Giuliani A., & Morini A. (2017). Sviluppo delle capacità di problem solving nella scuola secondaria di primo grado e apprendimento lo-gico-linguistico e logico-matematico. Adattamento e validazione in Italia «della versione breve» del Test della «Personalità Creativa» (TCD-As). Gior-nale Italiano della Ricerca Educativa, 10, 115-128.
Moscovici, S., Argyle, M., Beauvois, J.-L., & Doise, W. (2005). Psychologie sociale des relations à autrui. Paris: Armand Colin.
Noom, M. J., Dekovic, M., & Meeus, W. (2001). Conceptual analysis and meas-urement of adolescent autonomy. Journal of Youth and Adolescence, 30(5), 577-595.
https://doi.org/10.1023/A:1010400721676
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.
OECD (2015). Skills for social progress: The power of social and emotional skills. Paris: OECD Publishing.
https://doi.org/10.1787/9789264226159-en
Pellerey, M. (2017). Soft skill e orientamento professionale. Roma: CNOS-FAP.
Phillips, D., & Zimmermann, M. (1990). The developmental course of perceived competence and incompetence among competent children. In R. Sternberg & J. Kolligian (Eds.), Competence considered (pp. 41-66). New Haven, CT: Yale University Press.
Phillips, J., & Springer, F. (1992). Extended national youth sports program 1991-92 evaluation highlights, part two: Individual Protective Factors In-dex (IPFI) and risk assessment study. Report prepared for the National Col-legiate Athletic Association. Sacramento, CA: EMT Associates (Un-published).
Pilotte, W. J., & Gable, R. K. (1990). The impact of positive and negative item stems on the validity of a computer anxiety scale. Educational and Psycholog-ical Measurement, 50, 603-610.
https://doi.org/10.1177/0013164490503016
Pintrich, P. R., & Schunk, D. (1996). Motivation in education: Theory, research, and applications. Englewood Cliffs, NJ: Prentice-Hall.
Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.
https://doi.org/10.1515/9781400876136
Ross, J. A. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment, Research, and Evaluation, 11(10), 1-13.
Rovai, A. P. (2002). Sense of community, perceived cognitive learning, and persis-tence in asynchronous learning networks. The Internet and Higher Educa-tion, 5(4), 319-332.
https://doi.org/10.1016/S1096-7516(02)00130-6
Schmitt, N., & Stuits, D. (1985). Factors defined by negatively keyed items: The result of careless respondents? Applied Psychological Measurement, 9(4), 367-373.
https://doi.org/10.1177/014662168500900405
Schriesheim, C. A., & Hill, K. D. (1981). Controlling acquiescence response bias by item reversals: The effect on questionnaire validity. Educational and Psycho-logical Measurement, 41(4), 1101-1114.
https://doi.org/10.1177/001316448104100420
Scippo, S. (2021). Il metodo dell'uniformazione semplice per isolare l'effetto della formazione Montessori su apprendimenti disciplinari e caratteristiche so-cio-affettive. In G. Moretti, I. Vannini, & A. La Marca (a cura di), La ricerca educativa e didattica nelle scuole di dottorato in Italia (pp. 368-390). Lecce: Pensa Multimedia.
Shrauger, J. S., & Osberg, T. M. (1981). The relative accuracy of self-predictions and judgments by others in psychological assessment. Psychological Bulletin, 90, 322-351.
https://doi.org/10.1037/0033-2909.90.2.322
Stewart, T. J., & Frye, A. W. (2004). Investigating the use of negatively-phrased survey items in medical education settings: Common wisdom or common mistake? Academic Medicine, 79(10, Suppl.), S1-S3.
https://doi.org/10.1097/00001888-200410001-00006
PMid:15383379
Sullivan, K., & Hall, C. (1997). Introducing students to self-assessment. Assessment and Evaluation in Higher Education, 22(3), 289-306.
https://doi.org/10.1080/0260293970220303
Unione Europea, 2011. Transferability of skills across economic sectors: Role and im-portance for employment at European level. Luxembourg: EU Publications.
Vygotskij, L. S. (1934). Myšlenie i recˆ. Psichologičeskie issledovanija. Moscow: Socekiz (trad. it. Pensiero e linguaggio. Firenze: Giunti, 1966).
Waters, S., & Cross, D. (2010). Measuring students' connectedness to school, teachers, and family: Validation of three scales. School Psychology Quarterly, 25, 164-177.
https://doi.org/10.1037/a0020942
Wheeler, V. A., & Ladd, G. W. (1982). Assessment of children's self-efficacy for social interactions with peers. Developmental Psychology, 18, 795-805.
https://doi.org/10.1037/0012-1649.18.6.795
Wong, N., Rindfleisch, A., & Burroughs, J. (2003). Do reverse-worded items con-found measures in crosscultural consumer research? The case of the material values scale. Journal of Consumer Research, 30, 72-91.
https://doi.org/10.1086/374697
Zimprich, D., Perren, S., & Hornung, R. (2005). A two-level confirmatory factor analysis of a modified Rosenberg self-esteem scale. Educational and Psycho-logical Measurement, 65(3), 465-481.
https://doi.org/10.1177/0013164404272487
DOI: https://doi.org/10.7358/ecps-2021-024-scme
Copyright (©) 2021 Emiliane Rubat du Mérac – Editorial format and Graphical layout: copyright (©) LED Edizioni Universitarie
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Journal of Educational, Cultural and Psychological Studies (ECPS)
Registered by Tribunale di Milano (19/05/2010 n. 278)
Online ISSN 2037-7924 - Print ISSN 2037-7932
Research Laboratory on Didactics and Evaluation - Department of Education - "Roma Tre" University
Executive Editor: Gaetano Domenici - Associate Executive Editor & Managing Editor: Valeria Biasci
Editorial Board: Eleftheria Argyropoulou - Massimo Baldacci - Joao Barroso - Richard Bates - Christofer Bezzina - Paolo Bonaiuto - Lucia Boncori - Pietro Boscolo - Sara Bubb - Carlo Felice Casula - Jean-Émile Charlier - Lucia Chiappetta Cajola - Carmela Covato - Jean-Louis Derouet - Peter Early - Franco Frabboni - Constance Katz - James Levin - Pietro Lucisano - Roberto Maragliano - Romuald Normand - Michael Osborne - Donatella Palomba - Michele Pellerey - Clotilde Pontecorvo - Vitaly V. Rubtzov - Jaap Scheerens - Noah W. Sobe - Francesco Susi - Giuseppe Spadafora - Pat Thomson
Editorial Staff: Fabio Alivernini - Guido Benvenuto - Anna Maria Ciraci - Massimiliano Fiorucci - Luca Mallia - Massimo Margottini - Giovanni Moretti - Carla Roverselli
Editorial Secretary:Nazarena Patrizi
© 2001 LED Edizioni Universitarie di Lettere Economia Diritto