The present paper aims at assessing the linguistic and cultural adaptation and equivalency of the French, Farsi, Arabic and English versions of the NADINE1 Soft Skill Tests, which include four scales: interpersonal skills, organizational skills, personal skills, and entrepreneurial skills. The translation process included forward and backward translations. Three studies took place. In Study 1, both the source (English) and target versions of the tests were administered to 117 bilingual subjects. There were very few differences observed between the two versions of the tests, in all language versions, resulting in only a few items being revised. Study 2 aimed at determining the degree to which the items, the response scale and the instructions are comprehensible and culturally relevant to the target population. Three focus groups were formed, with 9, 9 and 7 participants respectively, who rated all items in terms of clarity and cultural relevance. The review resulted in minor changes in the instructions. In study 3, the Farsi, Arabic and French versions of the tests were administered to 120 migrants/refugees/asylum seekers. We calculated the discriminative power of each item and its contribution to the scale’s and subscale’s internal reliability coefficient. Moreover, we calculated Tucker’s congruence coefficient for each subscale to ensure cross-cultural equivalency. Items with poor discriminative power and items which reduced the reliability coefficients were eliminated. The remaining items had discriminative power between 0.20 and 0.80, while all reliability coefficients were above 0.70 (the vast majority being above 0.80). All Tucker’s phi coefficients were above the 0.90 cut-off point, with the exception of the Farsi-French and the Arabic-French comparison of one component. However, Tucker’s phi was above 0.80 which shows similarity. Results of all 3 studies support the case of linguistic and cultural equivalence between the different language versions of the tests. Therefore, all test versions are valid and reliable instruments that can be used by career practitioners to assess the level of soft skills of migrants, refugees and asylum seekers.
References
[1]
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). The Standards for Educational and Psychological Testing. Washington DC: AERA Publications.
Coolican, H. (2014). Research Methods and Statistics in Psychology (6th ed.). New York: Psychology Press. https://doi.org/10.4324/9780203769669
[4]
DeMaio, T., & Landreth, A. (2004). Do Different Cognitive Interview Techniques Produce Different Results? In S. Presser et al. (Eds.), Methods for Testing and Evaluating Survey Questionnaires (pp. 891-08). Hoboken, NJ: John Wiley and Sons. https://doi.org/10.1002/0471654728.ch5
[5]
European Commission (2018). Council Recommendation of 22 May 2018 on Key Competences for Lifelong Learning (Text with EEA Relevance) (pp. 1-13). ST/9009/ 2018/INIT OJ C 189, 4.6.2018, (BG, ES, CS, DA, DE, ET, EL, EN, FR, HR, IT, LV, LT, HU, MT, NL, PL, PT, RO, SK, SL, FI, SV). https://eur-lex.europa.eu/legal-content/EN/TXT/ ?uri=uriserv%3AOJ.C_.2018.189.01.0001.01.ENG
[6]
Furr, R. M. (2011). Scale Construction and Psychometrics for Social and Personality Psychology. New Delhi: Sage Publications. https://doi.org/10.4135/9781446287866
[7]
Hambleton, R. K. (2005). Issues, Designs, and Technical Guidelines for Adapting Tests into Multiple Languages and Cultures. In K. Hambleton, P. Merenda, & C. Spielberger (Eds.), Adapting Educational and Psychological Tests for Cross-Cultural Assessment (pp. 3-38). Mahwah, NJ: Lawrence Erlbaum. https://doi.org/10.4324/9781410611758
[8]
Harkness, J., Pennell, B. E., & Schoua-Glusberg, A. (2004). Survey Questionnaire Translation and Assessment. In S. Presser et al. (Eds.), Methods for Testing and Evaluating Survey Questionnaires (pp. 546, 453-473). Hoboken, NJ: John Wiley & Sons, Inc. https://doi.org/10.1002/0471654728.ch22
[9]
ILO International Labour Organization (2008). Conclusions on Skills for Improved Productivity, Employment Growth and Development. In International Labour Conference. Geneva: International Labour Office ILO. https://www.ilo.org/wcmsp5/groups/public/---ed_emp/---ifp_skills /documents/publication/wcms_103457.pdf
[10]
ILO International Labour Organization (2015). Labour Migration and Skills Matching. Labour Migration Highlights No. 6. Geneva: Labour Migration Branch and Skills and Employability Branch International Labour Office.
[11]
ILO International Labour Organization (2017). How to Facilitate the Recognition of Skills of Migrant Workers—Guide for Employment Service Providers. Geneva: ILO.
[12]
Irwing, P., & Hughes, D. J. (2018). Test Development. In The Wiley Handbook of Psychometric Testing (pp. 1-47). New Jersey: John Wiley & Sons Ltd. https://doi.org/10.1002/9781118489772.ch1
[13]
Kankaraš, M. (2017). Personality Matters: Relevance and Assessment of Personality Characteristics. OECD Education Working Papers, No. 157, Paris: OECD Publishing.
[14]
Kankaraš, M., & Moors, G. (2010). Researching Measurement Equivalence in Cross-Cultural Studies. Psihologija, 43, 121-135. https://doi.org/10.2298/PSI1002121K
[15]
McGorry, S. (2000). Measurement in a Cross-Cultural Environment: Survey Translation Issues. Qualitative Market Research: An International Journal, 3, 74-81. https://doi.org/10.1108/13522750010322070
[16]
Nunnally, J. C. (1978). Psychometric Theory (2nd ed.). New York: McGraw-Hill.
[17]
OECD (2005). Definition and Selection of Key Competencies—Executive Summary. https://www.oecd.org/pisa/35070367.pdf
OECD (2018). Skills on the Move: Migrants in the Survey of Adult Skills. OECD Skills Studies, Paris: OECD Publishing. https://doi.org/10.1787/9789264307353-en
[20]
Oremus, M., Cosby, J. L., & Wolfson, C. (2005). A Hybrid Qualitative Method for Pretesting Questionnaires: The Example of a Questionnaire to Caregivers of Alzheimer Disease Patients. Research in Nursing & Health, 28, 419-430. https://doi.org/10.1002/nur.20095
[21]
Parsons, F. (1909). Choosing a Vocation. Boston, MA: Houghton Mifflin.
[22]
Polit, D. F., & Beck, C. T. (2006). The Content Validity Index: Are You Sure You Know What’s Being Reported? Critique and Recommendations. Research in Nursing & Health, 29, 489-497. https://doi.org/10.1002/nur.20147
[23]
Presser, S., & Blair, J. (1994). Survey Pretesting: Do Different Methods Produce Different Results? Sociological Methodology, 24, 73-104. https://doi.org/10.2307/270979
[24]
Price, L. R. (2017). Psychometric Methods: Theory into Practice. New York: The Guilford Press
[25]
Sireci, S. G., Yang, Y., Harter, J., & Ehrlich, E. J. (2006). Evaluating Guidelines for Test Adaptations: A Methodological Analysis of Translation Quality. Journal of Cross-Cultural Psychology, 37, 557-567. https://doi.org/10.1177/0022022106290478
[26]
Society for Industrial and Organizational Psychology (2003). Principles for the Validation and Use of Personnel Selection Procedures. Bowling Green, OH: Author, Google Scholar.
[27]
Tanzer, N. K. (2005). Developing Tests for Use in Multiple Languages and Cultures: A Plea for Simultaneous Development. In R. K. Hambleton, P. F. Merenda, & C. D. Spielberger (Eds.), Adapting Educational and Psychological Tests for Cross-Cultural Assessment (pp. 235-264). Mahwah, NJ: Erlbaum.
[28]
The Adecco Group (2017a). The Labour Market Integration of Refugees’ White Paper: A Focus on Europe. White Paper 06/17, Adecco Group Publications. https://www.adeccogroup.com/wp-content/themes/ado-group/downloads /labour-market-integration-of-refugees-focus-europe.pdf
[29]
The Adecco Group (2017b). The Soft Skills Imperative. White Paper 1/2017. https://www.adeccogroup.com/wp-content/themes/ado-group/downloads /the-adecco-group-white-paper-the-soft-skills-imperative.pdf
[30]
Van de Vijver, F. J. R., & Leung, K. (1997). Cross-Cultural Psychology Series, Vol. 1. Methods and Data Analysis for Cross-Cultural Research. London: Sage Publications, Inc.
[31]
Willis, G., Schechter, S., & Whitaker, K. (2000). A Comparison of Cognitive Reviewing, Expert Review and Behavior Coding: What Do They Tell Us? In Proceedings of the Section on Survey Methods, American Statistical Association (pp. 28-37). Alexandria, VA: American Statistical Association.
