Adequate statistical power contributes to observing true relationships in a dataset. With a thoughtful power analysis, the adequate but not excessive sample could be detected. Therefore, this paper reviews the issue of what sample size and sample power the researcher should have in the EFA, CFA, and SEM study. Statistical power is the estimation of the sample size that is appropriate for an analysis. In any study, four parameters related to power analysis are Alpha, Beta, statistical power and Effect size. They are prerequisites for a priori sample size determination. Scale development in general and Factor Analysis (EFA, CFA) and SEM are large sample size methods because sample affects precision and replicability of the results. However, the existing literature provides limited and sometimes conflicting guidance on this issue. Generally, for EFA the stronger the data, the smaller the sample can be for an accurate analysis. In CFA and SEM parameter estimates, chi-square tests and goodness of fit indices are equally sensitive to sample size. So the statistical power and precision of CFA/SEM parameter estimates are also influenced by sample size. In this work after reviewing existing sample power analysis rules along with more elaborated methods (like Monte Carlo simulation), we conclude with suggestions for small samples in factor analysis found in literature.
References
[1]
Anderson, J. C., & Gerbing, D. W. (1984). The Effect of Sampling Error on Convergence, Improper Solutions, and Goodness-of-Fit Indices for Maximum Likelihood Confirmatory Factor Analysis. Psychometrika, 49, 155-173. https://doi.org/10.1007/BF02294170
[2]
Anderson, J. C., & Gerbing, D. W. (1988). Structural Equation Modeling in Practice: A Review and Recommended Two-Step Approach. Psychological Bulletin, 103, 411. https://doi.org/10.1037/0033-2909.103.3.411
[3]
Bandalos, D. L. (2014). Relative Performance of Categorical Diagonally Weighted Least Squares and Robust Maximum Likelihood Estimation. Structural Equation Modeling: A Multidisciplinary Journal, 21, 102-116. https://doi.org/10.1080/10705511.2014.859510
[4]
Bandalos, D. L., & Leite, W. (2013). Use of Monte Carlo Studies in Structural Equation Modeling. In G. R. Hancock, & R. O. Mueller (Eds.), Structural Equation Modeling: A Second Course (2nd ed., pp. 625-666). Charlotte, NC: IAP.
[5]
Barker, C., Pistrang, N., & Elliott, R. (2015). Research Methods in Clinical Psychology: An Introduction for Students and Practitioners (3rd ed.). Oxford, UK: John Wiley & Sons, Ltd.
[6]
Bentler, P. M., & Chou, C. P. (1987). Practical Issues in Structural Modeling. Sociological Methods & Research, 16, 78-117. https://doi.org/10.1177/0049124187016001004
[7]
Bentler, P. M., & Yuan, K. H. (1999). Structural Equation Modeling with Small Samples: Test Statistics. Multivariate Behavioral Research, 34, 181-197. https://doi.org/10.1207/S15327906Mb340203
[8]
Bollen, K. A. (1989). Structural Equations with Latent Variables. New York: Jon Wiley & Sons. https://doi.org/10.1002/9781118619179
[9]
Bollen, K. A., & Liang, J. (1988). Some Properties of Hoelter's CN. Sociological Methods & Research, 16, 492-503. https://doi.org/10.1177/0049124188016004003
[10]
Bollen, K. A., & Stine, R. A. (1992). Bootstrapping Goodness-of-Fit Measures in Structural Equation Models. Sociological Methods & Research, 21, 205-229. https://doi.org/10.1177/0049124192021002004
[11]
Boomsma, A. (1985). Nonconvergence, Improper Solutions, and Starting Values in LISREL Maximum Likelihood Estimation. Psychometrika, 50, 229-242. https://doi.org/10.1007/BF02294248
[12]
Boomsma, A., & Hoogland, J. J. (2001). The Robustness of LISREL Modeling Revisited. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural Equation Models: Present and Future. A Festschrift in Honor of Karl Jöreskog (pp. 139-168). Lincolnwood, IL: Scientific Software International.
[13]
Brown, T. A. (2015). Confirmatory Factor Analysis for Applied Research (2nd ed.). New York: The Guilford Press.
[14]
Byrne, B. M. (2012). Structural Equation Modeling with Mplus: Basic Concepts, Applications, and Programming (2nd ed.). New York: Routledge.
[15]
Cattell, R. B. (1978). The Scientific Use of Factor Analysis in Behavioral and Life Sciences. New York: Plenum. https://doi.org/10.1007/978-1-4684-2262-7
[16]
Chen, F., Bollen, K. A., Paxton, P., Curran, P. J., & Kirby, J. B. (2001). Improper Solutions in Structural Equation Models: Causes, Consequences, and Strategies. Sociological Methods and Research, 29, 468-508. https://doi.org/10.1177/0049124101029004003
[17]
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance. Structural Equation Modeling, 9, 233-255. https://doi.org/10.1207/S15328007SEM0902_5
[18]
Chumney, F. L. (2013). Structural Equation Models with Small Samples: A Comparative Study of Four Approaches (p. 189). College of Education and Human Sciences. http://digitalcommons.unl.edu/cehsdiss/189
[19]
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences.
[20]
Cohen, J. (1990). Things I Have Learned (so far). American Psychologist, 45, 1304-1312. https://doi.org/10.1037/0003-066X.45.12.1304
[21]
Cohen, J. (1992). A Power Primer. Psychological Bulletin, 112, 155-159. https://doi.org/10.1037/0033-2909.112.1.155
[22]
Comrey, A. L. (1988). Factor-Analytic Methods of Scale Development in Personality and Clinical Psychology. Journal of Consulting and Clinical Psychology, 56, 754-761. https://doi.org/10.1037/0022-006X.56.5.754
[23]
Comrey, A. L., & Lee, H. B. (1992). A First Course in Factor Analysis. Hillsdale, NJ: Lawrence Eribaum Associates.
[24]
Comrey, A. L., & Lee, H. B. (1992). Interpretation and Application of Factor Analytic Results.
[25]
Comrey, A. L., Backer, T. E., & Glaser, E. M. (1973). A Sourcebook for Mental Health Measures.
[26]
Coolican, H. (2014). Research Methods and Statistics in Psychology (6th ed.). New York, NY: Psychology Press.
[27]
Costello, A. B., & Osborne, J. (2005). Best Practices in Exploratory Factor Analysis: Four Recommendations for Getting the Most from Your Analysis. Practical Assessment Research & Evaluation, 10, 1-9.
[28]
Curran, P. J., Bollen, K. A., Paxton, P., Kirby, J., & Chen, F. (2002). The Noncentral Chi-Square Distribution in Misspecified Structural Equation Models: Finite Sample Results from a Monte Carlo Simulation. Multivariate Behavioral Research, 37, 1-36. https://doi.org/10.1207/S15327906MBR3701_01
[29]
DeVellis, R. F. (2017). Scale Development: Theory and Applications (4th ed.). Thousand Oaks, CA: Sage.
[30]
Dimitrov, D. M. (2012). Statistical Methods for Validation of Assessment Scale Data in Counseling and Related Fields. Alexandria, VA: American Counseling Association.
[31]
Ding, L., Velicer, W. F., & Harlow, L. L. (1995). Effects of Estimation Methods, Number of Indicators per Factor, and Improper Solutions on Structural Equation Modeling Fit Indices. Structural Equation Modeling: A Multidisciplinary Journal, 2, 119-143. https://doi.org/10.1080/10705519509540000
[32]
Du, H., Zhang, Z., & Yuan, K. (2017). Power Analysis for t-Test with Non-normal Data and Unequal Variances. In L. A. van der Ark et al. (Eds.), Quantitative Psychology, Springer Proceedings in Mathematics & Statistics (Volume 196, pp.373-380). Switzerland: Springer International. https://doi.org/10.1007/978-3-319-56294-0_32
[33]
Everitt, B. S. (1975). Multivariate Analysis: The Need for Data, and Other Problems. The British Journal of Psychiatry, 126, 237-240. https://doi.org/10.1192/bjp.126.3.237
[34]
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the Use of Exploratory Factor Analysis in Psychological Research. Psychological Methods, 4, 272-299. https://doi.org/10.1037/1082-989X.4.3.272
[35]
Fan, X., & Sivo, S. A. (2007). Sensitivity of Fit Indices to Model Misspecification and Model Types. Multivariate Behavioral Research, 42, 509-529. https://doi.org/10.1080/00273170701382864
[36]
Finch, H. W., Immekus, J. C., & French, B. F. (2016). Applied Psychometrics Using SPSS and AMOS. Charlotte, NC: Information Age Publishing Inc.
[37]
Floyd, F. J., & Widaman, K. F. (1995). Factor Analysis in the Development and Refinement of Clinical Assessment Instruments. Psychological Assessment, 7, 286-299. https://doi.org/10.1037/1040-3590.7.3.286
[38]
Forero, C. G., Maydeu-Olivares, A., & Gallardo-Pujol, D. (2009). Factor Analysis with Ordinal Indicators: A Monte Carlo Study Comparing DWLS and ULS Estimation. Structural Equation Modeling, 16, 625-641. https://doi.org/10.1080/10705510903203573
[39]
Gatignon, H. (2010). Confirmatory Factor Analysis. In Statistical Analysis of Management Data (pp. 59-122). New York, NY: Springer. https://doi.org/10.1007/978-1-4419-1270-1_4
[40]
Gorsuch, R. (1983). Factor Analysis. Hillsdale, NJ: L. Erlbaum Associates.
[41]
Guadagnoli, E., & Velicer, W. F. (1988). Relation of Sample Size to the Stability of Component Patterns. Psychological Bulletin, 103, 265-275. https://doi.org/10.1037/0033-2909.103.2.265
[42]
Hancock, G. R., & French, B. F. (2013). Power Analysis in Structural Equation Modeling. In G. R. Hancock, & R. O. Mueller (Eds.), Structural Equation Modeling: A Second Course (2nd ed., pp. 117-159). Charlotte, NC: IAP.
[43]
Hau, K. T., & Marsh, H. W. (2004). The Use of Item Parcels in Structural Equation Modeling: Non-Normal Data and Small Sample Sizes. British Journal of Mathematical and Statistical Psychology, 57, 327-351. https://doi.org/10.1111/j.2044-8317.2004.tb00142.x
[44]
Hoe, S. L. (2008). Issues and Procedures in Adopting Structural Equation Modeling Technique. Journal of Applied Quantitative Methods, 3, 76-83.
[45]
Hoelter, J. W. (1983). The Analysis of Covariance Structures: Goodness-of-Fit Indices. Sociological Methods & Research, 11, 325-344. https://doi.org/10.1177/0049124183011003003
[46]
Hoogland, J. J., & Boomsma, A. (1998). Robustness Studies in Covariance Structure Modeling: An Overview and a Meta-Analysis. Sociological Methods & Research, 26, 329-367. https://doi.org/10.1177/0049124198026003003
[47]
Hoyle, R. H. (1999). Statistical Strategies for Small Sample Research. New York, NY: Sage.
[48]
Hoyle, R. H., & Kenny, D. A. (1999). Statistical Power and Tests of Mediation. In R. H. Hoyle (ed.), Statistical Strategies for Small Sample Research (pp. 195-222). New York, NY: SAGE Publications.
[49]
Hu, L. T., & Bentler, P. M. (1999). Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria versus New Alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55. https://doi.org/10.1080/10705519909540118
[50]
Jaccard, J., & Wan, C. K. (1996). LISREL Approaches to Interaction Effects in Multiple Regression (No. 114). New York, NY: SAGE Publications. https://doi.org/10.4135/9781412984782
[51]
Jackson, D. L. (2001). Sample Size and Number of Parameter Estimates in Maximum Likelihood Confirmatory Factor Analysis: A Monte Carlo Investigation. Structural Equation Modeling, 8, 205-223. https://doi.org/10.1207/S15328007SEM0802_3
[52]
Jackson, D. L. (2003). Revisiting Sample Size and Number of Parameter Estimates: Some Support for the N:q Hypothesis. Structural Equation Modeling, 10, 128-141. https://doi.org/10.1207/S15328007SEM1001_6
[53]
Jöreskog, K. G., & Sörbom, D. (1993). LISREL 8: Structural Equation Modeling with the SIMPLIS Command Language. Scientific Software International.
[54]
Kelloway, E. K. (2015). Using Mplus for Structural Equation Modeling. Thousand Oaks, CA: Sage.
[55]
Kim, K. H. (2005). The Relation among Fit Indexes, Power, and Sample Size in Structural Equation Modeling. Structural Equation Modeling, 12, 368-390. https://doi.org/10.1207/s15328007sem1203_2
[56]
Kline, R. B. (2016). Principles and Practice of Structural Equation Modeling (4th ed.).
[57]
Lee, T., Cai, L., & MacCallum, R. (2012). Power Analysis for Tests of Structural Equation Models. In R. H. Hoyle (Ed.), Handbook of Structural Equation Modeling (pp. 181-194). New York: Guilford Press.
[58]
Loehlin, J. C. (2004). Latent Variable Models (4th ed.). Mahwah, NJ: Erlbaum.
[59]
Loehlin, J. C., & Beaujean, A. A. (2017). Latent Variable Models: An Introduction to Factor, Path, and Structural Equation Analysis. New York, NY: Taylor & Francis.
[60]
Lomax, R. G., & Hahs-Vaughn, D. L. (2013). An Introduction to Statistical Concepts. Abingdon-on-Thames: Routledge.
[61]
MacCallum, R. C., & Austin, J. T. (2000). Applications of Structural Equation Modeling in Psychological Research. Annual Review of Psychology, 51, 201-226. https://doi.org/10.1146/annurev.psych.51.1.201
[62]
MacCallum, R. C., & Hong, S. (1997). Power Analysis in Covariance Structure Modeling Using GFI and AGFI. Multivariate Behavioral Research, 32, 193-210. https://doi.org/10.1207/s15327906mbr3202_5
[63]
MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power Analysis and Determination of Sample Size for Covariance Structure Modeling. Psychological Methods, 1, 130-149. https://doi.org/10.1037/1082-989X.1.2.130
[64]
MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample Size in Factor Analysis. Psychological Methods, 4, 84-99. https://doi.org/10.1037/1082-989X.4.1.84
[65]
MacCallum, R., Lee, T., & Browne, M. W. (2010). The Issue of Isopower in Power Analysis for Tests of Structural Equation Models. Structural Equation Modeling, 17, 23-41. https://doi.org/10.1080/10705510903438906
[66]
Marsh, H. W., & Hau, K. T. (1999). Confirmatory Factor Analysis: Strategies for Small Sample Sizes. Statistical Strategies for Small Sample Research, 1, 251-284.
[67]
Marsh, H. W., Hau, K. T., Balla, J. R., & Grayson, D. (1998). Is More Ever too Much? The Number of Indicators per Factor in Confirmatory Factor Analysis. Multivariate Behavioral Research, 33, 181-220.https://doi.org/10.1207/s15327906mbr3302_1
[68]
Marsh, H. W., Wen, Z., & Hau, K. T. (2004). Structural Equation Models of Latent Interactions: Evaluation of Alternative Estimation Strategies and Indicator Construction. Psychological Methods, 9, 275-300. https://doi.org/10.1037/1082-989X.9.3.275
[69]
McDonald, R. P., & Marsh, H. W. (1990). Choosing a Multivariate Model: Noncentrality and Goodness of Fit. Psychological Bulletin, 107, 247-255. https://doi.org/10.1037/0033-2909.107.2.247
[70]
McQuitty, S. (2004). Statistical Power and Structural Equation Models in Business Research. Journal of Business Research, 57, 175-183. https://doi.org/10.1016/S0148-2963(01)00301-0
[71]
Mulaik, S. A. (1990). Blurring the Distinctions between Component Analysis and Common Factor Analysis. Multivariate Behavioral Research, 25, 53-59. https://doi.org/10.1207/s15327906mbr2501_6
[72]
Muthén, L. K., & Muthén, B. O. (2002). How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power. Structural Equation Modeling, 9, 599-620. https://doi.org/10.1207/S15328007SEM0904_8
[73]
Nevitt, J., & Hancock, G. R. (2001). Performance of Bootstrapping Approaches to Model Test Statistics and Parameter Standard Error Estimation in Structural Equation Modeling. Structural Equation Modeling, 8, 353-377. https://doi.org/10.1207/S15328007SEM0803_2
[74]
Nevitt, J., & Hancock, G. R. (2004). Evaluating Small Sample Approaches for Model Test Statistics in Structural Equation Modeling. Multivariate Behavioral Research, 39, 439-478. https://doi.org/10.1207/S15327906MBR3903_3
[75]
Newsom, J. T. (2018). Minimum Sample Size Recommendations (Psy 523/623 Structural Equation Modeling, Spring 2018). Manuscript Retrieved from upa.pdx.edu/IOA/newsom/semrefs.htm.
[76]
Nicolaou, A. I., & Masoner, M. M. (2013). Sample Size Requirements in Structural Equation Models under Standard Conditions. International Journal of Accounting Information Systems, 14, 256-274. https://doi.org/10.1016/j.accinf.2013.11.001
[77]
Nunnally, J. C., & Bernstein, I. H. (1967). Psychometric Theory (Vol. 226). New York, NY: McGraw-Hill.
[78]
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric Theory (McGraw-Hill Series in Psychology, Vol. 3). New York, NY: McGraw-Hill.
[79]
Preacher, K. J., & Coffman, D. L. (2006). Computing Power and Minimum Sample Size for RMSEA. http://quantpsy.org
[80]
Raykov, T. (2012). Scale Construction and Development Using Structural Equation Modeling. In R. H. Hoyle (Ed.), Handbook of Structural Equation Modeling (pp. 472-492). New York, NY: Guilford Press.
[81]
Saris, W. E., & Satorra, A. (1993). Power Evaluations in Structural Equation Models. In K. A. Bollen, & J. S. Long (Eds.), Testing Structural Equation Models (pp. 181-204). Newbury Park, CA: Sage.
[82]
Saris, W. E., & Stronkhorst, L. H. (1984). Causal Modelling in Nonexperimental Research: An Introduction to the LISREL Approach (Vol. 3). Sociometric Research Foundation.
[83]
Saris, W. E., Satorra, A., & Van der Veld, W. M. (2009). Testing Structural Equation Models or Detection of Misspecifications? Structural Equation Modeling, 16, 561-582.
[84]
Satorra, A., & Saris, W. E. (1985). Power of the Likelihood Ratio Test in Covariance Structure Analysis. Psychometrika, 50, 83-90. https://doi.org/10.1007/BF02294150
[85]
Schumacker, R. E., & Lomax, R. G. (2015). A Beginner’s Guide to Structural Equation Modeling (4th ed.). New York, NY: Routledge.
[86]
Schumacker, R. E., & Lomax, R. G. (2016). A Beginner’s Guide to Structural Equation Modeling (4th ed.). New York: Routledge.
[87]
Silvia, E. S. M., & MacCallum, R. C. (1988). Some Factors Affecting the Success of Specification Searches in Covariance Structure Modeling. Multivariate Behavioral Research, 23, 297-326. https://doi.org/10.1207/s15327906mbr2303_2
[88]
Singh, K., Junnarkar, M., & Kaur, J. (2016). Measures of Positive Psychology, Development and Validation. Berlin: Springer
[89]
Tabachnick, B., & Fidell, L. (2013). Using Multivariate Statistics. Boston, MA: Pearson Education Inc.
[90]
Tanaka, J. S. (1987). How Big Is Big Enough? Sample Size and Goodness of Fit in Structural Equation Models with Latent Variables. Child Development, 58, 134-146. https://doi.org/10.2307/1130296
[91]
Thomas, L. (1997). Retrospective Power Analysis. Conservation Biology, 11, 276-280. https://doi.org/10.1046/j.1523-1739.1997.96102.x
[92]
Thompson, B. (2004). Exploratory and Confirmatory Factor Analysis: Understanding Concepts and Applications. Washington DC: American Psychological Association.
[93]
Tinsley, H. E., & Tinsley, D. J. (1987). Uses of Factor Analysis in Counseling Psychology Research. Journal of Counseling Psychology, 34, 414-424. https://doi.org/10.1037/0022-0167.34.4.414
[94]
Velicer, W. F., & Fava, J. L. (1998). Affects of Variable and Subject Sampling on Factor Pattern Recovery. Psychological Methods, 3, 231-251. https://doi.org/10.1037/1082-989X.3.2.231
Wang, L. L., Watts, A. S., Anderson, R. A., & Little, T. D. (2013). Common Fallacies in Quantitative Research Methodology. In T. D. Little (Ed.), The Oxford Handbook of Quantitative Methods (pp. 718-758). New York: Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199934898.013.0031
[97]
Wheaton, B. (1987). Assessment of Fit in Overidentified Models with Latent Variables. Sociological Methods & Research, 16, 118-154. https://doi.org/10.1177/0049124187016001005
[98]
Wheaton, B., Muthen, B., Alwin, D. F., & Summers, G. F. (1977). Assessing Reliability and Stability in Panel Models. Sociological Methodology, 8, 84-136. https://doi.org/10.2307/270754
[99]
Widaman, K. F. (1993). Common Factor Analysis versus Principal Components Analysis: Differential Bias in Representing Model Parameters. Multivariate Behavioral Research, 28, 263-311. https://doi.org/10.1207/s15327906mbr2803_1
[100]
Wilcox, R. R. (2008). Sample Size and Statistical Power. In A. M. Nezu, & C. M. Nezu (Eds.), Evidence-Based Outcome Research: A Practical Guide to Conducting Randomized Controlled Trials for Psychosocial Interventions (pp. 123-134). New York, NY: Oxford University Press.
[101]
Williams, B., Onsman, A., & Brown, T. (2010). Exploratory Factor Analysis: A Five-Step Guide for Novices. Australasian Journal of Paramedicine, 8, 1-13.
[102]
Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013). Sample Size Requirements for Structural Equation Models: An Evaluation of Power, Bias, and Solution Propriety. Educational and Psychological Measurement, 73, 913-934. https://doi.org/10.1177/0013164413495237
[103]
Wothke, W. (1993). Nonpositive Definite Matrices in Structural Modeling. Sage Focus Editions, 154, 256-256.
[104]
Yu, C.-Y., & Muthén, B. (2002). Evaluating Cutoff Criteria of Model Fit Indices for Latent Variable Models with Binary and Continuous Outcomes. Doctoral Dissertation. http://www.statmodel.com/download/Yudissertation.pdf
[105]
Yuan, K.-H., & Bentler, P. M. (2000). Three Likelihood-Based Methods for Mean and Covariance Structure Analysis with Nonnormal Missing Data. Sociological Methodology, 30, 165-200. https://doi.org/10.1111/0081-1750.00078
