Quantitative Psychology Research Laboratory
Meta-analysis is becoming very popular in behavioral and medical research. We are interested in using the method in applied research to solve real problems.
So, S. H.-W., Siu, N. Y.-F., Wong, H.-L., Chan, W., & Garety, P. A. (2016). ‘Jumping to conclusions’ data-gathering bias in psychosis and other psychiatric disorders — Two meta-analyses of comparisons between patients and healthy individuals. Clinical Psychology Review, 46, 151-167.
The chi-square difference test plays a key role in multiple-group structural equation modeling (SEM) for testing invariance. We argue that the traditional approach of using the chi-square test for comparing two nested models sequentially would fail to control either type I or type II errors. Analysis and example show why chi-square-difference tests are easily misused in the context of multi-group analysis, and Monte Carlo results support the analysis. To overcome this issue, our study further proposes a test of equivalence as an alternative.
Yuan, K.-H., & Chan, W. (2016). Measurement invariance via multigroup SEM: Issues and solutions with chi-square-difference tests. Psychological Methods, 21, 405-426.
Yuan, K.-H., Chan, W., Marcoulides, G. A., & Bentler, P. M. (2016). Assessing structural equation models by equivalence testing with adjusted fit indices. Structural Equation Modeling, 23, 319-330.
Different indices have been proposed for measuring reliability of a test, including coefficient alpha, coefficient omega, the greatest lower bound reliability, and others. Among these, the coefficient alpha has been most widely used. However, it is known that coefficient alpha under-estimates the true reliability unless the items are tau-equivalent, and coefficient omega is deemed as a practical alternative to coefficient alpha in estimating measurement reliability of the total score. To see if the difference between alpha and omega is practically ignorable, we developed a method to examine the difference of coefficient alpha and omega statistically.
The R codes for testing the difference between reliability coefficients alpha and omega (Deng & Chan, 2017)
Deng, L., & Chan, W. (2017). Testing the difference between reliability coefficients alpha and omega. Educational and Psychological Measurement, 77, 185-203.