Invariance measure between two groups

[CarlMar]

Hi everyone,
I'm kinda stuck doing my analysis and I'd really appreciate some help since my only problem (for now) is pretty much that I can't find certain answer.
So I'm testing my two sample invariance using R Studio but I miss some important information. Basically I need to understand two thing to continue my work:

1. After using the measurementInvariance() function how do I understand where my invariance fail? I mean what index should I look at? CFI difference level?P value?It so how much should it be?
2. When I know where my invariance broke I use lavTestScore() but, once again, what index should I look to know what items should I set free?
   I really need some help if possible, thank you so much!

[technocrat]

Hi, and welcome.

It's hard to offer any more specific advice beyond pointing you to this page without a reproducible example, called a reprex. Also, if applicable, for future reference see homework policy.

[CarlMar]

Thank you fort your answer. Right now my main problem is about how to interpret the output I'm getting from r. More specifically how to understand what index should I consider to see what level of invariance fit is not good and how choose which item should I set free. Unfortunately the link you provide me leave that part out.
I'm sorry my english is not very good, hope I managed somehow to explain myself.
Thank you again!

[technocrat]

Hi, @CarlMar,

This is a worldwide forum, and English is a world language. That is convenient, but it also means that there's no one "good" English. Language is to communicate and even people in a country like mine, where most people grow up speaking English, there are enough differences to create misunderstandings. So, if I'm not understanding, it's our common problem, not your fault or mine! Ok?

I can help if I see your results and how you generated them, which requires a reproducible example, called a reprex. It doesn't have to be the same data that you are using. Any data that will generate the same form of input will do.

I understand that reprex doesn't always work with some code. If that's the case, provide representative data, preferably from a data() set, and the code you use to present your result. Then we can help with interpreting it.

[CarlMar]

Thank you for your patience! I try to post an exemple of what my problem:

                   Df   AIC   BIC     Chisq        Chisq diff Df    diff Pr(>Chisq)    
fit.configural 22 11404 13005  47.609                                  
fit.loadings   27 11768 10937  50.608      3.189       5  0.6792784    
fit.intercepts 32 11833 10965  81.241     48.679      5  1.092e-97 ***
fit.residuals  39 11545 10872 159.429     31.190     7  5.899e-05 ***
fit.means      41 11657 10064 107.094     15.093     2  0.0005909 ***
                                     cfi              rmsea        cfi.delta       rmsea.delta
fit.configural	             0.965    	0.056	NA	           NA
fit.loadings	             0.967         0.057         0.003         0.007
fit.intercepts	             0.907         0.075	0.057	0.026
fit.residuals	             0.871  	0.088	0.038	0.007
fit.means	                     0.854	       0.089	0.021	0.004

P value in less then 0.05 on fit.intercepts so should I try to free some item there? Or should I look to CFI and work on the fit.residual model? My sample is around 590 person for this type of invariance. But I also work with sample with more than 1000 people, so is it p value ok to look at to take decision?

than using lavtestscore() I get this:

         lhs    op    rhs          X2          df       p.value 
1	.p2.	==	.p29.	4.184	1	0.017
2	.p3.	==	.p30.	12.376	1	0.000
3	.p4.	==	.p31.	0.133	1	0.668
4	.p6.	==	.p33.	0.954	1	0.342
5	.p7.	==	.p34.	2.194	1	0.146
6	.p19.	==	.p46.	0.148	1	0.703
7	.p20.	==	.p47.	105.065	1	0.000
8	.p21.	==	.p48.	35.570	1	0.000
9	.p22.	==	.p49.	41.849	1	0.000
10	.p23.	==	.p50.	6.676	1	0.010
11	.p24.	==	.p51.	0.773	1	0.352
12	.p25.	==	.p52.	8.839	1	0.003

I'm assuming that p20.==.p47., .p21.==.p48., .p22.==.p49. are really bad but I how do I understand what item they referring to? And how do I decide how mant item should I set free (considering my scale only got 8 items)?
Thank you

[technocrat]

For the first part, the documentation provides an example

library(lavaan)
#> This is lavaan 0.6-5
#> lavaan is BETA software! Please report any bugs.
example(cfa)
#> 
#> cfa> ## The famous Holzinger and Swineford (1939) example
#> cfa> HS.model <- ' visual  =~ x1 + x2 + x3
#> cfa+               textual =~ x4 + x5 + x6
#> cfa+               speed   =~ x7 + x8 + x9 '
#> 
#> cfa> fit <- cfa(HS.model, data = HolzingerSwineford1939)
#> 
#> cfa> summary(fit, fit.measures = TRUE)
#> lavaan 0.6-5 ended normally after 35 iterations
#> 
#>   Estimator                                         ML
#>   Optimization method                           NLMINB
#>   Number of free parameters                         21
#>                                                       
#>   Number of observations                           301
#>                                                       
#> Model Test User Model:
#>                                                       
#>   Test statistic                                85.306
#>   Degrees of freedom                                24
#>   P-value (Chi-square)                           0.000
#> 
#> Model Test Baseline Model:
#> 
#>   Test statistic                               918.852
#>   Degrees of freedom                                36
#>   P-value                                        0.000
#> 
#> User Model versus Baseline Model:
#> 
#>   Comparative Fit Index (CFI)                    0.931
#>   Tucker-Lewis Index (TLI)                       0.896
#> 
#> Loglikelihood and Information Criteria:
#> 
#>   Loglikelihood user model (H0)              -3737.745
#>   Loglikelihood unrestricted model (H1)      -3695.092
#>                                                       
#>   Akaike (AIC)                                7517.490
#>   Bayesian (BIC)                              7595.339
#>   Sample-size adjusted Bayesian (BIC)         7528.739
#> 
#> Root Mean Square Error of Approximation:
#> 
#>   RMSEA                                          0.092
#>   90 Percent confidence interval - lower         0.071
#>   90 Percent confidence interval - upper         0.114
#>   P-value RMSEA <= 0.05                          0.001
#> 
#> Standardized Root Mean Square Residual:
#> 
#>   SRMR                                           0.065
#> 
#> Parameter Estimates:
#> 
#>   Information                                 Expected
#>   Information saturated (h1) model          Structured
#>   Standard errors                             Standard
#> 
#> Latent Variables:
#>                    Estimate  Std.Err  z-value  P(>|z|)
#>   visual =~                                           
#>     x1                1.000                           
#>     x2                0.554    0.100    5.554    0.000
#>     x3                0.729    0.109    6.685    0.000
#>   textual =~                                          
#>     x4                1.000                           
#>     x5                1.113    0.065   17.014    0.000
#>     x6                0.926    0.055   16.703    0.000
#>   speed =~                                            
#>     x7                1.000                           
#>     x8                1.180    0.165    7.152    0.000
#>     x9                1.082    0.151    7.155    0.000
#> 
#> Covariances:
#>                    Estimate  Std.Err  z-value  P(>|z|)
#>   visual ~~                                           
#>     textual           0.408    0.074    5.552    0.000
#>     speed             0.262    0.056    4.660    0.000
#>   textual ~~                                          
#>     speed             0.173    0.049    3.518    0.000
#> 
#> Variances:
#>                    Estimate  Std.Err  z-value  P(>|z|)
#>    .x1                0.549    0.114    4.833    0.000
#>    .x2                1.134    0.102   11.146    0.000
#>    .x3                0.844    0.091    9.317    0.000
#>    .x4                0.371    0.048    7.779    0.000
#>    .x5                0.446    0.058    7.642    0.000
#>    .x6                0.356    0.043    8.277    0.000
#>    .x7                0.799    0.081    9.823    0.000
#>    .x8                0.488    0.074    6.573    0.000
#>    .x9                0.566    0.071    8.003    0.000
#>     visual            0.809    0.145    5.564    0.000
#>     textual           0.979    0.112    8.737    0.000
#>     speed             0.384    0.086    4.451    0.000

^{Created on 2020-01-14 by the reprex package (v0.3.0)}

And there is a tutorial that came out yesterday! But that is many a how to operate, than how to intepret. For that we need to go to the paper

The second part of your question, using the same data, is represented by this reprex

library(lavaan)
#> This is lavaan 0.6-5
#> lavaan is BETA software! Please report any bugs.
HS.model <- '
  visual =~ x1 + b1*x2 + x3 
  textual =~ x4 + b2*x5 + x6 
  speed =~ x7 + b3*x8 + x9
  b1 == b2
  b2 == b3 '
 fit <- cfa(HS.model, data=HolzingerSwineford1939)
 lavTestScore(fit, cumulative = TRUE)
#> $test
#> 
#> total score test:
#> 
#>    test     X2 df p.value
#> 1 score 12.306  2   0.002
#> 
#> $uni
#> 
#> univariate score tests:
#> 
#>   lhs op rhs     X2 df p.value
#> 1  b1 ==  b2 12.060  1   0.001
#> 2  b2 ==  b3  0.947  1   0.330
#> 
#> $cumulative
#> 
#> cumulative score tests:
#> 
#>   lhs op rhs     X2 df p.value
#> 1  b1 ==  b2 12.060  1   0.001
#> 2  b2 ==  b3 12.306  2   0.002

^{Created on 2020-01-14 by the reprex package (v0.3.0)}

There's a lot to unpack here about the interpretation of the output. I'm still studying it. Some are fairly standard (lower AIC better than higher, ceteris paribus), but I'll need some more time to answer your specific questions.

In the meantime, I hope others can weigh in on the interpretation using these reprexs