Using R to automatically design internally consistent item scales?

Industry Pharma/Life Sciences
#1

Hi Everyone,

This is my first post on here, I've recently started to learn about R and Rstudio.

There are a lot of things in my job that I think it will be useful for, but the thing that interests me the most at the minute is finding out if I can use it to create internally consistent item scales automatically.

Just to explain the situation that leads up to the analysis - I send out 16 slider questions which all aim to measure the same thing ("adaptability" for example) to a sample of 300 participants. When I get the data back, I use Cronbachs α to create two 6-item scales (Named "Scale A" and "Scale B"), hopefully with α>.7. Out of the 16 slider items, 8 of them will be measuring one facet of adaptability, and 8 of them will be measuring another facet, so i need to make sure that when i create my two slider scales, they both have 3 items from each facet. A single item can't appear in both scales (so for example, item 3 cant be in Scale A and Scale B).

Currently I do this with a mixture of Excel/Jamovi (My company is too cheap to buy SPSS) - is there a way that I can do the above actions using a single R script, so that I can input the 16 items and it will output for me two 6-item scales, consisting of 3 items from each facet, with no one item being used more than once, with the highest cronbachs α possible?

I'm not asking if someone can write this script for me, my question is more around "is this theoretically possible to do?" before i start trying to figure it out myself

Looking forward to joining the community!

#2

Hi, welcome aboard
If I understand the question correctly, you simply want what in the psychometric literature is called parallel forms.

It should be relatively easy to do this. I'd suggest having a look at the {psych} package for basic psychometric calculations. Then it should be a matter of simple sorting. I doubt than you will get exactly the same means and variances but they should be close enough for practical work.

#3

Hi jrkrideau,

Thankyou very much for the warm welcome!

I've never thought of it that way, what I'm aiming for is to have parallel forms ( I think i completely missed this because the design process that lead to us having a second scale never had this objective)

I would never expect them to be exactly the same, as long as they are optimal in terms of internal consistency.

So for example, if the 16 items were named "Item_1", "Item_2" etc and lets say through some manual playing around i find that if i put items 1, 2, 4, 6, 9, 11 into Scale A and items 3. 5, 10, 13,14,16 into Scale B, I will get the following alphas

  • Scale A has an α of .71
  • Scale B has an α of .73

Is it possible to have an R script do the same task, however it would be able to pick up that if I swap item 9 from scale A with Item 14 from Scale B i would get these alphas

  • Scale A has an α of .73
  • Scale B has an α of .74

It would be a minor improvement over what I would have been able to achieve by playing around manually, but it would do it in a fraction of the time, and I would know that the items have been arranged to achieve the best possible alphas.

I will definitely have a play around with the {psych} package - Thank you!

#4

In computer science there is a term "Turing Complete", in laymans term this is saying that if its capable of being programmed in one system then it can be programmed in another. Generally speaking if you experience something being accomplished by a computer somewhere in the world, and you have access to sufficient data then you can assume that with sufficient skill you can compute it as others did. Obviously you will have to judge the depth of skill brought to bare. A hobbyist programmer will struggle to reproduce their own version of microsoft office suite with everyone of its features etc, but someones excel spreadsheet... generally this latter is very achievable

#5

I am having a blank on how to do what you want but you can generate all possible combinations of tests with combn()
EXAMPLE

set.seed(1234)
dat1  <- data.frame (ls = sample(LETTERS[1:16], 16, replace = FALSE), nn = sample(1:10, 16, replace = TRUE))

dat2  <-  as.data.frame(combn(dat1$ls, 8))

This gives us a 8 X 12870 item file.
combn() outputs a matrix but I put it in a data.frame for easier handling.