What questions have people used to work out whether someone is relative new to R vs an 'intermediate' R user vs an expert. Assume that they will be honest or have an interest in being streamed correctly. Naturally, anything beyond basic knowledge of R quickly becomes domain-specific, but I'd like to see how far we can get. I'm also concerned that I will mistake what I think is easy or hard with the order of knowledge that others learn.
Also particularly interested in scale, so questions that can be marked by people without R knowledge (or even by a computer) are great, whereas things that require a lot of subjective interpretation are less good.
Here are some of mine ordered roughly from easy to hard
What is the function to install a new package in R?
How do you open the help page for a function?
Give two ways to access a column foo in a data.frame named DF.
Write a line of R code that results in the error non-numeric argument to binary operator
Give an x and y such that x == y is FALSE but all.equal(x, y) is TRUE.
Why does is.integer(0) return FALSE?
(From the R Inferno) Without running the code, what does the following return? Why?
I am not sure if you need this kind of assessment. Wouldn't you rather know if applicants have the skills for the job you are offering? I don't care if you have a very deep understanding of R, can you go from a dirty horrible dataset to results? The best assessment of how well someone is doing in a job is giving them real work.
I like the first 4 ones, but I think the others are very specific and don't take into account the multitude of ways people come into R these days. If you have a intro to R class in university these would make perhaps sense, not because of the questions but more because of the way these curricula are often taught.
My ideas are very much shaped by the book 'How to Teach Programming (and Other Things)' by Greg Wilson (co-founded software carpentry).the online version of the book is free here.
I'm not convinced a task would be more useful. it's a lot slower to compare a large number of applicants, you need specialist people to judge the work, and it's also a lot slower for applicants to complete.
The test that my question is reaching for is more like a test for the start of a course than questions for a job application, or a test for interns where the interns are already hired but you want to offer some R training on the side. Think 100 or so students and you want a sense of the average skill and whether or not there's enough variance to justify multiple streams. In my case, it's far easier to change the job to match the skills than to find the skills to match a job.
You also need to look at it from the applicants' perspective. Frankly, I would take a negative view of a company's hiring process consisting of those questions, so you could miss out on some good candidates who have other options.
It depends whether you want to test their pure R ability, or their statistical ability. But overall I feel that the above questions test knowledge rather than understanding. Knowledge can be easily ascertained by quickly Googling. Understanding is what shows proficiency at something.
For their R ability questions as the above are okay, though I think I'd ask questions such as:
Create a vector of a certain length, with samples from a Normal distribution?
Find the arithmetic mean, the median and the IQR from a sample of data?
Create a function that returns the minimum from each row of a matrix?
For wider statistics and programming ability I'd present them with a dataset and ask them to perform exploratory data analysis, create visualisations and get some conclusions from data. This allows you to see how they approach tasks, their understanding of what they are doing and the quality of coding.
Though, I appreciate, depending on time and infrastructure, all of this might not be possible.
I would also say that questioning/marking should be done by people with R knowledge, if you are trying to determine an applicant's level of R knowledge.
I have been using R for a while and we are selling significant commercial services based on my poor knowledge of syntax.
BUT
What is the function to install a new package in R?
I use Rstudio - I click on two buttons name the package and the job is done - ie the syntax is abstracted from me.
If this were a legitimate question why not ask how to do it in in machine code - abstractions are tools and they can be your friend so long as you understand it is a precise shorthand?
PS the console reflects what I did at a deeper level.