After reading a David Robinson blog post last week, I've been reflecting on how programmers find out what they don't know. This is related to some common problems that new people have coming up with reproducible examples for stack overflow, or figuring out how to find information that they lack. It seems like when you're new to a field, one of the main problems is describing the gap in your knowledge precisely enough to get a useful answer from other users or the documentation. For instance if you've been writing code for a little while, you can identify that some behaviour is unexpected and also precisely describe the expectation that's not being met. These are two different skills and for new people the latter is pretty difficult because they don’t have solid enough intuition to describe their expectations precisely.
I think as people learn to program one of the main skills that they learn is how to identify their knowledge gaps, describe those gaps, and be able to tell when those gaps have been filled. My question is what are some teachable ways that you can help people develop that skill?
So far I’ve thought of four things which I’ve found personally important:
Vocabulary
A big part of learning is to use the same terms that the community uses for those things. For instance if you’re an excel user coming to R you probably use terms like “cells” or “formulas”. You might be able to precisely describe your problem using your old vocabulary, but that doesn’t help you find the answer in the field you’re trying to learn. So finding people to help translate your old knowledge into the new language is very important.
Stepping back to find anti-patterns
Sometimes people have expectations which are based on using software in an odd way. I run across a fair number of questions where someone is trying to find out how to do some really tricky data manipulation, but what’s really going on is that they went down the wrong path a few steps earlier and turned a simple problem into a complex one. This seems to be related to people using a mental model that they’re familiar with ina environment which is built around a different mental model. So asking questions like “what are you trying to do big picture” or “how does this problem fit into the analysis” are more helpful than giving an answer to the specific problem. What you want to know is often not what you need to know.
Learning things that everybody knows but nobody writes down
I think there’s a trend that the more people have a problem the more resources there are about how to fix that problem. The result of this is that if you don’t know the thing that “everybody” in a field knows you will have a terrible time trying to figure out the problem. In my case when I was learning R I ran across lots of people saying “or in the command line …. “ but I didn’t really know what the command line is or how to get there. Recognizing when something is common but undocumented helps a lot with the frustration of not finding helpful resources.
Focus on making new mistakes. I find that beginning programmers have a tendency to try the same thing over and over rather than trying different thing and seeing if the error they’re experiencing continues. One thing I’ve learned to do is actually write down the actions I’ve tried so that I can at least make new mistakes. This is a big step towards reproducibility because even if your actions don’t make a lot of sense to you, they can really help a more experienced person understand the context of your problem.
This seems related to something I run across a lot: when people try to define problems by outlining solutions. I am just as guilty as anyone else when it comes to this, but often instead of just saying "here is what is happening that I do not like", we start with "I think this is what is happening, and maybe this will fix it".
This is incredibly common in statistical consulting. "What's a statistical test for detecting an outlier?" "Why are you trying to detect an outlier?" "I have some odd values close to my spec and I want to remove them prior to capability analysis."
Not a real conversation I've had, but it could have been.
I tend to ignore those questioning instincts when answering questions about programming, so thanks for the reminder. There was a case recently when someone was asking about how to use paste when I worked out that he just didn't know about the %in% operator. Fortunately, I caught that one!
I think learning the vocabulary is incredibly important. However, I'm also a big proponent of using visuals. There's a selection bias wherein questions that are hard to describe using words simply don't get asked.
I know there are many people who aren't fans of GIFs, but in the 2years of TestFlighting an iPad app I use daily, the trifecta of crash logs/code, description of what seems to be happening, and a recording of the behaviour has made for much easier troubleshooting.
I also do a lot of scribbling on screenshots (though this isn't in lieu of a reprex or using actual code).
Others (esp. those on the receiving end of a higher volume of questions) might feel like this is terribly unhelpful…but, for me, it's been part of the vocabulary-building process.
Concept maps are a great tool for learners to figure out vocabulary, structure knowledge and see how smaller component skills/outcomes are related to each other in the bigger picture. @Hadley had a great concept map for how he structured a course he is teaching. Doing this collaboratively with students in the early going of learning (in nearly any field) can be a big help.
As to other questions that @gordon raised, and repeating myself from another thread: How Learning Works by Susan Ambrose is a fantastic resource to helping figure out your own answers to these questions by better understanding delving into the literature on teaching and learning.
Here's where I'm going to go full teacher mode and suggest Frayer diagrams for learning vocabulary.
They would likely need to be modified to fit into a programming context, but learning vocabulary in context is critical. I've found Frayer diagrams to be incredibly helpful in providing that context.
Here's a quick sketch of what one could look like for ggplot2():
This is on a small sheet of paper - if doing it "for real" I'd put it on at least a half sheet, so that I could write in more information and examples.
One of the most effective courses I've taught (HS Biology) was very clearly a Biology course, but I built the curriculum to be focused on building literacy and comprehension skills. I do firmly believe that we can take much of what works in K - 12 education and apply it to programming, and see far better results than the majority of programming education tools that currently exist.
@jakekaupp I second what you've said about Susan Ambrose' book, "How Learning Works." it is a very readable introduction to research-based teaching best practices, which distinguishes it from a lot of other "how to teach" books out there.
@mara, I'm reading a book right now called "Efficiency in Learning" by Ruth Clark that examines the research on cognitive load and how to minimize extraneous cognitive load in the design of teaching materials. According to her, a picture really is worth a thousand words: there's a lot of research that shows how, for complicated topics, giving learners a graphic or graphics can dramatically enhance the rate of learning. And Mara, "scribbling on screenshots" is also apparently a cognitive load "best practice," because it puts the annotation right next to where the learner needs to be paying attention. So you might find that book interesting reading, because it includes a lot of research that seems to confirm what you already suspect.
That sounds awesome! Ordered it and am super stoked to read it! I actually had a whole section about this I was going to include in some talk at some point, but ended up cutting it. It'll be great to learn more about it (and from a book, since papers lack a certain je ne sais quoi).
to the 
One would think that after colour-coding 144 interviews only to discover