Should I document my Exploratory Data Analysis?

In your experience, is it worth documenting each step of the EDA so that in the future when you look at the scripts you know what results are generated and what conclusions are reached?

This is a usecase for using an R Notebook/ markdown


If in doubt, it is always worth documenting your work even if it seems obvious to you at the moment.
My experience is, that it is always easier to get get rid of old documentation than it is to try to understand the work you have done a while ago.
And as Nir already pointed out, Notebooks and markdown are perfect for this, and you will certainly not regret using them.

1 Like @nirgrahamuk and @jms have noted, RMarkdown notebooks are the ideal medium for this.

If you're not thinking about keeping track of things, you won't keep track of things.
—Karl Broman

As well, I'd encourage you to read Randy Au's recent blog post " Let's Get Intentional About Documentation"

He makes the case that "documentation" is a loaded term and we'd be better served if we think about "recordkeeping" first.

There are three things to capture in your recordkeeping:

  1. The why and how of the decisions made

  2. How things work together (for this, I would stress that you include the details of the computational environment...which version of R and the packages you're using, folder structure, etc)

  3. How to make changes

1 Like

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.