We have document outline for .R script, .RMarkdown, but not for plain markdown files. I believe this is simple enough to support, and it should be very useful.
I probably suggested this in rstudio forum before but didn't see any update about this yet.
Perhaps I misunderstand, but Rmarkdown is a markdown document, just with support for R, bash, python, latex etc.. Also, Rmarkdown supports markdown documents.
document outline is the table of content view in right side of editor, enabled by the rightmost toolbar button in rmd editor. It will show the title/subtitle, chunk title, function name etc in hierarchy.
When you open a .md in rstudio, the document outline feature is not available. Since markdown document do have title/subtitle, an outline can be very helpful in navigating and overview.