The only other things I can think of are checking the Global or project settings.
Tools > Global Options
Tools > Project Options
The global options and project options both have some settings which control RMarkdown outputs. Screenshot of the global options pane:
You can also try adding this to your yaml heading (at the top of the rmarkdown document):
editor_options:
chunk_output_type: inline
If all else fails and your global options are set up to show tables and plots inline, a restart of RStudio may somehow magically fix the issue.