Thanks for the tips!
I did try theme:null before, but found that feel too "90s". Customizing a good theme is not easy, fortunately I found prettydoc. It solved my problem completely.
The html_document engine uses the Bootswatch theme library to support different styles of the document. This is a quick and easy way to tune the appearance of your document, yet with the price of a large file size (> 700KB) since the whole Bootstrap library needs to be packed in.
For package vignettes, we can use the html_vignette engine to generate a more lightweight HTML file that is meant to minimize the package size, but the output HTML is less stylish than the html_document ones.
So can we do BOTH, a lightweight yet nice-looking theme for R Markdown?
A simple document with some table took about 60k, support syntax highlighting.
This is good enough, and I can select from several themes.