flexdashboard is just a framework that help you create easily some dashboard based on template and easy-to-use Rmd feature. At the end, it will be html, js and css but the 'magic' is that you can produce a dashboard without knowing this.
By default, the dashboard will be created as only client side, self contained or not. That means that all the data will be pass through HTML and it could indeed be slow or too heavy if you have a lot of data.
You can easily create a flexdashboard with a server side, using shiny. You'll need to deploy on a server where R and shiny can be run, but you'll then be able to have your data loaded into the client side (html) when needed based on shiny widget interaction.
You can also notice that DT as the same possible behavior: by default it is client side only, but you can create a DT with server side. It allows to process the data for your DT on the server side with R ( ans thus not only in client with JS) - this is also useful for huge data.
You could find this server + client strategy for several widgets. All this is indeed web development even if Rmarkdown, shiny and htmlwidgets hide that to us.
At the end, I would go step by step and see what optimization if needed. On million row is not always big data, but yes it could be heavy to fit only in an html page without server.