Shiny depends on jQuery, so it's not possible to remove it, but you don't strictly need Bootstrap. You may want to investigate HTML templates as a way to create Shiny apps without Bootstrap.
A quick Google search turns up a few people who have experimented with Shiny and Vue.js. Personally, I've bookmarked this thorough post by Jaehyeon Kim as it covers quite a lot of ground.
You might also be interested in the ongoing work to integrate React.js and Shiny. A good introduction to this community is Alan Dipert's talk at rstudio::conf 2019 Integrating React.js and Shiny.