Here are few resources that might interest you.
Check out Shiny HTML Templates.
Custom message handlers will help you with this, specifically session$sendCustomMessage and Shiny.onInputChange or Shiny.setInputValue. Check out the Communicating with Shiny via JavaScript guide.
I would recommend Mozilla's JavaScript guide, as well as having a look through the html tutorials (for good semantic practices) and DOM guides (for a conceptual overview).
A bit about my experience - I started learning JavaScript when I when I wanted a specific actions to occur as a result of an event (button click for example). At the time, I didn't have much experience with other languages besides R (I come from the clinical research background). I started with looking at simple examples, reading guides, and practicing (lots of it and then repeating). Once I felt comfortable with the basics, I moved on to learning D3, data handling techniques in js, and then on to popular js libraries (reactJS, gatsbyJS, etc) for building static sites/apps. Recently, I've started learning using nodeJS and express to get familiar with backend work. It's been a few years, but taking that first step was a worth it.
Hope that helps! I'd be happy to answer any questions.
EDIT: forgot to add urls