Avoiding the Java dependency is the popular choice these days but if you need to work with it, the solution is to set your JAVA_HOME variable in one of your startup files (i.e. Rprofile.site, .Rprofile, Renviron.site or .Renviron)
# This is an example with my Java version, you have to change your path accordingly with your own Java version
Sys.setenv('JAVA_HOME'="C:/Program Files/Java/jdk-11.0.1/")