CRAN is only building windows binaries for 3 versions (devel, current, old release). 3.6 is too old and only source package is available. See CRAN Repository Policy
Binary packages are built for the current version of R: they may also be built for the last version in the previous series (e.g. R 3.1.3 when R 3.2.x is current) or for R-devel.
As mentioned above by @andresrcs , you can still install the source package using install_package() either by clicking on the box that should ask for installing source, or setting type = "source". Beware that you need some more tools on Windows to compile some packages from source - you need Rtools
See Chapter 9 Set up an R dev environment | What They Forgot to Teach You About R
Other package like remotes or pak can help to install packages efficiently (GitHub - r-lib/pak: A fresh approach to package installation)
Hope this helps