That is strange... when I compile this report into pdf - with the eisvogel.tex from the forked repo I linked above - on my computer I get both links underlined.
---
title: "sample document"
date: '2022-03-10'
output:
pdf_document:
latex_engine: xelatex
template: eisvogel.tex
---
## R Markdown
This is an [R Markdown document](https://www.google.com/). Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see <http://rmarkdown.rstudio.com>.
When you click the **Knit** button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this:
It seems that the issue you are facing may be more of "how to customize Eisvogel template to underline links" rather than "how to underline links in a generic markdown document".
I can help with the latter, but my knowledge of the Eisvogel template is limited - and at a quick glance it seems like a rather sophisticated template.