You can paste the html directly into an rmarkdown document. However, to have it render correctly you need to do one of the following:
self_contained: false in the YAML metadata:
data-external="1" as an attribute in the html:
<iframe src="https://open.spotify.com/embed/track/1dmES1X8l1AnFBy2gR3wYA" width="100%" height="380" frameBorder="0" allowtransparency="true" allow="encrypted-media" data-external="1"></iframe>
These options are described in this answer.
I confirmed that both options still work. However, note that the embedded music link will not appear in the RStudio html viewer that pops up when you knit the document. But if you click "open in browser" in the upper right corner, the music link will display properly.