Which format are you using ?
You could find Custom Blocks useful for that:
You would need to then style the blog according to your need. In HTML it is usually easier with CSS than in LaTeX, unless you are already familliar with LaTeX and available CTAN package that can help.
This is not a direct answer but I believe this is the best options you have in R Markdown feature to create such callout box.
You can also be interested in a new project called Quarto which has built in feature from some callout blocks
https://quarto.org/docs/authoring/callouts.html
If you are interested, you can have context of this new project: Quarto - FAQ for R Markdown Users