I don't think you need specific templates for this. Just make a floating TOC.
See the following illustration:
---
title: "dummy"
output:
html_document:
toc: yes
toc_float: true
---
# 1st heading of level 1
bla bla bla
# 2nd heading of level 1
bla bla bla
## 1st heading of level 2 under 2nd heading of level 1
bla bla bla
## 2nd heading of level 2 under 2nd heading of level 1
bla bla bla
### 1st heading of level 3 under 2nd heading of level 1
bla bla bla
### 2nd heading of level 3 under 2nd heading of level 1
bla bla bla
## 3rd heading of level 2 under 2nd heading of level 1
bla bla bla
This generates the following:
Hope this helps.