Oh you just want to show SQL code in your document with highlight ?
In that case, you can indeed omit the con - this works for me.
---
title: test
output: html_document
---
```{sql, eval = FALSE}
SELECT * FROM mtcars WHERE cyl = 4
```
but you won't be able to switch to TRUE as it will error. So this is not much different than just highlighting this chunk in markdown
---
title: test
output: html_document
---
```sql
SELECT * FROM mtcars WHERE cyl = 4
```
Is this what you are look for ?