In general, I find it hard to stop a SQL query (either MySQL or PostgreSQL) from within the R console, and I'm assuming that RStudio is using the console as well.
What I do as a workaround is run pg_admin or MySQL workbench so I can kill the query from there when I've messed up.