The error indicates that your RAM has been exhausted.
The first thing to try is to run from a clean session and only to load the objects you require. Memory can get clogged up if you have been running many things in the same session.
You could also close any other applications which are consuming large amounts of RAM.
If that doesn't work then you could use the far more memory efficient data.table package:
https://cran.r-project.org/web/packages/data.table/index.html
Otherwise you could look to install more RAM or run off a cloud server.