If you're dealing with datasets, consider using the data.table package. It's goal is to be fast and memory efficient, and it achieves that gloriously.
Also, check out The R Inferno, specifically chapters 2 (Growing Objects) and 3 (Failing to Vectorize).