I'd recommend one project per assignment, but either approach would work. If you do go with one project per assignment, there are two techniques you can use to ensure that the students have the right set of packages when they start an assignment:
If you use a Space for the course, you can create a Base Project Template, and put the set of common packages into it. Then any project created in the space via the New Project button will get that set of packages.
Another technique is to create a project for each assignment and include any packages that you'd like the students to have for that assignment. Then have the students start their work by making a copy.
Take a look at the Cloud Guide (https://rstudio.cloud/learn/guide), in particular the section titled "Using Private Spaces in Courses and Workshops", for more details on both these methods.