Packages can be loaded automatically in two ways, as @tjpalanca said, by the presence of a Rprofile startup file (that loads packages) but also by restoring your previous environment.
Rprofile type files can be found in three locations, the first one for the R version-level is called Rprofile.site and is located at R_HOME/etc/ being R_HOME the R's installation folder, the second one for the user-level called .Rprofile can be found at your user's HOME folder and the third one for the project-level also called .Rprofile can be found at your current working directory. Each one, if present, overwrites the previous one. You can check if any of these files are present and if the content is loading any packages.
Also, you can try disabling the "Restore .RData" option to avoid restoring your previous environment.