I don't remember if you have that option, if available, it should be listed as a 32bit program in your installed programs list in Windows and you would only need to click on "uninstall". I can't check my self because I only have the 64-bit version installed.
Anyways, even if you don't have that option, uninstalling R completely and reinstalling the 64-bit version only, shouldn't take much time so I would say, give it a try.