I would definitely not recommend uninstalling Shiny Server. This problem should be isolated from Shiny Server, and that you are still seeing the offline page is illustrative of a problem existing independent of the installation (since it looks like you have removed all of the files... I would have expected some files to turn up at /usr/lib/rstudio-server).
In any case, the fact that you are still seeing a file being served means we should be able to trace it from the networking side. I'm not an expert at this by any means, but I presume that the following should do the trick. Issue sudo netstat -lntp to see what processes are listening on what ports - you should find a PID attached to port 8787, which you can then investigate with ps -A | grep 'pid'
Now how to deal with it... The simple solution (that may resolve it in a not very insightful way) would be to kill that PID... I wonder if that would work even with rstudio-server installed. We probably should have tried this earlier! Maybe before killing, have a look at information about the job with:
sudo lsof -p PID
sudo ls -l /proc/PID
(Where PID is replaced with the process ID you found above). I'm curious what you will find. Ultimately, I expect that there is some rogue process running on that port, so killing it should resolve the issue unless there is a daemon restarting the process (there shouldn't be since you purged). In any case, the more info you can provide, hopefully the better documentation of the issue that will be available for any future individuals experiencing this issue!