Woops! So sorry for missing this, and thanks for the heads up!
I'll share this with our team and see if they can come up with any creative ideas. You're exactly right that we get into a bit of a sticky place here because (even though you can host a text file as a standalone app on RSC) you cannot host apps at / or /robots.txt.
Do you have a proxy of some stripe in front of RStudio Connect? That would be one option for injecting the robots.txt file that would not depend on us exposing such behavior. Unfortunately, I am not optimistic about potential for altering the __api__/me URL (which is an API request).
Apologies for your trouble (and the long delay!)!