That part defines the server response, yes, but you have to distinguish among developing, testing, and deploying an app.
For developing and testing (running the app locally) you don't need a server, but when you need to grant access to your app to many users at a time, over the public internet in a reliable and scalable way, then you do need a server.
Maybe your confusion comes from the fact that you can execute the app in your local system, and in some sense you could say it is been served from a 'local server' but that is not the same as using shiny-server which is an specialized service for deployment.