So from the looks of it, each dataset has a unique identifier, e.g. Abertamy = 2727 (you can see this by looking at the link for the name of the region https://www.obcepro.cz/obec/2727). The link for the spreadsheet download is "https://www.obcepro.cz/obec/download/excel/data/" followed by this identifier.
Best steps would be to use rvest to figure out what these unique identifiers are, loop through the page numbers to extract all of these identifiers and associate them with the village name.
You should be able to put these in a data.frame/tibble within rvest. Then write a function to take in an identifier and to download the data. read_excel() from the readr package can download files directly from a url.
You can even do this still within the same tibble if you use map(), and the data will be downloaded and stored in a list column in your harvested dataset.