When you render the document, you could use something like the code below to set the output document name based on whether the output includes the solutions or not. (The code is from my answer to a Stack Overflow question.) The parameter soln can be either TRUE or FALSE. This determines whether the rmarkdown document is rendered, respectively, with or without solutions, and also sets the output file name accordingly.
for (i in c(TRUE, FALSE)) {
rmarkdown::render("hw.Rmd",
params = list(soln = i),
output_file=ifelse(i, "Solutions.doc", "Homework.doc"))
}