You can directly divide thanks to vectorization.
matrix <- matrix(1:20, nrow = 4, ncol = 5)
vector <- c(1:4)
matrix / vector
#> [,1] [,2] [,3] [,4] [,5]
#> [1,] 1 5.000000 9.000000 13 17.000000
#> [2,] 1 3.000000 5.000000 7 9.000000
#> [3,] 1 2.333333 3.666667 5 6.333333
#> [4,] 1 2.000000 3.000000 4 5.000000
Created on 2020-08-26 by the reprex package (v0.3.0)