You are probably getting that error because your matrix contains text values. A matrix must be all text or all numeric, it cannot be a mixture. If you have a single text entry in the matrix, the entire matrix will be converted to text.
Let's take a step back and work on the data processing outside of Shiny. It is best to have the data processing and visualization worked out before trying to implement anything in Shiny because the reactive environment in Shiny makes debugging difficult.
Please give some details about what your data look like and the specific changes you want to make in the data. You may find helpful the link below about Reproducible Examples. Try to keep the example as simple as it can be and still be useful.
Are you sure you want your data in a matrix? The typical container for data is a data frame, though a matrix can be a good choice under the right circumstances.
I have also attached a link to the book R for Data Science. That is a good source for learning about general data processing and display.