I'm finding your question a little too vague to wrestle with.
Does the high COV impact the robustness of my models?
well, it makes fitting models non trivial. if all your variables were constant, what would there be to make a model out of ? if you outlaw high variance variables (whatever that means ) where does that leave you ?
Everything to do with model building is context dependent. i.e. Why are you building a model? you need to understand something scientifically, and to explain that to people ? you want to detect a disease in patients and are willing to accept some false postives so as to benefit from higher true positives, so that fewer folks dont slip through the net and miss treatment ?
what is the intent of the model building behaviour? Almost everything in model building world involves tradeoffs, so you need to be oriented to a goal in order to have a chance of picking a tradeoff that is favourable to your goal, if you are goaless then its almost arbitrary what model you make and will be satisfied with. At least in my understanding.