Have a look at calculating the geographic distance between coordinates .
It gives some pointers how to work with spatial data.
I like the sf package very much and would be surprised if it would not help you out.
If it does not work for you out-of-the-box, feel free to create a reprex with the first few lines of your data. That makes it easy for viewers to react.