I have left that organization. I don't have that code anymore. But I think now if I have re write it it would be at least 10x faster.
It would be great if you can share 10x faster code.
I am sorry I left the organization along with the code.
But I didn't use too many lapply previously but know I should be able to use purrr and avoid many loops. And I could also use furrr for multiple processing.