Has anyone else tested the performance of their Plumber endpoints over time? We have an endpoint that takes a few inputs, calls a database to get more info, then runs an H2O predict. Over the course of a day it seems to get slower and slower. We added debug messages and it is like the whole thing (data processing, data queries, H2O data transformation, H2O predict) slows down, not just one piece of the code. Restart Plumber and performance is good. We added a graph and it is consistent day after day and seems more time duration based than # of calls to the API.
A lot of it depends of your build environment. We've had stable API deployed for years without any trouble. The only time we had performance degradation is when we modified persisted object in memory and R had to reallocate more memory over time.
If you are on docker, maybe try with env OMP_NUM_THREADS=1.
The over time thing make me think there is something to do with memory management.
That (memory or object type growth over time) is what it feels like as well to me. I've been trying to figure out how to get visibility into that, so I created a plumber endpoint for debugging that I can call to log things like memory.profile() or gc() on demand so I can see how that compares between good/bad performing times.
Not on Docker, just an EC2 directly running Plumber. From an OS perspective the R process doesn't really appear to be growing in memory size.
Anyone else have ideas on performance metrics to look (like memory.profile() or gc()) at from within a long running R process like Plumber that might degrade over time.
This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.
If you have a query related to it or one of the replies, start a new topic and refer back with a link.