Automatically handle pipes causing OOM
Planned
Could there be a way to handle pipes that causes e.g. OOM issues on a node? I've been involved with several projects where the whole node restarts continuously due to memory issues like "too big total entity size". Could this, or similar issues, instead cause the pipe do disable, or some other way of handling it such that the pipe does not simply try again and again with the same issue? In my situation these errors causes the whole node to either stop working, or freezing, such that nothing properly works (or slows the node way down). I understand that this might make the issues harder to detect, but at least for me it's better than having a node that doesn't work
-
Official comment
We definitely want to improve these aspects of hops. Planned, but this is a set of multiple long-term fixes. Some of the things that we are considering are:
- Automatic reprocessing should kick in and reset the pipe if the memory thresholds are reached (instead of failing the pipe)
- Spill large temporary result sets to disk instead of loading them into memory.Comment actions
Please sign in to leave a comment.
Comments
1 comment