You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is there a way currently to set the heap size to a specific amount? Maybe from a config, or on the context object? Found this online for v8:
--max-old-space-size
I would also like us to add a way dispose of V8 memory if you want to do it by hand:
ctx = MiniRacer::Context.new
ctx.eval(javascript)
ctx.dispose # will free all underlying c++ assets (same as deallocate)
ctx.eval(javascript)
=> MiniRacer::ContextDisposedError
and perhaps
MiniRacer::Context.run do |ctx|
ctx.eval(javascript)
end
# everything is freed except for the 1 Ruby RVALUE.
For my specific purposes, (since I'm not in-control of the js code that's being executed by the context) I need to be able to completely halt execution when the memory cap is hit.
So preferably, it would be the memory equivalent of the timeout that's passed in on context create.
Guaranteed my players are writing code that looks like this:
var a = []
for(var i = 0; i < 10000000000000............; i++) a.push(i);
I figure the best bet for implementing something like that based on what you linked above, is to use the allocation callback and call stop when the mem hits the threshold. Probably using some c-side alloc size tracking, so we're not constantly re-querying.
I do like the context block-form and the dealloc stuff- luckily, memory is holding real well between runs :D
# MiniRacer::Platform.set_flags! max_old_space_size: 10
# todo find a way to get the above working in a test case.
context = MiniRacer::Context.new
assert_raises(MiniRacer::ScriptTerminatedError) { context.eval('var a = []; while(true) a.push(new Array(100000000));') }
put this into a quick test case, but not sure where to hook up the callback or how to forward the exception safely.
Also not sure how to set the space size flag just for the one test case. (as it's a platform option)
Got back into investigation of this and unfortunately it looks like there's no way to recover from the oom exceptions through the artificially set memory constraints :/
As far as I can tell, these platform limits are not in sync with failstate handling. My plan now is to register an alloc tracker/counter and call terminate like we do for timeouts.
Any thoughts are much appreciated
Proposal for implementation:
Alongside with 'timeout' context creation option, a max_mem option can be provided.
If the max_mem option is provided, the allocation callback will be registered. Allocation callback tracks amount allocated minus amount freed, if an alloc puts the net above max_mem, then TerminateExecution will be called in the same manner as timeout.
Isolate and context are left in the same state as a timeout. Non-fatal failure & further evals will be available.
I do think there might be a perf hit from the alloc callback, but the code impact will be only be applicable to those that provide the max_mem option.
Ok made some progress on this, but hitting up some roadblocks in a few places:
multithreading support for this option is going to be hard. There's no 'userdata' passed into the callback & it's not a functor, which would mean some sort of array-of-function-pointers-based system & thread-count limit, etc, etc. > for my purposes, I use process-based concurrency, so I don't need multi-threading here.
TerminateExecution is not working from the callback. :\
TerminateExecution looks like a timeout termination, need some way to differentiate between the two
by the way, these are a segfault when they happen. There is no way to recover from the v8 built-in OOM failures, even when an artificial max-memory amount is set with the platform variables.
Getting a bit more time on this, and looks like there's a tasty OOM handler in v8 5.4 -
Should I open a separate issue for upgrade to 5.4? @SamSaffron
Hope all is well!
Activity
SamSaffron commentedon Jun 8, 2016
I would love to do something, but we need to do a bunch of research here... first place I would like us to look:
Is all the memory V8 is allocating going through https://github.com/discourse/mini_racer/blob/master/ext/mini_racer_extension/mini_racer_extension.cc#L12-L20 ? If it is can we easily add:
If the block allocater does then we can probably simply use http://bespin.cz/~ondras/html/classv8_1_1HeapStatistics.html which is returned via http://bespin.cz/~ondras/html/classv8_1_1Isolate.html#add32e78544edaf8946ed9b328167e5e4
We need to do this first, to implement caps we would have to use: http://bespin.cz/~ondras/html/classv8_1_1Isolate.html#acff413b8633aa13f1308697c0ce8c5fa but that would be fairly tricky, what should we do when you reach the limit? burn the isolate? leave it around and shift it into error state?
I would also like us to add a way dispose of V8 memory if you want to do it by hand:
seanmakesgames commentedon Jun 8, 2016
That all looks good!
For my specific purposes, (since I'm not in-control of the js code that's being executed by the context) I need to be able to completely halt execution when the memory cap is hit.
So preferably, it would be the memory equivalent of the timeout that's passed in on context create.
Guaranteed my players are writing code that looks like this:
I figure the best bet for implementing something like that based on what you linked above, is to use the allocation callback and call stop when the mem hits the threshold. Probably using some c-side alloc size tracking, so we're not constantly re-querying.
I do like the context block-form and the dealloc stuff- luckily, memory is holding real well between runs :D
seanmakesgames commentedon Jun 30, 2016
I'll put something simple together for just the max-heap setting of this. Hopefully we can put something together that has reasonable performance.
wk8 commentedon Jun 30, 2016
@seanmakesgames : FWIW, MiniRacer now supports run time flags, so you could try doing something like this:
It seems to work, albeit there still is some work to be done to turn that into a friendly Ruby exception:
Seems like we'd want to register a error handler with V8, see https://github.com/v8/v8/blob/5.1.281.59/src/api.cc#L242-L244
wk8 commentedon Jun 30, 2016
Seems pretty straightforward to do => https://v8docs.nodesource.com/io.js-1.2/d5/dda/classv8_1_1_isolate.html#af8b475bb23fabe5b7708bbc7c572276f
seanmakesgames commentedon Jun 30, 2016
Awesome! Thanks Jean @wk8
I'll see what I can put together. :)
seanmakesgames commentedon Jul 1, 2016
put this into a quick test case, but not sure where to hook up the callback or how to forward the exception safely.
Also not sure how to set the space size flag just for the one test case. (as it's a platform option)
seanmakesgames commentedon Aug 5, 2016
Got back into investigation of this and unfortunately it looks like there's no way to recover from the oom exceptions through the artificially set memory constraints :/
The callback works, but there's no way to signal 'this is ok' -- this used to work, but is now removed:
https://bugs.chromium.org/p/v8/issues/detail?id=2638
https://bugs.chromium.org/p/v8/issues/detail?id=3060
http://stackoverflow.com/questions/16797423/how-to-handle-v8-engine-crash-when-process-runs-out-of-memory
As far as I can tell, these platform limits are not in sync with failstate handling. My plan now is to register an alloc tracker/counter and call terminate like we do for timeouts.
Any thoughts are much appreciated
seanmakesgames commentedon Aug 5, 2016
Proposal for implementation:
Alongside with 'timeout' context creation option, a max_mem option can be provided.
If the max_mem option is provided, the allocation callback will be registered. Allocation callback tracks amount allocated minus amount freed, if an alloc puts the net above max_mem, then TerminateExecution will be called in the same manner as timeout.
Isolate and context are left in the same state as a timeout. Non-fatal failure & further evals will be available.
I do think there might be a perf hit from the alloc callback, but the code impact will be only be applicable to those that provide the max_mem option.
seanmakesgames commentedon Aug 8, 2016
Ok made some progress on this, but hitting up some roadblocks in a few places:
In progress-PR here:
#36
Any support is much appreciated. Trying to full-launch in the next month. 🚢
seanmakesgames commentedon Aug 25, 2016
by the way, these are a segfault when they happen. There is no way to recover from the v8 built-in OOM failures, even when an artificial max-memory amount is set with the platform variables.
seanmakesgames commentedon Dec 16, 2016
Looks like the new way to handle this is w/ an interruptrequest & checking the heapsize
http://stackoverflow.com/questions/10285737/limiting-memory-of-v8-context/34695455#34695455
I'm worried about this though:
https://groups.google.com/forum/#!topic/v8-users/kl-Zz7UQls0
seanmakesgames commentedon Mar 7, 2017
Getting a bit more time on this, and looks like there's a tasty OOM handler in v8 5.4 -
Should I open a separate issue for upgrade to 5.4? @SamSaffron
Hope all is well!
SamSaffron commentedon Mar 7, 2017
Merge pull request rubyjs#18 from sqreen/release/0.2.5.0.1.beta1