Quantcast
Channel: Hacker News 50
Viewing all articles
Browse latest Browse all 9433

Feature #8339: Introducing Geneartional Garbage Collection for CRuby/MRI - ruby-trunk - Ruby Issue Tracking System

$
0
0

Comments:"Feature #8339: Introducing Geneartional Garbage Collection for CRuby/MRI - ruby-trunk - Ruby Issue Tracking System"

URL:https://bugs.ruby-lang.org/issues/8339


Feature #8339

Introducing Geneartional Garbage Collection for CRuby/MRI

Added by ko1 (Koichi Sasada)1 day ago. Updated about 2 hours ago.

Description

| One day a Rubyist came to Koichi and said, "I understand how to improve | CRuby's performance. We must use a generational garbage collector." Koichi | patiently told the Rubyist the following story: "One day a Rubyist came | to Koichi and said, 'I understand how to improve CRuby's performance..." | [This story is an homage of an introduction in a paper: | "A real-time garbage collector based on the lifetimes of objects" | (by Henry Lieberman, Carl Hewitt) | <http://dl.acm.org/citation.cfm?id=358147&CFID=321285546&CFTOKEN=10963356>] We Heroku Matz team developed a new generational mark&sweep garbage collection algorithm RGenGC for CRuby/MRI. (correctly speaking, it is generational marking algorithm) What goods are: * Reduce marking time (yay!) * My algorithm doesn't introduce any incompatibility into normal C-exts. * Easy to development Please read more details in attached PDF file. Code is: https://github.com/ko1/ruby/tree/rgengc How about to introduce this new GC algorithm/implementation into Ruby 2.1.0? Thanks, Koichi

gc-strategy-en.pdf(716.9 kB)ko1 (Koichi Sasada), 04/28/2013 03:19 am

History

Great, even a simpleton like me understood. So Dog.new.speak will get incinerated as soon as it finishes its "Bow wow", while senior objects have tenure. Btw. let me express thanks for 2.0, it's noticeably faster than 1.9.

Great work!!! Does RGenGC Ruby pass test-all?> How about to introduce this new GC algorithm/implementation into Ruby 2.1.0? +1

(2013/04/28 9:23), authorNari (Narihiro Nakamura) wrote:> Great work!!! Does RGenGC Ruby pass test-all? Sure. -- // SASADA Koichi at atdot dot net

Wow, nice! Is there a patch/branch around somewhere already to take a look at? Keep up the good work!

(2013/04/28 20:27), rkh (Konstantin Haase) wrote:> Wow, nice! Is there a patch/branch around somewhere already to take a look at?> Code is: https://github.com/ko1/ruby/tree/rgengc -- // SASADA Koichi at atdot dot net
On Sat, Apr 27, 2013 at 8:19 PM, ko1 (Koichi Sasada)<redmine@ruby-lang.org>wrote:> We Heroku Matz team developed a new generational mark&sweep garbage> collection algorithm RGenGC for CRuby/MRI.> (correctly speaking, it is generational marking algorithm)>> What goods are:>> * Reduce marking time (yay!)> * My algorithm doesn't introduce any incompatibility into normal C-exts.> * Easy to development>> Please read more details in attached PDF file.> Code is: https://github.com/ko1/ruby/tree/rgengc Great work! Are instances of user-level Ruby classes (`class MyClass; end`) marked as sunny by default as well? If not, would it be difficult? Also, I notice that generation/wb are stored as flags on the object. Wouldn't this undo the work of the new bitmap mark flag improvements in 2.0?
On Sat, Apr 27, 2013 at 8:19 PM, ko1 (Koichi Sasada)<redmine@ruby-lang.org>wrote:> We Heroku Matz team developed a new generational mark&sweep garbage> collection algorithm RGenGC for CRuby/MRI.> (correctly speaking, it is generational marking algorithm)>> What goods are:>> * Reduce marking time (yay!)> * My algorithm doesn't introduce any incompatibility into normal C-exts.> * Easy to development>> Please read more details in attached PDF file.> Code is: https://github.com/ko1/ruby/tree/rgengc>> How about to introduce this new GC algorithm/implementation into Ruby> 2.1.0?>> Thanks,> Koichi Another thing: What about passing the old value to OBJ_WB too? OBJ_WB(from, to, old)? In this implementation it would just be a no-op. There are some nice garbage collectors that you can implement if your write-barriers have old values too (e.g "An On-the-Fly Mark and Sweep Garbage Collector Based on Sliding Views").

(2013/04/28 21:40), Magnus Holm wrote:> Are instances of user-level Ruby classes (`class MyClass; end`) marked> as sunny by default as well? If not, would it be difficult? Not difficult (I think). I'll try ASAP.> Also, I notice that generation/wb are stored as flags on the object.> Wouldn't this undo the work of the new bitmap mark flag improvements in 2.0? Good question. I believe it is only a small affection (w/ some modification, I considered about that).> Another thing: What about passing the old value to OBJ_WB too? OBJ_WB(from, to, old)? In this implementation it would just be a no-op. There are some nice garbage collectors that you can implement if your write-barriers have old values too (e.g "An On-the-Fly Mark and Sweep Garbage Collector Based on Sliding Views"). Interesting. But I doubt that we can implement them (w/o incomplete WBs). Anyway, I need to read a paper. If there are good resources (slide pdf, etc) to know them, please tell them. -- // SASADA Koichi at atdot dot net

(2013/04/28 23:34), SASADA Koichi wrote:>> Another thing: What about passing the old value to OBJ_WB too? OBJ_WB(from, to, old)? In this implementation it would just be a no-op. There are some nice garbage collectors that you can implement if your write-barriers have old values too (e.g "An On-the-Fly Mark and Sweep Garbage Collector Based on Sliding Views").> Interesting. But I doubt that we can implement them (w/o incomplete> WBs). Anyway, I need to read a paper. If there are good resources (slide> pdf, etc) to know them, please tell them. Maybe I understand the snapshot idea. I don't think it is acceptable for CRuby (because CRuby moves memory areas!). But not for CRuby, it may considerable. -- // SASADA Koichi at atdot dot net

On Sun, Apr 28, 2013 at 6:07 PM, SASADA Koichi <ko1@atdot.net> wrote:> (2013/04/28 23:34), SASADA Koichi wrote:> >> Another thing: What about passing the old value to OBJ_WB too?> OBJ_WB(from, to, old)? In this implementation it would just be a no-op.> There are some nice garbage collectors that you can implement if your> write-barriers have old values too (e.g "An On-the-Fly Mark and Sweep> Garbage Collector Based on Sliding Views").> > Interesting. But I doubt that we can implement them (w/o incomplete> > WBs). Anyway, I need to read a paper. If there are good resources (slide> > pdf, etc) to know them, please tell them.>> Maybe I understand the snapshot idea.> I don't think it is acceptable for CRuby (because CRuby moves memory> areas!). But not for CRuby, it may considerable. In hindsight I think that the sliding views-technique is only required if you have multiple threads running at the same time. In CRuby there will only be a single thread running Ruby code so this becomes easier. For a concurrent tracer (tracing/marking objects as the Ruby thread runs) the most important thing is to get a consistent view of the heap. Example: (1) Tracer starts running in the background (2) a = object.thing; object.thing = nil (3) Tracer reaches `object`, doesn't find `a`, frees it (4) object.thing = a, dangling pointer If you have a WB that records previous values you just need to add the previous value to the marklist (e.g. `object.thing = nil` will add `a` to the marklist). If we have the sunny/shady-distinction it might be possible to (1) stop the world, (2) mark all roots, trace all shady objects (3) start the world (4) trace sunny objects concurrently. We just need a strategy for objects that become shady during the concurrent tracing. // Magnus Holm

(2013/04/29 1:19), Magnus Holm wrote:> In hindsight I think that the sliding views-technique is only required> if you have multiple threads running at the same time. In CRuby there> will only be a single thread running Ruby code so this becomes easier.> For a concurrent tracer (tracing/marking objects as the Ruby thread> runs) the most important thing is to get a consistent view of the heap. Please read my previous message: "I don't think it is acceptable for CRuby (because CRuby moves memory areas!)". Concurrent tracing needs an assumption that "do not move (free) memory area except sweeping timing". Current CRuby does. For example: "ary << obj". Yes, the CRuby's memory management strategy (assumption) is different from normal interpreters. -- // SASADA Koichi at atdot dot net

On Sun, Apr 28, 2013 at 6:29 PM, SASADA Koichi <ko1@atdot.net> wrote:> (2013/04/29 1:19), Magnus Holm wrote:> > In hindsight I think that the sliding views-technique is only required> > if you have multiple threads running at the same time. In CRuby there> > will only be a single thread running Ruby code so this becomes easier.> > For a concurrent tracer (tracing/marking objects as the Ruby thread> > runs) the most important thing is to get a consistent view of the heap.>> Please read my previous message: "I don't think it is acceptable for> CRuby (because CRuby moves memory areas!)".>> Concurrent tracing needs an assumption that "do not move (free) memory> area except sweeping timing". Current CRuby does.> For example: "ary << obj". Yes, the CRuby's memory management strategy> (assumption) is different from normal interpreters. I was not aware of that. What exactly does "moving memory areas" mean? Do you have any links?

"ko1 (Koichi Sasada)" <redmine@ruby-lang.org> wrote:> How about to introduce this new GC algorithm/implementation into Ruby 2.1.0? What is the expected performance impact for short-lived scripts and one-liners? I hope there is no (or very little) regression.

Also available in:AtomPDF


Viewing all articles
Browse latest Browse all 9433

Trending Articles