Performance Woes
I’ve been putting a lot of work into pure-perl RDF classes recently, and have been frustrated by some ongoing issues.
I should mention that the work basically re-implements a lot of what RDF::Core provides, mostly because I think there are a lot of problems with the design and code in RDF::Core (some of which prevent optimizations that would make RDF on perl really appealing), and the few suggestions and patches I’ve submitted to the RDF::Core maintainers have been met with resistance, rejection, or frustratingly long delays.
Being able to start from scratch provides a lot of nice opportunities to get the design and code right. I was really excited about using forward looking modules like Module::Pluggable and Moose to implement these new RDF classes. Unfortunately, I’ve run into a lot of performance problems with Moose (and also with Math::BigInt, but I’ll get to that).
When running some initial benchmarking code to determine where my code stands in relation to the existing perl RDF packages (RDF::Core and the perl bindings to RDF::Redland), I discovered that my code was running orders of magnitude slower than the competition. Profiling the code showed this:
%Time ExclSec CumulS #Calls sec/call Csec/c Name
94.3 40.64 159.52 631973 0.0000 0.0000 Class::MOP1
6.68 2.880 2.880 3315 0.0009 0.0009 DBI::st::execute
6.41 2.763 5.168 65400 0.0000 0.0001 Math::BigInt::new
…
1All functions in the Class::MOP namespace are grouped together.
This was certainly unwelcome. The meta object protocol code brought in by Moose was responsible for almost 95% of my program’s runtime. Ninety-five percent!
From the Class::MOP manpage:
This library in particular does it’s absolute best to avoid putting any drain at all upon your code’s performance. In fact, by itself it does nothing to affect your existing code. So you only pay for what you actually use.
Now, this may be true, but it seems like overkill to be using Moose mostly for the default constructor and accessor support, and paying this tremendous runtime cost. Class::MOP::Class::compute_all_applicable_attributes alone was responsible for 15% of the runtime. Another 24% was because of Class::MOP::Instance::new and Class::MOP::Class::construct_class_instance.
So out came the Moose code. Removing Moose mostly meant having to implementing my own constructors and accessors, so this took very little time. The biggest downside for me at the moment is in the loss of code conciseness.
Profiling the code again showed that Math::BigInt was now responsible for almost half of the runtime. Now, I saw this one coming. The problem involved having some simple 64-bit integer math, and needing to run on perl installations that don’t (necessarily) have 64-bit integers. Almost all machines perl runs on these days have some support for 64-bit integers, but many perls aren’t built to support them. This seems like a perfect problem that a module could address — support for 64-bit integer math without needing to use full BigInts — but an hour sifting through CPAN modules didn’t yield in anything promising.
I ended up implementing the code in XS, falling back to Math::BigInt if the machine doesn’t support 64 bit integers, but I’m not totally happy with this solution. I’m not sure my solution is totally portable, I’m not sure (yet) how to get Module::Build to work even if the compilation fails, and with the new XS code I’m having trouble using Devel::DProf to continue profiling.
Anyway, to wrap this up, the runtime of my code is now about half as fast as Redland and three-quarters as fast as RDF::Core — in the same general class to a first approximation (compared to the earlier orders-of-magnitude slower results).
I did start pulling out module dependancies to get here, but these results are without any optimization of my code. And the potential optimizations were what prompted this project. So I’ve got a lot of hope that this work could really pay off for working with RDF in Perl, both in terms of performance (expect to see much faster SPARQL queries soon) and in flexibility (I’ve got my eye on composite models and inferencing as next steps).
Comments
Stevan,
Performance isn’t the most important thing to me at this point. I started using Moose fully expecting an “early adopter” phase, but 95% runtime make continued development impractical.
Are there docs anywhere for using immuutable classes? I’d be interested in seeing if that helped the situation, as it seemed that most of the time spent in Class::MOP was at run time, not compile time.
Posted by: kasei on September 24th, 2006 9:26 PMKasei,
Well without seeing the code you benchmarked against I could not say for sure. However based on the 3 methods you highlighted (Class::MOP::Class::compute_all_applicable_attributes, Class::MOP::Instance::new and Class::MOP::Class::construct_class_instance), I would suspect you were either creating a lot of object instances or had several classes you were loading. As for how immutability would help you, both the compute_all_applicable_attributes and construction of the Class::MOP::Instance are memoized in an immutable class, and their runtime cost is then signifigantly reduced. Unfortunately there are very few docs on the immutable features at the moment, and it is only officially supported by Class::MOP and does not handle all of the Moose features completely yet (roles in particular). If you are still interested in trying to make Moose work, I would be happy to work with you on testing this out, just let me know.
Posted by: Stevan on September 25th, 2006 8:36 AM
Well, I will be the first one to tell you that Moose has some performance issues. However, we have not even begun optimizing Moose yet, so this is not a permanent condition.
While Class::MOP does try to stay out of the way of your code, it does have to bootstrap itself, so a very large portion of the performance cost in Moose and Class::MOP is in the compile time phase. Aside from a few key places, the runtime is rather speedy, and you really only pay for the features you use.
As for what is being done about this, we are currently focusing on theoretical correctness and the correctness of the roles implementation in particular (remember, no one has actually implemented true 100% feature complete roles yet, so this is virgin territory). Once we feel we are complete, we will optimize, and optimize vigorously. We have already implemented immutable (or closed) classes in Class::MOP and saw speeds comparable (and in some cases exceeding) that of hand coded Perl. Making your classes immutable eliminates a large portion of the runtime MOP overhead, but only somewhat helps the compile time. For compile time optimizations, we are exploring Module::Compile and using .pmc files which will likely eliminate much of the compile time bootstrap costs.
But all this aside, Moose is not yet 1.0 and is in what I consider to be an “early adopter” phase. If it is performance speed you crave, please come back in about 2-3 months. If you can live with the slowness for a while, then please stick around.
Posted by: Stevan on September 24th, 2006 8:08 PM