Frameworks Round 3 - TechEmpower Blog

You are viewing a single entry. Please also check out the most recent entries.

We've previously posted two rounds of results of benchmarking many web application platforms and frameworks. The community's response remains strong! We have really enjoyed your comments, advice, questions, criticism, and pull requests. Speaking of pull requests, we received tests for several additional frameworks since Round 2 and we have posted Round 3. Thanks to those contributions, the number of tests has grown to over 50. With that breadth we decided to move the project's results to a stand-alone site separate from this blog. View Round 3 results now. Round 3 notes and observations Thanks to enhancements made to Wrk by its author, wg, the tests in Round 3 are time-limited rather than request-limited. In previous rounds, each test ran 100,000 requests, which meant execution time spanned from seconds to hours depending on the framework. With Round 3, all tests run for 1 minute each. Timed tests means Wrk's computation of latency statistics is more accurate for high-performance frameworks. The community contributed numerous framework tests giving us coverage of several platforms we were missing in previous rounds. Round 3 includes Snap on Haskell; Elli and Cowboy on Erlang; Openresty on Lua; Tornado on Python; Onion on C; Slim, Codeigniter, Phreeze, Kohana, Lithium, Laravel, Silex, Fuel, and Symphony2 on PHP; Grizzly-Jersey and Play1 on Java; Scalatra, Lift, Unfiltered, and Finagle on Scala. The full results table is huge. We'll work to add better filtering controls for later rounds. Although we can't say for certain how many rounds this project will see, we have no plans to stop. Round 4 is already planned. If we're missing your favorite frameworks, we would love to receive a pull request. We have heard from a contributor who is working on a set of .NET/Mono tests, so we are optimistic that Round 4 will finally include .NET! In this round, we've tested Go and WebGo using Go 1.1 at the community's strong recommendation. The JSON tests on i7 have improved dramatically, with Go at just slightly over 200,000 requests per second. However, while Go is among the leaders in the JSON test, its newly added database tests are among the worst. The Go community is helping diagnose the problem and we expect its database performance to be improved in Round 4 or soon thereafter. In addition to Go, several other newly added frameworks now exceed 200,000 requests per second for the JSON test on i7: Finagle, Onion, and OpenResty join Round 2's leaders, Netty, Servlet, and Gemini. In database tests, the newly added Openresty test demonstrates extremely efficient database connection pooling and simple query execution. Openresty takes the lead for multiple queries on EC2. Play1 was added alongside its successor (Play 2) in Round 3. The Play1 implementation isn't able to successfully complete the database tests without errors. Based on community dialogue we've observed, we anticipate this problem will be resolved for a later round. In our tests, Slim and CodeIgniter appear to be the champions among the PHP frameworks. Symfony2 clocks in with lower performance than Cake.Comments A reader has posted this blog entry to Hacker News, and we invite you to comment there. Thanks! Very big thank-yous to all of the following who contributed to Round 3: Skamander (who has been especially generous with framework contributions), huntc, bitemyapp, sp1d3rx, keammo1, stevely, Licenser, torhve, Falmarri, kardianos, RaphaelJ, shenfeng, brendanhay, wsantos, greg-hellings, christkv, pakunoda, sidorares, PerfectCarl, tarndt, m20o, trautonen, jasonhinkle, gregwebs, and bakins. Thanks also to everyone who has e-mailed us and participated in the Hacker News conversations.

About this blog

Here, the programmers at TechEmpower share our thoughts and opinions. We discuss technology, development processes, the challenges faced by our clients, and the solutions we've put to use on real projects.

Subscribe to our blog's feed.

Questions or comments? Contact us.

Entries

April 2013 (2)March 2013 (2)January 2013 (2)