JMeter 5.1: What It Is & How It Works

In December of 1998 the Apache Software Foundation released an open source performance tool called JMeter to little fanfare in the industry. In those days, proprietary tools like Mercury (HP) LoadRunner, Borland’s Silk Performer, Compuware’s QALoad and IBM’s Rational Performance tool dominated the market and continued to do so for almost a decade – with all of them consolidating under Microfocus along the way. Open source tools like Open STA, The Grinder, and Pylot were present but usage of JMeter dwarfed all of this combined. Essentially, JMeter is a 100% pure Java desktop application whose main function is to measure performance of websites through features such as load testing functional behaviour.  So, why did JMeter become so popular? Why has it lasted nearly 2 decades while most open source and even some very popular proprietary tools have faded away? Simply put – it works. It gets the job done and provides just enough functionality to provide a good basic load test. Are there tools that provide more functionality and that can scale better for large tests? Yes. Are there tools that have a better scripting interface and provide better support for modern SPA websites? Yes. There isn’t really any area where JMeter is the best in class. Except for the one that may matter most – value. For the price there is nothing better. The main reason JMeter has stayed relevant is that it has stayed close to the load and performance testing community. It also garnered a passionate fan base early that kept development moving forward. I have personally seen a proprietary tool get shifted away from the community. The tool starts out meeting a genuine need, the community gets very excited and word spreads that a new tool is available. However, the business of making money will always come to the fore. Companies that are willing to pay will drive the direction of the tool even if that direction is not what the majority of the community is doing. As a result, the tool starts to become a specialized tool that is very good for a small subset of the community, but doesn’t fit quite as well with the rest of the load and performance testing world. This would be OK if it was an open source tool, but when you are paying money for a tool you really want something that meets most, if not all, of your needs. The result is that these tools now have to change  and adapt to support JMeter scripts! For example, SOASTA updated CloudTest to support importing JMeter scripts directly into CloudTest. Why is it that after 10 years of telling prospects they need to move away from JMeter do they now embrace JMeter? Simple. JMeter is not going anywhere. All the new features that are included with 5.1 just emphasize how important this tool is to the load and performance testing community. Here are some of the JMeter features that will really help many in the load and performance community:

• JDBC testing has been improved with ability to set init SQL statements and add compatibility with JDBC drivers that do not support QueryTimeout
• Search Feature in View Results Tree allows searching for text / regexp in Request/Responses/Headers/Cookies/
  • This feature is huge (said in a Donald Trump voice) because searching for response and requests is at the heart of script creation. This will allow you create correlations much faster. This will save hours of work when creating scripts!
• Validation features, in one click run a selection of Thread Groups with 1 user, no pause and 1 iteration
  • Can I get a high five? Validating changes can be very time consuming. Most tools will let you do one line as a time or the whole script. With this you can select which you want to run as a 1 time iteration.
• Several reporting capabilities
  • There are some real time reporting capabilities and now some very nice HTML reporting features with zoomable charts and sortable columns. The graphics also have a much nicer look and feel.

Entire companies are being built around JMeter (BlazeMeter, Flood IO and Octoperf, etc..). Recently I was speaking with the founder of one of these companies and they have spent considerable time and money building a very functional tool that has at its core JMeter. They are not alone. Many are seeing the opportunity to support the largest load and performance testing community on the planet. It is the proverbial ‘if you can’t beat ‘em, join ‘em’ move. I have used JMeter for several projects over the years and here is my verdict: it has worked wonderfully for some projects and given me nightmares for others. For example, any test at scale in the cloud will cause you pain unless you use a company like Blazemeter, Flood IO or Octoperf. Trying to use JMeter with Single Page Applications can really drive you crazy. There is no perfect tool out there, they all have their strengths and weaknesses. But JMeter has proven that it stands up there with the rest of the major players. It will continue to play a major role in the load and performance testing community. At TPC we have worked with nearly every tool out there and know the wrinkles, warts and beauty marks of them all. We can help you find the best tool for the best job. Reach out to us!