Head of Client Strategy
October 3, 2024
We’ve been building teams overseas for over a decade. Download our definitive guide to hiring international software developers.
By all measures, LoadRunner has stood the test of time over the history of performance testing. Originally known as StormRunner when it was first developed by Mercury Interactive back in 1993, LoadRunner is still used by enterprise performance engineering teams all over the world. Over the years, the product has also endured a multitude of ownership changes. Mercury Interactive was sold to Hewlett-Packard in 2006. The assets were sold again to MicroFocus in 2017 - which was then acquired by OpenText in 2023.
To its credit, LoadRunner has remained a stable, reliable product despite all the shifts. In our performance engineering practice at Perform, we’ve had clients who used it for decades before making a switch to a more contemporary solution. Some still use the platform, and quite successfully at that.
For our clients who decided to move away from LoadRunner, many of them did so for the same sets of reasons. In this article, we’ll share the parallels we’ve observed and offer a comparison to Tricentis’ NeoLoad - which we view as a leading LoadRunner alternative for enterprise use cases.
At Perform, we don’t advocate for clients to change performance testing platforms just for the sake of change. There needs to be sound business and technical reasoning for investing in new tools, along with implementation and training services. If these three factors apply to your team, it might actually make more sense to continue with the status quo (assuming you receive the same level of service after the OpenText acquisition).
If the target application is in ‘maintenance mode’ and isn’t undergoing continual changes, it’s much easier to maintain your repository of LoadRunner scripts. This is often the case for legacy applications where changes are made in a slow-moving, waterfall environment.
Assuming you have well-established procedures for tasks like scripting, test execution, analysis, and reporting, it might not be worth disrupting the continuity
If your team is also highly proficient with LoadRunner and hasn’t used other frameworks before, you’ll need to account for a learning curve on a new solution. Based on your project workload, it might not be worth investing in change management.
When the three conditions we just reviewed are met, sticking with the ‘tried & true’ status quo is a safe bet. However, the “This is just how we do things” mentality - even in the face of inefficiencies and limitations - isn’t a case for resistance to change.
As we continue, we’ll examine the functional differences and similarities between LoadRunner vs. NeoLoad. First, let’s review the three top reasons teams start considering a change to their approach in the first place.
If one of more of the following considerations apply to your team, it’s at least worth evaluating alternatives.
LoadRunner was developed and refined well before more fast-paced Agile methodologies became common. While LoadRunner still shines in most waterfall environments, teams tend to struggle with script maintenance once their organizations move from quarterly release cycles to biweekly or continuous releases. When scripting becomes such a burden that it takes away performance engineers from higher-impact activities, it’s worth considering alternatives.
If you’re starting a new complex project and anticipate the need to generate new performance test suites, first contemplate how much effort will be required to craft LoadRunner scripts. Other performance platforms utilize low-code test design that make scripting much more efficient (and easier to maintain as the project evolves). At Perform, ran an experiment where the same 5 tests were designed - first with NeoLoad, and then with LoadRunner. We found that NeoLoad offered a 70% improvement in efficiency.
When performance engineers are tasked with supporting a wide variety of applications, their bandwidth can be stretched thin. As we’ve established, it can be hard to balance test maintenance with test design when using LoadRunner. This can ultimately lead to bottlenecks that can prove costly when performance testing is needed. Low-code and no-code solutions like NeoLoad not only make existing performance engineers more productive, they also make performance testing accessible for other team members. This leads us to our next point.
When considering alternatives for any sort of engineering or testing tool, you have to weigh pros and cons. You also need to consider factors like feature parity, or in this context - what capabilities would you have to give up if you move away from LoadRunner? While NeoLoad has a different approach to certain tasks, it still offers depth that even the most experienced LoadRunner veterans can appreciate.
Let’s take a look.
LoadRunner exclusively relies on scripting, which means it’s only accessible for skilled performance engineers with programming expertise. Make no mistake, performance engineering skillsets are not made redundant by NeoLoad’s low-code approach. Instead, it first takes the pain out of activities like test case scripting and maintenance. Performance engineers can still flex their technical acumen to extend NeoLoad’s capabilities with customizations that do require programming knowledge.
Here’s the takeaway. The low-code approach allows business users and other technical users to contribute to performance testing. While there’s still a learning curve, this can be reduced with proper onboarding and training (which we’ll soon discuss). Ultimately, this means that performance engineers can spend more time on high-impact activities, which is always a win for any business
If LoadRunner is supporting a wide breadth of applications - from legacy monoliths to mobile apps built microservices - you’ll quickly rule out most performance frameworks. Tools like Grafana K6 are adept at testing for ecommerce apps and API’s, but isn’t meant for use cases like complex enterprise applications such as SAP or Oracle. Oftentimes, LoadRunner is deployed in environments where a variety of systems need to be accounted for. Given NeoLoad’s ability to test against almost any kind of application, it’s a strong candidate to replace LoadRunner.
At Perform, we always encourage our clients to make sure performance testing frameworks don’t operate in a silo. It’s important to integrate your tests into your CI/CD pipeline for most efficient workflows throughout the delivery cycle. When you can easily configure smaller tests to run during the development cycle, performance testing can ‘shift left’. This is important for making sure you’re not over-reliant on load testing in the final stages of a test cycle.
Shifting right is also key. By integrating NeoLoad with APM solutions like Dynatrace, Splunk, and New Relic - you can aggregate performance data and production data together in a more holistic view. This unlocks what you can think of as ‘proactive performance monitoring’, which helps organizations detect performance issues prior to final releases.
Oftentimes, our LoadRunner customers have a need to test for baseline traffic on a continual basis and test for spikes on a more periodic basis. While most performance testing providers have license models that support this, NeoLoad does so in a simple and effective manner. In essence, you can purchase ‘Virtual User’ (VU) blocks - which can be used for continuous performance testing. You’re also able to purchase blocks of ‘Virtual User Hours’ (VUH’s).
You can think of VUH blocks as a pool of resources that can supplement your VU’s. Let’s say you have 500 virtual users for on-demand testing, but you want to run a 1-hr test to simulate 5,000 users on your application. The pool of Virtual User Hours enable you to temporarily scale testing up (without having to over-purchase VU’s that you don’t need on a regular basis).
Aside from scripting, maintaining, and running tests - performance engineers naturally spend a fair bit of time on reporting. This makes strong analytics capabilities a must-have, especially for organizations with more complex environments. Aside from integrating performance data with APM data, NeoLoad also offers a robust reporting experience that makes troubleshooting and root cause analysis much easier.
The change management associated with migrating from LoadRunner can be difficult to tackle on your own. You’ll need to allot time for migration, training, configuration - and so on. This also underscores the importance of selecting a performance testing platform that has a fast time-to-value. If you purchase a platform that’s cheaper, but also takes 3-6 more months to implement, it’s going to be detrimental to your business in the long run. This also leads us to our final topic for this article.
There’s an old saying that goes like, “If you want to go fast, go by yourself. If you want to go far, go with others.” We slightly modify this during discussions about moving away from complex solutions like LoadRunner:
Here’s a few considerations to keep in mind when selecting an MSP to assist you during your migration from LoadRunner
At Perform, our team has extensive expertise on both LoadRunner and NeoLoad. It does take a degree of adjustment (especially if you’ve been using LoadRunner for a number of years), but we’re able to correlate the similarities and differences between the platforms. Once you’re able to ‘connect the dots’ between each solution, it’s much easier to learn the ins and outs of using NeoLoad.
As a Preferred Neoload Solutions Partner, we’ve helped more than a hundred companies over the years navigate change management through a tightly refined implementation process. Here’s an example of a baseline project plan that’s delivered over the course of 4-8 weeks:
Of course, not every project is the same. Sometimes, they need to be altered based on factors such as the # of target applications, # of scripts, and the # of team members that need hands-on training. For this reason, every migration and training engagement starts with 1-2 scoping sessions to properly understand a client’s needs.
This leads us to the next consideration.
When purchasing any new platform, you’ll need to consider who will actually be using - or ‘owning’ - the tool on a given basis. In the context of performance engineering tools, this includes roles like admins and end users. It also extends to architects who can help define the strategy for how the solution will be used.
This is important because a proper training plan should account for the skill levels of everyone involved in the training exercises. An experienced provider will tailor training to the client and make sure ‘they meet the customer where they are’. Here’s a few important considerations that factor in project plans:
Based on the answers to questions like these, we can adjust a project plan so it’s more bespoke to an individual client.
Aside from areas like training and migration strategy, a project plan to implement NeoLoad should also consider integrations. This includes configuration for tools like your CI/CD pipeline and integrations to APM solutions like we discussed earlier. For some customers, this is more of a ‘one and done’ exercise. In cases where more extensive customizations are needed, it’s often best to start with a few basic CI/CD configurations and save more complex workflows for a subsequent project.
With Tricentis’ migration utility, customers are able to convert most types of LoadRunner tests into new NeoLoad tests. In some cases, it can be easier to just redesign a test in NeoLoad. During this process, it’s important to make sure transactions are similar so that you can mimic how your tests execute in LoadRunner. When we help customers navigate migration, we walk through the test design process and provide hands-on training so team members are comfortable doing so without our assistance in the future.
For the sake of brevity, we’ll use reporting and analytics as a stopping point. By the end of implementation process, teams should be able to view reports that consolidate performance and APM data in a single view. It’s also important to have reporting set up so end users and managers can easily view results for the latest builds. As part of our approach, we also enable clients to navigate NeoLoad reporting interface to leverage data to quickly analyze root causes when issues inevitably arise.
The primary purpose of a implementation & training plan to move from LoadRunner to NeoLoad is to enable a swift migration with minimal disruption to day-to-day business operations. After this phase is completed, clients often identify more use cases for leveraging NeoLoad. This might include expanding usage for other applications or driving additional coverage for the initial apps. Or, additional training might be needed for new hires or other team members who would like to contribute to performance engineering.
Whether you’re contemplating NeoLoad as a solution for your business - or you want to ‘shift NeoLoad into 6th gear - you’re more than welcome to book a time with our team to discuss your project. Feel free to pick an open timeslot for a strategy call using this link.
e.
Shane Shelton
Sr. Director
Application Performance and Development Operations, McGraw Hill Education
Founded by engineers - for engineers.
Expert consulting and staffing for software engineering at scale.
