Load test tools are software assessment software and equipment that observes and understands the behavior of certain applications when under specific expected loads. Essentially, they are able to illustrate the performance of an application at both normal and peak conditions. This is especially prevalent within web testing, in which performance is often related to smaller web clients and large loads of concurrent users. The following paper observes a number of functions of two load testing tools, JMeter and Locust. JMeter is an Apache-owned property that aims to analyze and measure a diverse set of services, though it focuses on web applications. Locust is an open-source testing tool but is more scriptable and scalable, making it easier to manipulate to the tester’s needs. Within this paper, a literature review of a number of articles will be utilized to determine the ways in which JMeter and Locust interact with load tests for small or medium-sized web client applications through methodologies that include concurrent virtual users and its effects on increases or decreases in the usage of hardware consumption for memory.
JMeter’s ability to interact with concurrent users and the effect on hardware consumption can be observed through the offered testing models provided by the tool. The testing tool provides a number of load models, of which the closed model is able to construct a process by which concurrent users are limited to prevent crashing. On the other hand, the open model continues to observe the flow of virtual user arrivals to a website despite anything else happening on the tested system. As such, JMeter creates a space in which the users are independent of each other, and in the case of SUT slowing, the flow of new users will continue (Gomes Rodrigues et al., 2019). In the case of memory consumption, JMeter provides additional functions through their JProfiler software, which runs technical tests. The testing tool is able to provide memory dumps of the application at the beginning and end of the test. JMeter does so to ensure that the more recent version of the tested website or application does not consume as much memory as its prior version. A number of methods exist to change the memory consumption that occurs during load tests within the JMeter tool. These include excluding listeners from JMeter scripts, increasing Java heap size, and tune JMeter SaveServices, which can reduce the amount of data being written during a load test. The solutions mentioned hint at the sources of memory consumption that may occur during load testing for small or medium-sized web client applications. The overflow is often related to monitoring servers and controllers, recording sampler information, and the reduction of local memory consumption. As such, JMeter does observe substantial excessive usage in terms of hardware and memory consumption but is able to navigate these issues with a number of alternative functions.
The Python-based scenarios and open-source nature of Locust have been noted to allow more freedom in writing specialized tests. Another aspect of Locust that affects memory consumption in the case of concurrent user traffic is its design to generate and observe heavy workloads. This includes high virtual user traffic, specific behaviors, and other factors. Its high-performance testing is characterized by the creation of swarm-based sources of user traffic, which can then also simulate a random workload towards the SUT. Its procedures are also based on events and an approach that is asynchronous. In more detail, it is able to utilize the Gevent coroutine as the basis of this function. Due to Gevent coroutines Python-based networking and library, Locust is able to form performance testing that consumes decreased CPU and hardware memory resources (Shrivastava, 2020). This is a direct effect of the Locust’s ability to minimize memory consumption in relation to running concurrent user testing loads. Essentially, the application is able to provide a representation of a large number of users while still maintaining the option to run complex tests on typical and non-advanced hardware. Some of the processes Locust will be able to perform during the load include scripts that define customized virtual user classes and their behaviors based on tasks of the web client. This enables Locust to collect additional information about both users and user classes, such as metrics, pass or fail statuses, or other features. This can be done through the protocol, database, and HTTP clients within Locust’s built-in functions.
Both web testing tools feature their abilities to handle large workloads of big pools of concurrent users, diverse functions and behaviors, and the effects on memory consumption. While Locust has been noted to be inherently designed and more adept at handling complex tasks while maintaining high user traffic, JMeter utilizes a number of fixes that can increase or specialize the testing to the tester’s desired results. A study that evaluated a number of web testing tools, including Apache’s JMeter and Locust, was able to submit similar workloads to both tools in order to observe execution time, memory overflow and consumption, and relevance to SUT. Locust was able to consistently provide lower execution times for an identical workload as JMeter, though the differences were not excessive (Sharma & Kumar, 2020). Despite the minimal difference, a faster execution time can signal the fact that Locust is more capable of executing either higher workloads or more complex tasks with the same effort as JMeter. Locust also provided higher results in cumulative performance analysis, and lowest memory uses over a number of experimental studies. This is a direct result of the lightweight Gevent coroutine, but the average CPU usage and generated loads did not differ substantially among web testing tools. This indicated that while JMeter may have lacked memory preservation and execution time, it is equally capable of simulating and analyzing heavy workloads that are provided to Locust. The study also noted that in terms of pure HTTP requests, JMeter was able to generate the highest performance but continued to depict increased memory consumption. While Locust is able to create complex tasks alongside high workloads, there were noticeable decreases in throughput as the complexity of tests increased. This could be the result of Locust’s custom client reliance on external libraries when sending specialized request protocols. Although both web-testing tools proved to be able to manage high workloads of concurrent users when testing small to medium-sized web client applications, both have noticeable and unique drawbacks. In terms of the least usage of memory consumption during testing, Locust tends to rate over JMeter, primarily due to a more flexible design and the lightweight Python-based script model. In conclusion, while the workload management is equal, tasks that require the least possible memory consumption, as would often be necessary for a small web client application that may run on typical hardware, Locust is more likely to produce effective results.
References
Gomes Rodrigues, A., Demion, B., & Mouawad, P. (2019). Master Apache JMeter – from Load testing to DevOps. Packt Publishing.
Sharma, P. S., & Kumar, Y. (Eds.). (2019). A Pragmatic Evaluation of Stress and Performance Testing Technologies for Web-Based Applications. Institute of Electrical and Electronics Engineers. Web.
Shrivastava, S. (2020). Comprehensive Review of Load Testing Tools. International Research Journal of Engineering and Technology, 7(5), 3392-3395. Web.