As more businesses have turned towards IT based solutions, it has resulted in continuous development of large scale & complex applications; which are expected to serve large number of users. Most of users from this era have near-zero tolerance level for poor or slow performing applications. Statistics tell that slow performance of any application leaves a very negative impact on end users; which ultimately affects the business revenue.
To tackle these factors, it is must to have performance testing as an integral part of test plan. We can define Performance testing as “a non-functional software testing technique which determines responsiveness, stability, reliability and resource usage of system under a certain user load”.
Performance testing can be further categorized into different types; which are conducted by testing teams as per project requirements. Before going into details of performance testing types, we will first look into common performance issues found in software applications and the factors that can be used to identify these performance issues.
a) Load Time: Load time is mostly associated with the time taken by a page to load or the initial time which any desktop, mobile application takes to start completely. This time should be as low as possible. According to industry standards it should be less than 5 seconds in case of web pages and less than 5-10 seconds in case of most mobile and desktop applications.
b) Response Time: Response time usually indicates the performance of an individual transaction or query. It is the time which starts from sending a request till the time of receiving response. Lower response time indicates good performance and higher response time indicates performance issues in application which need to be traced.
c) Performance Bottlenecks: Some common performance bottlenecks are related to CPU, memory and network utilization. To trace and resolve bottlenecks different approaches and tools can be used at different levels.
- at application level, developers can look into code to find poor algorithms and code inefficiencies
- at database level, database administrators can use profilers and improve queries
- at operating system level, system engineers can use performance monitors to find resource usage and ensure that it remains below the industry standard of 60%
Performance Testing Types:
Since the nature of performance issues vary from system to system; therefore we have various performance testing types which are normally performed by professional performance testers to identify performance issues. All these testing types have been given unique names as per their approaches but ultimately all these testing types come under the umbrella of performance testing.
Performance Test: A performance test is any test that can be used to measure performance, capacity, reliability, scalability and/or response/throughput time of software application.
Load Test: It is the type of performance testing which is used to understand performance behavior of any system under a specific user load for specific period of time.
In this approach, load is constantly increased on the system until the threshold value reaches. Increasing load means increasing the number of concurrent users and the transactions they are performing. This approach helps in identifying performance metrics like response time, throughput time, resource utilization etc. Similar load tests can be executed before and after code changes to check the difference in performance metrics. Such usage of load test is also termed as ‘Benchmark performance testing’.
For example, we have a load test for total 1000 concurrent users for total time span of 30 minutes. So, this test can be started from 10 users which can be constantly increased to 1000 concurrent users in time span of 30 minutes.
Stress Test: It is the type of performance test which is used to measure stability of system by testing it on its upper capacity limits and then pushing it beyond its capacity; while keeping the hardware resources constant. Because of this approach, it is considered a negative type of performance testing; as the objective here is to break the system under the extreme / unexpected conditions.
Software teams use this performance testing approach to identify the system components which fail first under soak test. These components are then tuned and optimized to either do not fail at all or recover more gracefully under extreme conditions.
Spike Test: Spike test is often considered a subset of stress test as in this form of testing; stress on system is put by increasing the number of users suddenly by a very large amount. Spike test can be defined as type of performance test; which is carried out to validate the performance characteristics of system when the load volumes are repeatedly increased beyond expected levels within short periods of time.
The objective of spike test is to determine whether the system will be able to sustain the sudden user load beyond expectation.
Soak Test: Soak test also known as endurance test, is a type of performance test in which load is put on system for a longer duration of time. Soak test duration can range from couple of hours to even a day.
Soak testing is performed because an application/server may work well for an hour or two, but may start giving issues if continuous load is exerted for a longer period of time. These issues normally arise because of memory leaks or because of hardware limitations and soak testing is very useful in finding such issues.
Capacity Test: Capacity test is a common type of performance testing which is used to determine how many users application can handle easily before its performance becomes unstable or unacceptable.
Capacity performance testing is often considered very important as it helps in determining the number of users which the application/server can handle easily. Results from capacity tests can be used to further optimize the usage of hardware resources or tune the application as per requirements.
In this blog, I did a brief overview of Performance testing and its major types. Performance testing is a vast domain and certainly there are many other areas which must be explored to understand the performance testing from its core. In future I will be exploring key software performance metrics and top performance testing tools like JMeter and LoadRunner. If you have a request for specific topic or have any feedback on this blog then do share it in comments section.Arif Masood