What is a performance test

  • Use tools to simulate a large number of user operations to verify that the system can withstand the load, identify potential performance problems, analyze and solve problems
  • Value of performance testing: ensure system performance, provide good user experience, find out weak links of system performance as far as possible, and help to optimize performance.

Performance Test Process

  • Business learning: Understand system functions
  • Requirement analysis: analyze the non-functional requirements of the system, delineate the scope of performance testing, and understand the performance indicators of the system
  • Job evaluation: Break down and evaluate the workload
  • Design Model: Business model – Mapping – Test model
  • Plan preparation: Planned test work (test scope, human input, duration, work content, risk assessment, risk response strategy, etc.)
  • Script development: Record or write performance test scripts (develop test flapper test programs or tools)
  • Test environment: Server and load machine
  • Test data preparation: Prepare master data and business data of the system under test according to the data model
  • The test execution
  • Defect management
  • Performance analysis
  • Performance tuning
  • The test report
    • Performance test major deliverables
      • The test plan
      • The test script
      • The test program
      • Test report or periodic test report
  • Review: Review the contents of the performance report, identify problems and assess the risk of going online.

Performance Testing elements (key concerns)

  • Assessment system, requirements analysis

    It is not entirely dependent on requirements documentation, but also requires the relevant personnel (o&M/product) to provide specific requirements data and perform secondary analysis ->> real performance requirements

  • Scenario design, use case design
    • Use case Design (select features that need to be performance tested)
    • Scenario design (effective organization of test cases — comprehensive allocation of users, execution time, execution ratio, etc., according to business distribution, business volume, business period and business role)
  • Test execution, whether it passes (performance indicators: response time RT, throughput TPS, transaction success rate, stability, memory leakage, hardware indicators

    , others (database, middleware, cache, JVM)
    ,>
  • Performance diagnostic optimization

Performance testing from a performance perspective

  • Server Hardware Performance
  • Develop performance targets based on requirements and historical data
  • Build performance through model
  • Perform performance analysis on development code framework and hardware framework
  • Benchmarks for development releases
  • Perform software performance acceptance and stability testing
  • Production environment configuration and optimization
  • Develop test cases for performance tests
  • Develop scenario design for performance testing
  • Coordinate with other departments
  • Specific performance analysis

Performance testing tool

  • Loadrunner (business) at HP
  • Apache JMeter (open source)
  • The Grinder (open source)…

Related terms

  • Load: Simulates the process by which business operations stress the server
  • Performance Testing: Simulate user load to test whether the system’s response time, throughput and other indicators meet Performance requirements under load
  • Load Testing: Under certain hardware and software conditions, the maximum number of users == that can be supported by increasing the Load (different virtual users) under the condition of meeting performance indicators ==. TPS\ RT\ CPU Using\Mem Using
  • Configuration Testing: obtain performance information under different configurations by means of Testing -> allocate resources reasonably to improve system operation efficiency
  • Stress Testing: under certain software and hardware conditions, by means of high load to make the server resources in the limit state, test system == in the limit state for a long time running stability == (index: TPS/RT/CPU Using/Mem Using)
  • Stability Testing: : Under certain hardware and software conditions, the system can run under certain load for a long time to determine whether the system runs stably on the premise of meeting performance indicators
  • TPS: Number of transactions completed per second
  • RT/ART (Respnse Time/ Average) : response Time/average response Time
  • PV (Page View) : indicates the number of Page accesses per second
  • Vuser Virtual user: a Virtual user that simulates real service logic steps
  • Concurrency concurrent
    • Narrow sense: all users make the same request to the system at the same time –>> Test database and program handling of concurrent operations (for performance/load/stress/stability test scenarios)
    • Generalized: Multiple users make requests or operations on the system, but these requests or operations can be different (for mixed scenarios, stability testing scenarios)
  • Scenario: the business processing process of real users is simulated —- A collection of a series of actions based on transactions, scripts, virtual users, running Settings, running plans, monitoring, and analysis is constructed
  • Think Time: Simulates the Time between pauses during actual operations by regular users
  • Standard Deviation (Std. Deviation) : The smaller the standard Deviation — the smaller the fluctuation, the more stable the system; The greater the standard deviation — the greater the fluctuation, the more unstable the system

The performance test passed the standard

  • Contains:
    1. Server Performance
    2. The front end performance
    3. User Experience Performance
  • Through standard