Testing GIS Applications.flv

Performance testing can lead to proactive resolution of functional and system related issues before they impact end users and allow for identification of bottlenecks and/or underutilized resources for redeployment. To ensure a successful enterprise implementation, the following testing steps should be performed:

Application Functional Testing - The application functional testing is a crucial step in a successful GIS implementation. It is important that the application be complete, solid, sound and representative of defined business requirements.
System Tuning - Testing will expectedly reveal areas of improvement; however, some best practices should be implemented. This may include evaluation of the map documents, data location and data structure modifications.
Test Plan Development - A comprehensive test plan with input from domain/functional experts, documented user workflows and documented system configuration is imperative to testing success with effective and practical results.
Test Script Creation - Test Scripts are necessary so that tests can be re-run as required in a repeatable and controlled approach beginning with testing single user interactions and then scaling the testing to a simulated load test. Insertion of user transactions, user think time, pacing of user tasks as well as parameterizing transaction inputs are included in the scripts.
Testing Execution - Testing is performed using one or more load testing tools all while ensuring only target applications to be tested are running, as well as consistency in application data and the system configuration throughout the test cycles. This will produce reliable test results.
Analysis of Results - Analyze, compare and correlate key measures such as response times, and system behavior, evaluate unexpected results caused by test case or component failure and assess capacity thresholds and limitations.

Typically, simple web applications and services are relatively straightforward to script and performance test. However, rich client applications, e.g. ArcMap or web applications based on ADF API are more challenging to test. The following table ranks the relative level of testing for each application architecture:

Category Effort Considerations
Testing Level of Effort by Application Architecture
Rich Client Applications (ArcMap) High Requires scripting screen pixel clicks
Web Applications (JavaScript, Flex and Silverlight API) Medium-Low Stateless calls direct to the REST interfaces are easy, but emulating user movement within the application interface may be more challenging.
Web Applications (.NET and Java ADF) High
Requires managing application state and Ajax

Mobile Applications Medium
Services Low REST, and SOAP services are the simplest to test directly

Validate system configurations match deployment plans
Functional testing
It is important that the application be complete, solid, sound and representative of defined business requirements and functional specifications
Before testing ensure best practices and tuning are implemented such as
MXD tuning
Database tuning
A comprehensive test plan is imperative to testing success and includes
Interview application specialists
Define workflows and transaction to be tested, including think time
Define target user load, e.g. concurrent users, transactions per hour, and usage patterns
Define weights to usage patterns
Document configuration of the test environment
Verify success criteria
Define required metrics to be reported
Develop tests script as per Test Plan specification
Include validation rules
Select adequate test client machine
Define required metrics to be collected
Implement step load for virtual users
Implement step duration
Implement pacing
Initial run verifies the load test parameters yield expected behavior
Verify test results with additional test runs
The final step is to analyze, compare and correlate key measures such as response times, and system behavior, evaluate unexpected results caused by test case or component failure and assess capacity thresholds and limitations.
Review transaction response times
Identify dominant transactions by longest response times
Review Tier Metrics and Identify potential bottleneck Tier
Web servers
Application Servers
Database Servers
Review system metrics for each machine and identify potential hardware resource bottlenecks
CPU
RAM
DISK
Network
Review system and application matrix (including errors) for each tier and identify potential tier bottleneck
Test Client
Web servers
GIS Servers
Database Servers
Develop reports summarizing information

Category:

Science & Technology

Tags:

• Independent Software testing
• QA
• Quality Assurance
• Outsourcing
• Offshore
• product testing
• functional
• third party
• Localization
• system
• testing