What is Soak testing?

Soak testing is running a system at high levels of load for prolonged periods of time. A soak test would normally execute several times more transactions in an entire day (or night) than would be expected in a busy day, to identify any performance problems that appear after a large number of transactions have been executed.

Also, it is possible that a system may ‘stop’ working after a certain number of transactions have been processed due to memory leaks or other defects. Soak tests provide an opportunity to identify such defects, whereas load tests and stress tests may not find such problems due to their relatively short duration.



The above diagram shows activity for a certain type of site. Each login results in an average session of 12 minutes duration with and average eight business transactions per session.

A soak test would run for as long as possible, given the limitations of the testing situation. For example, weekends are often an opportune time for a soak test. Soak testing for this application would be at a level of 550 logins per hour, using typical activity for each login.

The average number of logins per day in this example is 4,384 per day, but it would only take 8 hours at 550 per hour to run an entire days activity through the system.

By Starting a 60 hour soak test on Friday evening at 6 pm (to finish at 6am Monday morning), 33,000 logins would be put through the system, representing 7½ days of activity. Only with such a test, will it be possible to observe any degradation of performance under controlled conditions. 

Some typical problems identified during soak tests are listed below: 
Serious memory leaks that would eventually result in a memory crisis, Failure to close connections between tiers of a multi-tiered system under some circumstances which could stall some or all modules of the system. 

Failure to close database cursors under some conditions which would eventually result in the entire system stalling. Gradual degradation of response time of some functions as internal data-structures become less efficient during a long test.

Apart from monitoring response time, it is also important to measure CPU usage and available memory. If a server process needs to be available for the application to operate, it is often worthwhile to record it's memory usage at the start and end of a soak test. It is also important to monitor internal memory usages of facilities such as Java Virtual Machines, if applicable.

Long Session Soak Testing
When an application is used for long periods of time each day, the above approach should be modified, because the soak test driver is not Logins and transactions per day, but transactions per active user for each user each day.

This type of situation occurs in internal systems, such as ERP and CRM systems, where users login and stay logged in for many hours, executing a number of business transactions during that time. A soak test for such a system should emulate multiple days of activity in a compacted time-frame rather than just pump multiple days worth of transactions through the system.

Long session soak tests should run with realistic user concurrency, but the focus should be on the number of transactions processed. VUGen scripts used in long session soak testing may need to be more sophisticated than short session scripts, as they must be capable of running a long series of business transactions over a prolonged period of time.
Test Duration

The duration of most soak tests is often determined by the available time in the test lab. There are many applications, however, that require extremely long soak tests. Any application that must run, uninterrupted for extended periods of time, may need a soak test to cover all of the activity for a period of time that is agreed to by the stakeholders, such as a month. Most systems have a regular maintenance window, and the time between such windows is usually a key driver for determining the scope of a soak test.

A classic example of a system that requires extensive soak testing is an air traffic control system. A soak test for such a system may have a multi-week or even multi-month duration.

PACING Calculation

Pacing Calculation :

D = Duration of the test(test window/time frame)

B = Baseline time(total time taken by 1 Vu to complete 1 whole iteration)

T = Total amount of Think time in the script

I = Expected/Target iteration

R = Residual time of the test window.

R = ( D - (T + B)*I)

P = Pacing interval

Dividing the residual time by target iteration gives pacing interval

Hence:

P = R/I

D is pacing time.
(T + B)*I represents the duration of a scenario
and P is the waiting time before the next scenario

Calculating Pacing Time/Think Time to achieve 50 TPS with an average response time of 0.5 seconds with
 total of 100 Users

Let us start with calculating total number of transactions in an hour.
1 sec = 50 trnx
3600 sec = x
x = 3600 * 50 = 180000 trnx/hour by 100 Users

We have total number of users given as 100. Let us see how to calculate how many transactions each user will perform.
Total Number of Users = 100
Each User will perform 180000/100=1800 transactions/hour

Since, every transaction is taking on an average 0.5 seconds, let us see how much time is required to complete the each user transactions.
To complete 1800 trnx it will take 1800*0.5 = 15 minutes

So now, let us see how much think-time is required to complete the required number transactions per User in an hour.
1800 transactions will complete in 15 minutes
hence, 45 minutes of thinktime is required in between 1800 transactions (i.e. 45*60 = 2700 seconds of thinktime required in between 1800 transactions (per user))
2700 seconds = 1800 trnx
x = 1 trnx
x = 1.5 seconds think time need to include

Let us see how much time is required to complete each iteration.
Total time required to complete each Iteration = x + 0.5 seconds
= 1.5 + 0.5
= 2 seconds

Verification: Let us verify if our above calculation is correct.
Total time = 1800 * 2
= 3600 seconds
= 1 Hr

So, Each User will perform 1800 transactions where we will provide 2 seconds for each Iteration to complete.

Random Virtual User Pacing in LoadRunner:
I guess there's always a first time. I had never used LoadRunner's random virtual user (VU) pacing earlier, but am working on a project that will need to use this feature for the first time. And as I thought about it a little more, I may start using it more frequently now. Here's how it happened

This is one of the rare projects that provided me with excellent documentation - not only system documentation like the system specs, API Guides etc but also performance requirements like actual production volume reports and capacity models that estimated the projected volumes.

The capacity models estimated the maximum transaction load by hour as well as by minute (max TPM). What I needed to do was take maximum hourly load, divide it by 60 to get a per minute transactional load and use this as the average TPM. The idea was to vary the VU pacing so that over the whole duration of test, average load stays at this Average TPM but it also reaches the Max TPM randomly.

For example, if the maximum hourly transaction rate is 720 requests and maximum TPM is 20, the average TPM will be 720/60 = 12 and I will need to vary the pacing so that the load varies between 4TPM and 20TPM and averages to around 12TPM.

The Calculation:

To vary the transactional load, I knew I had to vary the VU Pacing randomly. Taking above example, I had to achieve 12TPM and I knew the transactions were taking around 1-2 seconds to complete. So I could have the pacing of around 120 seconds if I needed to generate a fixed load of 12TPM with a 5 second Ramp-up and 24 users.

Script	TPM	Number of VUs	Pacing (sec)	Ramp Up
Script 1	12	24	120	1 VU/5sec

So now to vary the TPM to x with the same 24 virtual users, I will need to have a pacing of 24*60/x. I got this from an old-fashioned logic which goes in my head this way:
24 users with a pacing of 60 seconds generate a TPM of 24
24 users with a pacing of 120 seconds generate a TPM of 24 * 60/120
24 users with a pacing of x seconds generate a TPM of 24 * 60/x

So using above formula, to vary the load from 20 to 4TPM I will need to vary the VU pacing from 72 to 360. So now we have:

Script	TPM	Number of VUs	Pacing (sec)	Ramp Up
Script 1	4 to 20	24	Random (72 to 360)	1 VU/5sec

 

Of course, there's a caveat. The range of 72 to 360 seconds has an arithmetic mean of 216. 120 is actually the harmonic mean of the 2 numbers. So the actual variation in TPM will depend on the distribution of random numbers that LoadRunner generates within the given range. If it generates the numbers with a uniform distribution around the arithmetic mean of the range, then we have a problem.

I ran a quick test to find this out. I created an LR script and used the rand() function to generate 1000 numbers between the range with the assumption that LR uses a similar function to generate the random pacing values.

int i;
srand(time(NULL));
for (i=0;i<1000 br="" i="">lr_output_message("%d\n", rand() % 289 + 72);
}

And of course, the average came out close to the arithmetic mean of 72 and 360, which is 216.

So with the assumption that the function used by LoadRunner for generating random pacing values generates numbers that are uniformly distributed around the arithmetic mean of the range, we'll need to modify the range of pacing values so that the arithmetic mean of the range gives us the arithmetic mean of the TPM that we want...phew. What it means is that the above pacing values need to be modified from 72 to 360 (arithmetic mean = 216) to 72 to 168 (arithmetic mean = 120). However, this gives us the TPM range of 20 to 8.6 TPM with a harmonic mean of 12TPM.

But I'll live with it. I would rather have the average load stay around 12TPM. So here are the new values. Note the asterisk on TPM. I need to mention in the test plan that the actual TPM will vary from 8.6 to 20TPM with an average of 12TPM.

Script	TPM*	Number of VUs	Pacing (sec)	Ramp Up
Script 1	4 to 20	24	Random (72 to 168)	1 VU/5sec

Bugzilla-A Bug tracking tool

What is Bugzilla?

Bugzilla is a bug tracking system developed at mozilla.org.

How do enter a bug in Bugzilla?
To enter a bug, through "Enter a new bug" link from the main Bugzilla page. This will take you to a product 

selection screen.

What happens once enter a bug?
After you enter a bug, mail is sent both to you and the QA department. A member of the QA department will

 verify that they can reproduce your bug.

How do search a bug?
To search a bug, through "Query" link from the main Bugzilla page.

How do submit a patch?
The new Bugzilla system supports the attachment of patches, test cases, and various other forms of file types

 directly from the bug report screen. Just click on "Create an attachment"

Are cookies required in Bugzilla?
Yes.

How can you view your assigned bugs?
We can view the assigned bugs though “My Bugs” link.

How can you generate bug report?
We can generate bug report through “Report” link.

When Bugzilla was released?
Bugzilla was released in 1998.

Which language was written Bugzilla first time?
Bugzilla was originally written in "TCL".

Who developed the Bugzilla?
Terry Weissman

Which language Bugzilla written?
"Perl"

What are the Bugzilla fields?
Bugzilla have 11 fields like-

1. Product
2. Component
3. Version
4. Platform
5. OS
6. Priority
7. Severity
8. Assigned To
9. URL
10. Summary
11. Description

How can you edit your account in Bugzilla?
We can use "User Preferences" link.

How can you add new Product in Bugzilla?
We can use "Product" link for adding new product.

How can you add Components of product in Bugzilla?
We can use "Components" link for adding components in Bugzilla.

How can edit version of any product in Bugzilla?
We can use "Edit Versions" link.

Perfmon to capture the Process Performance of server or system.

 How do we track and log system and process information on a Windows operating system?

 Windows system tools available for monitoring almost every type of performance, including CPU, memory,
 file system and network usage. You do not have to rely merely on the Task Manager.

There is a Windows utility called PerfMon (Performance Monitor) that has the ability to graph and log
 performance metrics for specific processes, as well as set alarms and timing for performance monitoring 
and logging.

Steps to View and Log Performance Data Using Windows 7 Performance Monitor:

1.Make sure LabVIEW is open.

2.Click on the Start Menu and click Run.

3.Type perfmon into the Run command prompt and click OK.



4.You will then see the Performance Monitor pop up.



5.Click on the green ‘+’ sign near the top of the Performance Monitor Window to bring up the Add Counters
 window.

6.On the left side,choose which counters to add and click Add >>. Your counters should now appear under 
Added Counters. Click OK.



There are many options for what counters to add. A few that may be of special note when dealing with
 LabVIEW performance issues are:

Memory
Processor

Process (shown above): Under process, one can choose specific programs to monitor. In the upper left
 Window, you can select the aspects of the process that you’d like to monitor such as % Processor Time or 
Virtual Bytes. In the lower left hand window, you can select which process you’d like to monitor. Above, 
LabVIEW has been selected.

Now you should be able to see all chosen counters updating on the graph

In order to log this data to file, right-click on Performance Monitor and select New»Data Collector Set.

Type in a name for your data set and press Next.

Complete the rest of the steps including choosing the location you want to save your log file.
When you want to start logging the performance data, right-click on your Data Collector Set and select Start.
 To stop logging, right-click and select Stop.





Note: PerfMon provides ActiveX properties and methods, allowing you to control it through another ADE
 and even use it as an embedded control in an application.

Playback Load Runner through Fiddler

If you are concerned that your Load Runner script is not emulating user behavior correctly and want to 

make sure it is download all the correct files etc, then you can connect Load Runner to Fiddler and play the

script so that fiddler captures the HTTP calls. For those that don’t know Fiddler is a debugging proxy that 
captures and displays the HTTP requests between a browser and the server (it does more but that is a quick 

summary). An example output is shown below:

You can then run VuGen or (don’t do it at load) through the controller by setting the proxy under runtime
 options.

Proof of Concept-POC

A proof of concept (POC) is a demonstration whose purpose is to verify that certain concepts or theories 
have the potential for real-world application. POC is therefore a prototype that is designed to determine
 feasibility, but does not represent deliverables.Proof of concept is also referred to as proof of principle.

A proof of concept (POC) or a proof of principle is a realization of a certain method or idea to demonstrate
 its feasibility,or a demonstration in principle, whose purpose is to verify that some concept or theory has
 the potential of being used. A proof of concept is usually small and may or may not be complete.

Parameterization

What is Parameterization

• Replacing hard coded values in the script is called Parameterization.
• Parameterization helps in :
  • Reducing script size
  • Avoiding cache effect

Type of Parameters
1. Date/Time – Whenever we have to replace a date value with a parameter, Date/Time parameter is used. Any post with past date is not valid. To keep it updated, Date/Time parameter provides flexibility to get the current or future date. If past date is needed, it handles that too.

2. Group Name -We can generate a parameter on the basis of group that we select on controller for the script while execution. This parameter will only work while running the script on controller.

3. Iteration Number – This replaces the parameter with current iteration number. This is generally used to build some logic. For example- when we want some code in script to be executed alternatively. For this, we will use the iteration number to check whether it is even or odd number and for one of the condition we will execute the function.

4. Load Generator Name – We can also generate parameter while executing the script on controller on the basis of load generator name on which that script is running. This parameter only works while running the script on controller.

5. Vuser ID – When we run the script on controller, it assigns a unique id to each virtual user that emulate during the execution. This parameter type is used –

• To print the Vuser ID in an external file for script-debugging purpose.
• To segregate transaction volume based on Vuser ID

6. File – Some time we want to pass the specific value in the script. In such cases, we use file and enter the values that want to use during execution. LR provides options to run the script with provided list sequentially or randomly on next iteration.
In few cases we want to use a set of values passed to the script. In such cases, we can use same file for the other parameter value as well.

7. Random Number – As per need, Vugen also generates random value from the provided range.

8. Unique value – In few situations, script is not allowed to pass any duplicate value. In such cases, unique parameter is used  to avoid failures due to duplicate value,.

9. User Defined function – Such parameter calls a function whose return value replaces the parameter name.

10. XML – XML Parameter Types are used for multiple valued data contained in an XML structure.  XML parameters are widely used with Web Service scripts and with SOA services.

Difference between correlation and parameterization

In Correlation the Dynamic Value is Generated by the Server.

In Parametrization the Value will be given by the User.

Brief:

When one wants to perform the script’s actions like submit using different values from those recorded.

 To do this, we can replace the recorded values with parameters.This is known as parameterizing the script.

Eg: Try to Login Yahoo Mail or Gmail with Multiple Username and Password in Load Runner.

Many applications and Web sites identify a session by the current date and time. If you try to replay a script,

 it will fail, because the current time is different than the recorded time.

Correlation is a concept where we can capture the dynamic data when it is generated and used

 throughout the script.

Eg: In IRCTC website, Login and check the URL. The URL consist of Session Value which is dynamically 

Generated by the Server and even developer cannot tell that "What the dynamic value is going to generated".

Web Functions in Loadrunner

Function Name	Description
web_add_auto_filter	Sets criteria to include or exclude URLs when downloading
web_add_auto_header	Adds a customized header to all subsequent HTTP requests.
web_add_cookie	Adds a new cookie or modifies an existing one.
web_add_cookie_ex	Adds a cookie with specified add behavior.
web_add_filter	Sets criteria to include or exclude URLs when downloading.
web_add_header	Adds a customized header to the next HTTP request.
web_cache_cleanup	Cleans the content of the cache simulator.
web_cleanup_auto_headers	Removes all previous customized headers from HTTP requests.
web_cleanup_cookies	Removes all cookies currently stored by the Vuser.
web_concurrent_end	Marks the end of a concurrent group.
web_concurrent_start	Marks the beginning of a concurrent group.
web_convert_date_param	Converts a date string stored in a parameter to a different format.
web_convert_from_formatted	Converts a string created with one or more data format extensions back to the source format.
web_convert_param	Converts an HTML parameter to a URL or text.
web_convert_to_formatted	Formats data using one or more data format extensions.
web_custom_request	Allows you to create a custom HTTP request with any method supported by HTTP.
web_disable_keep_alive	Disables Keep-Alive HTTP connections.
web_dump_cache	Save the Vuser cache to a file.
web_enable_keep_alive	Enables Keep-Alive HTTP connections.
web_find	Searches inside an HTML page for a specified text string.
web_get_int_property	Returns specific information about the previous HTTP request.
web_global_verification	Searches for text string in all subsequent requests.
web_global_verification_pause	Suspends the specified checks set by web_global_verification.
web_global_verification_resume	Resumes the specified checks suspended by web_global_pause.
web_image	Emulates a mouse click on the image defined by the attributes.
web_image_check	Verifies the presence of a specified image inside an HTML page.
web_load_cache	Load the Vuser cache from a file.
web_link	Emulates a mouse click on the link that is defined by the attributes.
web_reg_add_cookie	Registers a search for a text string and adds a cookie if the text is found.
web_reg_cross_step_download	Registers a request to continue processing script steps without waiting for the server to terminate the response.
web_reg_find	Registers a search for a text string for the next Action function.
web_reg_save_param	Deprecated: Use web_reg_save_param_ex or web_reg_save_param_xpath.Saves dynamic data from an HTML page to a parameter.
web_reg_save_param_ex	Registers a request to save dynamic data located between specified boundaries to a parameter.
web_reg_save_param_regexp	Registers a request to save dynamic data that matches a regular expression to a parameter..
web_reg_save_param_xpath	Registers a request to find dynamic data in an HTML buffer that has been formatted as XML and save the data to a parameter.
web_remove_auto_filter	Removes a filter on download content.
web_remove_auto_header	Stops adding a specific header to subsequent HTTP requests.
web_remove_cookie	Removes the specified cookie.
web_report_data_point	Specifies a data point and adds it to the test results.
web_revert_auto_header	Stops adding a specific header to subsequent HTTP requests, but generates Implicit headers.
web_save_header	Saves request and response headers to a variable.
web_save_param_length	Saves the length of a parameter.
web_save_timestamp_param	Saves the current timestamp.
web_set_certificate	Causes a script to use a specific certificate that is listed in the Internet Explorer registry.
web_set_certificate_ex	Specifies location and format information of a certificate and key file.
web_set_max_html_param_len	Sets the maximum length of retrieved dynamic HTML information.
web_set_max_retries	Sets the maximum number of retries for an Action step.
web_set_option	Sets a Web option.
web_set_proxy	Specifies that all subsequent HTTP requests be directed to the specified proxy server.
web_set_proxy_bypass	Specifies the list of URLs that scripts access directly, i.e., not via the specified proxy server.
web_set_proxy_bypass_local	Specifies whether or not the proxy server should bypass local addresses.
web_set_secure_proxy	Specifies that all secure HTTP requests be directed to the specified secure proxy server.
web_sjis_to_euc_param	Converts an SJIS string to EUC and assigns it to a parameter.
web_set_sockets_option	Sets an option for secure sockets.
web_set_timeout	Specifies the maximum amount of time that a script waits for a specified operation to be performed.
web_set_user	Specifies a login string for a Web server.
web_submit_data	Performs an "unconditional" or "contextless" form submission.
web_submit_form	Performs a POST request; submits forms.
web_switch_net_layer	Specifies a network layer for replay. (WAP only)
web_url	Performs a GET request; receives pages or images.

web_reg_save_param_regexp

Do you have a correlation which you can’t solve because the values of the left and right boundary are dynamic? Correlation is an essential part of performance test scripting and there are plenty of different challenges with correlation. Imagine having a value of “GraphA123567EndGraphA” and the goal is to correlate 123567
From the example above, the left and right boundaries would be “LB=GraphA” “RB=EndGraphA”
What if the word GraphA is dynamic and can be anything from GraphA-GraphZ?
There is a solution at hand!
Using web_reg_save_param_regex will allow the user to grab a correlation value using dynamic left and right boundaries. This function uses the power of regular expressions, below are a few examples:
Example 1:
Source: “GraphA123567EndGraphA”
Solution: web_reg_save_param_regexp(“ParamName=CorrValue”, “RegExp=\“Graph[A-Za-z]\”, \“([0-9]+)\”, \“EndGraph[A-Za-z]\””, LAST);
Result: 123567
Example 2:
Correlate the values from a drop down list of a form
Source: dropdown >>> red, blue, green
Solution: web_reg_save_param_regexp(“ParamName=ColourList”, “RegExp=option=[0-9]+>([A-Za-z])”, “Ordinal=All”, LAST);
Result:

• {ColourList1}=red
• {ColourList2}=blue
• {ColourList3}=green

Example 3:
Correlate up till the end of 642
Source: J\u002blsGd3zj1qdP\u002bvk0vDRaKyJFde5tCa6spDEy08SNab1hP8j5GTs4j6\u002f\u002bTqOwvxMHEQZLWd\u002btu8NlHJrVAarIQ==|634998513832503642″];
Solution: web_reg_save_param_regexp(“ParamName=SecurityString”,”RegExp=\”([A-Z0-9a-z\\\\+]+==\\|[0-9]+)\”\\];”,LAST);
Result:J\u002blsGd3zj1qdP\u002bvk0vDRaKyJFde5tCa6spDEy08SNab1hP8j5GTs4j6\u002f\u002bTqOwvxMHEQZLWd\u002btu8NlHJrVAarIQ==|634998513832503642
Example 4:
Correlate only “634998513832503642” Source:

<em>J\u002blsGd3zj1qdP\u002bvk0vDRaKyJFde5tCa6spDEy08SNab1hP8j5GTs4j6\u002f\u002bTqOwvxMHEQZLWd\u002btu8NlHJrVAarIQ==|634998513832503642"];</em>

Solution:

web_reg_save_param_regexp("ParamName=SecurityString", "RegExp=\"[A-Z0-9a-z\\\\+]+==\\|([0-9]+)\"\\];", LAST);

Result: 634998513832503642
So what is a Regular Expression?
Also known as regex, a regular expression is a search string which enables matching of a string. Think of it as an advance searching function which can pick out values from a string of multiple characters.
Examples of regex:

• \d matches a single digit
• \w matches a single word (including alphanumeric characters and underscore)
• [A-Z]+ matches any word which is upper case
• [a-z]+ matches any word which is lower case
• [0-9]+ matches any numeric value

There are other alternatives to web_reg_save_param_regexp. However these functions are limited and not as flexible.
LB/DIG RB/DIG – # will be a wildcard for a numeric value“session_id_##”

• Adding LB/IC/DIG will ignore case
• “LB/IC/DIG=session_id_##=” (e.g. Session_id_20)

LB/ALNUM or RB/ALNUM – ^ will be a wildcard for an alphanumeric value

• ALNUMIC – ignore case
• ALNUMLC – match only lower case
• ALNUMUC – match only upper case

SaveOffSet

• If there is a dynamic value for a boundary e.g. “session_id_2” (3,4,5)
• SaveOffSet = 2 (to cover “2=”)
• Web_reg_save_param(“SessionID”, “LB=session_id_”, “RB=\””, “SaveOffSet=2”, LAST);

LR implementation

• PERL based
• LR 11 does not support multiple capture groups however this is now supported in LR 11.52 (example below)

Example Multiple Capture Groups
Source: rows”:[[“NW,RO,RA","DLY","10/07/2011","10/17/2011","10/01/2011","RA","Y","FR","AMEA","AC","1945","50","50","AC 100IOSH-08","UserDefined","10/07/2011","Reassigned"..."
Solution: web_reg_save_param_regexp("ParamName=ParamValue","RegExp=rows":\[\[“[^"\r\n]*”,”([A-Z]{3})”,”[^"\r\n]*”,”[^"\r\n]*”,”[^\/]+\/[\d]+?\/2011″,”[A-Za-z]*”,”[^"\r\n]*”,”[^"\r\n]*”,”([^"\r\n]*)”,”[^"\r\n]*”,”([^"\r\n]*)”,LAST);
Result:

• {ParamValue1} = DLY
• {ParamValue2} = AMEA
• {ParamValue3} = 1945

Web_Set_Max_Html_param_Len ()

How can we update the length of the parameter? Web_Set_Max_Html_param_Len ().

Default session id length – 256 Characters

To update length of the parameter

Web_set_max_html_length (“1024“) function is used to set the maximum length to the parameter to update the length of the parameter

Types of performance testing?

Load testing:Load testing is conducted to verify that your application can meet your desired performance objectives; these performance objectives are often specified in a service level agreement (SLA). A load test enables you to measure response times, throughput rates, and resource-utilization levels, and to identify your application’s breaking point, assuming that the breaking point occurs below the peak load condition.

purpose:To verify application behavior under normal and peak load conditions.

Stress testing – involves testing an application under extreme workloads to see how it handles high traffic or data processing .The objective is to identify breaking point of an application

Purpose:To determine or validate an application’s behavior when it is pushed beyond normal or peak load conditions.

Endurance testing – is done to make sure the software can handle the expected load over a long period of time.

Spike testing – tests the software’s reaction to sudden large spikes in the load generated by users.

Volume testing – Under Volume Testing large no. of. Data is populated in database and the overall software system’s behavior is monitored. The objective is to check software application’s performance under varying database volumes.

Scalability testing – The objective of scalability testing is to determine the software application’s effectiveness in “scaling up” to support an increase in user load. It helps plan capacity addition to your software system.Capacity testing:Capacity testing is conducted in conjunction with capacity planning, which you use to plan for future growth, such as an increased user base or increased volume of data. For example, to accommodate future loads, you need to know how many additional resources (such as processor capacity, memory usage, disk capacity, or network bandwidth) are necessary to support future usage levels.
Capacity testing helps you to identify a scaling strategy in order to determine whether you should scale up or scale out.

Purpose:To determine how many users and/or transactions a given system will support and still meet performance goals.

Types of VUsers

Types of VUsers

LoadRunner has various types of Vusers. Each type is designed to handle different aspects of today's client/server architectures.

You can use the Vuser types in any combination in a scenario in order to create a comprehensive client/server test. The following Vuser types are available: 

The Vuser types are divided into the following categories:

All Protocols: a list of all supported protocols in alphabetical order. 

1.  Application Deployment Solution: For the Citrix protocol. 
2. Client/Server: For DB2 CLI, DNS, MS SQL, ODBC, Oracle (2-tier), Sybase 
3. Ctlib, Sybase Dblib, and Windows Sockets protocols. 
4. Custom: For C templates, Visual Basic templates, Java templates, 
5. Javascript and VBscript type scripts. 
6.  Distributed Components: For COM/DCOM, Corba-Java, and Rmi -Java protocols. 
7. E-business: For FTP, LDAP, Palm, Web (HTTP/HTML), Web Services, and the dual Web/Winsocket protocols. 
8.  Enterprise Java Beans: For EJB Testing and Rmi-Java protocols. 
9. ERP/CRM: For Baan, Oracle NCA, Peoplesoft-Tuxedo, Peoplesoft 8, 
10. SAPGUI, SAP-Web, and Siebel (Siebel-DB2CLI, Siebel-MSSQL, Siebel-Web, and Siebel-Oracle) protocols. 
11.  Legacy: For Terminal Emulation (RTE). 
12.  Mailing Services: Internet Messaging (IMAP), MS Exchange (MAPI), POP3, and SMTP. 
13.  Middleware: Jacada and Tuxedo (6, 7) protocols. 
14.  Streaming: For MediaPlayer and RealPlayer protocols. 
15.  Wireless: For i-Mode, VoiceXML, and WAP protocols. 

Unlike a WinRunner workstation which emulates a single user's use of a client, LoadRunner can emulate thousands of Virtual Users.

Load generators are controlled by VuGen scripts which issue non-GUI API calls using the same protocols as the client under test. But WinRunner GUI Vusersemulate keystrokes, mouse clicks, and other User Interface actions on the client being tested Only one GUI user can run from a machine unless LoadRunner Terminal Services Manager manages remote machines with Terminal Server Agent enabled and logged into a Terminal Services Client session.

During run-time, threaded vusers share a common memory pool. So threading supports more Vusers per load generator.

The Status of Vusers on all load generators start from "Running", then go to "Ready" after going through the init section of the script. Vusers are "Finished" in passed or failed end status. Vusers are automatically "Stopped" when the Load Generator is overloaded.

No additional license is needed to monitor standard web (HTTP) servers (Apache, IIS, and Netscape).

To use Web Services Monitors for SOAP and XML, a separate license is needed, and vUsers require the Web Services add-in installed with Feature Pack (FP1)

lr_save_datetime function in Loadrunner

LoadRunner stores date/time parameters locally on the LoadRunner PC. This means that opening a script
 on another users PC, where the date/time parameter is not stored, can cause corruption to the date/time 
parameters in a script. You know that this has happened when a date/time parameter in this format
 [%Y%m%d%H%M%S] is sent to the server rather than the format that you intended.

Due to this problem, It is recommend that the lr_save_datetime function is used to save date or time
 values to a string. The string can then be used in the script and the script is portable between PCs because
 we are no longer relying on the locally stored parameters.

Sample code

lr_save_datetime("%d/%m/%y", DATE_NOW, "DDMMYY");
lr_output_message("Today's Date is %s",lr_eval_string("{DDMMYY}"));

lr_save_datetime("%d/%m/%Y", DATE_NOW, "DDMMYYYY");
lr_output_message("Today's Date is %s",lr_eval_string("{DDMMYYYY}"));

lr_save_datetime("%B %d %Y", DATE_NOW, "Today");
lr_output_message("Today is %s",lr_eval_string("{Today}"));

lr_save_datetime("%B %d %Y", DATE_NOW + ONE_DAY, "Tomorrow");
lr_output_message("Tomorrow is %s",lr_eval_string("{Tomorrow}"));

lr_save_datetime("%A", DATE_NOW - ONE_DAY, "Yesterday");
lr_output_message("Yesterday was %s",lr_eval_string("{Yesterday}"));

lr_save_datetime("%A", DATE_NOW, "Today");
lr_output_message("Today is %s",lr_eval_string("{Today}"));

lr_save_datetime("%A", DATE_NOW + ONE_DAY, "Tomorrow");
lr_output_message("Tomorrow is %s",lr_eval_string("{Tomorrow}"));

lr_save_datetime("%X", TIME_NOW , "Time");
lr_output_message("The time is %s (in this locale's date format)",lr_eval_string("{Time}"));

lr_save_datetime("%X", TIME_NOW + ONE_HOUR , "Time");
lr_output_message("In one hour it will be %s",lr_eval_string("{Time}"));

lr_save_datetime("%A", DATE_NOW + ONE_DAY, "Tomorrow");
lr_output_message("Tomorrow is %s",lr_eval_string("{Tomorrow}"));

lr_save_datetime("%j", DATE_NOW, "Today");
lr_output_message("Today is day %s in the year",lr_eval_string("{Today}"));

lr_save_datetime("%Z", DATE_NOW, "TimeZone");
lr_output_message("This machine is in the '%s' time zone",lr_eval_string("{TimeZone}"));

lr_save_datetime("%p", DATE_NOW, "AMPM");

if (strcmp (lr_eval_string("{AMPM}"),"PM")!=0)
{
lr_output_message("Good Morning");
}
else
lr_output_message("Good Afternoon");

SCOM (System Center Operations Manager) Monitoring tool

System Center Operations Manager 2012 – the complete application monitoring solution

For many years Operations Manager has delivered infrastructure monitoring, providing a strong foundation on which we can build to deliver application performance monitoring. It is important to understand that in order to provide the application level performance monitoring, we must first have a solid infrastructure monitoring solution in place. After all, if an application is having a performance issue, we must first establish if the issue is due to an underlying platform problem, or within the application itself.

A key value that Operations Manager 2012 delivers is a solution that uses the same tools to monitor with visibility across infrastructure AND applications.

To deliver application performance monitoring, we provide 4 key capabilities in Operations Manager 2012:
Infrastructure monitoring – network, hardware and operating system
Server-side application monitoring – monitoring the actual code that is executed and delivered by the application
Client-side application monitoring – end-user experiences related to page load times, server and network latency, and client-side scripting exceptions
Synthetic transaction – pre-recorded testing paths through the application that highlight availability, response times, and unexpected responses

Configuring application performance monitoring

So it must be hard to configure all this right? Lots of things to know, application domain knowledge, settings, configurations? Rest assured, this is not the case! We make it incredibly easy to enable application performance monitoring!

1. Define the application to monitor.



2. Configure server-side monitoring to be enabled and set your performance thresholds



3. Configure client-side monitoring to be enabled and set your performance thresholds



And that’s it, you’re now set to go. Of course setting the threshold levels is the most important part of this, and that is the one thing we can’t do for you… you know your application and what the acceptable performance level is.
Configuring an application performance dashboard in 4 steps

It’s great that we make the configuration of application performance monitoring so easy, but making that information available in a concise, impactful manner is just as important.

We have worked hard to make the creation of dashboards incredibly easy, with a wizard driven experience. You can create an application level dashboard in just 4 steps:

1. Choose where to store the dashboard



2. Choose your layout structure. There are many different layouts available.



3. Specify which information you want to be part of your dashboard.



4. Choose who has access to the dashboard. As you will see a little later in this article, publishing information through web and SharePoint portals is very easy.



And just like that, you’ve created and published an application performance monitoring dashboard!

Anyone who has either worked in IT, or been the owner of an application knows the conversations and finger pointing that can go on when users complain about poor performance. Is it the hardware, the platform, a code issue or a network problem?

This is where the complete solution from Operations Manager 2012 really provides an incredible solution. It’s great that an application and associated resources are highly available, but availability does not equal performance. Indeed, an application can be highly available (the ‘5 nines’) but performing below required performance thresholds.

The diagram below shows an application dashboard that I created using the 4 steps above for a sample application. You can see that the application is available and ‘green’ across the board. But the end users are having performance issues. This is highlighted by the client side alerts about performance.






Deep Insight into application performance

Once you know that there is an issue, Operations Manager 2012 provides the ability to drill into the alert down to the code level to see exactly what is going on and where the issue is.




Reporting and trending analysis

An important aspect of application performance monitoring is to be able to see how your applications are performing over time, and to be able to quickly gain visibility into common issues and problematic components of the application.

In the report shown below, you can see that we can quickly see areas of the application we need to focus on, and also understand how these components are related to other parts of the application, and may be causing flow-on effects.




Easily make information available
With Operations Manager 2012, we have made it very easy to delegate and publish information across multiple content access solutions. Operations staff have access to the Operations Manager console, and we can now easily publish delegated information to the Silverlight based Operations web console and also to SharePoint webparts.