Non Functional Requirement (NFR)
The most important points to consider when writing an NFR are:
1. Qualified – It should describe in words what is required. This is important, spreadsheets with boxes
that have numbers in them are only useful if everyone agrees on the same assumptions – this is rarely
the case as a box with 1 second in it can mean a lot of things to a lot of people.
2. Defined – For example, a response can be measured at various points, there are lots of layers that a
request can pass through and the very last point in the stack is when it hits the user’s eyeball. It is
important to define which layer we are working in. Typically, in load testing you want to remove the
client and the network and only focus on server response times (different people have different bandwidths
and different PCs so we remove those variables).
3. Quantified – We need actual numbers to work to. The best number to use is the 90th percentile as this
prevent the minority skewing the majority.
4. Flexibility – NFRs are not written in stone and can be changed. But we must have a line in the sand to
start with. This line can be, and usually is, moved but having it defined prevents either needless tuning or,
worse, insufficient tuning.
5. Volumetrics – Now that we know how fast we want the system to respond we have to decide under
what conditions we want it to do this. This is known as defining the ‘load’ to be tested to and is most often
expressed in terms of business requests per second/minute. For end to end performance testing it should
be complete and describe traffic from all sources. For component testing it should set the scope for what
is to be tested. If the system is complicated then this data can be defined in a separate document and
referenced.
An example:
“As a business owner, I want 85% of Login requests to be served within 650ms, 95% within 850ms and
none over 1200ms where a response is defined as a HTTP 200 return code from the server to Apache and
does not include any client side or external network (internet) processing. Requests must be made against
a fully populated database that is a replica of Production and this level of performance should be
maintained for 1 hour without degradation during a simulation of the peak load as defined in the ‘Project
Volumetrics’ document which includes 2.6 Login requests per second via 220 concurrent threads.”
1. Qualified – It should describe in words what is required. This is important, spreadsheets with boxes
that have numbers in them are only useful if everyone agrees on the same assumptions – this is rarely
the case as a box with 1 second in it can mean a lot of things to a lot of people.
2. Defined – For example, a response can be measured at various points, there are lots of layers that a
request can pass through and the very last point in the stack is when it hits the user’s eyeball. It is
important to define which layer we are working in. Typically, in load testing you want to remove the
client and the network and only focus on server response times (different people have different bandwidths
and different PCs so we remove those variables).
3. Quantified – We need actual numbers to work to. The best number to use is the 90th percentile as this
prevent the minority skewing the majority.
4. Flexibility – NFRs are not written in stone and can be changed. But we must have a line in the sand to
start with. This line can be, and usually is, moved but having it defined prevents either needless tuning or,
worse, insufficient tuning.
5. Volumetrics – Now that we know how fast we want the system to respond we have to decide under
what conditions we want it to do this. This is known as defining the ‘load’ to be tested to and is most often
expressed in terms of business requests per second/minute. For end to end performance testing it should
be complete and describe traffic from all sources. For component testing it should set the scope for what
is to be tested. If the system is complicated then this data can be defined in a separate document and
referenced.
An example:
“As a business owner, I want 85% of Login requests to be served within 650ms, 95% within 850ms and
none over 1200ms where a response is defined as a HTTP 200 return code from the server to Apache and
does not include any client side or external network (internet) processing. Requests must be made against
a fully populated database that is a replica of Production and this level of performance should be
maintained for 1 hour without degradation during a simulation of the peak load as defined in the ‘Project
Volumetrics’ document which includes 2.6 Login requests per second via 220 concurrent threads.”
No comments:
Post a Comment