Testing in the enterprise: how hard can it be?

You get a patch or an upgrade for your firewall from the vendor. They say it fixes that one problem that’s been bugging you for the last few months. But you can’t just install the update, right? You have to test it first.

No problem. You upgrade the backup firewall, throw together a few scripts to hit it with traffic and use a packet sniffer to make sure the right stuff comes out the other end. Simple, right? Maybe not.

Before you go live, you have to make sure that the patch:

1. fixes the problem,
2. doesn’t break anything else, and
3. still gives you the throughput and performance you need.

Will your script verify the function and performance of all the features your organisation depends on? Will it generate stateful TCP conversations? Both inbound and outbound traffic flows? SSL in an HTTPS flow?

As you dig deeper, you find more questions than answers. If you find a problem, will your vendor accept the results of your homegrown test scripts? How do you know the problem isn’t with your scripts instead of the device? For example, can you demonstrate how much transaction latency is created by the traffic generation process and how much by the firewall?

The testing process is more than just discovering performance limits. Testing is about delivering the assurance and confidence that a system will perform at a given level. At the core of testing is the concept of acceptance, which can be achieved by testing for real-world scenarios.
Vendor data sheets promise that their network devices will perform at very high speeds. However, real-world performance depends on how the device is used - what the actual traffic flows are on the hardware and software in use. And despite what the data sheet says, one thing you know for sure: they didn’t test it on your network.

Your network was designed specifically to address the needs of your organisation, its geographic reach, its mix of users, applications and mission-critical processes, its requirements for response time, throughput, redundancy and failover.

Enterprise network and application testing: how hard can it be? Actually, if you have the right tools, it can be fairly painless. For more information on how to tame the complexity of testing in the enterprise, download the Spirent white paper, New Ideas for Enterprise Testing.