First public network security tests to provide guidance for building more secure mobile network

Crossbeam and Spirent Communications have announced results of the industry’s first public network security tests that emulate more than a million simultaneous mobile subscribers using their smartphones and tablets. Conducted by EANTC, an internationally recognised independent test centre, the tests evaluated the effectiveness of network security devices and how they perform under extremely demanding real-world conditions. Test results showcase what is necessary to credibly validate security performance and provide reliable guidance for mobile operators as they seek to build network security infrastructure that can handle growing user demand for 4G/LTE networks.

Spirent, a leader in networks, devices and services testing; Crossbeam, a provider of high-performance network security platforms; and EANTC collaborated to define a set of real-world test scenarios that emulated the performance impact of network security being enabled in a mobile network environment.

“With so much misleading information about security product performance and the unpredictability of threats and traffic demands, there has been no clarity around what to test for and what methodology to use,” said Chet Gapinski, vice president, product development at Crossbeam. “Any single measurement, such as network throughput, is meaningless unless tested and measured in combination with other metrics under real-world conditions. We saw an important opportunity to work with Spirent and EANTC on a set of real-world tests to move the industry forward to provide more accurate, uniform guidance.”

In order to replicate the incredible traffic loads that carriers experience at the mobile internet gateway (Gi/SGi), the Spirent Avalanche was used to emulate more than 1 million users concurrently accessing the mobile data network with their connected devices. The traffic was sent through a single Crossbeam X-Series X80-S platform running Check Point software, where it underwent a series of firewall and IPS (intrusion prevention system) inspections, as well as network address translation (NAT).

Measuring subscriber happiness

The tests were designed to not only understand how a variety of security devices affect the performance on mobile network infrastructure - but also how these results translate into subscriber satisfaction and quality of experience. Due to the intense processing required to secure mobile data traffic, the user experience can be dramatically impacted through slow page load times, extended response times and connection instability.

“There is no silver bullet for measuring security performance,” said Jeff Schmitz, vice president of the Networks & Applications group at Spirent. “No one calls their provider to complain about connections per second. Instead, our goal was to create a test solution that truly emulates what happens on mobile operator networks and provides realistic guidelines for building secure networks that can also support millions of connected devices with a high-quality user experience.”

The real-world traffic model was custom built to emulate a modern-day mobile environment with more than 1 million active users. This included using a 95% majority of HTTP sessions representing a range of user profiles. In addition, email (SMTP and POP3) connections were emulated alongside smartphone OS over-the-air updates.

Using Spirent’s Avalanche layer 4-7 test solution, EANTC validated the Crossbeam X-Series at 106 Gbps with stateful firewall, IPS and NAT enabled, with 1 million simultaneously active users, 4 million active TCP connections and 242,000 new connections per second.

In each test scenario - including firewall only, firewall and NAT, firewall and IPS, and NAT and IPS - every performance metric measured showed consistent service establishment performance at scale. Moreover, the user-experience measurements demonstrated the X-Series did not drop a single connection or request, and maintained a very low URL response time of 10 ms.

“We challenged the Crossbeam X-Series mobile security gateway with rigorous, objective and realistic performance tests,” said Carsten Rossenhövel, managing director, EANTC. “Mobile operators require dependable firewall performance and scalability figures for their fast-growing 3G and LTE services. Crossbeam’s solution worked flawlessly and surpassed our expectations in the various mobile deployment scenarios we tested.”

To download a full report on the test results, visit www.crossbeam.com/why-crossbeam/performance.