Why test your 3G/4G mobile network? 10 things you need to know

Q: We are planning to migrate to cellular technology’s next-generation global standard: LTE. My highly qualified team of engineers has ample experience with designing and implementing mobile networks. If we have their stamp of approval, is it really necessary to test our newly proposed LTE solution any further?

A: While LTE offers higher data rates, better service quality and lower operational costs, it also represents the most drastic evolutionary step in several years and requires much more care than other recent wireless technology upgrades. LTE relies on new, complex evolving standards and some network operators who plan early LTE deployments are transitioning from older technologies that must interoperate seamlessly. Implementing this generational advancement successfully involves managing complex and multiple risks. What looks good on paper may not be in sync with the evolving complex realities of LTE. The three key vectors of successful LTE network deployment are: speed - high-throughput in the user plane with minimum loss, latency and jitter; connectivity - minimum session drops with Any-G to Any-G seamless mobility; application awareness - optimisation for QoS and QoE. Findings from Nemertes Research state that launching a technology initiative without testing can completely wipe out the anticipated operational and cost benefits. Proper and timely performance testing before launch assures the delivery of LTE’s promise.

Q: What is Any-G to Any-G, and why is Any-G to Any-G mobility testing important?

A: Any-G to Any-G means the capability of any generation of mobile technology being able to interact seamlessly with another generation (ie, 2G to 3G). LTE is the first technology to offer mobile roaming fully compatible with all previous generations of mobile network. Connectivity is critically important. Many mobile subscribers have switched carriers because of connection reliability, including a lack of seamless handoffs while roaming. Having a high session drop is unacceptable and can mean lost customers. Network deployments and modifications must assure a clean and seamless transition from one technology to the other and back. Any-G to Any-G mobility testing addresses this challenge.

Q: What are quantifiable benefits of performance testing my mobile network before launch?

A: First, testing can verify network equipment manufacturer product claims, acquiring consistent, comparable statistics to provide objective comparison of each tested vendor. In addition, not all networks will comprise equipment from a single vendor, so testing allows you to find performance data between different vendors. Second, you are able to establish network/device performance under load, identifying break points and allowing issues to be resolved prior to the subscriber experience. Third, you ensure network services and infrastructure meet current Standards requirements. Fourth, you gain budget savings with successful first-time seamless system integration. On a related budgetary point, equipment is expensive and testing helps optimise your expenditure, allowing you to know maximum capacity for future planning, assuring you don’t come up short in terms of capacity. And finally, you fortify customer satisfaction by delivering the highest service quality possible.

Q: We are a well-established WiMAX operation, have no plans to migrate to any new technology, yet we are planning to upgrade our network with new devices that will support new services we’ll be offering our customers. We believe testing is unnecessary. Do you agree?

A: With its mature and rich ecosystem, numerous experts state WiMAX is expected to undergo a surge in popularity in the near future. Currently, with 502 networks in 142 countries, a recent WiMAX.com survey reports that by 2013 there will be over 120 million subscribers. As WiMAX reaches its potential in a growing market share - offering innovative services such as open internet anywhere at true broadband speeds - it’s critical that past deployment challenges not impair future adoptions. The full benefits of WiMAX’s broad and innovative services cannot be delivered until performance and quality of service (QoS) are assured. It is therefore recommended that at each enhancement to any WiMAX network, testing be an integral part of the implementation plan.

Q: At a high level, what aspects of a mobile network deployment are potentially vulnerable to risk and would benefit by from predeployment performance testing?

A: Vendor selection: Testing network equipment against your requirements and specifications for performance and configurations, features delivered and bugs is critical to ensure the device works exactly as you planned. Test data also empowers your vendor negotiations. Network design: Proper testing challenges existing designs where specialists offer expert input for new designs. Network implementation: The correct testing methodology and experienced test team identifies all critical issues and roadblocks before you go live. Risks such as incompatible devices or protocols are addressed prior to launch. Network security: 100% effectiveness in security is 100% useless if it uses all your CPU preventing intrusion, slowing throughput in the process. Testing identifies these issues before there is a problem.

Q: In mobile network testing, why is test scalability important and what are some of the elements of testing in this space?

A: Because of network complexity, it’s not necessary to test just when you make major changes, but also for firmware updates and other seemingly minor changes that have the potential to seriously impact network performance. The scalability of test libraries is critical to successful regression testing, and at-a-glance pass/fail indications ensure a fully functional network after every network modification. It’s imperative that the test partner (or test tool) you select be fully scalable in testing session loading, rate, interoperability and handoffs, and cover node emulation to complete your network functionality for full-scale testing, including roaming, billing, policies, authentication and Any-G interoperability. The most effective testing strategies involve testing all the categories above separately, as well as all together, to provide the most reliable and useful test result data.

Q: What network performance tests are recommended for a successful implementation of LTE?

A: Typical LTE tests: maximum users; maximum user activation rate; maximum session loading (simultaneous activations and deactivation for a long period of time); maximum handoff rates (Inter-MME, Inter-SGW or Intra-SGW); maximum data throughput (stateless and stateful data flows). 4G wireless tests: MME node, SGW node, PGW node or end-to-end. Test network data collection: max number of users; max activation/deactivation rate; maximum activations/deactivations per second; maximum handoffs per second; maximum BPS and maximum PPS.

Q: What are the main advantages of testing high speed for LTE?

A: The new type of services that will be carried over the LTE network will be very demanding in terms of quality. New emerging services such as Hi-Def video and interactive mobile gaming are growing in robust popularity. Thus, mobile backhaul testing for loss, latency and jitter are key to ensure the QoS and quality of experience (QoE).

Q: I recognise the need to test my evolved packet core deployment for performance and mobility issues. I’ll just have my in-house team take over with the general performance testing guidelines I learned about here, right?

A: Best practices clearly indicate testing, before a mobile network deployment goes live, is an effective practice to mitigate costly risks. However, relying on in-house engineering resources, which may lack the required up-to-date expertise and experience, can produce unreliable test results. The choice of a qualified test partner is a critical choice, especially when deadlines are short, budgets tight and margin for error is zero. Employing a partner with a team of test experts conversant in the challenges in the field of mobile network testing is essential. Also, since the cost of the required test equipment can be prohibitive, selecting a partner that also provides rental options offers added advantages.

Q: What specific qualifications should I look for when choosing a third-party test partner?

A: First, they must be a neutral party, without a motivation to influence the test results, as may be the case with a system integrator testing their own solution. Second, testing should be a core competency of the partner, not an ad hoc solution offered on request. This means the partner is an established global name in the test and measurement industry, with verifiable experience and expertise, and a team of quality test engineers. The testing team should have a holistic understanding of networks and be able to articulate testing benefits/ROI vs risk and have an established delivery process. A test partner’s qualifications are further strengthened if they have extensive expertise in lab and test automation, and can provide references of successful engagements. The most advantageous test partners that provide test rental equipment are those who supply devices manufactured by the same company.