RESOURCES

Network Testing for NEMs

Monitoring Optimization for Data Centers

Network Emulation Resources

Product Spotlight

Watch the 5200 Net Tool Optimizer Product Overview

(3 mins. flash required)

WATCH NOW »

TESTIMONIAL

Case Study:  Microsoft Windows Server and Windows Vista Upgrade Testing

Background

The Tolly Group is a US-based (Boca Raton, FL) provider of third-party validation services for vendors of IT products, components, and services.  Tolly has worked with a variety of hardware and software vendors of all sizes to test and validate performance of a wide range of technical products.

Challenge

During 2008, The Tolly Group conducted a thorough test of new Microsoft products, including Microsoft Windows Server 2008 (32-bit) and Microsoft Windows Vista Ultimate SP1 (32-bit). Microsoft made significant improvements to the networking components of these operating systems. Specifically, they integrated policy-based QoS to manage network bandwidth, upgraded the TCP/IP stack in each product, and enhanced Server Message Block (SMB) 2.0, which is the file sharing protocol used by Windows-based computers.

The goal of the analysis was to quantify the performance improvement IT managers can expect by upgrading existing Microsoft Windows Server 2003 R2 (32-bit) and Microsoft Windows XP SP2 (32-bit) installations to these new products. Specific topics that were evaluated included network throughput and task completion time for common tasks on various WAN and LAN scenarios. Tolly wanted to accurately quantify how the new software products performed under a range of real world LAN and WAN environments.  In order to successfully benchmark the differences between the technologies, Tolly had to effectively emulate a high-speed 10 Mbps WAN with 50-ms of delay, a 2 Mbps T1/E1 WAN with 150 ms of latency, a 1 Gbps LAN with 1 ms of delay, and a 1 Gbps LAN with 5 ms of delay. Test repeatability was a key requirement since 48 network and task scenerios were tested. 

The testing team selected a market leader, the Anue GEM: 10/100/Gigabit Ethernet Network Emulator, to handle the required emulation activities.  They chose the Anue product because it can quickly and easily emulate numerous network scenarios simultaneously, with a high degree of accuracy and repeatability. While emulating all four network types as indicated above, tests were run against the following product combinations on each configuration:

• Windows Server 2003 and Windows XP

• Windows Server 2003 and Windows Vista

• Windows Server 2008 and Windows XP

• Windows Server 2008 and Windows Vista.

The specific tests included:

• Copying Microsoft Office files over a WAN

• Opening Microsoft Office files over a WAN

• Supporting Mobility in the workplace

• Performance with portals

• Copying Microsoft Office files over a LAN

• Opening Microsoft Office files over a LAN

For data collection and analysis purposes, the team also included a PC running Wireshark network protocol analyzer.  Each Operating System was set up and  connected as follows:

• The Anue GEM Network Emulator was connected directly to a laptop that controls the GEM, both Windows server boxes, and a Finisar Cu GIGABIT IL/1 Shadow Tap.

• The PC running Wireshark and both client systems (XP and Vista) were connected to the Finisar Shadow Tap, effectively enabling connection of all systems through the Tap and GEM.

All tests were run under four scenarios as described above.

Results

The Anue GEM solution enabled The Tolly Group to successfully emulate real-world WANs and LANs at different bandwidths and latencies, revealing very clearly how the network affects the software performance in all 48 scenario combinations. It was also key to discovering exactly how much of a performance difference the new Microsoft products can deliver on any of the three networking technologies.

Test results revealed significant benefits for upgrading only the client systems to Windows Vista while still operating the server on Windows Server 2003. Specifically, throughput improved by 2X – 3X in various tests, and time-to-completion of key tasks was reduced by up to 60% vs. Windows XP.

The tests also showed even further performance improvements with Windows Server 2008 implemented in tandem with the upgrade to Vista. With both OS’s installed, the tests revealed improvements in both throughput and time-to-completion in excess of 3X.

The full report, published by Microsoft Coroporation, can be accessed here.