How We Test VPNs – CNET

Whether you’re using a virtual private network to hide your online activity from your ISP, bypass firewalls, avoid censorship or simply stream your favorite sports and entertainment, CNET’s expert-tested, hands-on VPN reviews will help you quickly choose the best service for you. 

A virtual private network routes your internet traffic through an encrypted tunnel to a secure VPN server in a location of your choosing — keeping your online activity private and giving you broader access to the internet. With a VPN connection, outside entities like your internet service provider, network administrators, government snoops and hackers are unable to see what you’re doing online. And because the VPN routes your encrypted traffic through a different location and hides your IP address in the process, you can bypass firewalls to access blocked sites on your network and unblock geographically restricted content online.

How we test VPNs

Comparing VPN services can be confusing, given the range of options and each one’s specialized features and pricing structures. Choosing the right VPN is especially important if you have heightened privacy needs. That’s why CNET has developed a detailed and rigorous VPN testing and reviews process to help quickly guide you toward a confident decision. All of the VPNs we review are meticulously evaluated on everything from features to usability, speeds, server network, privacy policy, ownership, jurisdiction, price and everything in between. 

We always incorporate hands-on testing, in addition to research and interviews with VPN representatives and experts, into our reviews. While it’s a lengthy process, we stand by the time we put into our VPN reviews because that allows us to deliver an authentic assessment that’s thorough, cuts through the hype and helps you make an informed choice. CNET’s team of VPN experts includes Attila Tomaschek and Moe Long. Together, we have nearly two decades’ worth of experience using, testing, researching and reviewing dozens of VPNs. We’ve run thousands of speed tests, written hundreds of articles and had expert commentary on VPNs and digital privacy published in various online publications.   

Our testing and reviews process takes into consideration the many reasons to use a VPN and focuses on what you need to know about the VPNs we review. All the VPNs we review are selected by our team of experts and are assessed with complete editorial independence. To be considered by our experts for a spot on CNET’s best VPNs list, a VPN must meet our criteria for privacy, speed and value. It must also be easy to use, unblock various streaming services and be available on most major platforms. To earn an Editors’ Choice designation, a VPN must consistently demonstrate exceptional capacity in one or more of those areas. 

How we test VPN speeds

Using a VPN will slow your internet speeds. This can be a result of various factors, but it’s mostly because the VPN encrypts your traffic and routes it through a different server location — both of which take time and contribute to the speed loss. Slow speeds can negatively affect things like streaming, gaming, video conferencing and even general browsing. VPN speeds are important for many activities, so one of the main things you’d want to know about a potential VPN service is how much you can expect it to slow you down.

VPN speed testing is an inexact science, but when we evaluate a VPN’s speeds, we do our best to replicate the conditions in which the typical VPN user would use their VPN. With the help of CNET’s Labs team (a dedicated team of data and product testing experts) in Louisville, Kentucky, we first measure our internet speeds on a residential Wi-Fi network without connecting to the VPN. This is to get a baseline reading for comparing the speeds we get through various VPN server locations. Though we report on the actual speed numbers we achieve with and without the VPN, we focus mainly on the percentage of speed lost through the VPN connection. For example, we’d report a VPN cutting our speeds by 17% rather than just saying we got 345.23 Mbps through the VPN’s German servers. Network speeds vary wildly, and the network(s) we’re testing on may be significantly faster or slower than the network you’re using. Framing speed performance in terms of percentage loss helps you translate the exact speeds to your internet connections. 

We use the popular speed testing platform developed by Ookla, a network testing firm based in Seattle, to run three rounds of speed tests using OpenVPN and another three on WireGuard (or the equivalent proprietary protocol, if applicable) on both Windows and MacOS. We run tests using OpenVPN and WireGuard because they are two of the most popular VPN protocols. We recommend OpenVPN for optimal privacy and WireGuard for optimal speeds. Each round of speed tests includes three tests without the VPN, then three tests each to six different server locations: New York, the United Kingdom, France, Germany, Singapore and Australia. Including the baseline internet speed tests, we run a minimum of 252 speed tests on each VPN we review. Once we’ve collected all of our speed test readings, we take the averages and calculate the average speed loss. 

In addition to reporting the average speed loss, we highlight which protocols, server locations and operating systems delivered the fastest speeds. We also call out any inconsistencies or anomalies we detect. For example, we’ll note in our review if we saw any major speed fluctuations from one test to another, or if we noticed significantly faster speeds to a distant location than we did to a location closer to our physical testing location(s). We’ll also recognize VPNs that deliver consistent speeds throughout the testing process.

Many VPNs can cut your speeds by 50% or more, but we look for VPNs that cut speeds by 20% or less — those providers can qualify for placement on our list of the fastest VPNs.

How we evaluate VPN pricing

When we evaluate a VPN’s cost, we look at the price of the service itself and the overall value it delivers through its features and performance. We take into consideration all of the subscription plan options available, but we generally don’t recommend signing on with a VPN provider for more than a year at a time. Things tend to shift rapidly in the VPN industry, and performance can fluctuate as VPN companies consolidate, server networks expand or contract, policies change and features come and go. We also check which payment options are available and whether the VPN offers a money-back guarantee or a free trial (or even a free version). And if a money-back guarantee includes any stipulations, we make sure to call it out. 

As with any product or service, the overall quality and performance of the VPN should be commensurate with the price you pay for it. This is why we consider every aspect of the VPN when evaluating its value, including its feature set, server network, number of simultaneous connections and ease of use. We check the VPN’s help section for tutorials and help articles and we actively engage with the customer support team to assess how responsive and effective it is. We test the VPN’s streaming capabilities through various server locations on several different streaming services, including Netflix, Disney Plus, Hulu and Amazon Prime Video. We test the service on all available platforms and operating systems to assess its compatibility and consistency across platforms. If applicable, we test any “quick connect” feature to ensure it works as intended and connects us to an optimal server. We also look out for and actively test any unique features that the VPN may offer.

To be considered for a spot on CNET’s list of the Best Cheap VPNs, the VPN must deliver a premium level of performance at a budget-friendly price.

How we evaluate VPN privacy

Because privacy is the often biggest selling point of a VPN, we pay special attention to and scrutinize everything about a VPN’s privacy posture. Evaluating a VPN’s privacy protections involves deep research, frequent communication with VPN representatives (whether via email, video conference or both) and meticulous hands-on testing. 

We carefully examine the VPN’s privacy policy, with special attention to things like logging practices and data collection and sharing. Even though VPN ownership is often murky territory, we carefully research the company’s corporate structure and history, making sure to flag anything potentially dubious. We want to make sure VPNs are as transparent as possible and that you know the entity behind the VPN you’re entrusting with your online privacy. If anything is unclear or if we uncover anything suspect in the VPN’s corporate structure or policies, we make sure to reach out to the company for clarification.

We look for things like third-party security audit reports and transparency reports — how many have been published and how often? We also look for any court cases the VPN company has been involved with — with special attention to cases where authorities have requested activity logs. The country from which the VPN operates is also an important consideration, so we research the VPN’s jurisdiction to see what privacy laws are in place and whether the country is part of the 14 Eyes data-sharing alliance. 

At a minimum, a VPN should have a no-logging policy and offer AES 256-bit encryption, DNS leak protection and a kill switch. We put each VPN’s kill switch feature through its paces to make sure it’s doing what it’s supposed to be doing and we run multiple DNS leak tests on various VPN servers using leak testing sites like ipleak.net. We also check to see whether the VPN still offers outdated and vulnerable VPN protocols like PPTP — and if so, we make sure to flag it because it can give users a false sense of security when using the VPN. 

We look for VPNs that offer a diskless, RAM-only server infrastructure as well as advanced privacy features like obfuscation, multihop connections, split-tunneling, ad/malware blocking and Tor over VPN. Whenever possible, we thoroughly test these features, along with any additional privacy features the VPN may offer that go above and beyond.

A VPN must offer excellent privacy protections, deliver stable connections and go beyond the call of duty in terms of transparency in order to earn its stripes as one of CNET’s top VPNs.

Leave a Reply