Best Vacuum Cleaners of 2024 – CNET

How we test robot vacuums

Our method for evaluating robot vacuums is straightforward, yet grueling. There are two types of tests we run. The first trial is to figure out how well a robot covers the floor while it’s cleaning. We built an industry-standard testing room as specified by the International Electrotechnical Commission, just for this purpose. The IEC is an international standards body responsible for managing robot vacuum testing procedures, among other things, for vacuum manufacturers.

A robot vacuum cleans around table legs. A robot vacuum cleans around table legs.

Enlarge Image

A robot vacuum cleans around table legs. A robot vacuum cleans around table legs.

Obstacles in our test room mimic what robot vacuums run into in the real world.

Tyler Lizenby/CNET

Inside this room are objects designed to simulate typical obstacles a robot vac encounters for navigation as it cleans. These obstacles include wall edges, table and chair legs, couches and other furniture, and so on, plus bare tile and hardwood floors, as well as carpet. We mount LED lights to the top of each vacuum cleaner. The dimensions of the lights correspond to the measured nozzle width of each particular robot vacuum we test.

As robots move through the room while cleaning, a camera overhead captures a long-exposure image of the entire room in low light. That photo will then have a light trail, created by the LEDs, that shows the exact areas where the robot traveled (and its nozzle position) during its runtime. We can also see areas of the floor where the vacuum may have missed or gotten stuck. You can see the navigation results of all the robot vacuums in our test group in the gallery below.

The second type of test reveals exactly how much physical debris a vacuum is able to pick up off of the floor. To mimic dirt of small particle size, we use a mixture of play-sand and landscaping sand. For bigger particle soil, we use grains of uncooked black rice. Robots then run in straight line mode across three types of flooring (low-pile carpet, medium-pile carpet and hardwood bare floors).

Three flooring surfaces used in the vacuum tests. Three flooring surfaces used in the vacuum tests.

Enlarge Image

A tape measure being used to build the custom testing tools. A tape measure being used to build the custom testing tools.

Our custom-built tool lets us match soil area to a robot vacuum’s nozzle width.

Tyler Lizenby/CNET

We conduct three cleaning runs (at minimum) on each floor type. We also perform cleaning tests with sand and rice separately. That comes to at least 18 tests per robot vac. We weigh the robot’s dust bin both before and after each run. From there we can calculate the percentage of debris pickup for every cleaning run and the average amount of soil a machine manages to remove. Additionally we run anecdotal (visual) pet hair tests for each robot, on all three floor types.

A robot vacuum being placed on a hardwood mat for testing. A robot vacuum being placed on a hardwood mat for testing.

Enlarge Image

A before and after picture of our floor test. A before and after picture of our floor test.

We run tests in a straight line across all three floor types.

Brian Bennett/CNET

Just like in robot tests, the width of the test bed is proportional to the vacuum’s nozzle width. We measure this width ourselves. We also use nozzle width, plus the flooring type, to calculate the soil density for each test, per IEC guidelines. We use the same soil types here as well; sand, rice and pet hair. We perform three runs (at minimum) on each floor type. We also test suction power with sand and rice separately. That comes to at least 18 tests per vacuum. We weigh the vacuum’s dust bin both before and after each run.

From there we can calculate the percentage of dirt and debris pickup for every run and the average amount of soil a vacuum manages to remove. Additionally, we run anecdotal (visual) pet hair tests for each vacuum, on all three floor types to help us select the best cordless vacuum.

Leave a Reply