Tested: iPhone 4 (iOS 4.2.1 vs. 4.3) and Android phones

The best way to test battery life is to get a robot to do it. In lieu of a robot, I drafted a computer for the task. It’s just an immobile robot anyway, isn’t it?

(Credit:
Eric Franklin/CNET)

Earlier this week, we posted some very interesting tablet testing results. Today, it’s smartphones.

Over the last few weeks we’ve tested a number of smartphones in a multitude of ways. If you’re looking for talk time tests, however, check the full review of each phone. The tests detailed here will focus on real-world speed as well as audio and video battery life.

How we test smartphones

In the CNET Labs, we currently run four different tests to evaluate the performance of smartphones.

Battery life
We evaluate battery life by continually running a movie file on the smartphone until its battery dies. We do the same with audio using a full album set on repeat all.

For audio battery, each phone runs with its default mobile network on; for video battery, the phones are run in Airplane mode.

During the movie battery-drain, we set each
tablet to Airplane mode and adjust its respective brightness to 140 candelas per square meter (cd/M2) or as close to that number as is possible.

For the
iPhone, we run the iPhone version of “Toy Story 3.” For Android phones, a 720p version of “Toy Story 3” is run. The reason we chose 720p for Android was that most Android phones are capable of running 720p video and we wanted the best quality video represented.

On iPhone, we ran the movie through its
iPod app; for Android, we used the movie player, mVideoPlayer, as it provides a much-needed repeat video function that not all native Android movie players include.

Site-loading speed
We used GiantBomb.com as our Web site of choice; it doesn’t use Flash or have many dynamic elements. Each tablet was connected to the same closed network (not its mobile network) with no other devices on it, with the router about 5 feet away. The test began the moment we pressed Enter, with the end of the test signified by the disappearance of the browser’s progress bar. We measured speed in seconds, with a lower number indicating faster performance.

We used the latest version of each respective phone’s OS. Obviously, for the iPhone 4, we used both version 4.2.1 and 4.3 of iOS.

Boot time

Rebooting your phone can sometimes solve issues you might be having with apps not functioning correctly, so it’s good to know how long it take to get back to action. The test begins when we press the power button and ends when we see the lock screen on iPhone and after the internal storage has been set up on Android phones.

Analysis

We definitely saw some interesting results here. In our tests, the Verizon iPhone 4 had much longer audio battery life than its ATT counterpart. This likely has a lot to do with the Verizon iPhone 4 having an easier time maintaining a mobile network signal than the AtT version does. It’s likely, then, if you live in an area like San Francisco where ATT coverage isn’t the best, you may see longer battery life with a Verizon iPhone 4 than with AtT’s.

Also, apparently iOS 4.3 makes a definite difference in Web speed and it’ll be interesting to see what effect the OS’ improved Safari speed has on the iPad, since the Xoom beat it in our previous tests.

As for Android phones, the Nexus S has the longest video battery life, but the Atrix is–by a slim margin–the fastest at loading Web sites.

Overall, although iOS 4.3 speeds up the iPhone 4, it’s still not quite as fast as the Atrix or Nexus S. What that means to you will depend on what effect losing 1 to 2 seconds while surfing has on your schedule.

We’ll be updating this post with new results as we receive them, so keep an eye out.