Transcript
Guy Podjarny
@guypod
http://www.guypo.com/
The Mobile Difference . In Numbers
Akamai Confidential
Guy Podjarny
@guypod
http://www.guypo.com/
The Mobile Difference . In Numbers
Mobile Isn’t Desktop
. It’s different in many ways…
. This presentation focuses on sizing the gap
. How big is the gap between Mobile & Non-Mobile?
. How much does it impact performance?
. We can’t cover everything, so we’ll focus on three topics:
. Mobile CPU != Desktop/Laptop CPU
. Cellular Networks != Broadband Networks
ⓒ2012 Akamai Faster ForwardTM
. Mobile Browsers != Desktop Browsers
ⓒ2012 Akamai Faster ForwardTM
Akamai Confidential
Mobile Computing Power
!=
Desktop/Laptop Computing Power
Akamai Confidential
Mobile Computing Power
!=
Desktop/Laptop Computing Power
Apple Laptop vs. Mobile, CPU/RAM
ⓒ2012 Akamai Faster ForwardTM
Fast Mobile Browsing is all the rage
ⓒ2012 Akamai Faster ForwardTM
Better CPU matters in Benchmarks
ⓒ2012 Akamai Faster ForwardTMhttp://www.anandtech.com/show/5810/samsung-galaxy-s-iii-performance-preview
Quad Core, ~1.5 Ghz
Dual Core, ~1 Ghz
Perf of iOS5 across different hardware
Sunspider\'JS\'Benchmark,\'iOS\'5\'Devices\'
iPhone\'4S,\'2x800\'Mhz\'
iPhone\'4,\'800\'Mhz\'
iPhone\'3GS,\'600Mhz\'
ⓒ2012 Akamai Faster ForwardTM
0\"500\"1000\"1500\"2000\"2500\"3000\"3500\"4000\"4500\"5000\"
4737\'
3545\'
2250\'
25%
Better
37%
Better
Test Time!
ⓒ2012 Akamai Faster ForwardTM
Test Time!
.
Question:
Does mobile computing power impact page load time?
.
Tested Sites: Alexa US Top 100
.
Test Tool: Mobitest
.
Details:
. Test iPhone 4, iPhone 4S and iOS Simulator on 2-yr old MacBook Pro
. Note: iOS Simulator uses the actual hardware
ⓒ2012 Akamai Faster ForwardTM
. Tested overnight
. High speed Cable connection, WiFi
Faster ForwardTM ⓒ2012 Akamai
Results: Hardware Load Time Impact
Measure
iPhone 4
vs.
iPhone 4S
iPhone 4S
vs.
iOS Simulator
Sunspider 37% 90%
Load Time 15% 48%
0\"
500\"
1000\"
1500\"
2000\"
2500\"
3000\"
3500\"
4000\"
iPhone\"4,\"800\"Mhz\" iPhone\"4S,\"2x800\"Mhz\" 24yr\"old\"MBP,\"2x2.7\"Ghz\"
Sunspider*&*Load*Time*Per*Device*
Sunspider\"
Load\"Time\"
CPU Difference - Conclusions
. Mobile CPU is still underpowered compared to desktop
. It indeed impacts web performance, even for regular sites
. Impact on load time is lighter than on benchmarks
. Roughly 25-50% of impact on benchmarks
. Computing power is more than just CPU
. At least GPU and RAM also have an impact
. Stick around for Ariya’s Presentation!
ⓒ2012 Akamai Faster ForwardTM
. Recommendation: Test on old devices too
Akamai Confidential
Cellular Network
!=
Broadband Network
Akamai Confidential
Cellular Network
!=
Broadband Network
Claim: Cellular Networks are SSSLLLOOOOWWWW
Low
Bandwidth
High Latency
High Packet
Loss
Volatility
Bandwidth
http://www.flickr.com/photos/pasukaru76/4228438307/
TM
Cellular Network Speed . Wikipedia
Protocol Max Downlink Max Uplink
GPRS 56-115 Kbps
CDMA200 (1X) 153.6 Kbps
Edge 236.8 Kbps
3G Bar 2 Mbps stationary client
384 Kbps moving client
EV-DO 2.5-4.9 Mbps 0.15-1.8 Mbps
EDGE Evolution 1.6 Mbps 0.5 Mbps
HSDPA 2 Mbps 200 Kbps
HSPA+ 14.4 Mbps
ⓒ2012 Akamai Faster ForwardTM
4G Bar 1 Gbps stationary client
100 Mbps moving client
HSPA+ Release 11 21-672 Mbps 5.8-168 Mbps
LTE 100-300 Mbps 50-75 Mbps
WiMax 37-365 Mbps 17-376 Mbps
You only get those speeds it if you’re this guy…
ⓒ2012 Akamai Faster ForwardTMhttp://www.flickr.com/photos/23081877@N02/2798378080/
PCWorld/Novarum
. 3G
. Download: 0.59 . 3.84 Mbps
. Upload: 0.56 . 1.44 Mbps
. 4G
. Download: 2.81 . 9.12 Mbps
. Upload: 0.97 . 5.86 Mbps
ⓒ2012 Akamai Faster ForwardTM
. Date: Apr, 2012
Source (PCWorld): http://bit.ly/JX7MQw
Mobile Network Speed, Akamai SOTI, Q4 2011
ⓒ2012 Akamai Faster ForwardTM
http://www.akamai.com/stateoftheinternet/
Average Broadband Speeds
ⓒ2012 Akamai Faster Forward http://www.akamai.com/stateoftheinternet/
http://www.netindex.com/download/
Cellular Network Speed . Conclusions
. Theoretical limits are NOT the reality
. Cellular bandwidth definitely less than broadband
. There is no single number for cellular network speed
. But we need SOME numbers for testing…
. So here’s a SWAG at numbers to use for testing
. Merging many data points, and rounding numbers for convenience
ⓒ2012 Akamai Faster ForwardTM
Test Case Download (Kbps) Upload (Kbps)
Poor Cell Network 400 250
Average Cell Network 1200 600
Latency
TM
Broadband Latency (FCC)
“Fiber-to-the-home services provided 17 ms round-trip
latency on average, while cable-based services averaged
28 ms, and DSL-based services averaged 44 ms.”
mai Fast
DSL 44ms
http://www.fcc.gov/measuring-broadband-america
Connection
Type RTT
Fiber 17ms
Cable 28ms
Ofcom study, UK Mobile Networks, May ‘11
“Latency… was an average of 117 milliseconds. Nearly two
Static Tests:
thirds of tests took more than 100 milliseconds”
“Average latency was 192 milliseconds and around a third
User Panel:
of tests delivered latency of higher than 200 milliseconds”
ⓒ2012 Akamai Faster ForwardTM
http://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf
Cisco Cloud Index, 2010-2015
“Global average fixed latency is 125 and average mobile
latency is 290… Global mobile consumer average latency is
307.3 ms”
ⓒ2012 Akamai Faster ForwardTM
Yep, Mobile Latency is different
Latency(by(Connec,on(Type(
. Chart showing latency on Mobile vs Broadband, Ofcom
350\"
300\"
250\"
200\"
150\"
100\"
50\"
ⓒ2012 Akamai Faster ForwardTM
numbers
0\"
Fiber\"Cable\"DSL\"Mobile\"Mobile\"Mobile\"Mobile\"
(Ofcom\"Tests)\"(Ofcom\"(Cisco)\"Consumer\"
Panel)\"(Cisco)\"
Cellular Network Latency - Conclusions
. Not a pretty sight…
. MUCH higher than broadband latency
. Measured averages around 100-300 ms
. Updated table for suggested testing numbers:
ⓒ2012 Akamai Faster ForwardTM
Test Case Download
(Kbps)
Upload
(Kbps)
Latency
(ms)
Poor Cell Network 500 250 300
Average Cell Network 1200 600 200
Test Time!
ⓒ2012 Akamai Faster ForwardTM
Test Time!
.
Question:
How much impact does the difference in Bandwidth & Latency
have on web performance?
.
Tested Sites: Alexa US Top 100
.
Test Tool: WebPageTest
.
Details:
. Increase Latency, Download & Upload Bandwidth separately
ⓒ2012 Akamai Faster ForwardTM
. Set no limits on other two variables
. Run each test 3 times, use median result
Download BW Impact on Load Time
ⓒ2012 Akamai Faster ForwardTM
DSL
Cable FIBER
Poor Cell Conn
Avg Cell Conn
Upload BW Impact on Load Time
ⓒ2012 Akamai Faster ForwardTM
DSL
Cable FIBER
Poor Cell Conn Avg Cell Conn
Latency Impact on Load Time
ⓒ2012 Akamai Faster ForwardTM
DSL
CableFIBER
Poor Cell Conn
Avg Cell Conn
Bandwidth/Latency Difference - Conclusions
. Upload difference doesn’t matter much
. Download difference DOES matter
. Latency difference matters the most
. Tip #1: Test using Cell Network speeds
. Know what you’re getting
. Tip #2: Reduce number of requests
ⓒ2012 Akamai Faster ForwardTM
. Especially blocking requests, e.g scripts, CSS
. Tip #3: Reduce number of bytes
Packet Loss
Packet Loss . Broadband vs Mobile
Faste
Broadband
Mobile
Average Packet Loss
0.017%
0.6%
Packet Loss, Akamai/MIT Research
. Tested 149 Million connections in 51 countries
. Compared PL for mobile networks from non-mobile networks
. Most connections have no packet loss (Median PL is 0%).
. 75th percentile usually zero too
. Average connection packet loss: 2.5%
. Fixed network average is 2.2%
. Implies connections are usually fine, but sometimes terrible
. The average PL for the non-zero half is 5%
. Suggests “small” connections
Faster ForwardTM
have higher Packet Loss
. HTTP Connections are often small
http://pam2012.ftw.at/papers/PAM2012paper6.pdf
Packet Loss - Conclusions
. Packet Loss is significantly higher in Mobile, But…
. It’s small (even in Mobile)
. It usually doesn’t happen (even in Mobile)
. Smaller connections lose more packets
. Recommendations:
. Don’t worry about Packet Loss too much…
. Include some Packet Loss in your tests
. Ideal: Lots of caching, good user-feedback on error and no SPOFs
ⓒ2012 Akamai Faster ForwardTM
SWAG Test Case Download
(Kbps)
Upload
(Kbps)
Latency
(ms)
Packet
Loss
Poor Cell Network 500 250 300 1%
Average Cell Network 1200 600 200 0%
Apple Network Link Conditioner
ⓒ2012 Akamai Faster ForwardTM
850/420 Kbps
190ms RTT 780/330 Kbps
200ms RTT
1% Packet Loss
780/330 Kbps
200ms RTT
Network Link Conditioner in iOS 6? (via @jedisct1)
ⓒ2012 Akamai Faster ForwardTM
Volatility
TM
Different protocols lead to different speeds
ⓒ2012 Akamai Faster ForwardTMhttp://www.ericsson.com/res/docs/2012/traffic_and_market_report_june_2012.pdf
Signal Strength affects download speed
ⓒ2012 Akamai Faster ForwardTMhttp://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf
Different Carriers have different average speeds
ⓒ2012 Akamai Faster ForwardTM
Source: Ofcom
http://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf Source (PCWorld): http://bit.ly/JX7MQw
Volatility - Conclusions
. Stop Guessing!
. You can’t predict your user’s speeds, today or in the future
. Little value in doing something “only on cellular”
. Strategy #1: Aim at the lowest common denominator
. Design for a usable experience in a slow network
. Enhance the page when possible
. Strategy #2: Adapt to Network Conditions
. E.g. reduce image quality when conditions are bad
ⓒ2012 Akamai Faster ForwardTM
. Not easy to do, but doable
. Check your options with your Server/Host/CDN
Cellular Networks - Summary
. Cellular Networks are kinda crappy
. Can’t live with ‘em, can’t live without ‘em…
. You can assume latency will be high
. And download won’t be too awesome
. Don’t worry about Packet Loss & Upload rate
. You have bigger things to worry about
. Make Peace with it . you can’t predict cellular network speed
. Best bet is to aim low, and adapt to network conditions
ⓒ2012 Akamai Faster ForwardTM
SWAG Test Case Download
(Kbps)
Upload
(Kbps)
Latency
(ms)
Packet
Loss
Poor Cell Network 500 250 300 1%
Average Cell Network 1200 600 200 0%
Akamai Confidential
Mobile Browser
!=
Desktop Browser
Akamai Confidential
Mobile Browser
!=
Desktop Browser
Actually, they’re not that different…
. Mobile & Desktop browsers are actually similar
. Many share the same underlying engines (e.g. WebKit)
. Some share the browser itself (e.g. Safari, Chrome, Firefox)
. How are they different, then?
. HTTP Pipelining
. Connection Models
. Cache Sizes
. Fragmentation
ⓒ2012 Akamai Faster ForwardTM
HTTP Pipelining
ⓒ2012 Akamai Faster ForwardTM
Mobile Software: Pipelining
. HTTP Pipelining is around since HTTP 1.1
. Send multiple requests on connection before receiving response
. Most useful in high latency environment
ⓒ2012 Akamai Faster ForwardTM
.Opera Only (Default)
.Chrome, Firefox (Opt-in)
49
. Big in Mobile
. All big mobile browsers
.iOS 5+, Android, Opera
. Hardly used on Desktop
Network Capture
. Samsung Galaxy S
. Max Conn: 12
. Conn Per Host: 12
. Max Piped Reqs: 6
. Max “In-Flight” Reqs: 72 (!!!)
. Full Details:
Faster Fo
http://www.guypo.com/technical/http-p
request-distribution-algorithms/
50
Test Time!
ⓒ2012 Akamai Faster ForwardTM
Test Time!
.
Question:
Does HTTP Pipelining help make websites faster?
.
Tested Sites: Alexa US Top 500
.
Test Tool: WebPageTest
.
Details:
. Use Firefox agent, which supports pipelining
. Turn pipelining on and off
ⓒ2012 Akamai Faster ForwardTM
. Set bandwidth to DSL, Cable & FIOS
. Increase Latency to see if impact changes
Faster ForwardTM ⓒ2012 Akamai
Test Results . Pipelining has little effect…
0\"
2,000\"
4,000\"
6,000\"
8,000\"
10,000\"
12,000\"
14,000\"
50\" 100\" 200\" 300\"
Load%Time,%DSL%Bandwidth% Pipelining\"Off\"
Pipelining\"On\"
0\"
2,000\"
4,000\"
6,000\"
8,000\"
10,000\"
50\" 100\"
Load%Time,%Cable%Bandwidth% Pipelining\"Off\"
Pipelining\"On\"
0\"
2,000\"
4,000\"
6,000\"
8,000\"
10,000\"
12,000\"
50\" 100\" 200\" 300\"
Load%Time,%FIOS%Bandwidth% Pipelining\"Off\"
Pipelining\"On\"
Why doesn’t pipelining have an effect?
. Are you sure pipelining was used?
. Yes…
confirmed via tcpdump and various other metrics
. I don’t know for sure, but I can guess…
. 18 domains referenced on a page (average)
. 9 domains used only for one request (average)
. 3-4 domains used only for two requests (average)
. Browsers prefer new connection to piping a request
. Other bottlenecks…
. Saw similar results for SPDY in recent test
ⓒ2012 Akamai Faster ForwardTM
. Details: http://bit.ly/MptaBV
. Conclusion: Not much you can/should do…
Akamai Confidential
Summary
Akamai Confidential
Summary
Summary
. Those were A LOT of numbers…
. Mobile is truly different, and it matters for web performance
. Who saw that coming?!
. Main differences for Web Performance:
. CPU delta, but its effect is not linear
. Network delta in Latency & Download BW, not upload/Packet-Loss
. Main takeaways for web developers:
. Test performance on old devices too
. Test with at least average and poor cell network performance
. Provide a decent experience in the worst of conditions
ⓒ2012 Akamai Faster ForwardTM
. Remember - we only looked at a handful of aspects…
Akamai Confidential
Guy Podjarny
@guypod
http://www.guypo.com/
The Mobile Difference . In Numbers
Thank You!
Questions?
Akamai Confidential
Guy Podjarny
@guypod
http://www.guypo.com/
The Mobile Difference . In Numbers
Thank You!
Questions?