“Hunch” Study : Network Calls and their Effect on Mobile Apps

Every good mobile engineer has questions that they wish they could answer, if they only had the time. One of those questions, related to concurrent network calls, goes something like: “How many calls can my app do and not impact app performance?” Until recently, no one had actually measured to figure it out — then we did!

Hop in the time machine with us and we’ll explain how it happened . . .

Timehop is a nostalgic social networking app with tens of millions of downloads on iOS and Android. We worked with the Timehop team to understand how technical choices impact users’ experiences within their iOS app, and identified suboptimal areas to focus and improve upon. For purposes of this calculation, we focused on 3 days in October during which we collected data globally from over 7.5 million experiences, spread across over 2.5 million devices.

Let’s start with the common culprits.  SDKs are often the primary hunch for the cause of suboptimal experiences on mobile.  Knowing the hunch, most app developers still initialize *all* their SDKs on startup (somewhat of a standard practice).  Like all apps, Timehop embeds a lot of SDKs; 47 to be exact, ~16 from 3rd parties vendors.  (Please note that SDKs are not limited to 3rd parties, like analytics, ads, Google, FB, Twitter and the like, but are also many of the libraries that we use out of the box, like Bolts, Appirater, and SocketRocket). Starting the app with this many SDKs obviously impacts the quantity of network calls that occur during start-up.  

When initializing the app with all of these SDKs *plus* your own 1st party network calls, you’ll get a logjam, and many of these calls (3rd or 1st party) will block your app from progressing. In Timehop’s case, we focused on startup time and a very slow benchmark of 5 seconds.  We figured-out that the number of slow startups increased to 6% with ~6 concurrent network calls.  The number increased to 13% of startups with 10+ concurrent network calls.  Getting a bit more granular, we helped Timehop sort out that a user with any network call of over 2.5 seconds during startup correlated to an increase of +27% chance of that user leaving the app.

Further, we found that 3.3% of users were experiencing a slow startup (over 5 seconds), and 2.9% had a stall (e.g. what amounted to a freeze from the user’s perspective) at startup. We were also able to pinpoint that Mixpanel and StatHat were the primary sources of HTTP errors during startup. Timehop has been able to take this info, tweak their startup, and reduce user churn as a result.

The moral of the story? With a little time, good technology performance becomes good business.

You can’t change how many, or how fast, network calls occur, BUT you can control how well you have built your app. Embrace.io cures app blindness.  We make your app’s performance visible for each user and segment, leading to better user experiences, and improved business outcomes.

We are here to help. Let us know how we can for you.

–    Eric

 

There’s more to life than just crashes!

Mobile teams are singularly focused on crashes. Understandably so — they are the simplest to identify and diagnose. BUT crashes account for less than 1% of your issues! You still find errors and your users are still complaining.

With mobile apps, we are conditioned to think of all performance issues as ‘crashes’. That’s because a) there’s a tendency for users and business people alike to rush to the term ‘crash’ when it may not really apply, and b) there are limited tools on-par with their desktop counterparts for identifying and diagnosing mobile app performance issues.

So, even though the performance issue may deal with an inhibited experience (the app is taking so long time to load that it appears frozen), it’s treated like a crash. Ask yourself:
When you fix a crash, how often does it really impact the experience of any user?
How often is it in the background?
How often does it affect a small number of users who don’t contribute to the app or app community?
How often does a user just re-open the app anyway?

The truth is, most crashes are a fender-bender, but we treat them like 50 car pile-ups.

Instead of solving every issue, shouldn’t we be focused on solving the most important ones? If you are constantly distracted with minor performance issues that affect just a few low-value users, then you’re not spending time improving experience and heading off a serious error that could kill-off lots of high-value users.

The logic is the same whether you’re an engineer or a business person: focus on what matters and de-emphasize what doesn’t. This brings us right back to the need for mobile-first tools purpose-built to help engineers and their mobile teams quickly assess and resolve performance issues.

One of the unique things that Embrace does is let engineers stitch together user sessions. So, if the user experienced a crash, but then just immediately re-opened the app and all was fine, maybe it’s not worth solving that problem right now. We surface that data and visualize it so the engineer, product manager, or other member of the team can easily dig in and decide the appropriate course of action.

This ensures the most mission critical performance issues get the attention they deserve, relieves pain for engineers, and keeps users happy. And that’s what it’s all about.