Let me state this clearly, again: data latency sucks. This is especially true when working on Firefox: a nicely crafted piece of software that ships worldwide to many people. When something affects the experience of our users we need to know and react fast.
The story so far…
We started improving the latency of the data coming from Firefox, in the previous quarters, and got to the point where the majority of pings reach our servers within 1 hour, instead of days (latest Beta only): there’s an extremely satisfying plot by :chutten about that!
However, this change does not help too much with the data latency of users who just installed Firefox (or created a new profile), don’t trigger a subsession split and usually suspend their computer instead of shutting Firefox down. Their first chunk of data would come either at their local midnight or after they wake their computer again. And this could take hours or days (on weekends).
Welcome, ‘new-profile’ ping!
In bug 1120370 we introduced a new ping type: ‘new-profile’. This one-time ping is sent from Firefox Desktop 30 minutes after the browser is started, on the first session of a newly created profile. If the first session of a newly-created profile was shorter than 30 minutes, it gets sent at shutdown using the pingsender. We don’t sent the ping immediately after Telemetry completes initialization to give our users enough time to tweak their data collection preferences.
How does the data look?
The data coming in from the Beta channel looks promising: we’re seeing signals from new profiles way earlier and the pings are sent at the expected times.
By looking at the initial Beta data we were able to see that most (~78%) of these new pings were sent at shutdown, implying that the first session lasted less than 30 minutes. It’s also interesting to notice that we’re receiving multiple new-profile pings from an handful (~0.81%) of profiles. This percentage is low enough to not be a problem, at least for now.
What now?
The new-profile ping is the latest addition to our Swiss army knife for dealing with data latency in Firefox. Having this new ping will also allow us to further investigate the bot profiles phenomenon: presumably scripted instances of Firefox that are run once and never again.
But we’re not done yet! When this new ping will hit the Release channel we will be able to discuss enabling the pingsender for the first browsing session as well (currently disabled, bug 1354482), improving the overall latency of our pings even more. And let’s not forget about the upcoming update ping too… exciting times!