Getting Firefox data faster: the shutdown pingsender

The data our Firefox users share with us is the key to identify and fix performance issues that lead to a poor browsing experience. Collecting it is not enough if we don’t manage to receive the data in an acceptable time-frame. My esteemed colleague Chris already wrote about this a couple of times: data latency sucks. But we can fix that.

Why is there latency, anyway?

The bulk of measurements we collect (histograms, scalars, events, …) are sent through the main-ping. This ping is generated at different times during the browsing session, including shutdown. The “shutdown” main-ping, which accounts for about ~80% of all the pings we receive, once generated, is not sent to our servers until the next Firefox restart. Depending on the user habits and the day of the week, this could be anything between a few minutes to a few days (see the CDF plot below): way too much! One of my team’s goal for this year is to reduce this latency, allowing developers to take decisions and iterate quickly.

… 

 

How We Built It: The First Ever Firefox Hardware Report

By Alessio Placitelli, Ali Almossawi, and Rebecca Weiss
Cross-posted from Medium

We have just released the Firefox Hardware Report, a report of the hardware used by the Firefox release user base. You can read the announcement here. The Firefox team believes that this report will be very valuable to developers, particularly those who build for the web. Web developers need to know what platforms and hardware are being used to inform their decisions when they are building and upgrading applications.

As you may know, Firefox is built not just by the paid contributors of Mozilla but by an amazing community of volunteer contributors. When it comes to data, we believe that our users are also contributing by providing data to us about their client hardware and activity. The Firefox Hardware Report is a way to demonstrate the value of the data that our users have provided.

This article will describe how the Firefox Hardware Report works, from the manner in which the device-level data is measured to the process by which the data is prepped and processed to produce this report.

… 

 

Build Firefox for Android on Ubuntu and test it on Windows!

Sadly, right now (and until bug 1169873 is fixed), it’s not possible to build Firefox for Android on Windows. That’s not nice, especially if you need to track down some Android-only failures triggered by your code 🙂

Until recently I was able to run my Android Virtual Device within my Ubuntu Virtual Machine (yeah, tricky). Then something broke in OracleVM (ticket 12941), preventing me from starting the Android emulator.

Here comes the good news: there’s a way to spawn an Android emulator on Windows, build Firefox for Android on a VM and then run it. Yay!

Let’s go straight to the action: the HOST prefix is used for actions performed on the Windows system, and GUEST for actions on the Linux VM.

  • HOST – Install the Android SDK (no studio) and the Android Platform Tools.
  • HOST – Set the ANDROID_HOME environment variable to the installation path of the Android SDK.
  • HOST – Set ANDROID_SDK_ROOT to %ANDROID_HOME%
  • HOST – Add %ANDROID_HOME%\tools;%ANDROID_HOME%\platform-tools at the end of your PATH environment variable.
  • HOST – Check that adb.exe can be run from a command prompt.
  • HOST – Install an SSH daemon. OpenSSH over Cygwin is a nice choice, there’s even an installation guide on that 🙂
  • HOST – Run ./mach android-emulator –verbose –wait from the Mozilla source directory. This command will automagically fetch the AVD and run it when done.
  • HOST – Run the adb devices command. The emulator should be shown there.
  • GUEST – Try to telnet into the emulator, to see if everything is running correctly: telnet <HOST IP> 5037 (10.0.2.2 in my case, but that depends on your HOST-GUEST network configuration).
  • GUEST – Run adb kill-server to stop any server running on the guest.
  • GUEST – Install autossh, which monitors and re-establishes the SSH connection when it fails: sudo apt-get install autossh
  • GUEST – Now the fun part. We forward the local 5037 port to the 5037 port on the Windows host. The adb daemon is listening on that port. autossh -nNL5037:localhost:5037 -oExitOnForwardFailure=yes -o PreferredAuthentications=password -o PubkeyAuthentication=no <HostUsername>@<HOST IP>
  • GUEST – Running adb device on the guest machine will show the emulator running on the Windows host.
  • GUEST – Build and run Firefox for Android! ./mach build && ./mach package && ./mach install && ./mach run

Yes, this also works with ./mach test and all its happy friends!

Last, but not least, thank you to Stephen Niedzielski for your great answer on SO.