The Battle of Session Restore – Season 1 Episode 3 – All With Measure
July 17, 2014 § 4 Comments
Plot For the second time, our heroes prepared for battle. The startup of Firefox was too slow and Session Restore was one of the battle fields.
When Firefox starts, Session Restore is in charge of restoring the browser to its previous state, in case of a crash, a restart, or for the users who have configured Firefox to resume from its previous state. This entails numerous activities during startup:
- read sessionstore.js from disk, decode it and parse it (recall that the file is potentially several Mb large), handling errors;
- backup sessionstore.js in case of startup crash.
- create windows, tabs, frames;
- populate history, scroll position, forms, session cookies, session storage, etc.
It is common wisdom that Session Restore must have a large impact on Firefox startup. But before we could minimize this impact, we needed to measure it.
Benchmarking is not easy
When we first set foot on Session Restore territory, the contribution of that module to startup duration was uncharted. This was unsurprising, as this aspect of the Firefox performance effort was still quite young. To this day, we have not finished chartering startup or even Session Restore’s startup.
So how do we measure the impact of Session Restore on startup?
A first tool we use is Timeline Events, which let us determine how long it takes to reach a specific point of startup. Session Restore has had events `
sessionRestoreInitialized` and `
sessionRestored` for years. Unfortunately, these events did not tell us much about Session Restore itself.
The first serious attempt at measuring the impact of Session Restore on startup Performance was actually not due to the Performance team but rather to the metrics team. Indeed, data obtained through Firefox Health Report participants indicated that something wrong had happened.
d2` in the graph measures the duration between `
firstPaint` (which is the instant at which we start displaying content in our windows) and `
sessionRestored` (which is the instant at which we are satisfied that Session Restore has opened its first tab). While this measure is imperfect, the dip was worrying – indeed, it represented startups that lasted several seconds longer than usual.
Upon further investigation, we concluded that the performance regression was indeed due to Session Restore. While we had not planned to start optimizing the startup component of Session Restore, this battle was forced upon us. We had to recover from that regression and we had to start monitoring startup much better.
A second tool is Telemetry Histograms for measuring duration of individual operations, such as reading sessionstore.js or parsing it. We progressively added measures for most of the operations of Session Restore. While these measures are quite helpful, they are also unfortunately very unstable in real-world conditions, as they are affected both by scheduling (the operations are asynchronous), by the work load of the machine, by the actual contents of sessionstore.js, etc.
The following graph displays the average duration of reading and decoding sessionstore.js among Telemetry participants:
Difference in colors represent successive versions of Firefox. As we can see, this graph is quite noisy, certainly due to the factors mentioned above (the spikes don’t correspond to any meaningful change in Firefox or Session Restore). Also, we can see a considerable increase in the duration of the read operation. This was quite surprising for us, given that this increase corresponds to the introduction of a much faster, off the main thread, reading and decoding primitive. At the time, we were stymied by this change, which did not correspond to our experience. We have now concluded that by changing the asynchronous operation used to read the file, we have simply changed the scheduling, which makes the operation appear longer, while in practice it simply does not block the rest of the startup from taking place on another thread.
One major tool was missing for our arsenal: a stable benchmark, always executed on the same machine, with the same contents of sessionstore.js, and that would let us determine more exactly (almost daily, actually) the impact of our patches upon Session Restore:
This test, based on our Talos benchmark suite, has proved both to be very stable, and to react quickly to patches that affected its performance. It measures the duration between the instant at which we start initializing Session Restore (a new event `
sessionRestoreInit`) and the instant at which we start displaying the results (event `
With these measures at hand, we are now in a much better position to detect performance regressions (or improvements) to Session Restore startup, and to start actually working on optimizing it – we are now preparing to using this suite to experiment with “what if” situations to determine which levers would be most useful for such an optimization work.
Evolution of startup duration
Our first benchmark measures the time elapsed between start and stop of Session Restore if the user has requested all windows to be reopened automatically
As we can see, the performance on Linux 32 bits, Windows XP and Mac OS 10.6 is rather decreasing, while the performance on Linux 64 bits, Windows 7 and 8 and MacOS 10.8 is improving. Since the algorithm used by Session Restore upon startup is exactly the same for all platforms, and since “modern” platforms are speeding up while “old” platforms are slowing down, this suggests that the performance changes are not due to changes inside Session Restore. The origin of these changes is unclear. I suspect the influence of newer versions of the compilers or some of the external libraries we use, or perhaps new and improved (for some platforms) gfx.
Still, seeing the modern platforms speed up is good news. As of Firefox 31, any change we make that causes a slowdown of Session Restore will cause an immediate alert so that we can react immediately.
Our second benchmark measures the time elapsed if the user does not wish windows to be reopened automatically. We still need to read and parse sessionstore.js to find whether it is valid, so as to decide whether we can show the “Restore” button on about:home.
We see peaks in Firefox 27 and Firefox 28, as well as a slight decrease of performance on Windows XP and Linux. Again, in the future, we will be able to react better to such regressions.
The influence of factors upon startup
With the help of our benchmarks, we were able to run “what if” scenarios to find out which of the data manipulated by Session Restore contributed to startup duration. We did this in a setting in which we restore windows:
and in a setting in which we do not:
Interestingly, increasing the size of sessionstore.js has apparently no influence on startup duration. Therefore, we do not need to optimize reading and parsing sessionstore.js. Similarly, optimizing history, cookies or form data would not gain us anything.
The single largest most expensive piece of data is the set of open windows – interestingly, this is the case even when we do not restore windows. More precisely, any optimization should target, by order of priority:
- the cost of opening/restoring windows;
- the cost of opening/restoring tabs;
- the cost of dealing with windows data, even when we do not restore them.
Now that we have information on which parts of Session Restore startup need to be optimized, the next step is to actually optimize them. Stay tuned!
any bugs filed on the last three points ?
Bug 1034036 is about point 1. The other two have not been filed yet.
[…] 24 juin : Vient d’embarquer dans Firefox Nightly 33, un nouveau composant de restauration de session, par David Teller ; RÉSOLU RÉGLÉ bogue 883 609.La bataille de la restauration de session – Saison 1 Épisode 3 – Tout en mesure […]
This is my average Firefox 30 setup on Kubuntu amd64 :
First window : 158 tabs ( ten of them pinned )
Second window : 22 tabs ( four of them pinned )
Third window : 136 tabs
sessionstore.js –> 766.7 KiB
Bookmarks : 280
Addons : 49 ( two of them inactive )
Plugins : 11 ( two of them inactive )
I have set the browser to show my windows and tabs from previous sesion at start … and also … Don’t load tabs until selected.
Health report says my average start-up time is around 10 seconds … … I have never measured it, but I would say it actually takes maybe a minute to ‘get ready’ … … … and half a minute to shut down …
Update : 101 seconds to get ready ( with active internet connection ) .
81 seconds to get ready ( with NO internet connection ) curiously Firefox uses 800MiB with 0 tabs loaded.
29 seconds to shutdown.
Either way I love Firefox. I really appretiate your team’s work …