The Battle of Session Restore – Season 1 Episode 3 – All With Measure

July 17, 2014 § 2 Comments

Plot For the second time, our heroes prepared for battle. The startup of Firefox was too slow and Session Restore was one of the battle fields.

When Firefox starts, Session Restore is in charge of restoring the browser to its previous state, in case of a crash, a restart, or for the users who have configured Firefox to resume from its previous state. This entails numerous activities during startup:

  1. read sessionstore.js from disk, decode it and parse it (recall that the file is potentially several Mb large), handling errors;
  2. backup sessionstore.js in case of startup crash.
  3. create windows, tabs, frames;
  4. populate history, scroll position, forms, session cookies, session storage, etc.

It is common wisdom that Session Restore must have a large impact on Firefox startup. But before we could minimize this impact, we needed to measure it.

Benchmarking is not easy

When we first set foot on Session Restore territory, the contribution of that module to startup duration was uncharted. This was unsurprising, as this aspect of the Firefox performance effort was still quite young. To this day, we have not finished chartering startup or even Session Restore’s startup.

So how do we measure the impact of Session Restore on startup?

A first tool we use is Timeline Events, which let us determine how long it takes to reach a specific point of startup. Session Restore has had events `sessionRestoreInitialized` and `sessionRestored` for years. Unfortunately, these events did not tell us much about Session Restore itself.

The first serious attempt at measuring the impact of Session Restore on startup Performance was actually not due to the Performance team but rather to the metrics team. Indeed, data obtained through Firefox Health Report participants indicated that something wrong had happened.

Oops, something is going wrong

Indicator `d2` in the graph measures the duration between `firstPaint` (which is the instant at which we start displaying content in our windows) and `sessionRestored` (which is the instant at which we are satisfied that Session Restore has opened its first tab). While this measure is imperfect, the dip was worrying – indeed, it represented startups that lasted several seconds longer than usual.

Upon further investigation, we concluded that the performance regression was indeed due to Session Restore. While we had not planned to start optimizing the startup component of Session Restore, this battle was forced upon us. We had to recover from that regression and we had to start monitoring startup much better.

A second tool is Telemetry Histograms for measuring duration of individual operations, such as reading sessionstore.js or parsing it. We progressively added measures for most of the operations of Session Restore. While these measures are quite helpful, they are also unfortunately very unstable in real-world conditions, as they are affected both by scheduling (the operations are asynchronous), by the work load of the machine, by the actual contents of sessionstore.js, etc.

The following graph displays the average duration of reading and decoding sessionstore.js among Telemetry participants: Telemetry 4

Difference in colors represent successive versions of Firefox. As we can see, this graph is quite noisy, certainly due to the factors mentioned above (the spikes don’t correspond to any meaningful change in Firefox or Session Restore). Also, we can see a considerable increase in the duration of the read operation. This was quite surprising for us, given that this increase corresponds to the introduction of a much faster, off the main thread, reading and decoding primitive. At the time, we were stymied by this change, which did not correspond to our experience. We have now concluded that by changing the asynchronous operation used to read the file, we have simply changed the scheduling, which makes the operation appear longer, while in practice it simply does not block the rest of the startup from taking place on another thread.

One major tool was missing for our arsenal: a stable benchmark, always executed on the same machine, with the same contents of sessionstore.js, and that would let us determine more exactly (almost daily, actually) the impact of our patches upon Session Restore:Session Restore Talos

This test, based on our Talos benchmark suite, has proved both to be very stable, and to react quickly to patches that affected its performance. It measures the duration between the instant at which we start initializing Session Restore (a new event `sessionRestoreInit`) and the instant at which we start displaying the results (event `sessionRestored`).

With these measures at hand, we are now in a much better position to detect performance regressions (or improvements) to Session Restore startup, and to start actually working on optimizing it – we are now preparing to using this suite to experiment with “what if” situations to determine which levers would be most useful for such an optimization work.

Evolution of startup duration

Our first benchmark measures the time elapsed between start and stop of Session Restore if the user has requested all windows to be reopened automatically

restoreAs we can see, the performance on Linux 32 bits, Windows XP and Mac OS 10.6 is rather decreasing, while the performance on Linux 64 bits, Windows 7 and 8 and MacOS 10.8 is improving. Since the algorithm used by Session Restore upon startup is exactly the same for all platforms, and since “modern” platforms are speeding up while “old” platforms are slowing down, this suggests that the performance changes are not due to changes inside Session Restore. The origin of these changes is unclear. I suspect the influence of newer versions of the compilers or some of the external libraries we use, or perhaps new and improved (for some platforms) gfx.

Still, seeing the modern platforms speed up is good news. As of Firefox 31, any change we make that causes a slowdown of Session Restore will cause an immediate alert so that we can react immediately.

Our second benchmark measures the time elapsed if the user does not wish windows to be reopened automatically. We still need to read and parse sessionstore.js to find whether it is valid, so as to decide whether we can show the “Restore” button on about:home.

norestoreWe see peaks in Firefox 27 and Firefox 28, as well as a slight decrease of performance on Windows XP and Linux. Again, in the future, we will be able to react better to such regressions.

The influence of factors upon startup

With the help of our benchmarks, we were able to run “what if” scenarios to find out which of the data manipulated by Session Restore contributed to startup duration. We did this in a setting in which we restore windows:size-restore

and in a setting in which we do not:

size-norestore

Interestingly, increasing the size of sessionstore.js has apparently no influence on startup duration. Therefore, we do not need to optimize reading and parsing sessionstore.js. Similarly, optimizing history, cookies or form data would not gain us anything.

The single largest most expensive piece of data is the set of open windows – interestingly, this is the case even when we do not restore windows. More precisely, any optimization should target, by order of priority:

  1. the cost of opening/restoring windows;
  2. the cost of opening/restoring tabs;
  3. the cost of dealing with windows data, even when we do not restore them.

What’s next?

Now that we have information on which parts of Session Restore startup need to be optimized, the next step is to actually optimize them. Stay tuned!

How to be an evil start-up CEO

April 1, 2014 § 3 Comments

Being the CEO of a start-up is fun. Being evil and mischievous is fun. Completely destroying one’s life dream is fun. However, reaching expertise in all three requires considerable subtlety. Here are a few notes for the day I decide to become an evil mischievous start-up CEO.

  1. I will keep in mind that my main currencies are time and credibility, both inside and outside my startup. Therefore, I will make my best to maintain that credibility and save that time.
  2. For this reason, although I pay them, I will describe my employees as trusted colleagues. I will, however, treat them as incompetent children.
  3. Conversely, to ensure credibility, I will encourage my trusted colleagues to worship me.
  4. For some reason, my relationship with trusted colleagues tends to alter when trusted colleagues realize that I lie to them. Which is why I will use threats and dissimulation to ensure that they do not.
  5. Being worthy of worship, I am the sole holder of the truth. Consequently, everything I have just told my investors or prospects is true. Trusted colleagues who fail to base their reality upon my truth will be punished.
  6. I will have trusted advisors, be they COO, CTO, CSO, CFO, GPU, tech leads, mentors, janitors, nannies or anything else. Listening to them is important. However, I know better, so there is no need to take anything they say into account.
  7. Being a CEO is all about taking decisions quickly. For this reason, I will avoid smoking pot or drinking alcohol. I will remain on coke.
  8. One of the roles of my trusted advisors is to help me differentiate the real world from my imagination. Do they wonder aloud whether my Reality Adjustment Factor is misaligned? Well, that is the sign that I should put them on coke, too.
  9. I will realize that some of my trusted advisors might be polite. Therefore, if one of them asks “er… are you really, really sure?”, I will take this as a hint that they may be politely inquiring about my being high on LSD. Since I am actually high on coke, there is nothing to worry about.
  10. If I have to divide my start-up in teams, I can ensure that teams can work in complement of each other. Of course, I can also ensure that teams will be at each other’s throat, which is much more amusing, especially if I live in a country where pitbull fighting is illegal. If I do organize my start-up as a pitbull fighting ring, this will, of course, open the possibility of taking bets to determine which team goes down first.
  11. Nothing motivates trusted colleagues quite as much as calling their colleagues “stupid”, “lazy” or “incompetent”, except perhaps calling them “stupid”, “lazy” or “incompetent” within earshot of said colleagues.
  12. Also, nothing motivates trusted colleagues quite as much as non-existent deadlines for imaginary clients on fabulous contracts. My trusted colleagues will be permitted to thank me for such managerial prowesses.
  13. I will claim that Microsoft, or Google, or Mozilla, or Apple, or Amazon, or Facebook, are a bunch of incompetent morons. They merely got to #1 in their respective sectors while I intend to do nothing less than revolutionize the world!
  14. In the spirit of encouraging team work, I will occasionally let a trusted colleague put his/her name along with mine as a co-author/inventor/creator of the work they have done. This way, in case of problem, it will be easy to find someone to take the blame.
  15. I could organize my company so that decisions are taken at the appropriate level by my trusted colleagues. However, this is clearly inefficient, so all decisions, no matter how small, must go through me. Should this be tiresome, I reserve the right to turn off my cellphone while I sip a Piña Colada in the sun.
  16. I realize that some of my trusted colleagues will be better than me at something. With time, many might end up better than me at most things. I could save face by deciding to take this as a proof of my recruiting skills, but this is hardly as satisfying as belittling their achievements and skills and then firing them.
  17. If I need to get rid of one of my trusted colleagues, I will have three options. The first one is to offer conditions to that trusted colleague for leaving. The second one is to fire the trusted colleague. The last one is to mount a cabal inside my start-up to get that trusted colleague to leave in disgust. The cabal-oriented solution will prove much more diverting, as I will be able to watch the cascade of consequences, plots and counter-plots, the consequent loss of time, productivity and morale, and I will be able to find out just exactly how much a lawyer charges for defending my company in court.
  18. At some point, my dream project will near completion. By then, I will have dreamed up tons of new features, and it only makes sense to start piling them up until requirements dwarf what has been realized so far. My trusted colleagues will certainly complete these final few features in a matter of days. Additionally, by the time my trusted colleagues are done, I can certainly have dreamed up a few new features. Ain’t that great?
  19. Also, a project approaching completion signals that I can reassign everybody to another project. The project will certainly find a way to finish itself.
  20. It may happen that my project cannot have all of the following features: working, released, everything I dreamed it to be. The first two features are not really important, so I can remove either.
  21. If a project has failed, or if the company has pivoted, I will inform my trusted employees. Of course, I might decide to keep the news for after the end of an ongoing death-march-to-finish-the-project-in-emergency. Just imagine the look on their face.
  22. This is also true for my commercially-oriented trusted colleagues. Just imagine them telling to their prospects that everything they have promised until this date is false.
  23. As a CEO, I will be approached by countless people with ideas. With a little effort, this will give me the opportunity to pivot as often as twice a week. My trusted colleagues need the exercise.
  24. Being an avid Mac user, I can do as well as Steve Jobs. Even better, being a Facebook user, I can also do as well as me Mark Zuckerberg. Where else could you find a CEO as exceptional as me?
  25. I will occasionally take breaks, or even vacations. Whenever I do go on vacation, however, I will make sure to keep this a secret. Just think how funny it will be when they realize how much time they have wasted coming by every few minutes to check if I had arrived in the office.
  26. While this is my company, not all trusted colleagues may realize that its money is mine to use as I see fit and that I can invest it in my luxury vacations. Some of them might conclude that I am abusing everybody’s trust, work and livelihood. The simplest solution is to fire them, but I should check with a lawyer whether I could also sue them.
  27. While I may have to lie to potential clients and investors, I will refrain from lying to (alleged) mobsters and secret services. My health matters too much to me.
  28. Also, should I be faced with (alleged) mobsters and secret services, and should I determine that the people in front of me are morons and/or easy milk cows, I will refrain from making this realization obvious to said idiots.
  29. Some of my colleagues will insist that we should not release a product that we have not tested. If they refuse to use our product because they hate it, it means that the product is ready for release. If I refuse to use our project because I hate it, it means that we should restart from scratch, re-implement everything in two weeks, then fire my trusted colleagues.
  30. From time to time, I will feel like taking a scapegoat among my trusted colleagues, because this is a cheap way to vent out frustration at failed projects or meetings. After all, there is no way said trusted colleague could figure out that competition would pay more and grab the experience gained at my company.
  31. If I attempt to enter a market saturated of products that are free-to-use and just as good as mine – possibly even dominated by Google, Microsoft, Apple or Mozilla – my investors and trusted colleagues should not worry, as I have a secret weapon: I am the smartest person in the world.
  32. If my product requires that user give up on their existing data, about their existing code, about their existing infrastructure, I will not be too surprised when users rather decide to give up on my product. That is because users are stupid.
  33. If potential users start describing my sole product as vaporware, I can just retort that Microsoft got around with delivering vaporware for more than 10 years. Of course, Microsoft got to #1 before doing so, but we are almost there ourselves.
  34. Occasionally, I will make mistakes and choose wrong paths for my company. The most amusing manner of getting away with mistakes is to pretend that my trusted colleagues have disobeyed me and taken bad initiatives. Doing so is, of course, a mistake and places the company on a wrong path, which makes things recursively more amusing [1].
  35. My trusted colleagues will often be occupied with non-productive trivialities such as building my product, signing paychecks or hunting for clients. I will have to remind them regularly that I am the sole efficient worker in this company.
  36. As my Reality Distortion Field can tell you, there is no difference between “clients”, “contacts” and “people who have looked at the website”. For instance, if our website gets 30,000 hits per month (or per day), we surely have 30,000 paying users.
  37. Occasionally, my trusted colleagues will reach milestones, possibly even releasable/deployable versions of our product. These milestones/versions will probably not match my dream ideas. The simplest way to deal with such imperfections is to treat these unworthy versions with disgust, refuse to give them a name or number and retroactively add some set of features that must absolutely be completed before the milestone is hit.
  38. Occasionally, I will be unhappy with some of my trusted colleagues, but not sufficiently to fire them. I will assigning them to a punishment project, designed solely for maximal pain. That will teach them!
  39. If one of my projects succeeds, I will know – and remind everybody – that I am the sole reason for this success.
  40. If one of my ideas succeeds, I will attempt to remember if, by any chance, one of my trusted colleagues may have spent some time attempting to convince me that this was a good idea, before this was my idea. If so, I will make sure that he/she understand how the first idea was bad and mine was a stroke of genius. If this is not sufficient, I will fire him/her.
  41. My main argument for selling to potential clients will be that they have been idiots to not use my product/service. With my help, however, they can become intelligent enough to know that they should buy my wares. Am I not generous?
  42. Although I may not have heard of weird concepts such as revision control systems, bug trackers or customer relationship managers, if one of my trusted colleagues informs me that they cannot work without such exotic software, I will simply dismiss my trusted colleague as incompetent. After all, I could do the work without such niceties, so they cannot be really useful, can they?
  43. Once I know of weird concepts such as revision control systems, bug trackers or customer relationship managers, I should probably build one. After all, how hard can it be? Additionally, I am so much smarter than everybody who has built one before us. The world will be so grateful that they will beg to use our product, once we have gotten around to building it.
  44. Should I decide that we need to reinvent yet another revision control systems, bug tracker or customer relationship managers and to use it, I will make sure that the product is tested, at least marginally, before it hosts our critical data. Of course, if my trusted colleagues insist that the product does not work, there is always time to ignore their feedback. Otherwise, how could I watch the revision control system lose all the source code of the revision control system?
  45. Should investors ever decide to audit my company with the perspective of buying it, I will adjust my Reality Perception Filter to ensure that the esteemed idiots rake. This will, of course, not prevent me from complaining to my trusted colleagues that all this work was for naught and for so little money. I would not want them to waste their health by envying me too much.
  46. Of course, to ensure credibility, I will make sure that said trusted colleagues will not receive one cent from the sale. Their health really is that important to me.
  47. As my company will undoubtedly be #1 soon, I will make sure that my salary corresponds to that rank. Unfortunately, trusted colleagues will have to satisfy themselves with salaries slightly below the minimum wage, to ensure the survival of the company. Remember, we are just a start-up.
  48. I might I come from a commercial background, but I know technical matters better than my trusted technical colleagues. After all, I know Excel.
  49. I might I come from a technical background, but I know commercial matters better than my trusted commercial colleagues. After all, I know Excel.
  50. It is a well-known fact that trusted colleagues are wimps and that they burnout faster than matches. I will keep vigilant for such occurrences, not only because it is fun to watch one trusted colleague turn into an improductive bag of nerves that the mere mention of my name will send into fits of dementia, but also because with the proper balance of (mis)management, such burnouts can propagate faster than forest fires.

Thanks for the contributions of Team TGCM, Team Malsain, Team Dixous, Team HokutoNoOpa. No animals were hurt in the process, but I intend to remedy this in part 2, to be released soon.


[1] Or coinductively more amusing, if you want to be precise.

Where Am I?

You are currently browsing entries tagged with startup at Il y a du thé renversé au bord de la table.

Follow

Get every new post delivered to your Inbox.

Join 29 other followers