Today is an international day of fighting against web censorship.

Against SOPA (in the US), Hadopi (in France), and all the bills that assume that freedom of expression and freedom of information can be cast aside whenever they come in the way of profit.

Learn more.

Mozilla is working with numerous educators, professors and researchers across the world, both to bring open-source, the open web and web technologies into the classroom, and to bring the contributions of students and their mentors to the world. You can be a part of this, and your field does not have to be Computer Science.

Mozilla in the classroom – and beyond

Mozilla? Certainly, you have heard about us. Mozilla is best known for its work on the Firefox web browser and the Thunderbird mail client but that is not all there is to Mozilla, far from it. The activities of Mozilla reach largely beyond a few software products. The objective of the Mozilla project is to promote an open, participative web. For this purpose, we have developed and promoted Firefox, Thunderbird, open standards, add-ons, and countless experiments and innovations. For this purpose, we want to work with you.

This is a call for academics and students. We would like to work with you, with your classrooms and with your labs.

Mozilla is already present in numerous classrooms and research laboratories around the world, thanks to collaboration of numerous educators, professors, researchers and students. We deliver courses, we provide teaching material, but most importantly, we provide opportunities for students and their mentors: the opportunity to make impacting contributions to the web, to open-source and sometimes to society, the opportunity to tinker with technologies used by hundreds and millions of people, and to deliver research results to an audience consisting in the the whole web, and also, most importantly, the opportunity to learn and teach as part of an exciting, participatory and fun community.

So, whether you are an academic or a student, if you want to work alongside us, please contact contact us.

Topics of interest

At its core, Mozilla is an open-source community. Our work at Mozilla covers many topics. Some of these topics are extremely technical, some do not even involve a computer.

Here are but a few.

Open-source (Software Engineering, IT, CS Research, UX)…

• client-side programming in C, C++, JavaScript, Java;
• server-side programming in Python, JavaScript;
• concurrent and system-oriented programming;
• implementation of programming languages;
• static analysis, rewriting and semantics or programming languages;
• instrumentation, dynamic analysis;
• distributed, scalable databases;
• distributed, scalable web services;
• client-side web security;
• server-side web security;
• network security;
• network maintenance;
• applied cryptography;
• implementation of cryptographic protocols;
• establishing and maintaining mirrors;
• service-oriented architectures;
• low-level, 2d graphics programming;
• low-level, 3d graphics programming;
• implementing typography and layouts;
• defining typography and layouts;
• developing user interfaces;
• designing user interfaces;
• web applications;
• video game development;
• build systems;
• tools for developers;
• data mining from bugzilla, survey results, etc;
• implementing web standards;
• designing candidate web standards;
• privacy analysis;
• operating system development;
• …

… community (Internationalization, Events, Representing Mozilla)

• internationalizing Mozilla and other community applications;
• internationalizing Mozilla and other community websites;
• organizing local or global open-source events;
• … or web events;
• … or technology events;
• participating to local or global events;
• teaching the open web to non-technical audiences;
• teaching the open web to technical audiences;
• teaching web technologies;
• organizing local communities;
• spreading Mozilla and other open-source technologies;
• providing technical support in the local language;
• representing Mozilla in a highschool/university/…;
• communicating with local media;
• analyzing industry news;
• advocating open web or open-source to local companies, organizations and communities;
• interacting with other technology communities;
• writing articles on open-source and the open web;
• … or filming documentaries;
• … or anything else you can think of!

What the Mozilla community can do for you

On some topics, Mozilla can provide you with teaching/learning material or get you in touch with Mozillians who would love to come to your classroom and deliver interventions or lectures.

More importantly, we can help shape student projects and we can contribute to mentoring them. Do not limit yourself – with enough dedication, student projects can change the world.

If you are interested, please get in touch.

You can reach me by e-mail at mozilla.com, as dteller.

One of the key components of OS.File is the Schedule API, a tiny yet powerful JavaScript core designed to considerably simplify the development of asynchronous modules. In this post, we introduce the Schedule API.

Introduction

In a previous post, I introduced OS.File, a Mozilla Platform library designed make the life of developers easier and to help them produce high-performance, high-responsiveness file management routines.

In this post, I would like to concentrate on one of the core items of OS.File: the Schedule API. Note that the Schedule API is not limited to OS.File and is designed to be useful for all sorts of other modules.

Overview

One of the most important aspect of any user-facing application is its responsiveness. Contrary to intuition, however, responsiveness is much less a matter of speed than one of tempo: not necessarily attempting to perform tasks as fast as possible, but rather attempting to perform them in a manner that does not disrupt the user experience. In the Mozilla Platform, disrupting the user experience is synonymous with freezing the user interface.

Avoiding this is the art of asynchronous, non-blocking operations, and this is the reason we built the Schedule API.

The Schedule API is a tiny JavaScript module designed to simplify considerably the development of asynchronous code. It is based on Joseph Walker’s excellent Promises API [1] and it lies at the core of OS.File’s asynchronous operations.

Before detailing the full API, let us start with a few simple examples. Consider a common situation: some treatment needs to be done soon, but not immediately, as this would otherwise disrupt the user experience. For this purpose, JavaScript offers function setTimeout, which schedules the execution of the treatment to later:

function doItLater() {
  window.setTimeout(function() {
    var result = fibonacci(10);
  });//Schedule the execution for later
}//Wait, how do we get the result?

However, setTimeout offers no help for informing the rest of the code that the execution is complete. While there are many solutions to this problem – I am sure that dozens can be found on StackOverflow – perhaps the most developer-friendly is that of promises:

function doItLater() {
  var promise = new Promise();
  window.setTimeout(function() {
    var result = fibonacci(10);
    promise.resolve(result);
  });//Schedule the execution for later
  return promise;//And return immediately a promise.
}

With this code, we have introduced the promise of a future result. Once we have obtained a promise, we can simply add a reaction that will be triggered once the promise is resolved:

doItLater().then(function(result) {
   alert("I have just finished computing Fibonacci: "+result);
})

Or we can chain promises to obtain other promises:

var promise2 = doItLater().chainPromise(function(result) {
   return sieve(result);
})

Note that these two extracts are compatible with each other. If we write both, we will be informed once the first treatment is complete and a second treatment will also take place immediately, one whose result is promised as promise2.

Unfortunately, a few problems appear with our code. Firstly, window.setTimeout is unfortunately not available in all contexts. While workarounds are available for non-UI modules and background threads, writing any reusable asynchronous is a somewhat painful that can quickly lead to unreadable code. Chaining such treatments becomes even worse.

The second issue is that our call fibonacci(10) might well be quite long itself and might very well disrupt the user experience if we do nothing about it.

Let us start by addressing the first issue, with the Schedule module:

Components.utils.import("resource://gre/modules/schedule.jsm");

Function Schedule.soon replace most uses of window.setTimeout:

function doItLater() {
  return Schedule.soon(function() {
    return fibonacci(10);
  });
}

Or, more concisely:

function doItLater() {
  return Schedule.soon(fibonacci, 10);
}

Or, to delay the execution of the task by 100ms:

function doItLater() {
  return Schedule.soon({delay: 100}, fibonacci, 10);
}

The Schedule API also extends Promises with a new method soon:

var promise2 = doItLater().soon(function(result) {
   return sieve(result);
})

Just like chainPromises, soon adds new treatments that are triggered only once the promise is resolved. However, and by opposition to chainPromises, these treatments are executed later, without disrupting the user experience.

With the combination of Schedule and Promises, it is easy to see how we could write a version of the Fibonacci function that is computed progressively:

//Compute the fibonacci function of n, as a background task
function fibo(n) {
  if (n == 1 | n == 0) {
    return Schedule.alreadyComplete(1);
    //Promise a result and deliver it immediately
  } else {
    var tasks = [];//Launch two tasks
    tasks.push(fibo(n - 1));
    tasks.push(fibo(n - 2));
    return Promise.group(tasks).soon(function(results) {
       return results[0] + results[1];
    })
  }
}

Nice, readable and non-disruptive. Plus, for debugging purposes, or if you wish to add some visual feedback, you can attach additional observers to promises.

The API

So far, we have made use of the following functions of the Promises API:

• Promise;
• Promise.resolve;
• Promise.prototype.chainPromise;
• Promise.prototype.then;
• Promise.group;

I am trying to convince Joseph to blog about this very nice API, so you should be able to learn more about it on his blog, one of these days.

We have also made use of the following functions of module Schedule:

• Schedule.soon;
• Promise.prototype.soon;
• Schedule.alreadyComplete.

While these methods can already be very useful to help you write or rewrite asynchronous code, the Schedule API goes a little bit further and introduces Scheduling Categories and Rendez-vous points.

Categories

While interleaving tasks is a good way to ensure that all tasks progress, some tasks need a little tighter control. Typically, the result of successive operations on a file depend on their order of execution – closing a file before reading from it is generally not a very good idea. Similarly, for performance reasons, an application should generally avoid reading from several files simultaneously, as this can slow down not only the application but the whole system.

In the Schedule API, this is materialized as Scheduling Categories. When several tasks are placed in the same category, any call to Schedule.soon or Promise.prototype.soon guarantees that these tasks will be executed in the order in which they were added and that only one of these task is executed at a time. This does not prevent tasks from other categories from being interleaved.

If, for instance, we wish to serialize the computation of Fibonacci’s function, we may declare a category as follows:

var Fibo = new Schedule.Category("Fibonacci's function");

and replace the final function call of the previous listing with

    soon({category: Fibo}, function(results) {
       return results[0] + results[1];
    })

Thus categorized, Fibonacci’s functions will take less resources to compute, and will be even nicer to the rest of the application and the system, though the total computation will probably last a little longer.

Rendez-vous

One of the original reasons for which we started developing the Schedule API was to simplify asynchronous initialization on the Mozilla Platform and ensure that whichever parts of initialization were not strictly required at start-up could be simply and safely postponed.

For this purpose, the Schedule API provides one additional function: Schedule.on. This function lets developers register code to be executed only once some condition is met, as follows:

Schedule.on("startup-complete").soon(delayedInitializer);

With this line, function delayedInitializer will be executed only once “startup-complete” is resolved. Note that, by opposition to DOM-style events, if “startup-complete” has already been resolved, delayedInitializer will be immediately scheduled for execution. This simple change solves a number of refactoring headaches.

By the way, Schedule.on("startup-complete") is just a promise. To resolve it, simply call resolve:

Schedule.on("startup-complete").resolve();

And that’s it. Welcome to the Schedule API!

Status

Everything that was presented in this post is implemented, unit-tested – although not battle-tested – used in OS.File, and currently under review. There is no ETA yet, due to dependencies, but I hope that we will be able to push it soon.

There is one important feature that I wish to add at some point: multi-threading. While there are limitations to what we can transmit between threads, there is no reason for which we could not schedule code for execution not on the main thread, but on a pool of workers. There will be limitations, but I believe that I have a pretty good idea of how to implement this, and if I succeed, this can expand considerably our horizons. I have small chunks of code written, but no ETA for a prototype.

Wrapping this up

The Schedule API is a simple module (only 5 public functions) designed to make our life easier when writing asynchronous code or refactoring code for asynchronicity.

I hope that you will find it useful.

[1] Thanks to Panos for point this code to me. I had previously reimplemented my own brand of Promises, but having two of them in the Mozilla Platform codebase was simply pointless.

edit Clarified the fact that calling fibonacci(10) in a setTimeout will not magically make everything non-disruptive. Thanks for the feedback, Axel.

Summary

The Mozilla Platform keeps improving: JavaScript native file management is an undergoing work to provide a high-performance JavaScript-friendly API to manipulate the file system.

The Mozilla Platform, JavaScript and Files

The Mozilla Platform is the application development framework behind Firefox, Thunderbird, Instantbird, Camino, Songbird and a number of other applications.

While the performance-critical components of the Mozilla Platform are developed in C/C++, an increasing number of components and add-ons are implemented in pure JavaScript. While JavaScript cannot hope to match the speed or robustness of C++ yet (edit: at least not on all aspects), the richness and dynamism of the language permit the creation of extremely flexible and developer-friendly APIs, as well as quick prototyping and concise implementation of complex algorithms without the fear of memory errors and with features such as higher-level programming, asynchronous programming and now clean and efficient multi-threading. If you combine this with the impressive speed-ups experienced by JavaScript in the recent years, it is easy to understand why the language has become a key element in the current effort to make the Mozilla Platform and its add-ons faster and more responsive at all levels.

Many improvements to the JavaScript platform are pushing the boundary of what can be done in JavaScript. Core Modules, strict mode and the let construct are powerful tools that empower developers to produce reusable, clean and safe JavaScript libraries. The Mozilla Platform offers XPConnect and now js-ctypes, two extremely powerful technologies that let privileged JavaScript maskerade as C/C++ and get access to the low-level features of the platform. Other technologies such as the Web Workers expose low-level operating system features through fast, JavaScript-friendly APIs (note that the Mozilla Platform has exposed threads and processes to JavaScript at least since 2005 – Web Workers are faster, nicer, and play much more nicely with the runtime, in particular with respect to garbage-collection and the memory model).

Today, I would like to introduce one such improvement: native file management for JavaScript, also known as OS.File.

Since JavaScript has become a key component to the Mozilla Platform, the Mozilla Platform needs a great library for manipulating files in JavaScript. While both XPConnect and JS-ctypes can (and have been) used for this purpose, our objective, with this library, is to go way beyond the file management APIs that has been exposed to JavaScript so far, regardless of the platform, in terms of:

• expressiveness;
• integration with the JavaScript side of the Mozilla Platform;
• operating system-level features;
• performance;
• extensibility.

This library is a work in progress by the Mozilla Performance Team, and we have good hope that a fully working prototype will be available by early January. Not everything is implemented yet and all sorts of adjustments can yet be made based on your feedback.

Once we have delivered, it is our hope that you will use this library for your future works on the Mozilla Platform, whether you are extending the Mozilla Platform, developing an add-on or an application, or refactoring some existing feature.

Let me emphasize that this is a Mozilla Platform API (hence the “OS” prefix), not a Web API. By opposition to the HTML5 File object, this API gives full access to the system, without any security limitation, and is definitely not meant to be scriptable by web applications, under any circumstance.

Manipulating files, the JavaScript way

Reading from a file

Let us start with something simple: reading from a file.

First, open the library:

Components.utils.import("resource://gre/modules/osfile.jsm");

OS.File is a JavaScript module, in other words it is shared between all users in the same thread. This is particularly important for speed, as this gives us the ability to perform aggressive caching of certain data.

Once you have opened the module, you may read your file:

var fileName = "/home/yoric/hello";
var contents = OS.File.openForReading.using(fileName, function(myFile) {
  return myFile.readString()
});

This extract:

• opens file "/home/yoric/hello" for reading;
• reads the contents of the file as a string (assuming ASCII encoding);
• closes the file;
• reports an error if anything wrong has happened either during opening or during reading;
• places the result in variable contents.

This short listing already demonstrates a few interesting elements of the API. Firstly, notice the use of function using. This function performs scope-bound resource management to ensure that the file is properly closed once it has become unneeded, even in presence of errors. This has roughly the same role as a finally block in Java or a destructor on a C++ auto-pointer. I will return to the topic of resource management later. For the moment, suffices to say that closing a file through using or method close is optional but recommended, as open files are a limited resource on all operating systems.

Had we decided to entrust JavaScript to close the file by itself at some point in the future, we could have simply written:

var fileName = "/home/yoric/hello";
var contents = OS.File.openForReading(fileName).readString();

Secondly, consider OS.File.openForReading. As its name suggests, this function/object serves to open an existing file for reading, and it fails if the file does not exist yet. The API provides such functions for all common scenarios, all of which accept optional flags to customize Unix-style file rights, Windows-style sharing properties and other Unix- or Windows-style attributes. Alternatively, function/object/constructor OS.File is the general manner of controlling all details of file opening.

The extracts above do not demonstrate any feature that could not have been achieved with XPConnect. However, let us briefly compare our extracts with an XPConnect-based implementation using similar lines:

• the OS.File implementation consists of 2 to 4 lines, including resource cleanup and error-handling / a comparable XPConnect-based implementation requires about 30 lines;
• the OS.File implementation works both in the main thread or in a background thread / a comparable XPConnect-based implementation works only in the main thread;
• benchmarks are not available yet, but I have hope that the OS.File implementation should be slightly faster due to a lower overhead and an optimized implementation of readString;
• in case of error, the OS.File implementation raises an exception with constructor OS.File.Error / the XPConnect-based implementation raises a generic XPConnect exception;
• if the file does not exist, the OS.File implementation raises an error while executing OS.File.openForReading / the XPConnect-based implementation raises an error later in the process;
• if executed on the main thread, the OS.File implementation will print a warning.

Note that OS.File manipulates this and closures in the JavaScript fashion, which makes it possible to make our extracts even more concise, as follows:

var fileName = "/home/yoric/hello";
var contents = OS.File.openForReading.using(fileName, function() {
  return this.readString();
});

or, equivalently,

var fileName = "/home/yoric/hello";
var contents = OS.File.openForReading.using(fileName,
  OS.File.prototype.readString);

Of course, OS.File is not limited to strings. Indeed, to return a typed array, simply replace readString with readBuffer. For better performance, it is also possible to reuse an existing buffer. This is done by replacing readBuffer with readTo.

Also, OS.File is not limited to reading entire files. Indeed, all read/write functions accept an optional argument that may be used to determine a subset of the file that must be read:

var fileName = "/home/yoric/hello";
var contents = OS.File.openForReading.using(fileName,
  {fileOffset: 10, bytes: 100},
  OS.File.prototype.readString);

Well-known directories

The operations we have demonstrated so far use an hard-coded path “/home/yoric/hello”. This is not a very good idea, as this path is valid only under Linux, but not under Windows or MacOS. Therefore, we certainly prefer asking the Mozilla Platform to select the path for us. For this purpose, we may replace the first line with:

var fileName = OS.Path.home.get("hello");

This extract:

• uses global object OS.Path (part of library OS.File);
• requests the path to the user’s home directory;
• requests item "hello" at this path.

The extract demonstrates a few things. Firstly, the use of OS.Path. This object contains paths to well-known directories, and can be extended with new directories. Each path has constructor OS.Path, and supports a method get that serves to enter into files/directories. Secondly, the use of OS.Path as a path for functions of module OS.File: any function of this module accepts an OS.Path in place of a hard-coded directory.

Note that OS.Path objects are purely in-memory constructs. Building an OS.Path does not cause any call to the file system.

As previously, something similar is feasible with XPConnect. Comparing with a XPConnect-based implementation, we may notice that:

• the OS.File implementation consists of 1 line / a comparable XPConnect-based implementation consists of 1 to 4 lines, depending on the use of additional libraries;
• the OS.File implementation works both in the main thread and in a background thread / again, XPConnect works only in the main thread;
• benchmarks are not available yet, but I have hope that the OS.File implementation should be slightly faster due to a lower overhead and use of caching.

Behaving nicely

The operations we have demonstrated so far are synchronous. This is probably not problematic for file opening, but reading a large file synchronously from the main thread is a very bad idea, as it will freeze the user interface until completed. It is therefore a good idea to either send the operation to a background thread or to ensure that reading takes place by small chunks.

OS.File supports both scenarios by integrating with (work-in-progress) libraries Promise and Schedule, both of which will be introduced in another post, once their API has stabilized.

The first step to reading asynchronously is to open library Promise. We will take the opportunity to open Schedule

Components.utils.import("resource://gre/modules/promise.jsm");
Components.utils.import("resource://gre/modules/schedule.jsm");

Now that the module is open, we may use asynchronous reading and asynchronous writing functions:

var promisedContents = OS.File.openForReading(fileName).
    readString.async();

This operation schedules progressive reading of the file and immediately returns. Note that we do not close the file, as this would stop reading, probably before the operation is complete. The result of the operation, promisedContents, is a Promise, i.e. a variable that will eventually contain a value, and that may be observed or  polled, as follows:

promisedContents.onsuccess(function(contents) {
  console.log("It worked", contents);
});
promisedContents.onerror(function(error) {
  console.log("It failed", error);
});

Similarly, reading from a background thread is a simple operation:

var promisedContents = Schedule.bg(function() {
  importScripts("resource://gre/modules/osfile.jsm");
  var fileName = "/home/yoric/hello";
  return OS.File.openForReading.using(fileName, function(myFile) {
    return myFile.readAsString();
  });
);

The call to Schedule.bg “simply” sends a task to a background thread and ensures that any result, error, etc. is routed back to the promise. The promised value itself is used exactly as in the previous example.

Once again, we may compare to the XPConnect-based implementation;

• OS.File-based implementation of asynchronous reading takes 3 lines including opening, closing, resource management / general XPConnect-based implementation of asynchronous reading takes about 10-15 lines, although reading from a hard-coded path or a resource inside the Mozilla Platform can be reduced to 5-6 lines;
• OS.File implementation of background reading takes 5 lines / XPConnect does not expose sufficient features to permit permit background, although such features could certainly be implemented in C++ and exposed through XPConnect;
• OS.File-based implementation only works for files / XPConnect-based implementation works for just about any construction;
• benchmarks are not available, but I have hope that the OS.File implementation should be faster than the XPConnect-based implementation due to a less generic implementation and a lower overhead;
• the promises used in the OS.File-based implementation encourages writing code in natural order, in which the code that uses a value appears after the code that fetches the value / XPConnect-based implementation encourages backwards coding, in which the function that uses a value appears before the code that fetches the value (aka “asynchronous spaghetti programming”).

API summary

The API defines the following constructors:

• OS.File – all operations upon an open file, including reading, writing, accessing or altering information, flushing, closing the file;
• OS.Dir – all operations upon an open directory, including listing its contents, walking through the directory, opening an item of the directory, removing an item of the directory;
• OS.Path – all operations on paths which do not involve opening a directory, including concatenation, climbing up and down the tree ;
• OS.File.Error – all file-system related errors.

and the following global objects:

• OS.File – opening a file, with or without auto-cleanup;
• OS.Dir – opening a directory;
• OS.Path – well-known directories and files.

Speed

Writing fast, cross-platform, file manipulation code is a complex task. Indeed, some platforms accelerate opening a file from a directory (e.g. Linux), while other platforms do not have such operations (e.g. MacOS, Windows). Some platforms let applications collect all information regarding a file with a single system call (Unix), while others spread the work through several system calls (Windows). The amount of information that may be obtained upon a file without having to perform additional system calls varies from OS to OS, as well as the maximal length of a path (e.g. under Windows, the value of MAX_PATH is false), etc.

The design of OS.File takes this into account, as well as the experience from the previous generations of file manipulation APIs in the Mozilla Platform (prfile and nsIFile/nsILocalFile), and works hard to minimize the number of system calls required for each operation, and to let experts fine-tune their code for performance. While benchmarking is not available yet, we have good hope that this will make it possible to write IO code that runs much faster, in particular on platforms with slow file systems (e.g. Android).

In addition, although this should have a much smaller impact, OS.File uses as bridge between C++ and JavaScript the JSAPI, which is, at the moment of this writing, the fastest C++-to-JavaScript bridge on the Mozilla Platform.

Responsiveness

Speed is not sufficient to ensure responsiveness. For this purpose, long-running operations are provided with asynchronous variants that divide the work in smaller chunks to avoid freezing up the thread. The API does not enforce the use of these asynchronous variants, as experience shows that such a drastic choice is sometimes too constraining for progressive refactoring of synchronous code towards better asynchronicity.

Every operation can be backgrounded thanks to the Schedule module. At the time of this writing, it is not possible to send a file from a thread to another one, but we have a pretty clear idea of how we can do this, so this should become possible at some point in the future.

What now?

As mentioned, this is a work in progress. I am currently hard at work on building a complete prototype by the end of December, with the hope of landing something soon afterwards. I expect that benchmarking will continue after this stage to fine-tune some low-level choices and improve the API. If you wish to follow progress – or vote for this feature – we have a Bugzilla tracking bug on the topic, and a whole host of subbugs.

Note that this API will not replace nsIFile, although once it has landed, some of our JavaScript code will progressively from nsIFile to OS.File.

If you have any feedback, now is a great time to send it. Would you use this API? Would you need certain specific or obscure feature that is currently missing in the Mozilla Platform or that risks being lost?

In future posts, I will introduce further examples and detail some of the choices that we have made to ensure the best possible speed on all platforms.

Stay tuned!

Comme tous les ans, Mozilla propose des stages en informatique, orientés Développement, R&D ou Recherche. Selon le sujet, le stage peut vous emmener à Paris, aux États-Unis, au Canada, en Chine…

À propos de Mozilla

La Fondation Mozilla est une association à but non-lucratif, fondée pour encourager un Internet ouvert, innovant et participatif. Vous avez probablement entendu parler de Mozilla Firefox, le navigateur open-source qui a réintroduit sur le web les standards ouverts et la sécurité, ou de Mozilla Thunderbird, le client de messagerie multi-plateforme, open-source et extensible. Les activités de Mozilla ne s’arrêtent pas à ces deux produits et se prolongent à de nombreux projets pour le présent et l’avenir, tels que :

• Boot-to-Gecko, système d’exploitation totalement ouvert et construit par la communauté, pour les téléphones portables, tablettes et autres machines communicantes ;

• SpiderMonkey, une famille de Machines Virtuelles conçues pour l’analyse statique et dynamique, la compilation et l’exécution des langages web, en particulier JavaScript ;

• DeHydra et JSHydra, outils d’analyse statique pour les langages C++ et JavaScript ;

• Rust, un nouveau langage de programmation conçu pour le développement d’applications système parallèles sûres ;

• WebAPI, un ensemble d’outils qui permettent d’étendre les capacités des applications web au-delà de celles des applications traditionnelles, la sécurité et la confidentialité en plus ;

• Gecko, le moteur de rendu extensible et portable pour le HTML, le XML et les interfaces graphiques, qui a permis Firefox, Thunderbird et de nombreuses autres applications ;

• BrowserID, une technique innovante qui fournit aux utilisateurs et aux développeurs les outils cryptographiques pour assurer l’identification sur le web, sans compromettre la vie privée, la simplicité ou la sécurité ;

• les fonctionnalités Mozilla Services de gestion d’identité par le Cloud ;

• et d’autres encore…

À propos de vous

Mozilla proposes plusieurs stages dans ses installations à travers le monde sur de nombreux sujets.

Votre profil :

• vous voulez faire du web un endroit meilleur, sur lequel chacun peut naviguer et contribuer en toute sécurité, sans avoir à craindre pour sa sécurité ou sa vie privée ;
• vous souhaitez prendre part à un projet utilisé par plus de 33% de la population du web ;
• vous voulez que votre travail soit utile à tous et visible par tous ;
• vous avez de fortes compétences en Algorithmique et en Informatique ;
• vous avez de fortes compétences dans au moins l’un des domaines suivants :
• systèmes d’exploitation ;
• réseaux ;
• géométrie algorithmique ;
• compilation ;
• cryptographie ;
• analyse statique ;
• langages de programmation ;
• extraire des informations pertinentes à partir de données exotiques ;
• algorithmique distribuée ;
• le web en tant que plate-forme ;
• interactions avec les communautés du logiciel libre ;
• toute autre compétence qui, à votre avis, pourrait nous servir.
• sur certains sujets, un excellent niveau d’Anglais peut être indispensable ;
• les stages sont généralement prévus pour des étudiants M1 ou M2 mais si vous arrivez à nous impressionner par vos réalisations ou par vos connaissances, le diplôme n’est pas indispensable.

Si vous vous reconnaissez, nous vous invitons à nous contacter. En fonction du sujet, les stages peuvent vous emmener à Paris, Mountain View, San Francisco, Toronto, Taipei, ou d’autres lieux à travers le monde.

Les meilleurs stagiaires peuvent espérer un contrat freelance, un CDI ou/et une bourse de doctorat.

Pour nous contacter

Pour toute question, contactez :

• pour tout ce qui concerne les stages chez Mozilla, Julie Deroche (à mozilla.com, jderoche) – Mozilla Mountain View, College Recruiting ;
• pour les stages à Paris, David Rajchenbach-Teller (à mozilla.com, dteller) – Mozilla Paris, Développeur / Chercheur.

JavaScript is a dynamic language. However, by borrowing a few pages from static languages – and a few existing tools – we can considerable improve reliability and maintainability.

« Writing one million lines of code of JavaScript is simply impossible »

(source: speaker in a recent open-source conference)

JavaScript is a dynamic language – a very dynamic one, in which programs can rewrite themselves, objects may lose or gain methods through side-effects on themselves on on their prototypes, and, more generally, nothing is fixed.

And dynamic languages are fun. They make writing code simple and fast. They are vastly more suited to prototyping than static languages. Dynamism also makes it possible to write extremely powerful tools that can perform JIT translation from other syntaxes, add missing features to existing classes and functions and more generally fully customize the experience of the developer.

Unfortunately, such dynamism comes with severe drawbacks. Safety-minded developers will tell you that, because of this dynamism, they simply cannot trust any snippet, as this snippet may behave in a manner that does not match its source code. They will conclude that you cannot write safe, or even modular, applications in JavaScript.

Many engineering-minded developers will also tell you that they simply cannot work in JavaScript, and they will not have much difficulty finding examples of situations in which the use of a dynamic language in a complex project can, effectively, kill the project. If you do not believe them, consider a large codebase, and the (rather common) case of a large transversal refactoring, for instance to replace an obsolete API by a newer one. Do this in Java (or, even better, in a more modern mostly-static language such as OCaml, Haskell, F# or Scala), and you can use the compiler to automatically and immediately spot any place where the API has not been updated, and will spot a number of errors that you may have made with the refactoring. Even better, if the API was designed to be safe-by-design, the compiler will automatically spot even complex errors that you may have done during refactoring, including calling functions/methods in the wrong order, or ownership errors. Do the same in JavaScript and, while your code will be written faster, you should expect to be hunting bugs weeks or even months later.

I know that the Python community has considerably suffered from such problems during version transitions. I am less familiar with the world of PHP, but I believe this is no accident that Facebook is progressively arming itself with PHP static analysis tools. I also believe that this is no accident that Google is now introducing a typed language as a candidate replacement for JavaScript.

That is because today is the turn of JavaScript, or if not today, surely tomorrow. I have seen applications consisting in hundreds of thousands of lines of JavaScript. And if just maintaining these applications is not difficult enough, the rapid release cycles of both  Mozilla and Chrome, mean that external and internal APIs are now changing every six weeks. This means breakage. And, more precisely, this means that we need new tools to help us predict breakages and help developers (both add-on developers and browser contributors) react before these breakages hit their users.

So let’s do something about it. Let’s make our JavaScript a strongly, statically typed language!

Or let’s do something a little smarter.

JavaScript, with discipline

At this point, I would like to ask readers to please kindly stop preparing tar and feathers for me. I realize fully that JavaScript is a dynamic language and that turning it into a static language will certainly result in something quite disagreeable to use. Something that is verbose, has lost most of the power of JavaScript, and gained no safety guarantees.

Trust me on this, there is a way to obtain the best of both worlds, without sacrificing anything. Before discussing the manner in which we can attain this, let us first set objectives that we can hope to achieve with a type-disciplined JavaScript.

Finding errors

The main benefit of strong, static typing, is that it helps find errors.

• Even the simplest analyses can find all syntax errors, all unbound variables, all variables bound several times and consequently almost all scoping errors, which can already save considerable time for developers. Such an analysis requires no human intervention from the developer besides, of course, fixing any error that has been thus detected. As a bonus, in most cases, the analysis can suggest fixes.
• Similarly trivial forms of analysis can also detect suspicious calls to break or continue, weird uses of switch(), suspicious calls to private fields of objects, as well as suspicious occurrences of eval – in my book, eval is always suspicious.
• Slightly more sophisticated analyses can find most occurrences of functions or methods invoked with the wrong number of arguments. Again, this is without human intervention. With type annotations/documentation, we can move from most occurrences to all occurrences.
• This same analysis, when applied to public APIs, can provide developers with more informations regarding how their code can be (mis)used.
• At the same level of complexity, analysis can find most erroneous access to fields/methods, suspicious array traversals, suspicious calls to iterators/generators, etc. Again, with type annotations/documentation, we can move from most to all.
• Going a little further in complexity, analysis can find fragile uses of this, uncaught exceptions, etc.

Types as documentation

Public APIs must be documented. This is true in any language, no matter where it stands on the static/dynamic scale. In static languages, one may observe how documentation generation tools insert type information, either from annotations provided by the user (as in Java/JavaDoc) or from type information inferred by the compiler (as in OCaml/OCamlDoc). But look at the documentation of Python, Erlang or JavaScript libraries and you will find the exact same information, either clearly labelled or hidden somewhere in the prose: every single value/function/method comes with a form of type signature, whether formal or informal.

In other words, type information is a critical piece of documentation. If JavaScript developers provide explicit type annotations along with their public APIs, they have simply advanced the documentation, not wasted time. Even better, if such type can be automatically inferred from the source code, this piece of documentation can be automatically written by the type-checker.

Types as QA metric

While disciples of type-checking tend to consider typing as something boolean, the truth is more subtle: it quite possible that one piece of code does not pass type-checking while the rest of the code does. Indeed, with advanced type systems that do not support decidable type inference, this is only to be expected.

The direct consequence is that type-checking can be seen as a spectrum of quality. A code can be seen as failing if the static checking phrase can detect evident errors, typically unbound values or out-of-scope break, continue, etc. Otherwise, every attempt to type a value that results in a type error is a hint of poor QA practice that can be reported to the developer. This yields a percentage of values that can be typed – obtain 100% and get a QA stamp of approval for this specific metric.

Typed JavaScript, in practice

Most of the previous paragraphs are already possible in practice, with existing tools. Indeed, I have personally experienced using JavaScript static type checking as a bug-finding tool and a QA metric. On the first day, this technique has helped me find both plenty of dead code and 750+ errors, with only a dozen false positives.

For this purpose, I have used Google’s Closure Compiler. This tool detects errors, supports a simple vocabulary for documentation/annotations, fails only if very clear errors are detected (typically syntax errors) and provides as metric a percentage of well-typed code. It does not accept JavaScript 1.7 yet, unfortunately, but this can certainly be added.

I also know of existing academic work to provide static type-checking for JavaScript, although I am unsure as to the maturity of such works.

Finally, Mozilla is currently working on a different type inference mechanism for JavaScript. While this mechanism is not primarily aimed at finding errors, my personal intuition is that it may be possible to repurpose it.

What’s next?

I hope that I have convinced you of the interest of investigating manners of introducing static, strong type-checking to JavaScript. In a second part, I will detail how and where I believe that this can be done in Mozilla.

Now, Google has unveiled the specifications of Dart, as well as library documentation. Neither will be sufficient to answer all questions, but they give us an opportunity to look at some of the technical sides of the problem. As a programming language researcher/designer and a member of the web browser community, I just had to spend some quality time with the Dart specifications.

So, how’s Dart? Well, let’s look at it.

What Dart is

Dart is a programming language and a Virtual Machine. As a programming language, Dart positions itself somewhere in the scope between scripting/web development and application development. From the world of application development, Dart brings

• clean concurrency primitives that would feel at home in Scala, Clojure or Erlang – including a level of concurrent error reporting;
• a clean module mechanism, including a notion of privacy;
• a type system offering genericity, interfaces and classes;
• compilation and a virtual machine;
• a library of data structures;
• no eval();
• data structures that do not change shape with time.

From the world of scripting/web development, Dart brings:

• usability in any standards-compliant browser, without any plug-in (although it will work better in a plug-in and/or in Chrome);
• DOM access;
• emphasis on fast start-up;
• a liberal approach to typing (i.e. types are optional and the type system is incorrect, according to the specifications);
• dynamic errors;
• closures (which are actually not scripting/web development related, but until Java 8 lands or until Scala, F# or Haskell gain popularity, most developers will believe that they are).

Where Dart might help

Web development has a number of big problems. I have trolled written about some of them in previous posts, and Dart was definitely designed to help, at least a little.

Security

JavaScript is interpreted, can be written inline in html and supports eval(). By opposition, Dart code is compiled. Dart does not have eval() and Dart code is not written inline in html. Consequently, Dart itself offers a smaller attack surface for cross-site scripting.  Note that Dart can still be used as a component for a XSS targeting the document itself, and that using Dart does not prevent an attacker from using JavaScript to inject XSS in the page.

Safety and Code Hygiene

Out-of-the-box, JavaScript does not offer any static or hybrid typing. Dart offers (optional, hybrid) typing. This is a very useful tool for helping developers and developer groups find errors in their code quickly.

JavaScript offers prototype-based object-oriented programming, without explicit private methods/fields. By opposition, Dart offers modules, classes (with support for private methods/fields) and interfaces. Again, very useful for providing abstractions that do not leak [too much].

For historical reasons, JavaScript offers weird and error-prone scoping and will let developers get away without realizing that they are dereferencing undefined variables. Dart does away with this. Again, this is a good way to find errors quickly.

Libraries

Out-of-the-box, JavaScript does not provide data structures, or much in the way of libraries. By opposition, Dart provides a few data structures and libraries.

Exceptions

For a long time, JavaScript exceptions were not extensible. Eventually, it became possible to define new kinds of exceptions. However, JavaScript still doesn’t support matching the exception constructor, by opposition to what almost all other programming languages do. Dart makes no exception and allows matching upon the exception constructor. This makes exception-handling a little nicer and debugging exception traces a little more robust.

Concurrency

For a long time, JavaScript did not provide any form of concurrency primitive. Recent versions of JavaScript do offer Workers. Similarly, Dart offers Isolates, with a paradigm very similar to Workers. Where Workers are always concurrent, Isolates can also be made non-concurrent, for better performance at the expense of reactivity. Initialization and error-reporting are also a little different, but otherwise, Isolates and Workers are quite comparable.

Speed

Dart promises better speed than JavaScript. I cannot judge about it.

Niceties

Dart offers “string interpolation” to insert a value in a string. Nice but not life-altering. Also, out-of-the-box, JavaScript DOM access is quite verbose. By opposition, Dart provides syntactic sugar that makes it a little nicer.

Where Dart might hinder

Vendor control/adoption

The single biggest problem with Dart is, of course, its source. To get the VM in the browsers, Google will have to convince both developers and other browser vendors to either reimplement the VM by themselves or use a Google-issued VM. This is possible, but this will be difficult for Google.

The open vehicle for this is to convince developers to us Dart for server-side programming – where Dart will be competing with Java, Scala, C#, F#, Python, JavaScript, Erlang and even Google’s Go – and for client-side programming by getting through JavaScript – which will severely hinder performance, safety and security.

The vendor controlled vehicle will be to integrate the VM in Chrome and Android and encourage developers targeting the Chrome Market and Android Market to use Dart. Some speculate that this is a manner for Google to get rid of the Java dependency on the Android Market. In this case, of course, there will be little competition.

Libraries and documentation

JavaScript has a host of libraries and considerable documentation. I will admit that much of the documentation one may find around the web is not good (hint: use Mozilla’s documentation, it is the only reliable source I have found), but that is still infinitely more than what Dart can provide at the moment.

In other words, for the moment, Dart cannot take advantage of the special effects, the game-building libraries, the streaming libraries, etc. that have been developed for JavaScript. This, of course, is something that Google has the resources to change relatively fast, but, by experience, I can tell that many developers are averse to relearning.

Doing it without Dart

Security

We’re not going to get rid of XSS without some effort, even with Dart. However, making sure that JavaScript offers an attack surface no larger than Dart is easy: forbid eval() and forbid any inline JavaScript. It would be quite easy to add an option to HTML documents to ensure that this is the case. Note that this option remains necessary even if all the code is written in Dart, as Dart does not prevent from injecting JavaScript.

Code Hygiene

Out-of-the-box, JavaScript does not offer any support for static/hybrid typing. However, Google has demonstrated how to add static typing with the Google Closure Compiler and Mozilla has demonstrated how to add hybrid typing with Dynamic Type Inference. Both projects indicate that we can obtain something at least as good as Dart in JavaScript, without altering/reinventing the language.

Out-of-the-box, JavaScript does not offer modules. However, Mozilla has been offering a module system for JavaScript for years and new versions of the language are in the process of standardizing this.

Also, while classes and private fields are probably the least surprising techniques for application developers coming to the web, developers used to dynamic or functional languages know that closures and prototypes are essentially equivalent. So, this is essentially a matter of taste.

Finally, clean, lexical scoping will be welcomed by all developers who know what these words mean and quite a few others. Fortunately, it is also coming to JavaScript with recent versions of the language.

Concurrency

Isolates are nice. Workers are nice. Isolates are a little easier to set-up, so I would like to see an Isolate-like API for Workers. Other than that, they are essentially equivalent.

Speed

So far, Google has always managed to deliver on speed promises with V8, so I would tend to believe them. However, recent improvements in JavaScript analysis also promise to analyze away all the cases that can make JavaScript slower than Java, and I also tend to believe these promises. Consequently, I will venture no guess about it.

Libraries

It is a shame that JavaScript does not come with more libraries. However, many frameworks are available that implement standard data structures and more.

Exceptions

Dart exceptions are a little nicer than JavaScript exceptions, there no doubt about that. However, making JavaScript exceptions as good as Dart exceptions would be quite simple. The only difficulty is getting this improvement into the standard.

Niceties

String interpolations are nice to have, but not really life-altering. If necessary, they can trivially be implemented by a pre-processor. CoffeeScript might already do it, I’m not sure. Adding this to the JS standard might be tricky, for reasons of backwards compatibility, but there is not much to it.

Dart-style DOM access is nice, too. However, adding this to JavaScript would be quite trivial, in particular with next-generation DOM implementations such as dom.js.

The result

I admit that I am a little disappointed. When Dart was announced, I was hoping something truly evolutionary. So far, what I have found out is a nice language, certainly, but not much more. While Dart is definitely better in many aspects than today’s JavaScript, given the current evolution of JavaScript, none of these aspects is a deal-breaker. However, several aspects of Dart (in particular, typing and exceptions) indicate a good direction in which I believe JavaScript should evolve, and I hope that the presence of Dart can get JavaScript standardization moving faster.

If we consider I my opinion, there are three ways that Google can get Dart adopted on the web:

• make it the default choice for Android & Chrome development;
• provide a set of killer libraries for the web, that work on all browsers but are truly usable only with Dart (DirectX anyone? something Cocoa-style, perhaps?);
• spend Google-sized budgets on adoption (PR, marketing, GSoC, open-source projects, etc.).

Nevertheless, for the moment, I will keep far away from Dart and look hopefully at Scala-GWT, WebSharper or Ocsigen.

We can make Bugzilla more newbie-friendly.

edit Thanks for pointing me to mentored bugs, Josh!

The context

Earlier today, a discussion got me thinking about how we can lower the entry barriers for people who want to contribute to Mozilla in a technical or semi-technical way but do not know where to start – by “technical or semi-technical”, I have in mind developers, web developers, translators, and more generally people who need to commit some form of patch to any of the Mozilla subprojects.

There are many things that we can improve, but for this post, I would like to concentrate on Bugzilla tweaks that I believe could be useful. These tweaks come from a simple remark: for all contributors Bugzilla is, in fact, a form of contribution console. Unfortunately, for the moment, I see the following problems:

• Bugzilla doesn’t help newcomers find easy entry points;
• Bugzilla doesn’t guide newcomers towards their first patch;
• Bugzilla doesn’t help old-timers help newcomers.

So, let’s see how we could improve Bugzilla.

Making newcomers visible and encouraging old-timers to help them

I believe that the first thing we need to do is to make sure that newbies are more visible to old-timers. For this purpose, we can provide an account attribute “I’m a beginner, please be gentle”, which appears in the profile, ticked by default, and remains there until unticked. Every message, attachment or patch provided by a newbie is clearly labelled as such.

This label may come with a link to the wiki with suggestions for old-timers wanting to help the newbie.

Similarly, we can encourage old-timers to adopt a newbie. For this purpose, we can provide an account attribute “Adopt a newbie”, with a list of accounts. Every message, attachment or patch provided by an adopted newbie is automatically tracked by the adopter.

Note As far as I understand, these features are already essentially implemented. I’m just proposing a small tweak.

Having Bugzilla tell stories

Bugzilla is a nice place for reporting bugs and discussing how to solve it. I propose a simple tweak to make Bugzilla tell the story of a patch. Let me show you one possible manner of doing this:

Each of these link should help a newcomer on one step towards getting a patch submitted and accepted:

1. “Help me set up for contributing” points to some of the wiki pages we already have containing project-specific setup documentation. For Mozilla coding bugs, this explains how to build Mozilla. For localization bug, this explains how to setup whichever tools are required, etc. This is set up on a per-component basis.
2. “Help me contact the developers” points to the component-specific wiki page with all the contact information (IRC channel, forum, mailing-list, module owner e-mail address, etc.).
3. “Help me find project doc” points to the project-specific wiki page with all the documentation.
4. “Help me submit a contribution” points to a beginner-oriented wiki page with the information on submitting. For mozilla-central, for instance, beginners should be able to find the two lines that will let them export everything they have done as one hg patch.
5. “What’s next” points to a wiki page detailing the submission approval process, the various forums, IRC channels, etc. to which the contributor can participate.

This is for beginners. As you may have seen, there is another toolbar, designed for experienced developers/community members:

This toolbar is designed to keep track of activity on bugs that the user follows:

1. “Unconfirmed” lists the number of UNCO bugs followed by the user.
2. “In progress” lists the number of NEW bugs followed by the user.
3. “Review” lists the number of patches pending review in bugs followed by the user.
4. “Mentoring” lists the unread messages and patches submitted by newbies adopted by the user.

Clicking on each link points to a bugzilla search of the corresponding bugs.

Making easy bugs easy to discover

New contributors need to find good entry points. For this purpose, we have project-specific lists of good first bugs. However, at the moment, the lists I could find are both hard to discover and unmaintained. So, let’s integrate this in Bugzilla. As it turns out, it’s quite easydone already (and better than what I suggested initially).

I suggest adding two tags: “easy” and “average”. Every bug can be annotated as “easy” if a developer believes that it’s a good first bug or “average” if a developer believes that it’s a bug that requires some experience but no architectural overview.

Once this is done, we can link from all our wikis and contributor engagement pages to Bugzilla searches with all the “easy” and all the “average” bugs, both project-wide and by component.

Submission add-on

This one is more of a dream. I believe that, at some point, we could develop an add-on that can be installed directly fro the Bugzilla toolbar, and whose role is to provide a wizard to help users:

• install the appropriate source management tool;
• once the tool is installed, submit a patch for review.

Initially, this add-on covers only mozilla-central. Later, upgrades to this add-on and/or project-specific add-ons may be added for projects not covered by mozilla-central (typically projects hosted on github).

Inspiration: want to do better than (otherwise excellent) github “pull request”

Badges!

• First bug report tagged “new”? Earn a badge!
• First patch submitted? Earn a badge!
• First patch accepted? Earn a badge!
• Same thing for the marks of 10, 50, 100 …

Any thoughts?

Ok, this lists the tweaks that I can think of to improve newbie experience with contributions. What do you think? Any way we can make this better?

Three years ago, I was an academic researcher, studying the best manner to certify binaries uploaded to clouds and ensure their security. Less than three weeks ago, I was working at MLstate, leading the R&D team and striving to change the face of web programming languages and web security. Today, I am part of Mozilla, as a proud member of the Performance team, where we take steps, each day, to make Firefox, Thunderbird and the Mozilla platform faster and more responsive.

We, at Mozilla, have considerable work ahead of us. Because the web needs to be defended, kept open and compatible with privacy, and, on some of these fronts, only Mozilla stands.

An exciting time, indeed – and an exciting way of doing my part!

I will try and blog about ongoing developments as soon as I find a minute.

Edit It seems that I misredacted some of the announces that were sent. Just to clarify, I am not the leader of the Performance team. That person is Taras Glek.