Here is an example using the Unix libc:

// Open the C library
let libcCandidates = [
  'libSystem.dylib',// MacOS X
  'libc.so.6',      // Linux
  'libc.so'         // Android, B2G
];
let libc;
for each(let candidate in libcCandidates) {
  libc = ctypes.open(candidate);
  if (libc) {
    break;
  }
}

// Import some functions from libc
let open = libc.declare("open", ctypes.default_abi,
  /*return int*/ ctypes.int,
  /*const char* path*/ctypes.char.ptr,
  /*int oflag*/ ctypes.int
  /*int mode*/ ctypes.int);
let read = libc.read("read", ctypes.default_abi,
  /*return ssize_t*/ ctypes.ssize_t,
  /*int fildes*/ ctypes.int,
  /*void *buf*/ ctypes.voidptr_t,
  /*size_t nbytes*/ ctypes.size_t);
let close = libc.read("close", ctypes.default_abi,
  /*return int*/ ctypes.int,
  /*int fd*/ ctypes.int);

// Now use them
let myfile = open("/etc/passwd", 0, 0);
if (myfile == -1)
  throw new Error("Could not open file");
// ...

If you are familiar with XPConnect, the mechanism generally used in the Mozilla Platform for letting JavaScript and C++ interact, you can see that using js-ctypes to call native code directly is much nicer than adding a C++ XPCOM/XPConnect layer. From what I hear, it seems to be also much faster, as XPConnect needs to perform expensive magic to ensure that memory is properly passed between JavaScript and C++. In addition, this selfsame memory magic now prevents XPConnect from being executed from threads other than the main thread, which makes js-ctypes the only manner of doing any system access from worker threads.

Now, js-ctypes nicely solves the issue of calling native code from JavaScript. However, JavaScript and C are very different languages, with very different paradigms, so getting them to coexist requires a little more than simply the ability to place calls or convert values. In particular, C has:

• manual resource management (memory must be released, file descriptors must be closed, locks must be released, etc.);
• no language-level mechanism for error management (a task smaller than a process cannot be killed because of an error).

By contrast, Javascript has:

• automated memory management, but no support for managing automatically resources other than memory (no user-level finalization or scoped resources mechanism);
• several language/vm-level mechanisms that can kill a task in non-trivial manners (exceptions, “this script is busy”, etc.)

Unfortunately, putting all of this together makes it quite difficult to write JavaScript code that manipulates C resources without leaking. Such leaks can cause both performance issues (memory leaks, in particular, tend to slow down the whole system) and hard-to-track errors (leaking file descriptors can prevent the application from opening any new file, or, under Windows, can prevent the application from reopening some files that were improperly closed, while leaking locks can completely freeze an application).

Introducing C data finalization

For this reason, we have recently added a new features to js-ctypes, designed to add automated resource management to JavaScript: C data finalization.

Specifying a finalizer is simple:

function openfile(path, flags, mode) {
  let fd = open(path, flags, mode);
  if (fd == -1) {
    throw new Error("Could not open file " + path);
  }
  return ctypes.CDataFinalizer(fd, close);
}

What this code does is ensure that, whenever the file descriptor is garbage-collected, function close is called, releasing the C resources represented by that file descriptor. This value is C data with a finalizer, aka CDataFinalizer.

You can use it just as you would use the C data through js-ctypes:

let myfile = openfile("/etc/passwd", 0, 0);
let result = read(myfile, myarray, 4096); // Read some data
// Wherever required, |myfile| is automatically converted to
// the underlying integer value.
// Once |myfile| has no reference, it will (eventually) be
// closed.

It is, of course, possible (and strongly recommended) to close the file manually to ensure that resources are immediately available for the process and the rest of the system:

let myfile = openfile("/etc/passwd", 0, 0);
// ...
// ... do whatever you wish to do with that file
let result = myfile.dispose(); // This calls |close|.

// From this point, |myfile| cannot be converted to the underlying
// integer value anymore. Any attempt to do so will raise an
// exception.

Or, an equivalent but more verbose solution, using forget:

let myfile = openfile("/etc/passwd", 0, 0);
// ...
// ... do whatever you wish to do with that file
let fd = myfile.forget();
// From this point, |myfile| cannot be converted to the underlying
// integer value anymore. Any attempt to do so will raise an
// exception.
let result = close(fd);

This mechanism is, of course, not restricted to file descriptors. It has been used with success to other data structures, including malloc-allocated strings.

Details and caveat

JavaScript does not feature finalization and might never do so. There are good reasons for this: finalization considerably complicates the garbage-collector and introduces the possibility of subtle bugs and leaks that the various JS implementors do not want to inflict to their users (if you are curious, two of the main problems are resurrection of dead references and finalization of cyclic data structures).

Consequently, C data finalizers are not full-featured finalizers. Indeed, the main limitation of C data finalizers is that its first argument must be a C value and its second argument must be a pointer to a C function – for the above mentioned reasons, letting users specify any JavaScript function as a finalizer would open a can of worms that nobody really wants to see crawling around.

Also, before using a finalizer, you should be aware that JavaScript garbage-collection is not necessarily deterministic – during the testing phase of CDataFinalizer, we have encountered memory errors caused by developers (ok, I will confess, that was me, sorry guys) making invalid assumptions about just when values would be garbage-collected. Let me emphasize this: any hypothesis you make about when a value is finalized is bound to be regularly false. In other words, C data finalizers should be used as a last line of defense, not as the default mechanism for recovering resources.

Still, C data finalizers are a powerful mechanism that make manipulation of C values with JavaScript much more reliable. Indeed, it is one of the core mechanisms used pervasively by the OS.File library.

edit As per Steve Fink’s suggestion, I have emphasized that users should not rely on the behavior of garbage-collection/finalization, and clarified the can of worms.

A small class hierarchy has been added to MFBT, the “Mozilla Framework Based on Templates” which contains some of the core classes of the Mozilla Platform. This hierarchy introduces general-purpose and specialized classes for scope-based resource management. When it applies, Scope-based resource management is both faster than reference-counting and closer to the semantics of the algorithm, so you should use it

The codebase of Mozilla is largely written in C++. While C++ does not offer any form of automatic memory management, the (sometimes scary) flexibility of the language has allowed numerous projects to customize the manner in which memory and other resources are managed, and Mozilla is no exception. Largely, the Mozilla C++ codebase uses reference-counting, to provide automatic memory management in most cases.

While reference-counting is quite imperfect, and while future versions of Mozilla might possibly use other forms of memory management, it is also a very useful tool for such a large codebase. However, in some cases, reference-counting is just too much. Indeed, in a number of simple cases, we prefer the simpler mechanism of scope-based resource management, that is both more predictable, faster and more resource-efficient – at the cost of not being able to scale up to the more complex cases for which reference-counting or even more powerful mechanisms become much more suited.

Scope-based resource management is designed to handle resources that should be cleaned-up as soon as you leave a given scope (typically, the function), regardless of how you leave it (by reaching the end, with a break, a return or an exception).

The following extract illustrates the use of scoped resource allocation:

// returns true in case of success, false in case of error
bool copy(const char* sourceName, const char* destName, size_t bufSize) {
   ScopedFD source(open(sourceName, O_RDONLY));
   if (source.get() == -1) return false;

   ScopedFD dest(open(destName, O_WRONLY|O_CREAT, 0600));
   if (dest.get() == -1) return false;
     // source is closed automatically

   ScopedDeleteArray buf(new char[bufSize]);
   if (buf.get() == NULL) return false;
     // source, dest are closed automatically

   while (true) {
     const int bytesRead = read(source.get(), buf.rwget(), bufSize);
     if (bytesRead == 0) break;
     if (bytesRead == -1) return false;
       // source, dest, buf are cleaned-up

     const int writePos = 0;
     while (writePos < bytesRead) {
       const int bytesWritten = write(dest.get(), buf.get(),
                                      bytesRead - writePos);
       if (bytesWritten == -1) return false ;
         // source, dest, buf are cleaned-up
       writePos += bytesWritten;
     }
   }

   return true;
      // source, dest, buf are cleaned-up
}

As you can see, the main point of these scope-based resource management classes is that they are cleaned up automatically both in case of success and in case of error. In some cases, we wish to clean up resources only in case of error, as follows:

// returns -1 in case of error, the destination file descriptor in case of success
int copy(const char* sourceName, const char* destName, size_t bufSize) {
   ScopedFD source(open(sourceName, O_RDONLY));
   if (source.get() == -1) return -1;

   ScopedFD dest(open(destName, O_WRONLY|O_CREAT, 0600));
   if (dest.get() == -1) return -1;
      // source is closed automatically

   ScopedDeleteArray buf(new char[bufSize]);
   if (buf.get() == NULL) return -1;
      // source, dest are closed automatically

   while (true) {
     const int bytesRead = read(source.get(), buf.rwget(), bufSize);
     if (bytesRead == 0) break;
     if (bytesRead == -1) return -1;
      // source, dest, buf are cleaned-up

     const int writePos = 0;
     while (writePos < bytesRead) {
       const int bytesWritten = write(dest.get(), buf.get(),
                                      bytesRead - writePos);
       if (bytesWritten == -1) return -1 ;
        // source, dest, buf are cleaned-up
       writePos += bytesWritten;
     }
   }

   return dest.forget();
   // source and buf are cleaned-up, not dest
}

While both examples could undoubtedly be implemented with reference-counting or without any form of automated resource management, this would either make the source code much more complex and harder to maintain (for purely manual resource management) or make the executable slower and less explicit in terms of ownership (for reference-counting). In other words, scoped-based resource management is the right choice for these algorithms.

Now, the Mozilla codebase has offered a few classes for scope-based resource management. Unfortunately, these classes were scattered throughout the code, some of them were specific to some compilers, and they were generally not designed to be reusable.

We have recently starting consolidating these classes into a simple and extensible hierarchy of classes. If you need them, you can find the root of this hierarchy, as well as the most commonly used classes, on mozilla-central, as part of the MFBT:

• ScopedFreePtr<T> is suited to deallocate C-style pointers allocated with malloc;
• ScopedDeletePtr<T> is suited to deallocate C++-style pointers allocated with new;
• ScopedDeleteArray<T> is suited to deallocate C++-style pointers allocated with new[];
• root class Scoped<Trait> and macro SCOPED_TEMPLATE are designed to make it extremely simple to define new classes to handle other cases.

For instance, class ScopedFD as used in the above examples to close Unix-style file descriptors, can be defined with the following few lines of code:

struct ScopedFDTrait
{
public:
  typedef int type;
  static type empty() { return -1; }
  static void release(type fd) {
    if (fd != -1) {
      close(fd);
    }
  }
};
SCOPED_TEMPLATE(ScopedFD, ScopedFDTrait);

So, well, if you need scoped-based resource management, you know where to find it!

I will blog shortly about the situation in JavaScript.

Save as .epub (Firefox add-on)

(Kevin CORRE, Benjamin ROCHER, Elie AHUMA, Sylvestre ANTOINE – Université d’Orléans, MIAGE + IRAD)

Objective Add the following feature to Firefox, as an add-on: Save a page and its resources as one file, using open standard .epub. This open-standard file can then be transferred to just about any device, edited with LibreOffice, etc.

Current status I have seen version 0.1 on addons.mozilla.org, although it seems to have disappeared in the meantime. I suppose it still needs a little polish, but, hey, it’s a good start.

Follow this project This project lives on github.

Detect use of the wrong account (Thunderbird add-on)

(Baptiste MEYNIER, Johan JANS, Maxime DENOYER, Mustapha OUCHEIKH – Université d’Orléans, MIAGE + IRAD)

Objective Add the following feature to Thunderbird, as an add-on: Detect that a message is being sent to a correspondant using the wrong account (e.g. using a professional account for a personal message or a personal account for a professional message).

Current status The project has reached an important milestone and has become usable. I have the impression that no version has been released, which is a shame, but it already offers some useful features. I really hope that the students continue the project and turn it into a complete add-on.

Follow this project This project lives on github (currently on a non-master branch).

Simplify the addition of several alarms for the same event in Lightning (Thunderbird add-on)

(Loïc LE MÉRO aka Morkai – Université d’Évry, MIAGE 2)

Objective Lightning offers the ability to add several alarms for the same event (e.g. 1 day before then 15 minutes before). Improve the user interface to make this more discoverable.

Current status Code is complete, what is left is to submit it upstream.

Follow this project This project lives on Bugzilla.

Extend Lightning alarms (Thunderbird add-on)

(Anto DOMINIC PAUL – Université d’Évry, MIAGE 2)

Objective Lightning offers the ability to attach alarms to an event. Extend this feature to make it possible to play a music or execute a script when the alarm is triggered.

Current status I have seen a working version. Not 100% about security, but it looks very promising. Guys, I hope you can finish that alarm, I want to use it.

Follow this project This project lives on Bugzilla.

Handle resources in Lightning events (Thunderbird add-on)

(Julien LACROIX – Université d’Évry, MIAGE 2)

Objective Add the ability to attach resource requirements to events: a picnic requires food (one resource), drinks (one resource), cutlery (one resource), etc… Who will bring them? Also, add the ability to attach a geolocation to events, to help finding the way. Who brings the beer?

Current status It works! And it was submitted as an add-on on for Thunderbird.

Follow this project This project lives on github.

Remind me that I need to reply within 24h/remind me that I expect a reply within 24h (Thunderbird add-on)

(Vincent LEGUEVEL, Mickael MAINGE – Université d’Évry, MIAGE 2)

Objective Add the ability to mark a message as “I need to answer within …” / “I expect an answer within …”. Nag the user as long as she hasn’t sent or received the reply.

Current status A little disappointed. A subset of the features is here, but as far as I can tell, not unified as one single add-on.

Follow this project This project lives on two Bugzilla bugs: need to send / expect to receive.

Let us see what has changed.

Save as .epub (Firefox add-on)

(Kevin CORRE, Benjamin ROCHER, Elie AHUMA, Sylvestre ANTOINE – Université d’Orléans, MIAGE + IRAD)

Objective Add the following feature to Firefox, as an add-on: Save a page and its resources as one file, using open standard .epub. This open-standard file can then be transferred to just about any device, edited with LibreOffice, etc.

Current status Unfortunately, I have not had contact from the students in a few weeks. Looking at the source code, I have the impression that it is rather advanced, and that they are only a few hours of work away from a first version. I am counting on you guys! Also, do not forget that I want you to communicate publicly.

Follow this project This project lives on github.

Detect use of the wrong account (Thunderbird add-on)

(Baptiste MEYNIER, Johan JANS, Maxime DENOYER, Mustapha OUCHEIKH – Université d’Orléans, MIAGE + IRAD)

Objective Add the following feature to Thunderbird, as an add-on: Detect that a message is being sent to a correspondant using the wrong account (e.g. using a professional account for a personal message or a personal account for a professional message).

Current status Still experimenting. A first version of the user interface is in place, some experiments on how to react to the “send” event, a database (empty so far, afaict), but no learning of accounts yet. Hey guys, I hope you manage to complete that project before the deadline.

Follow this project This project lives on github.

Simplify the addition of several alarms for the same event in Lightning (Thunderbird add-on)

(Loïc LE MÉRO aka Morkai – Université d’Évry, MIAGE 2)

Objective Lightning offers the ability to add several alarms for the same event (e.g. 1 day before then 15 minutes before). Improve the user interface to make this more discoverable.

Current status I have seen a working demo. Next step is submitting the patch to Lightning. Keep up the good work!

Follow this project This project lives on Bugzilla.

Extend Lightning alarms (Thunderbird add-on)

(Anto DOMINIC PAUL – Université d’Évry, MIAGE 2)

Objective Lightning offers the ability to attach alarms to an event. Extend this feature to make it possible to play a music or execute a script when the alarm is triggered.

Current status Unclear. I have not seen any working code yet. Anto?

Follow this project This project lives on Bugzilla.

Handle resources in Lightning events (Thunderbird add-on)

(Julien LACROIX – Université d’Évry, MIAGE 2)

Objective Add the ability to attach resource requirements to events: a picnic requires food (one resource), drinks (one resource), cutlery (one resource), etc… Who will bring them? Also, add the ability to attach a geolocation to events, to help finding the way. Who brings the beer?

Current status I understand that we now have working demos. I do not know about packaging as an add-on yet.

Follow this project This project lives on github.

Remind me that I need to reply within 24h/remind me that I expect a reply within 24h (Thunderbird add-on)

(Vincent LEGUEVEL, Mickael MAINGE – Université d’Évry, MIAGE 2)

Objective Add the ability to mark a message as “I need to answer within …” / “I expect an answer within …”. Nag the user as long as she hasn’t sent or received the reply.

Current status I have seen demo of a small subset of the features. It’s a good start, guys, but I certainly hope that you will deliver something complete by the deadline.

Follow this project This project lives on two Bugzilla bugs: need to send / expect to receive.

Reading it, I wonder. Here is article 11, for instance:

« Without prejudice to its law governing privilege, the protection of confidentiality of information sources, or the processing of personal data, each Party [= each country] shall provide that, in civil judicial proceedings concerning the enforcement of intellectual property rights, its judicial authorities have the authority, upon a justified request of the right holder, to order the infringer or, in the alternative, the alleged infringer, to provide to the right holder or to the judicial authorities, at least for the purpose of collecting evidence, relevant information as provided for in its applicable laws and regulations that the infringer or alleged infringer possesses or controls. Such information may include information regarding any person involved in any aspect of the infringement or alleged infringement and regarding the means of production or the channels of distribution of the infringing or allegedly infringing goods or services, including the identification of third persons alleged to be involved in the production and distribution of such goods or services and of their channels of distribution. »

Do I understand correctly that, to comply with such orders:

• a library must keep watch over its uers, so as to be able to identify those who might be implied in alleged infringements?
• a university must keep watch over its professors, so as to be able to identify those who might be implied in distributing documents and exceeding fair use?
• … or a highschool? or a kindergarten?
• … and no element of proof is required from the right holder?

Have I mentioned that I am a little worried by ACTA?

Du coup, je lis l’article 11 et je m’interroge :

« Without prejudice to its law governing privilege, the protection of confidentiality of information sources, or the processing of personal data, each Party [= chaque pays] shall provide that, in civil judicial proceedings concerning the enforcement of intellectual property rights, its judicial authorities have the authority, upon a justified request of the right holder, to order the infringer or, in the alternative, the alleged infringer, to provide to the right holder or to the judicial authorities, at least for the purpose of collecting evidence, relevant information as provided for in its applicable laws and regulations that the infringer or alleged infringer possesses or controls. Such information may include information regarding any person involved in any aspect of the infringement or alleged infringement and regarding the means of production or the channels of distribution of the infringing or allegedly infringing goods or services, including the identification of third persons alleged to be involved in the production and distribution of such goods or services and of their channels of distribution. »

Ai-je bien compris que, pour pouvoir obéir à ce type d’injonction :

• une bibliothèque doit surveiller ses empruntants, pour pouvoir identifier ceux qui pourraient être impliqués dans de la photocopie de livres ?
• une université doit surveiller ses enseignants, pour pouvoir identifier ceux qui pourraient distribuer des polycopiés sans autorisation ?
• … ou un lycée, un collège, une maternelle ?
• … ainsi que les intervenants invités le temps d’un cours ou d’un séminaire ?
• … le tout sur simple soupçon ?

Ai-je mentionné qu’ACTA m’inquiète quelque peu ?

Note to the students I can only blog if you tell me your current status!

Note to other students Do you want to take part in an open-source / open web project? Then feel free to contact me, I will be glad to help you or to introduce you to someone who might. You can find me on Tweeter (ImYoric), by e-mail, on mozilla.com (dteller), or by IRC, on irc.mozilla.org, channel #introduction (Yoric).

Save as .epub (Firefox add-on)

(Kevin CORRE, Benjamin ROCHER, Elie AHUMA, Sylvestre ANTOINE – Université d’Orléans, MIAGE 2)

Objective Add the following feature to Firefox, as an add-on: Save a page and its resources as one file, using open standard .epub. This open-standard file can then be transferred to just about any device, edited with LibreOffice, etc.

Current status Early stage of coding. The first items of the user interface are in place, as well as some experiments regarding how to create an .epub file.

Follow this project This project lives on github.

Detect use of the wrong account (Thunderbird add-on)

(Baptiste MEYNIER, Johan JANS, Maxime DENOYER, Mustapha OUCHEIKH – Université d’Orléans, MIAGE 2)

Objective Add the following feature to Thunderbird, as an add-on: Detect that a message is being sent to a correspondant using the wrong account (e.g. using a professional account for a personal message or a personal account for a professional message).

Current status Early stage of coding. The first items of the user interface are in place, as well as some experiments regarding how to react to the user clicking on “send”.

Follow this project This project lives on github.

Simplify the addition of several alarms for the same event in Lightning (Thunderbird add-on)

(Loïc LE MÉRO aka Morkai – Université d’Évry, MIAGE 2)

Objective Lightning offers the ability to add several alarms for the same event (e.g. 1 day before then 15 minutes before). Improve the user interface to make this more discoverable.

Current status (unknown, waiting for Loïc to tell me).

Follow this project This project lives on Bugzilla.

Extend Lightning alarms (Thunderbird add-on)

           (Anto DOMINIC PAUL – Université d’Évry, MIAGE 2)

Objective Lightning offers the ability to attach alarms to an event. Extend this feature to make it possible to play a music or execute a script when the alarm is triggered.

Current status (unknown, waiting for Arno to tell me).

Follow this project This project lives on Bugzilla.

Handle resources in Lightning events (Thunderbird add-on)

(Julien LACROIX – Université d’Évry, MIAGE 2)

Objective Add the ability to attach resource requirements to events: a picnic requires food (one resource), drinks (one resource), cutlery (one resource), etc… Who will bring them? Also, add the ability to attach a geolocation to events, to help finding the way. Who brings the beer?

Current status Early stages of coding. First prototype of geolocation implemented, and work on requirements added.

Follow this project This project lives on github.

Remind me that I need to reply within 24h/remind me that I expect a reply within 24h (Thunderbird add-on)

(Vincent LEGUEVEL, Mickael MAINGE – Université d’Évry, MIAGE 2)

Objective Add the ability to mark a message as “I need to answer within …” / “I expect an answer within …”. Nag the user as long as she hasn’t sent or received the reply.

Current status (unknown, waiting for Vincent or Mickael to tell me more)

Follow this project This project lives on two Bugzilla bugs: need to send / expect to receive.

Today is an international day of fighting against web censorship.

Against SOPA (in the US), Hadopi (in France), and all the bills that assume that freedom of expression and freedom of information can be cast aside whenever they come in the way of profit.

Learn more.

Mozilla is working with numerous educators, professors and researchers across the world, both to bring open-source, the open web and web technologies into the classroom, and to bring the contributions of students and their mentors to the world. You can be a part of this, and your field does not have to be Computer Science.

Mozilla in the classroom – and beyond

Mozilla? Certainly, you have heard about us. Mozilla is best known for its work on the Firefox web browser and the Thunderbird mail client but that is not all there is to Mozilla, far from it. The activities of Mozilla reach largely beyond a few software products. The objective of the Mozilla project is to promote an open, participative web. For this purpose, we have developed and promoted Firefox, Thunderbird, open standards, add-ons, and countless experiments and innovations. For this purpose, we want to work with you.

This is a call for academics and students. We would like to work with you, with your classrooms and with your labs.

Mozilla is already present in numerous classrooms and research laboratories around the world, thanks to collaboration of numerous educators, professors, researchers and students. We deliver courses, we provide teaching material, but most importantly, we provide opportunities for students and their mentors: the opportunity to make impacting contributions to the web, to open-source and sometimes to society, the opportunity to tinker with technologies used by hundreds and millions of people, and to deliver research results to an audience consisting in the the whole web, and also, most importantly, the opportunity to learn and teach as part of an exciting, participatory and fun community.

So, whether you are an academic or a student, if you want to work alongside us, please contact contact us.

Topics of interest

At its core, Mozilla is an open-source community. Our work at Mozilla covers many topics. Some of these topics are extremely technical, some do not even involve a computer.

Here are but a few.

Open-source (Software Engineering, IT, CS Research, UX)…

• client-side programming in C, C++, JavaScript, Java;
• server-side programming in Python, JavaScript;
• concurrent and system-oriented programming;
• implementation of programming languages;
• static analysis, rewriting and semantics or programming languages;
• instrumentation, dynamic analysis;
• distributed, scalable databases;
• distributed, scalable web services;
• client-side web security;
• server-side web security;
• network security;
• network maintenance;
• applied cryptography;
• implementation of cryptographic protocols;
• establishing and maintaining mirrors;
• service-oriented architectures;
• low-level, 2d graphics programming;
• low-level, 3d graphics programming;
• implementing typography and layouts;
• defining typography and layouts;
• developing user interfaces;
• designing user interfaces;
• web applications;
• video game development;
• build systems;
• tools for developers;
• data mining from bugzilla, survey results, etc;
• implementing web standards;
• designing candidate web standards;
• privacy analysis;
• operating system development;
• …

… community (Internationalization, Events, Representing Mozilla)

• internationalizing Mozilla and other community applications;
• internationalizing Mozilla and other community websites;
• organizing local or global open-source events;
• … or web events;
• … or technology events;
• participating to local or global events;
• teaching the open web to non-technical audiences;
• teaching the open web to technical audiences;
• teaching web technologies;
• organizing local communities;
• spreading Mozilla and other open-source technologies;
• providing technical support in the local language;
• representing Mozilla in a highschool/university/…;
• communicating with local media;
• analyzing industry news;
• advocating open web or open-source to local companies, organizations and communities;
• interacting with other technology communities;
• writing articles on open-source and the open web;
• … or filming documentaries;
• … or anything else you can think of!

What the Mozilla community can do for you

On some topics, Mozilla can provide you with teaching/learning material or get you in touch with Mozillians who would love to come to your classroom and deliver interventions or lectures.

More importantly, we can help shape student projects and we can contribute to mentoring them. Do not limit yourself – with enough dedication, student projects can change the world.

If you are interested, please get in touch.

You can reach me by e-mail at mozilla.com, as dteller.

One of the key components of OS.File is the Schedule API, a tiny yet powerful JavaScript core designed to considerably simplify the development of asynchronous modules. In this post, we introduce the Schedule API.

Introduction

In a previous post, I introduced OS.File, a Mozilla Platform library designed make the life of developers easier and to help them produce high-performance, high-responsiveness file management routines.

In this post, I would like to concentrate on one of the core items of OS.File: the Schedule API. Note that the Schedule API is not limited to OS.File and is designed to be useful for all sorts of other modules.

Overview

One of the most important aspect of any user-facing application is its responsiveness. Contrary to intuition, however, responsiveness is much less a matter of speed than one of tempo: not necessarily attempting to perform tasks as fast as possible, but rather attempting to perform them in a manner that does not disrupt the user experience. In the Mozilla Platform, disrupting the user experience is synonymous with freezing the user interface.

Avoiding this is the art of asynchronous, non-blocking operations, and this is the reason we built the Schedule API.

The Schedule API is a tiny JavaScript module designed to simplify considerably the development of asynchronous code. It is based on Joseph Walker’s excellent Promises API [1] and it lies at the core of OS.File’s asynchronous operations.

Before detailing the full API, let us start with a few simple examples. Consider a common situation: some treatment needs to be done soon, but not immediately, as this would otherwise disrupt the user experience. For this purpose, JavaScript offers function setTimeout, which schedules the execution of the treatment to later:

function doItLater() {
  window.setTimeout(function() {
    var result = fibonacci(10);
  });//Schedule the execution for later
}//Wait, how do we get the result?

However, setTimeout offers no help for informing the rest of the code that the execution is complete. While there are many solutions to this problem – I am sure that dozens can be found on StackOverflow – perhaps the most developer-friendly is that of promises:

function doItLater() {
  var promise = new Promise();
  window.setTimeout(function() {
    var result = fibonacci(10);
    promise.resolve(result);
  });//Schedule the execution for later
  return promise;//And return immediately a promise.
}

With this code, we have introduced the promise of a future result. Once we have obtained a promise, we can simply add a reaction that will be triggered once the promise is resolved:

doItLater().then(function(result) {
   alert("I have just finished computing Fibonacci: "+result);
})

Or we can chain promises to obtain other promises:

var promise2 = doItLater().chainPromise(function(result) {
   return sieve(result);
})

Note that these two extracts are compatible with each other. If we write both, we will be informed once the first treatment is complete and a second treatment will also take place immediately, one whose result is promised as promise2.

Unfortunately, a few problems appear with our code. Firstly, window.setTimeout is unfortunately not available in all contexts. While workarounds are available for non-UI modules and background threads, writing any reusable asynchronous is a somewhat painful that can quickly lead to unreadable code. Chaining such treatments becomes even worse.

The second issue is that our call fibonacci(10) might well be quite long itself and might very well disrupt the user experience if we do nothing about it.

Let us start by addressing the first issue, with the Schedule module:

Components.utils.import("resource://gre/modules/schedule.jsm");

Function Schedule.soon replace most uses of window.setTimeout:

function doItLater() {
  return Schedule.soon(function() {
    return fibonacci(10);
  });
}

Or, more concisely:

function doItLater() {
  return Schedule.soon(fibonacci, 10);
}

Or, to delay the execution of the task by 100ms:

function doItLater() {
  return Schedule.soon({delay: 100}, fibonacci, 10);
}

The Schedule API also extends Promises with a new method soon:

var promise2 = doItLater().soon(function(result) {
   return sieve(result);
})

Just like chainPromises, soon adds new treatments that are triggered only once the promise is resolved. However, and by opposition to chainPromises, these treatments are executed later, without disrupting the user experience.

With the combination of Schedule and Promises, it is easy to see how we could write a version of the Fibonacci function that is computed progressively:

//Compute the fibonacci function of n, as a background task
function fibo(n) {
  if (n == 1 | n == 0) {
    return Schedule.alreadyComplete(1);
    //Promise a result and deliver it immediately
  } else {
    var tasks = [];//Launch two tasks
    tasks.push(fibo(n - 1));
    tasks.push(fibo(n - 2));
    return Promise.group(tasks).soon(function(results) {
       return results[0] + results[1];
    })
  }
}

Nice, readable and non-disruptive. Plus, for debugging purposes, or if you wish to add some visual feedback, you can attach additional observers to promises.

The API

So far, we have made use of the following functions of the Promises API:

• Promise;
• Promise.resolve;
• Promise.prototype.chainPromise;
• Promise.prototype.then;
• Promise.group;

I am trying to convince Joseph to blog about this very nice API, so you should be able to learn more about it on his blog, one of these days.

We have also made use of the following functions of module Schedule:

• Schedule.soon;
• Promise.prototype.soon;
• Schedule.alreadyComplete.

While these methods can already be very useful to help you write or rewrite asynchronous code, the Schedule API goes a little bit further and introduces Scheduling Categories and Rendez-vous points.

Categories

While interleaving tasks is a good way to ensure that all tasks progress, some tasks need a little tighter control. Typically, the result of successive operations on a file depend on their order of execution – closing a file before reading from it is generally not a very good idea. Similarly, for performance reasons, an application should generally avoid reading from several files simultaneously, as this can slow down not only the application but the whole system.

In the Schedule API, this is materialized as Scheduling Categories. When several tasks are placed in the same category, any call to Schedule.soon or Promise.prototype.soon guarantees that these tasks will be executed in the order in which they were added and that only one of these task is executed at a time. This does not prevent tasks from other categories from being interleaved.

If, for instance, we wish to serialize the computation of Fibonacci’s function, we may declare a category as follows:

var Fibo = new Schedule.Category("Fibonacci's function");

and replace the final function call of the previous listing with

    soon({category: Fibo}, function(results) {
       return results[0] + results[1];
    })

Thus categorized, Fibonacci’s functions will take less resources to compute, and will be even nicer to the rest of the application and the system, though the total computation will probably last a little longer.

Rendez-vous

One of the original reasons for which we started developing the Schedule API was to simplify asynchronous initialization on the Mozilla Platform and ensure that whichever parts of initialization were not strictly required at start-up could be simply and safely postponed.

For this purpose, the Schedule API provides one additional function: Schedule.on. This function lets developers register code to be executed only once some condition is met, as follows:

Schedule.on("startup-complete").soon(delayedInitializer);

With this line, function delayedInitializer will be executed only once “startup-complete” is resolved. Note that, by opposition to DOM-style events, if “startup-complete” has already been resolved, delayedInitializer will be immediately scheduled for execution. This simple change solves a number of refactoring headaches.

By the way, Schedule.on("startup-complete") is just a promise. To resolve it, simply call resolve:

Schedule.on("startup-complete").resolve();

And that’s it. Welcome to the Schedule API!

Status

Everything that was presented in this post is implemented, unit-tested – although not battle-tested – used in OS.File, and currently under review. There is no ETA yet, due to dependencies, but I hope that we will be able to push it soon.

There is one important feature that I wish to add at some point: multi-threading. While there are limitations to what we can transmit between threads, there is no reason for which we could not schedule code for execution not on the main thread, but on a pool of workers. There will be limitations, but I believe that I have a pretty good idea of how to implement this, and if I succeed, this can expand considerably our horizons. I have small chunks of code written, but no ETA for a prototype.

Wrapping this up

The Schedule API is a simple module (only 5 public functions) designed to make our life easier when writing asynchronous code or refactoring code for asynchronicity.

I hope that you will find it useful.

[1] Thanks to Panos for point this code to me. I had previously reimplemented my own brand of Promises, but having two of them in the Mozilla Platform codebase was simply pointless.

edit Clarified the fact that calling fibonacci(10) in a setTimeout will not magically make everything non-disruptive. Thanks for the feedback, Axel.