Asynchronous file I/O for the Mozilla Platform

October 3, 2012 § 18 Comments

The Mozilla platform has recently been extended with a new JavaScript library for asynchronous, efficient, file I/O. With this library, developers of Firefox, Firefox OS and add-ons can easily write code that behave nicely with respect to the process and the operating system. Please use it, report bugs and contribute.

Off-main thread file I/O

Almost one year ago, Mozilla started Project Snappy. The objective of Project Snappy is to improve, wherever possible, the responsiveness of Firefox, the Mozilla Platform, and now, Firefox OS, based on performance data collected from volunteer users. Thanks to this real-world performance data, we have been able to identify a number of bottlenecks at all levels of Firefox. As it turns out, one of the main bottlenecks is main thread file I/O, i.e. reading from a file or writing to a file from the thread that also runs most of the code of Firefox and its add-ons.

« Read the rest of this entry »

Getting file information with OS.File

July 31, 2012 § 2 Comments

OS.File keeps gaining new features.

Today, let me show you OS.File.stat and OS.File.prototype.stat, two data structures used to get information on a file, such as its size, its creation date or its nature.

How to

There are two ways to get information on a file.

The first technique is to simply call OS.File.stat with the path of the file you wish to open:

// File sessionstore.js in the user’s profile directory
let path = OS.Path.join(OS.Constants.Path.profileDir, "sessionstore.js");
let stat = OS.File.stat(path)

This returns a OS.File.Info object containing all the interesting information on the file.

if (stat.isDir) {
  dump("This is a directory\n");
} else if (stat.isSymLink) {
  dump("This is a symbolic link\n");
}
dump("The file contains " + stat.size + "bytes\n”);
dump("The file was created at " + stat.creationDate + "\n");
dump("The file was last accessed at " + stat.lastAccessDate + "\n");
dump("The file was last modified at " + stat.lastModificationDate + "\n");

Additionally, under Unix, some security information is available:

if ("unixOwner" in OS.File.Info.prototype) {
  dump("The file belongs to user " + stat.unixOwner +
    " in group " + stat.unixGroup +
    " and has mode " + stat.unixMode);
}

That’s it.

The second technique will let you get information on a file that is already opened:

let file = OS.File.open(path);
let stat = file.stat();

The result is exactly the same. Of course, file.stat() is faster if you have already opened the file, while OS.File.stat(path) if faster than opening the file, calling file.stat() then closing it.

Exercise

Let’s put OS.File.stat and OS.File.DirectoryIterator to good use for getting the list of all files in a directory, ordered by last modification date.

function sortedEntries(path) {
  // Get the list of all files in directory
  let iterator = new OS.File.DirectoryIterator(path);
  let entries;
  try {
    entries = [entry for (entry in iterator)];
  } finally {
    iterator.close();
  }

  // If we are under Windows, we have all information in entries already
  // We can make this happen without any further I/O
  if ("winLastModificationDate" in OS.File.DirectoryIterator.prototype) {
    return entries.sort(function compare(x,y) {
      return x.winLastModificationDate - y.winLastModificationDate;
    }
  } else {
    // On other systems, we have to call stat before we can order
    let sortable = [{entry: entry, stat: OS.File.stat(entry.path)} for (entry in entries)];
    // Array comprehension is cool
    let sorted = sortable.sort(function compare(x, y)) {
      return x.stat.lastModificationDate - y.stat.lastModificationDate;
    }
    return [x.entry for (x in sorted)];
  }
}

Note that OS.File.DirectoryIterator does not return special files “.” and “..”.

For bonus points, let’s do the same, but only for non-directory files in the directory:

function nonDirectoryEntries(path) {
  // Get the list of all files in directory
  let iterator = new OS.File.DirectoryIterator(path);
  try {
    for (let entry in iterator) {
      if (!entry.isDir) {
        // Generators are cool, too
        yield entry;
      }
    }
  } finally {
    iterator.close();
  }
}

function sortedEntries(path) {
  // Get the list of all non-directory files in directory
  let entries = nonDirectoryEntries(path);
  if ("winLastModificationDate" in OS.File.DirectoryIterator.prototype) {
    // ... as above
  } else {
    // ... as above
  }
}

We could of course remove directories after sorting, but removing it initially saves both computation time (we sort through a shorter array) and I/O (under non-Windows platforms, we only need to call stat on a smaller set of files).

Homework

As a Programming Language guy, I see an opportunity to develop this API into a nice Domain Specific Language that would let developers formulate queries and would let the engine generate OS-optimized functions to execute these queries.

For instance:

OS.File.Query.SelectFromDir().
  where({isDir: false}).
  sortedBy({lastModificationDate: true})
  // returns the above function, including the optimizations

 

OS.File.Query.SelectFromDir().
  where({path: /.*\.tmp^/, isSymLink:false}).
  sortedBy({creationDate: true})

I do not have plans to implement anything such at the moment, but this sounds like a nice student project. If you are interested, do not hesitate to drop me a line.

That’s all folks.

In the next entries of this blog, I expect to introduce, in no particular order:

  • path manipulation with OS.File;
  • reading/writing with encodings in OS.File;
  • off-main-thread async I/O for the main thread;

benchmarks.

Where Am I?

You are currently browsing entries tagged with system at Il y a du thé renversé au bord de la table.

Follow

Get every new post delivered to your Inbox.

Join 31 other followers