OCaml Batteries Included, release 0: Where it should all have begun

August 29, 2008 § Leave a comment

After a few discussions on IRC, by e-mail and on forums, I have come to realize that both the purpose of Batteries Included and what the development of Batteries involved were quite unclear to most people — and that we should probably have started our work in quite a different manner. All these discussions have prompted a few changes and the release of a first pre-version of Batteries Included, which you may find on the OCamlForge project (you may also browse source code here and API documentation here).

This release represents what we should have produced in the first place; a simple and uniform presentation layer on top of existing libraries.

What exactly is OCaml Batteries Included?

OCaml is a great language which suffers from its lack of standardization. The absence of a real standard library has led the OCaml community to produce many incompatible libraries with distinct conventions. In turn, both this lack of out-of-the-box features and this multiplicity of choices make it difficult to start developing with OCaml or to design new works for reusability. By contrast, commercial offerings such as Java or .Net come with a comprehensive toolbox which makes the life of programmers that much easier.

OCaml Batteries Included is an attempt to provide a uniform library environment for OCaml, both for beginners and for experienced developers, which should give programmers out-of-the-box support for most common tasks. Although OCaml Batteries Included has to be a library for technical reasons, the objective is not to create some brand new framework but rather to integrate existing works,  with dependencies towards existing libraries, and relying on the mechanisms of Findlib and GODI/Apt/Rpm for dependencies resolution.

OCaml Batteries Included is not a standard library: as far as we are concerned, the standard library is a combination of the base library provided by Inria and ExtLib, with possible other adjunctions if deemed necessary.

OCaml Batteries Included also doesn’t stop at the level of standard libraries. In time, we aim to provide

  • network management (most likely using OCamlNet)
  • concurrency (most likely using coThreads)
  • calendar management
  • distributed programming (possibly bulk synchroneous programming or Opis)
  • user-interface (underlying support undecided)
  • plotting
  • geometry
  • parsing and unparsing
  • database access
  • logging
  • image manipulation
  • xml manipulation
  • unit testing
  • pdf generation
  • reactive programming
  • localisation
  • safe marshalling

Finally, OCaml Batteries Included doesn’t stop at the level of libraries. We already give access to a few compile-time tools and future releases will also include syntax extensions. Expect things along the lines of:

  • various syntactic improvements (local opening of modules, extended exception management)
  • improved pattern matching (micmatch or Patterns)
  • management of free/bound names
  • domain-specific languages

Oh, and why does this work have to be a library? Because of that little matter of uniformity. At the moment, the OCaml base library provides channels for input and output and streams as a lazy data structure. ExtLib, on the other hand, provides the more generic input and 'a output for, well, input and output, along with enumerations as a lazy data structure. In both occurrences, the mechanism offered by ExtLib is more  open, more generic and more powerful. Per design, we plan to consider channels and streams as obsolete. However, for the moment, most libraries use the more standard channels and streams. In order to provide uniform APIs, we will need to program a compatibility layer. Fortunately, it’s usually easy to convert between channels and inputs/outputs (except for Printf, but we’ve solved that particular problem already). That’s the glue.

Release 0 and beyond

The main objective of release 0 is to get things (re)started on a clear path, so don’t expect any brand new feature. What this release contains, in addition to logistics, is access to:

  • the base library ;
  • ExtLib ;
  • OCamlDoc ;
  • Findlib ;
  • Godi-script.

Let’s look a little bit more at the details.

The first part is Standard, the candidate replacement for Pervasives:

  • Batlib
    • Standard

Standard is composed from the contents of Pervasives and ExtLib’s Std. This module should change relatively little in the future, besides, perhaps, the addition of a few very common data structures (à la Either).

Next, we have base data (typically numbers and characters):

  • Batlib
    • Data
      Numbers

      • Big_int
      • Complex
      • Int32
      • Int64
      • Nativeint
      • Num

      Text

      • Char
      • String [ExtLib’s extended version]
      • StringLabels

This version doesn’t contain anything Unicode-ish yet nor any of our additional number modules. We’ll get Unicode support whenever ExtLib finally adopts Camomile or drops the one module of Camomile already adopted. More modules are already on the repository and waiting to be submitted to ExtLib, including safe integers, custom operators, etc.

Then come containers

  • Batlib
    • Containers
      Collection-style stuff

      • Enum

      Generic persistent containers

      • Dllist
      • List [ExtLib’s extended version]
      • ListLabels
      • Map
      • MapLabels
      • Set
      • SetLabels

      Generic mutable containers

      • Array [ExtLib’s extended version]
      • Global
      • Hashtbl [ExtLib’s extended version]
      • HashtblLabels
      • RefList
      • Stack
      • Stream

      Specialized containers

      • BitSet
      • Buffer
      • Option
      • Lazy

Again, a number of things are waiting on the repository (or for acceptation by ExtLib) before we can add them. In the future, we’ll get improved streams (yes, although they’re obsoleted), lazy lists, etc. In time, I hope we can add things like red-black trees, generic AVLs, etc.

Let’s continue with meta-level stuff:

  • Batlib
    • Meta
      Access to language primitives

      • Marshal
      • Oo

      Interaction with other languages

      • Callback
      • Memory
      • Gc
      • Weak

      Internals

      • Obj
      • CamlinternalMod
      • CamlinternalOO

In a future version, we may eventually add Dynlink if/whenever a consistent interface permits its use from both byte-compiled and native code. There is also some space for SafeMarshal and for bindings to other programming languages.

Text manipulation, from lexing to pretty-printing, goes into module Batlib.Text, as follows:

  • Batlib
    • Text
      Parsing

      • Genlex
      • Lexing
      • Parsing
      • Scanf
      • Str

      Printing

      • Format
      • Printexc
      • Printf

Future versions will be enriched with parser combinator library, unicode lexing (Ulex) and interaction with other parsers than Yacc. In addition, Printf, Scanf, Printf and Genlex will be replaced by nearly-identical but more generic modules, while the future of Format remains undecided. Finally, this may be the right place to give access Camlp4.

Module Batlib.Util groups a few utilities which have no better place:

  • Batlib
    • Base64 [from ExtLib]
    • Digest
    • Random

No decision have been taken for the future of this module. Most likely, it will continue gathering otherwise-impossible-to-classify items, with features such as calendar management.

As OCaml offers support for concurrency, we publish this feature in a dedicated module Batlib.Concurrency:

  • Batlib
    • Concurrency
      • Threads
        • Condition
        • Event
        • Mutex
        • Thread

Future versions of Batlib.Concurrency will receive alternatives to Threads, such as coThreads or lwt, while Batlib.Concurrency.Threads itself will receive additional tools for use with threading, such as Concurrent Cells.

Other interactions with the system, from file management to signals, are grouped in Batlib.System:

  • Batlib
    • System
      Environment interaction

      • Arg
      • OptParse [from ExtLib]

      Streams

      • IO [from ExtLib]
      • Unzip [from ExtLib]

      System calls

      • Filename
      • Unix
      • UnixLabels
      • Sys

Future versions will complete this offering with more operations on files (as a replacement for those operations included in Pervasives), operating-system related utilities, such as are included already in OCamlBuild or Godi-script. There is also some space for operating-system specific modules.

The final module permits (some) interaction with the OCaml toolchain:

  • Batlib
    • Toolchain
      • Odoc_info [from OCamlDoc]
      • Findlib [from Findlib]
      • Godi_script [from Godi-script]

Future versions will give access to OCamlBuild’s modules and should eventually provide tools for causing and controlling compilations from within OCaml. While this is not our main objective, together with Dynlink, this may give us elements of a dynamically-compiled OCaml.

How can I get some stuff into Batteries Included?

As you have seen, we already have a number of plans for Batteries Included and there is room for more. Now, if you believe that some library should absolutely get loaded into Batteries, we suggest taking the following steps:

  • make sure that the license of the library is compatible with Batteries (chances are that if it’s open-source and if it’s OCaml, there won’t be any licensing issue, but be careful with the full GPL or the un-amended LGPL)
  • if this library is not in GODI yet, contribute a GODI package (or find someone to do so) — Debian or Fedora packages are nice, too, but for the time being, we are using GODI as a reference
  • if this library is new or not very well-known, find people to test it and review it
  • if the API of the library makes use of functions or data structures which are considered obsolete in Batteries or which are clearly redundant with respect to functions or data structures already present in Batteries, please provide a glue layer, hiding these functions or data structures behind an API which makes use of the functions or data structures favored by Batteries
  • if the API of the library makes use of closed data structures, please try and find a manner of opening them — if you don’t understand what I mean, compare the closed design of channel_in and channel_out to the open design input and 'a output, the closed design of streams to the open one of enumerations, etc.
  • design an OCamlBuild plug-in which will let users link against the API
  • contact us — for the time being, we’re using the main OCaml Mailing-List.

Also, feel free to contact us with any question or suggestion.

Tagged: , , , , , , , , , , , , , , , , , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

What’s this?

You are currently reading OCaml Batteries Included, release 0: Where it should all have begun at Il y a du thé renversé au bord de la table.

meta

%d bloggers like this: