What’s up with Batteries?

August 27, 2008 § 7 Comments

A few days ago, someone asked me whether there was more to OCaml Batteries Included than Lazy Lists. The answer is a definite yes. If you have looked at the repository recently, you may have seen that there is plenty more waiting for testing and for a release.

Let’s take a brief look at what is done and what is left to do.

The big picture

Batteries Included is not meant to end up as one single library. Batteries Included is meant to be a meta-library, depending on numerous existing libraries and presenting them with one consistent module hierarchy, with consistent documentation, consistent APIs, etc.

At the moment, we are centering our efforts on improving/completing Extlib. More will come later.

Waiting in the repository

Let’s start with data structures. Some come from ExtLib, other from OCaml or from smaller libraries and some are brand new. They are reorganized by functionality rather than by source and may be found in module Batlib.Containers:

  • Batlib
    • Containers
      Generic persistent containers

      • List / ListLabels  (extended from ExtLib’s)
      • Map / MapLabels (extended from OCaml’s)
      • PMap (untouched from ExtLib’s)
      • LazyList / LazyListLabels (brand new)
      • Set (untouched from OCaml’s)
      • Vect (a generic implementation of ropes — brand new)

      Generic mutable containers

      • Array (untouched from ExtLib’s)
      • BitSets (untouched from ExtLib’s)
      • Dlllist (doubly linked lists — untouched from ExtLib’s)
      • DynArray (dynamic arrays — untouched from ExtLib’s)
      • Enum / EnumLabels (enumerations aka iterators — vastly extended from ExtLib’s)
      • Hashtbl (untouched from ExtLib’s)
      • Queue (untouched from OCaml’s)
      • RefList (untouched from ExtLib’s)
      • Stack (untouched from OCaml’s)
      • Stream / StreamLabels (extended / adapted from SDFlow’s)

      Specialized containers

      • Buffer (untouched from OCaml’s)
      • Option (utilities for ‘a option — extended from ExtLib’s)
      • Result (an error monad — brand new, will be replaced by Catch me if you Can)
      • Global (global variables — untouched from ExtLib’s)
      • Ref (operations on references — brand new)
      • Lazy (untouched from OCaml’s)

Similarly, typical contents for these data structures may be found in module Batlib.Data, as follows:

  • Batlib
    • Data
      Numbers (all numbers come with uniformized custom operators, including range operators)

      • Bool (operations on booleans — brand new)
      • Complex (extended from OCaml’s)
      • Float (operations on floats — brand new)
      • Int (operations on integers — brand new)
      • Big_int (extended from OCaml’s)
      • SafeInt (overflow-aware integers — brand new)
      • Int32 (extended from OCaml’s)
      • Int64 (extended from OCaml’s)
      • Nativeint (extended from OCaml’s)

      Text (also with operators, when applicable)

      • Char (operations on characters, including operators — vastly extended from OCaml’s)
      • UChar (operations on Unicode characters — untouched from ExtLib’s)
      • String (extended from ExtLib’s)
      • UTF8 (UTF8-based strings — untouched from ExtLib’s)
      • Rope (UTF8-based immutable replacement for strings)

System-related operations, from file management to arguments to I/O come next, in module Batlib.System. Note that Printexc goes into that module, for no better reason than not fitting anywhere else:

  • Batlib
    • System
      Argument management

      • Arg (untouched from OCaml’s)
      • OptParse (untouched from ExtLib’s)

      Operating system

      • Sys (untouched from OCaml’s)
      • Unix / UnixLabels (untouched from OCaml’s)
      • Filename (untouched from OCaml’s)
      • Largefile (untouched from OCaml’s)

      I/O

      • Format (untouched from OCaml’s)
      • Printf (untouched from OCaml’s)
      • Printexc (untouched from OCaml’s)
      • Scanf (untouched from OCaml’s)
      • IO (extended from ExtLib’s)
      • Unzip (untouched from ExtLib’s)

Development of domain-specific languages is a big favorite in OCaml. This stuff goes into Batlib.Languages:

  • Batlib
    • Languages
      Lexers

      • Lexing (untouched from OCaml’s)
      • Genlex (programmatic generation of lexers — vastly extended from OCaml’s)

      Parsers

      • Parsing (untouched from OCaml’s)
      • ParserCo (a parser combinator library — brand new)
      • CharParser (char-related parsing utilities — brand new)

OCaml offers a few meta-level utilities, which don’t always appear in the documentation but are present nevertheless. Batteries Included publishes them as Batlib.Meta:

  • Batlib
    • Meta
      • Callback (untouched from ExtLib’s)
      • Gc (untouched from ExtLib’s)
      • Marshal (untouched from ExtLib’s)
      • Obj (untouched from ExtLib’s)
      • CamlinternalOO (untouched from ExtLib’s)
      • Oo (untouched from ExtLib’s)
      • Weak (untouched from ExtLib’s)

A few utilities are also presented in module Batlib.Util:

  • Batlib
    • Util
      • Random (extended from OCaml’s)
      • Base64 (untouched from ExtLib’s)
      • Digest (untouched from OCaml’s)

Last but not least, we have a candidate standard module (i.e. a replacement for Pervasives). This module contains:

  • everything from ExtLib’s Std
  • a few common operators for function composition, function application, arrow-style projections, pairing, currification etc.
  • everything from Pervasives (in the future, string conversion functions and I/O should disappear)
  • Enum (which become the standard currency between data structures, as well as the most common replacement for for loops)

What’s in store

While we already have our hands quite busy with just what has already been mentioned, more is coming.

  • An optimized reimplementation of ParserCo, along with more friendly error-reporting tools.
  • The Printf module and channels are on their way out (at least from the documentation) — Batteries Included offers alternatives (including a new implementation of the printf function) which are better in most respects and which we’re busy completing.
  • Despite the power of SDFlow, streams are also on their way out. We just need to finish implementing SDFlow’s features with Enum and to complete a nice syntax extension to replace streams with enumerations and stream parsers with our monadic parser combinator library. This is being worked on.
  • Camlp4 and ULex will join the standard library, somewhere in module Batlib.Languages.
  • Camomile will also join the standard library, somewhere between modules Batlib.Data and Batlib.Structures.
  • Type-conv will fit nicely into Batlib.Meta, just as Sexplib.

Want more?

  • A networking library, probably OCamlNet.
  • A better thread library, probably coThreads.
  • More pseudo-concurrency. Probably a superset of SDFlow.
  • Syntax extensions. More to be decided.

Bottom line

Lots of work has already gone into Batteries Included. Most likely, lot more will come.  Eventually, we hope to achieve JDK-like out-of-the-box usability for OCaml, even at the risk of JDK-like out-of-the-box bloat. Hopefully, we’re still quite far away from that.

If you wish to help, please don’t hesitate. There’s work for everyone.

Suggestions are also welcome, preferably in the request for features tracker. And if needed, don’t hesitate to discuss here or on our public forum.

About these ads

Tagged: , , , , , , , , , , , , , , , , , , , , , ,

§ 7 Responses to What’s up with Batteries?

  • Chris says:

    What do you mean by “The Printf module and channels are on their way out”? I haven’t heard anything to this effect.

  • Rich says:

    I’m a bit concerned that we’re not seeing patches coming upstream for the changes to (eg) extlib. I don’t think it’s helpful to fork extlib into a separate Batlib.* library.

    See: http://et.redhat.com/~rjones/how-to-supply-code-to-open-source-projects/

    & I also intend to write something on this general subject of the OCaml community and their difficulty with open source on my blog at some point.

  • yoric says:

    I’m a bit concerned that we’re not seeing patches coming upstream for the changes to (eg) extlib. I don’t think it’s helpful to fork extlib into a separate Batlib.* library.

    I concur that we should work more closely together — the idea is definitely not to make a separate library. However, I still don’t know whether the patches I submitted last year have been accepted. It’s a bit difficult to wait for answers before carrying on work.

  • yoric says:

    What do you mean by “The Printf module and channels are on their way out”? I haven’t heard anything to this effect.

    I should have phrased this differently: channels (as featured in Pervasives) and the functions of module Printf are expected to become second-class citizens in Batteries Included.

    Let me detail this a bit further: ExtLib (hence Batteries) offers [input] and [output], which are more powerful counterparts to OCaml’s channels. Similarly, Batteries contains a [printf] function more powerful than OCaml’s [printf], [eprintf], [ifprintf] and [fprintf] (or ExtLib’s [printf]).

    The only things missing for the moment are [bprintf], [sprintf] and [k*printf]. Unless I’m direly mistaken, these functions should be rather easy to write. We’ll thus obtain a full counterpart for module Printf, except this one uses the more generic [output] instead of channels.

  • Gaius says:

    Hi,

    This is great, but what about an interface to relational databases? Something equivalent to Perl’s DBI would be enormously useful, and certainly help OCaml into the mainstream. The lack of it keeps me in Python for “real work”.

    Thanks,

    Gaius

  • yoric says:

    This is great, but what about an interface to relational databases?

    Sure. I personally have never done any DB work with OCaml, which means I have very little idea of what exists. But if someone is willing to work on this, that seems like a very good idea.

    Do you wish to start a thread on either the OCaml mailing-list or our forums?

  • Rich says:

    > This is great, but what about an interface to
    > relational databases? Something equivalent to
    > Perl’s DBI would be enormously useful,…

    Is this for OCaml or for Batteries? OCaml has interfaces to every database around, and this is certainly no reason to stay on Python. If you’re using PostgreSQL, we even have type-safe database access which is a feature that doesn’t exist in any other language apart from the half-assed implementation in Microsoft C# called “LINQ”.

    http://caml.inria.fr/cgi-bin/hump.en.cgi?contrib=471

    http://caml.inria.fr/cgi-bin/hump.en.cgi?contrib=354

    http://caml.inria.fr/cgi-bin/hump.en.cgi?contrib=361

    http://caml.inria.fr/cgi-bin/hump.en.cgi?contrib=501

    http://caml.inria.fr/cgi-bin/hump.en.cgi?contrib=165

    http://www.dse.nl/~dario/projects/pgoctut/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

What’s this?

You are currently reading What’s up with Batteries? at Il y a du thé renversé au bord de la table.

meta

Follow

Get every new post delivered to your Inbox.

Join 33 other followers

%d bloggers like this: