What’s up with Batteries?

A few days ago, someone asked me whether there was more to OCaml Batteries Included than Lazy Lists. The answer is a definite yes. If you have looked at the repository recently, you may have seen that there is plenty more waiting for testing and for a release.

Let’s take a brief look at what is done and what is left to do.

The big picture

Batteries Included is not meant to end up as one single library. Batteries Included is meant to be a meta-library, depending on numerous existing libraries and presenting them with one consistent module hierarchy, with consistent documentation, consistent APIs, etc.

At the moment, we are centering our efforts on improving/completing Extlib. More will come later.

Waiting in the repository

Let’s start with data structures. Some come from ExtLib, other from OCaml or from smaller libraries and some are brand new. They are reorganized by functionality rather than by source and may be found in module Batlib.Containers:

• Batlib
  • Containers
    Generic persistent containers
    • List / ListLabels  (extended from ExtLib’s)
    • Map / MapLabels (extended from OCaml’s)
    • PMap (untouched from ExtLib’s)
    • LazyList / LazyListLabels (brand new)
    • Set (untouched from OCaml’s)
    • Vect (a generic implementation of ropes — brand new)

    Generic mutable containers

    • Array (untouched from ExtLib’s)
    • BitSets (untouched from ExtLib’s)
    • Dlllist (doubly linked lists — untouched from ExtLib’s)
    • DynArray (dynamic arrays — untouched from ExtLib’s)
    • Enum / EnumLabels (enumerations aka iterators — vastly extended from ExtLib’s)
    • Hashtbl (untouched from ExtLib’s)
    • Queue (untouched from OCaml’s)
    • RefList (untouched from ExtLib’s)
    • Stack (untouched from OCaml’s)
    • Stream / StreamLabels (extended / adapted from SDFlow’s)

    Specialized containers

    • Buffer (untouched from OCaml’s)
    • Option (utilities for ‘a option — extended from ExtLib’s)
    • Result (an error monad — brand new, will be replaced by Catch me if you Can)
    • Global (global variables — untouched from ExtLib’s)
    • Ref (operations on references — brand new)
    • Lazy (untouched from OCaml’s)

Similarly, typical contents for these data structures may be found in module Batlib.Data, as follows:

• Batlib
  • Data
    Numbers (all numbers come with uniformized custom operators, including range operators)
    • Bool (operations on booleans — brand new)
    • Complex (extended from OCaml’s)
    • Float (operations on floats — brand new)
    • Int (operations on integers — brand new)
    • Big_int (extended from OCaml’s)
    • SafeInt (overflow-aware integers — brand new)
    • Int32 (extended from OCaml’s)
    • Int64 (extended from OCaml’s)
    • Nativeint (extended from OCaml’s)

    Text (also with operators, when applicable)

    • Char (operations on characters, including operators — vastly extended from OCaml’s)
    • UChar (operations on Unicode characters — untouched from ExtLib’s)
    • String (extended from ExtLib’s)
    • UTF8 (UTF8-based strings — untouched from ExtLib’s)
    • Rope (UTF8-based immutable replacement for strings)

System-related operations, from file management to arguments to I/O come next, in module Batlib.System. Note that Printexc goes into that module, for no better reason than not fitting anywhere else:

• Batlib
  • System
    Argument management
    • Arg (untouched from OCaml’s)
    • OptParse (untouched from ExtLib’s)

    Operating system

    • Sys (untouched from OCaml’s)
    • Unix / UnixLabels (untouched from OCaml’s)
    • Filename (untouched from OCaml’s)
    • Largefile (untouched from OCaml’s)

    I/O

    • Format (untouched from OCaml’s)
    • Printf (untouched from OCaml’s)
    • Printexc (untouched from OCaml’s)
    • Scanf (untouched from OCaml’s)
    • IO (extended from ExtLib’s)
    • Unzip (untouched from ExtLib’s)

Development of domain-specific languages is a big favorite in OCaml. This stuff goes into Batlib.Languages:

• Batlib
  • Languages
    Lexers
    • Lexing (untouched from OCaml’s)
    • Genlex (programmatic generation of lexers — vastly extended from OCaml’s)

    Parsers

    • Parsing (untouched from OCaml’s)
    • ParserCo (a parser combinator library — brand new)
    • CharParser (char-related parsing utilities — brand new)

OCaml offers a few meta-level utilities, which don’t always appear in the documentation but are present nevertheless. Batteries Included publishes them as Batlib.Meta:

• Batlib
  • Meta
    • Callback (untouched from ExtLib’s)
    • Gc (untouched from ExtLib’s)
    • Marshal (untouched from ExtLib’s)
    • Obj (untouched from ExtLib’s)
    • CamlinternalOO (untouched from ExtLib’s)
    • Oo (untouched from ExtLib’s)
    • Weak (untouched from ExtLib’s)

A few utilities are also presented in module Batlib.Util:

• Batlib
  • Util
    • Random (extended from OCaml’s)
    • Base64 (untouched from ExtLib’s)
    • Digest (untouched from OCaml’s)

Last but not least, we have a candidate standard module (i.e. a replacement for Pervasives). This module contains:

• everything from ExtLib’s Std
• a few common operators for function composition, function application, arrow-style projections, pairing, currification etc.
• everything from Pervasives (in the future, string conversion functions and I/O should disappear)
• Enum (which become the standard currency between data structures, as well as the most common replacement for for loops)

What’s in store

While we already have our hands quite busy with just what has already been mentioned, more is coming.

• An optimized reimplementation of ParserCo, along with more friendly error-reporting tools.
• The Printf module and channels are on their way out (at least from the documentation) — Batteries Included offers alternatives (including a new implementation of the printf function) which are better in most respects and which we’re busy completing.
• Despite the power of SDFlow, streams are also on their way out. We just need to finish implementing SDFlow’s features with Enum and to complete a nice syntax extension to replace streams with enumerations and stream parsers with our monadic parser combinator library. This is being worked on.
• Camlp4 and ULex will join the standard library, somewhere in module Batlib.Languages.
• Camomile will also join the standard library, somewhere between modules Batlib.Data and Batlib.Structures.
• Type-conv will fit nicely into Batlib.Meta, just as Sexplib.

Want more?

• A networking library, probably OCamlNet.
• A better thread library, probably coThreads.
• More pseudo-concurrency. Probably a superset of SDFlow.
• Syntax extensions. More to be decided.

Bottom line

Lots of work has already gone into Batteries Included. Most likely, lot more will come.  Eventually, we hope to achieve JDK-like out-of-the-box usability for OCaml, even at the risk of JDK-like out-of-the-box bloat. Hopefully, we’re still quite far away from that.

If you wish to help, please don’t hesitate. There’s work for everyone.

Suggestions are also welcome, preferably in the request for features tracker. And if needed, don’t hesitate to discuss here or on our public forum.