Il y a du thé renversé au bord de la table » OCaml

Last lecture

yoric — Wed, 22 Apr 2009 14:17:33 +0000

Note: This post is written on the 79th day of strike of Universities. Despite the overwhelming consensus against these bills, the government has just passed the application decrees implementing the possibility of arbitrarily increasing the teaching charge of researchers, without need for any justification. The government obviously fails to see how much this will hurt Research. Simultaneously, the government has announced that, since the reform of primary and secondary schools cannot proceed in compliance with the government’s own decrees, it will simply proceed illegally. Once the shock is gone, expect increased strike actions. Expect Resarch strikes on publications, on patents, on contracts with the government or French companies. Expect difficulties with baccalauréat, exams and degrees.

Where headhunters had failed, the government has finally succeeded. Today was my last lecture.

As the government obviously doesn’t want Researchers to have the means and time to undertake Research, I have accepted a position in the private sector, where I should be able to pursue my work on semantics, security and functional concurrent/distributed programming languages.

While I’m glad to start in a position where I will have both more leeway and both students and engineers to work with me, I am saddened that the situation had to reach the point where I felt I had no choice.

Barring any accident, starting September 1st, you will be able to find me at MLState.

OCaml Batteries Included Beta 1

yoric — Mon, 06 Apr 2009 10:24:32 +0000

Note This post is written on the 64th day of University strikes in France. During these 64 days, the government has rejected any negociation on the core reasons for the strike, has attempted to discredit the contestation (if I understand correctly, I am both « improductive » and a « mask-wearing commando ») and has used police intimidation and repression. The strike continues. Quite possibly, there will be no university exams this year and no baccalauréat. If repression continues increasing, no one can tell for sure what will happen. Nothing good, for sure.

After days and nights of coding, debugging and fighting over naming conventions, the OCaml Batteries Included team is proud to announce OCaml Batteries Included Beta 1. You can find the binaries here, read the API documentation, the platform documentation, the release notes and the ChangeLog or the list of individual commits. A GODI package and a Debian/testing package are also available.

So, what’s new in Beta 1?

API freeze

Firstly, from now on, barring trivial changes and bugfixes, the API should be stable. So it’s a good time to start testing it with your projects and submitting bug reports.

Command-line tools

OCaml Batteries Included already had a drop-in replacement for ocamlc and ocamlopt, to automatically open, load and link all the necessary libraries and/or syntax extensions. To invoke them, use


$ ocamlfind batteries/ocamlc foo.ml


$ ocamlfind batteries/ocamlopt foo.ml

They behave essentially as their counterparts in the INRIA distribution.

Ok, that’s not new. What’s new is that OCaml Batteries Included now also has a drop-in replacement for ocaml, ocamldep and ocamlbuild. To launch the toplevel, run


$ ocamlfind batteries/ocaml

or to use it as an interpreter


$ ocamlfind batteries/ocaml foo.ml

Similarly, you may now use the ocamlbuild replacement


$ ocamlfind batteries/ocamlbuild foo.byte


$ ocamlfind batteries/ocamlbuild foo.native

What does this mean? Well, essentially, it means that you don’t have to do any complex configuration to take advantage of OCaml Batteries Included. Just replace every use of ocaml/ocamlc/ocamlopt/ocamlbuild with the corresponding drop-in replacement.

Toplevel

Since Alpha 2, the OCaml Batteries Included toplevel gives access to the manual, using directive #man. With Beta 1, a new directive #browse appears. This directive lets you insepct the contents of a module, regardless of whether the documentation has been generated:


$ rlwrap ocamlfind batteries/ocaml
        Objective Caml version 3.11.0

      _________________________________
     |       | |                       |
    [| +     | | Batteries Included  - |
     |_______|_|_______________________|
      __________________________________
     |                       | |        |
     | -    Type '#help;;'   | |      + |]
     |_______________________|_|________|

# #browse "List";;
module List :
  sig
    type 'a t = 'a list
    type 'a enumerable = 'a t
    type 'a mappable = 'a t
    val length : 'a list -> int
    val hd : 'a list -> 'a
    val tl : 'a list -> 'a list
    val is_empty : 'a list -> bool
    (*...*)
end

This directive is provided as a quick way of understanding complex and undocumented modules — say, Camlp4.

Printf replacement

OCaml has long provided a useful but awkward Printf module. A few versions ago, OCaml Batteries Included introduced a partial replacement for Printf, more extensible and more compsable. With OCaml Batteries Included Beta 1, we now have a complete replacement, which is both safer, lightweight and more extensible. Let’s take a look :


# let format = p"Let's print a string: %s\n";;
val format : (string -> 'a, 'a) Batteries.Print.format =
  {Batteries.Print.pattern = "Let's print a string:%(0)\n";
   Batteries.Print.printer = }
# Print.printf format "some text";;
Let's print a string: some text
- : unit = ()

Up to this point, there’s not much difference with the usual Printf module. But the nice thing is that this is completely extensible. It can be used just as well to print, say, lists:


# Print.printf p"Here's a list of integers: %{int list}\n" [1;2;3;4;5];;
Here's a list of integers: [1; 2; 3; 4; 5]
- : unit = ()

Of course, this works just as well with arrays, ropes, and, well, just about every single data structure, including your own. You can add new printing functions, overload printing locally, etc.

And more

The OCaml Batteries Included toplevel now supports a .ocamlinit file, which can take advantage of all the features of Batteries.
Previous versions of OCaml Batteries Included introduced syntactic construction
```
open List include Labels
```
to create a new local module List containing the code of List and that of Labels. Due to ambiguities, the syntax was changed to
```
open List with Labels
```
UTF-8 literal strings are now validated at compile-time
Conversion between Latin-1 literal strings and UTF-8 is now performed at compile-time
Modules Digest, Scanf, Format, Lexing now compatible with Batteries I/O
New out-of-the-box modules StringSet, IntSet, StringMap, IntMap, etc.
New module Future.Logger
The interpreter may now be called as a library
Improvements to Random, Enum, Bigarray, String, String.Cap, Set, PSet, …
Speed improvements in LazyList, Rope, String
Test suite
Build improvements, including parallel build, better detection of default directories
Numerous bugfixes
etc.

On-line interpreter for Batteries

yoric — Wed, 25 Mar 2009 08:57:05 +0000

Note This entry is written on the 51st day of nationwide strike in French Universities and Research laboratories. Still no sign of negociation from the government. Liquidation of the system continues.

I am happy to announce that the repository version of OCaml Batteries Included now has a full-featured (and working) interpreter.

Toplevel (aka REPL aka interactive loop)

The toplevel has been available for some time in Batteries Included, but it has recently seen a lot of polish.


~$ rlwrap ocamlfind batteries/ocaml
 Objective Caml version 3.11.0

  _________________________________
 |       | |                       |
[| +     | | Batteries Included  - |
 |_______|_|_______________________|
  _________________________________
 |                       | |       |
 | -    Type '#help;;'   | |     + |]
 |_______________________|_|_______|

# open Print;;
# printf p"Hello %rope!\n" r"world" ;;
Hello world!
- : unit = ()

Now is the right time to thank Warren Harris for the nice batteries logo and Jérémie Dimino for the new, extensible printf replacement.

Want to know more about that module?


# #browse "Print";;
module Print :
 sig
 type ('a, 'b, 'c) directive =
 (('c Extlib.InnerIO.output -> unit) -> 'b) -> 'a
 val literal : string -> ('a, 'a, 'b) directive
 type pattern = string
 val format :
 'a Extlib.InnerIO.output ->
 pattern -> ('a Extlib.InnerIO.output -> unit) array -> unit
 type ('a, 'b, 'c) format =
 ('a, 'b, 'c) Extlib.Print.format = {
 pattern : pattern;
 printer : pattern -> ('a, 'b, 'c) directive;
 }
 val printf : ('a, unit, unit) format -> 'a
 val eprintf : ('a, unit, unit) format -> 'a
 val fprintf : 'a Extlib.InnerIO.output -> ('b, unit, 'a) format -> 'b
 val kfprintf :
 ('a Extlib.InnerIO.output -> 'b) ->
 'a Extlib.InnerIO.output -> ('c, 'b, 'a) format -> 'c
 val rprintf : ('a, Extlib.Rope.t, string) format -> 'a
 val krprintf : (Extlib.Rope.t -> 'a) -> ('b, 'a, string) format -> 'b
 val sprintf : ('a, string, string) format -> 'a
 val ksprintf : (string -> 'a) -> ('b, 'a, string) format -> 'b
 val bprintf : Buffer.t -> ('a, unit, string) format -> 'a
 val kbprintf :
 (Buffer.t -> 'a) -> Buffer.t -> ('b, 'a, string) format -> 'b
 end

Ah, well, the signature may be a tad complex. But in fact, this was just a shameless plug for the new #browse directive, which I personally find quite useful. As usual, for more details, use #man to open the documentation.

Script interpreter

Now, until recently, we couldn’t use Batteries Included to execute a script. This feature has recently been added by Peng Zang, Jérémie Dimino and Yours Truly.

Which means that we can now run


$ bocaml shuffle.ml Joe Jack William Averell
Jack William Averell Joe

(I’m cheating here, I have an alias from bocaml to ocamlfind batteries/ocaml, I assume you should’nt have difficulties doing the same if you use Batteries often).

In other news, we’re getting very close to a Beta release. Stay tuned for more info!

OCaml Batteries Included, alpha 3

yoric — Fri, 06 Feb 2009 14:03:47 +0000

Dear programmers.

I am happy to inform you that, despite the in-progress liquidation of French Universities, OCaml Batteries Included Alpha 3 has landed. Barring any accident, this should be the final Alpha version, with a mostly stable API and module structure. You may now download it from the Forge. A GODI package is also available and a Debian package should follow soon. You may also read the documentation on-line.

So, what’s new with Alpha 3? Plenty of things, as you’ll see.

General

OCaml Batteries Included is now available for both OCaml 3.10 and OCaml 3.11. Using OCaml 3.11 bring several additional features.

Language

A new syntax extension allows merging several modules into one:
```
module List = List include Labels
```
gives name List to a new module containing both List and Labels. This extension should simplify the use of labels, exception-less extensions, etc.
General and extendable data structure comprehension:
```
[? i*i | i <- 1 -- 100 ; i mod 2 = 0]
```
produces an enumeration containing all the squares of even numbers between 1 and 100, while
```
[? String : c | c <- open Char in 'a' -- 'z']
```
produces string "abcdefghijklmnopqrstuvwxyz". This works just as well with arrays, Unicode strings, Unicode ropes, doubly-linked lists, dynamic arrays, etc.

(OCaml 3.11 only) It is now possible to write Unicode strings, Unicode ropes, read-only strings, write-only strings, etc. Similarly, it is now possible to pattern-match against these strings with a natural syntax:


# u"This is a Unicode string";;
- : Batteries.UTF8.t = u"This is some Unicode string"
# ro"This is a read-only string";;
- : [`Read] Batteries.String.Cap.t = ro"This is some Unicode string"
# r"This is a functional Unicode rope";;
- : Batteries.Rope.t = r"This is a functional Unicode rope"

Library

After public consultation (1, 2), the module hierarchy has been completely rewritten into a mostly flat hierarchy, as per common OCaml usage.
New abstract thread-safe modules, to allow extensibility to e.g. coThreads.
Thread-safe version of the I/O library.
Old-style ArrayLabels, ListLabels, etc. have been replaced with Array.Labels, List.Labels, etc.
Plenty of new functions in Enum, Array, Hashtbl, Pervasives, String, Rope, Bingarray, Dllist, DynArray, Arg, Sys.
The File module now permits using temporary files.
New compatibility layer with OCamlNet.
New version of the Unix module, now compatible with I/O.
New module Future.Path for manipulating paths.
New module CharEncoding for transcoding inputs/outputs.
New module Date for manipulating time.

Documentation

The new documentation allows browsing modules by topics. If this is successful, this will be generalized to browsing values, types, etc. by topics.
New documentation generator.
New explanations on a number of subjects.
A number of docfixes.

Bugfixes

A host of bugfixes, in particular wrt Enum.

Toplevel

Loading speed improved.
More robust implementation of the toplevel.
Pretty-printers for UChar, Rope, String.Cap, UTF8.t.
More robust implementation of the help system.

Examples

Beginning of reimplementation of Pleac in idiomatic OCaml Batteries Included.
Opening a web-browser from a program.
Text encoding transcoder.
Solution of the Euler problem.

Build system

It is now possible to configure whether documentation should be generated.
Documentation installation made more robust.

What now?

Well, as any Alpha version, this code needs testing, lots of testing. So please consider using it and posting both feedback, requests for features and bug reports.

Besides testing and fixing bugs, our next few steps consist in

improving the documentation and the on-line help
attempting to decrease the size of binaries produced with Batteries
integrating Delimited Overloading
integrating more network features (OCamlNet)
a preference system
improving configurability of the GODI package.

And progressively moving towards 1.0!

The next release should be Beta 1 and is planned for March.

Have fun!

Let’s do it with Batteries

yoric — Tue, 27 Jan 2009 20:15:03 +0000

Or, OCaml is a scripting language, too.

Note: These extracts use the latest version of Batteries, currently available from the git. Barring any accident, this version should be made public within the next few days.

A few days ago, when writing some code for OCaml Batteries Included, I realized that, to properly embed Camomile’s Unicode transcoding module, I would need to manually write 500+ boring lines, all of them looking like:


| `ascii -> Encoding.of_name "ASCII"

The idea behind that pattern matching was to define a type-safe phantom type for text encodings. Upon installation, Camomile generates a directory containing about 540 files, one per text encoding, and it seemed like a good idea to rely upon something less fragile than a string name.

Of course, writing this pattern-matching manually was out of the question: it was boring, error-prone, and while Batteries deserves sacrifices, it doesn’t quite deserve that level of mind-numbing activities. The alternative was to generate both the list of constructors and the pattern-matching code from the contents of the directory. I could have done it with some scripting language but that sounded like a good chance to test-drive the numerous new functions of the String module of Batteries (73 for 28 in the Base Library).

The main program

The structure of the program is easy: read the contents of a directory. For each file, do some treatment on the file name and print the result:


open Shell
foreach (files_of argv.(1)) do_something

Here, foreach is the same function as iter but with its arguments reversed. It’s sometimes much more readable. Instead of reading the contents of a directory with Shell.files_of, we could just as well have traversed the command-line arguments with args, or read the lines of standard input using IO.lines_of stdin.

Actually, we could just as well generalize to a (possibly empty) set of directories. For this purpose, we just need to map our function files_of to the enumeration of command-line arguments. This yields an enumeration of enumerations, which we turn into a flat enumeration with flatten. In my mind, that’s somewhat nicer and more readable than nested loops.

Our main program now looks like:


open Shell, Enum
foreach (flatten (map files_of (args ()))) do_something

Or, for those of us who prefer operators to parenthesis:


open Shell, Enum
(foreach **> flatten **> map files_of **> args ()) do_something

String manipulation

It’s now time to take a file name and turn it into

a nice constructor name
a file name without extension,

That second point is the easiest, so let’s start with it. We have a function Filename.chop_extension just for this purpose. So, if we were interested only in printing the list of files without their extension, we could define


let do_something x = print_endline (Filename.chop_extension x)

The first point is slightly trickier, as we need to

remove the extension from the file name (done)
prepend character ` (trivial)
replace any illicit character with _ (slightly more annoying, I know that the list of illicit characters which may actually appear in my list of files contains :, -, (, ) and whitespaces but I’d rather not go and check manually which other characters may turn out problematic)
prepend something before names which start with a digit, as digits cannot appear as the first character of an OCaml constructor (a tad annoying, too)
make everything lowercase, just because it’s nicer (trivial).

Let’s deal with the third item, it’s bound to be central. Let’s see, replacing characters could be done with regular expressions, something I dislike, or with function String.map. It’s nicer, type-safer, and it has a counterpart Rope.map for Unicode, if we ever need one. Now, functions Char.is_letter and Char.is_digit will help us determine which names are safe. Using them together, we obtain the following function:


open Char
let replace s = String.map (fun c -> if is_letter c || is_digit c then c else '_') s

Let’s solve the fourth item on our list. We need to check the first character of a string and to determine whether it’s a digit. Well, we already know how to do this. Let’s call our prefix p:


let clean_digit p s = if is_digit s.[0] then p^s else s

If we chain up everything, we obtain


let constructor p s = "`" ^ (if is_digit r.[0] then p^r else r)
    where         r = lowercase (String.map (fun c -> if is_letter c || is_digit c then c else '_') s)

I like this where syntax.

Format

Now that we have both our strings, we just need to be able to combine and print them. For this purpose, Printf is probably the most concise tool. Here, we can just write


let print s1 s2 = Printf.printf " | %s -> %S\n" s1 s2

We could parameterize upon the format used by printf and we’re bound to do this sooner or later, but let’s keep it simple for now.

The complete program


open Shell, Enum

foreach (flatten **> map files_of **> args ()) do_something
  where do_something s =
   let name = Filename.chop_extension s in Printf.printf " | %s -> %S\n" c name
     where c = "`" ^ (if Char.is_digit r.[0] then "codemap_"^r else r)
     where r = lowercase (String.map (fun c -> if Char.is_letter c || Char.is_digit c then c else '_') name)

I don’t know about you but I find this pretty nice, for a type-safe language. I’m sure it would have been possible to make something shorter in Perl or awk, and suggestions are welcome regarding how to improve this but I’m rather happy. And, once again, we’re not trying to beat Python, Perl or awk in concision, just to do something comparably good, because we already beat them by far in speed and safety.

So, what do you think?

OCaml Batteries Included: The Hierarchy, reloaded

yoric — Thu, 20 Nov 2008 20:12:01 +0000

Well, my previous post on the Hierarchy of OCaml Batteries Included certainly triggered reactions. Essentially, judging from these, the OCaml community doesn’t seem to want of a module hierarchy. So here’s a reworked version of the library layout, without hierarchy. Again, feedback is appreciated and should go to the OCaml mailing-list.

Batteries

Standard (automatically opened)
Legacy
1. Arg
2. Array
3. …
Future
1. Lexers
  1. C
  2. OCaml
I. Control
Exceptions
Return
Monad Interfaces for monadic operations
I.1. Concurrency
Concurrency Interfaces for concurrency operations
I.1.i. Built-in threads
Condition
Event
Mutex
RMutex
Thread
Threads A module containing aliases to Condition, Event…
I.1.ii. coThreads
CoCondition
CoEvent
CoMutex
CoRMutex
CoThread
CoThreads as Threads but with implementations coming from coThreads
I.1.iii. Shared memory
Shm_* Placeholders
II. IO
IO
1. BigEndian
GZip
Bz2
Zip
Transcode
III. Mutable containers
Array
1. Cap
  1. ExceptionLess
  2. Labels
2. ExceptionLess
3. Labels
Bigarray
1. Array1
2. Array2
3. Array3
Dllist
Dynarray
Enum
1. ExceptionLess
2. Labels
Global
Hashtbl
1. Make
  1. ExceptionLess
  2. Labels
IV. Persistent containers
Lazy
List
1. ExceptionLess
2. Labels
Map
1. Make
  1. ExceptionLess
  2. Labels
Option
1. Labels
PMap
PSet
RefList
1. Index
Queue
Ref
Set
1. Make
  1. ExceptionLess
  2. Labels
Stack
Stream
V. Data
Unit
V.1. Logical
Bool
BitSet
V.2. Numeric
Numeric Interfaces for number-related stuff
Big_int
Common
Complex
Float
Int
Int32
Int64
Native_int
Num
Safe_float placeholder
Safe_int
V.3. Textual
Text Definition of text-related interfaces
Buffer
Char
UTF8
Rope
UChar
String
StringText A module containing aliases to String and Char¹
RopeText As StringText but with implementations from Rope and UChar
UTF8Text As StringText but with implementations from UTF8 and UChar
1. Labels
VI. Distribution-related stuff
Packages
Compilers
VII. Internals
Gc
Modules
Oo
1. Private
Weak
1. Make
VIII. Network (placeholders)
URL
Netencoding
1. Base64
2. QuotedPrintable
3. Q
4. URL
5. Html
VIII.1. Http
Http
Http_client
Cgi_*
Httpd_*
MIME
VIII.2. Ftp
Ftp_client
VIII.3. Mail
Netmail
Pop
Sendmail
Smtp
VIII.4. Generic server
Netplex_*
VIII.5. RPC
Rpc_*
VIII.6. Languages
Genlex
Lexing
CharParser
UCharParser
ParserCo
1. Source
Parsing
Format
Printf
Str
PCRE place-holder
Scanf
1. Scanning
SExpr
IX. System
Arg
File
OptParse
1. Opt
2. OptParser
3. StdOpt
Path
Shell
Unix
1. Labels
Equeue
X. Unclassified
Digest
Random
1. State
Date placeholder

1: Actually a slightly modified version of Char to match signatures for Latin-1 and Unicode

Batteries: reworking the hierarchy (feedback wanted)

yoric — Tue, 18 Nov 2008 09:54:17 +0000

As readers interested in OCaml may know, we’ve been working for several months of OCaml Batteries Included. Early in the development, it appeared to us that, with the large number of modules involved, we would need a hierarchy of modules.

For instance, for the moment, we have a module System containing among other submodules IO (definition of i/o operations), File (definition of operations on files), Sys (the usual OCaml Sys module, soon to be expanded), etc. Therefore, before one may open and manipulate files, one has to do


open System.IO;;
open System.File;;

or, with the syntax extension we developed to alleviate this,


open System, IO, File;;

the syntax extension does a few other things which we’re not going to detail here — for one thing, it allows local opening of modules).

Now that we’ve reached Alpha 2 and are well on our way towards Alpha 3, we’ve decided that it’s time to rework our current hierarchy and make it shorter and more consistent. Before we proceed, we’d like some feedback from the community. Discussion will take place mostly on the OCaml mailing-list and on irc (server: Freenode, channel: #ocaml) but I preferred posting the hierarchy here for easier reading.

At the moment, our hierarchy looks roughly as follows:

Batteries (automatically opened)

Control
1. Concurrency
  1. Common
  2. Threads
    1. Condition
    2. Event
    3. Mutex
    4. Thread
2. Exceptions
3. Return
4. Monad
Data
1. Containers (common interfaces)
2. Mutable
  1. Array
    1. Cap
      1. Labels
      2. ExceptionLess
    2. Labels
    3. ExceptionLess
  2. Bigarray
    1. Array1
    2. Array2
    3. Array3
    4. Genarray
  3. Dllist
  4. Dynarray
  5. Enum
    1. ExceptionLess
  6. Global
  7. Hashtbl
    1. Make
      1. Labels
      2. ExceptionLess
  8. RefList
    1. Index
  9. Queue
  10. Ref
  11. RefList
  12. Stack
  13. Stream
3. Logical
  1. Bool
  2. BitSet
4. Numeric
  1. Big_int
  2. Common
  3. Complex
  4. Float
  5. Int
  6. Int32
  7. Int64
  8. Native_int
  9. Num
  10. Safe_int
  11. Unit
  12. Common
5. Persistent
  1. Lazy
  2. List
    1. ExceptionLess
    2. Labels
  3. Map
    1. Make
      1. ExceptionLess
      2. Labels
  4. Option
    1. Labels
  5. PMap
  6. PSet
  7. Set
    1. Make
      1. ExceptionLess
      2. Labels
6. Text
  1. Buffer
  2. Char
  3. UTF8
  4. Rope
  5. UChar
  6. String
    1. ExceptionLess
    2. Labels
Languages
1. CharParser
2. Genlex
  1. Languages
    1. Library
      1. C
      2. OCaml
3. Format
4. Lexing
5. ParserCo
  1. Source
6. Parsing
7. Printf
8. Scanf
  1. Scanning
9. SExpr
10. Str
Meta
1. Callback
2. CamlinternalMod
3. CamlinternalOo
4. Gc
5. Obj
6. Oo
7. Weak
  1. Make
Standard (automatically opened)
System
1. Arg
2. Compress
  1. GZip
3. File
4. Filename
5. IO
  1. BigEndian
6. Network (placeholder)
7. OptParse
  1. Opt
  2. OptParser
  3. StdOpt
8. Unix
  1. Labels
9. Sys
Toolchain
1. Execute
2. Findlib
Util
1. Digest
2. Random
  1. State
Legacy

One possible replacement has been drafted:

Control
1. Concurrency
  1. Common
  2. Threads
    1. Condition
    2. Event
    3. Mutex
    4. Thread
  3. Exceptions
  4. Return
  5. Monad
Data
1. Containers
  1. Common
  2. Array
    1. Cap
      1. ExceptionLess
      2. Labels
    2. ExceptionLess
    3. Labels
  3. Bigarray
    1. Array1
    2. Array2
    3. Array3
    4. Genarray
  4. Dllist
  5. Dynarray
  6. Enum
    1. ExceptionLess
    2. Labels
  7. Global
  8. Hashtbl
    1. Make
      1. ExceptionLess
      2. Labels
  9. Lazy
  10. List
    1. ExceptionLess
    2. Labels
  11. Map
    1. Make
    1. 1. ExceptionLess
      2. Labels
  12. Option
    1. Labels
  13. PMap
  14. PSet
  15. RefList
    1. Index
  16. Queue
  17. Ref
  18. RefList
  19. Set
  20. Make
    1. ExceptionLess
    2. Labels
  21. Stack
  22. Stream
2. Logical
  1. Bool
  2. BitSet
  3. Unit
3. Numeric
  1. Interfaces (formerly Common)
  2. Big_int
  3. Common
  4. Complex
  5. Float
  6. Int
  7. Int32
  8. Int64
  9. Native_int
  10. Num
  11. Safe_int
4. Text
  1. Buffer
  2. Char
  3. UTF8
  4. Rope
  5. UChar
  6. String
    1. Labels
Distro
1. Packages (formerly Findlib)
2. Tools (formerly Execute)
Internals
1. Callback
2. Gc
3. Mod (formerly CamlinternalMod)
4. Obj
5. Oo
  1. Private (formerly CamlinternalOo)
6. Weak
  1. Make
IO
1. BigEndian
2. Gzip
3. Bz2
4. Zip
5. Transcode (placeholder)
Network (placeholder)
Text (formerly Languages)
1. Lex
  1. Genlex
    1. Languages
      1. C
      2. OCaml
  2. Lexing
2. Parse
  1. CharParser
  2. UCharParser
  3. ParserCo
    1. Source
  4. Parsing
3. Pretty
  1. Format
4. Printf
5. Regexp
  1. Str
  2. PCRE (in the future)
6. Scanf
  1. Scanning
7. SExpr
8. XML (placeholder)
Standard (automatically opened)
Sys
1. Arg
2. File
3. OptParse
  1. Opt
  2. OptParser
  3. StdOpt
4. Path (formerly Filename)
5. Shell (formerly Sys)
6. Unix
  1. Labels
Util
1. Digest
2. Random
  1. State
Legacy

The overall goal, with this new version, is to make for

a shallower hierarchy
shorter module names
more consistent groups of modules.

What do you think of this replacement? Is it any better?

OCaml Batteries Included: Alpha 2 has landed

yoric — Mon, 10 Nov 2008 20:20:44 +0000

note: There seems to have been a WordPress bug. For some reason, the extended release notes on OCaml Batteries Included were replaced by something quite unrelated. My apologies for this.

Dear programmers, I am happy to inform you that the second alpha release of OCaml Batteries Included has landed. You may now download it from the Forge. A GODI package is also available and a Debian package should follow soon (you should be able to find the old one here) and you can read the documentation on-line.

So, what’s new in this release?

Documentation

All the quirks, bugs and erroneous links which plagued the documentation of Alpha 1 should now be gone.
Documentation has been extended, with more explanations regarding the use of Input/Output, more explanations on common functions of Pervasives, etc.
The top-level now has a on-line help feature. You should try it, it’s nice.
New examples, including an 8-liner gzip.

Toolchain

The top-level now works.
We now have a custom version of ocaml, ocamlc, ocamlopt and ocamlcp. To invoke them, use ocamlfind. For instance, for the new top-level, invoke ocamlfind batteries/ocaml. This should make your life simpler if you don’t use OCamlBuild.

Fundamentals

Improved Input/Output. In particular, inputs and outputs may now be garbage-collected, automatically closed by higher-order functions, closed manually, closed by their dependencies, or closed automatically when program stops. Speed of key I/O operations was also largely improved.
We now have a set of standard combinators for function composition. You won’t need anymore to redefine your |>, <|, etc.
Numerous improvements to Unicode, including improved type-safety, Unicode characters on par with Latin-1 characters, etc.
We are now very close to being completely rid of legacy channel_in and channel_out, in favor of the more abstract, more composable, safer and more feature-rich IO. In particular, the Unix module and the Standard module are now completely rid of these pesky channels.

Utilities

We now have transparent on-the-fly gzip compression/decompression — more formats are coming.
A nice little parser combinator library has been added. It works with either Latin-1, Unicode or custom tokens.
An OCaml program may now open a web browser to display a URI. In the future, this will be extended to opening arbitrary files with system-specific tools.
Appearance of a foreach loop (which is actually nothing more than Enum.iter, but we don’t have to tell that to beginners).

Boilerplate

Most data structures now support serialization to/deserialization from S-Expressions.
Most data structures now support printing to arbitrary outputs.

What now?

Well, now, we’d appreciate if you could download and install this library. We’d appreciate if you could test your programs against it. And we’d appreciate if you could file bug reports and request for features.

What’s next ?

We’re already at work on the next version.

Here’s an overview of what you can expect in the relatively near future:

Re-designed, simpler, module hierarchy (by the way, if you have suggestions, please drop us a line).
More syntax extensions, including the long-awaited finally.
Exception-less error management in addition to exceptions.
More on-the-fly (de)compression formats.
Further documentation.
Bindings for Camlp4.
Unicode transcoding I/O.
Further examples.
In-memory (de)compression.
File and directory management.
Shell programming.
Pattern-matching on lazy lists.
List (and other data structures) comprehension.
Compatibility with OCaml 3.11.

A taste of OCaml Batteries Included

yoric — Fri, 07 Nov 2008 18:14:08 +0000

I don’t know about you, but I have the feeling that many people are interested by OCaml Batteries Included but don’t dare try it yet, due to the Alpha status and the fact that it’s not available for their favorite Linux distribution yet. Well, it’s probably a healthy level of caution.

Of course, just reading the manual is probably not the best way of getting a feeling of OCaml Batteries Included.

So I’ve decided to take measures. From time to time, I’ll add here a few samples of what you can do with OCaml Batteries Included and how you can do it.

For today, let’s start with displaying the contents of a file. You know, Unix’s cat or MS-DOS’s type.


open System, IO, File

iter (fun x -> copy (open_in x) stdout) (args ())

That’s it. Three lines, one of them blank.

Now, for details: open System, IO, File opens three modules System (which contains all system-related functions, including input/output, file management, etc.), IO (the submodule of System containing all the operations on inputs and outputs), and File (the submodule of System containing all the necessary to open files for reading, writing, etc.).

Let’s move on to the last line. Function iter is defined in module Standard, which means that you don’t need to open any module to be able to use it. This function is a general imperative loop on enumerations. This is the equivalent of loop for-each in some languages. Perhaps I should detail what enumerations are: they are a read-and-forget data structure used pervasively in OCaml Batteries Included, and which replaces streams. By opposition to lists, arrays, etc., enumerations are built lazily and discarded as they are read, which makes them quite convenient for loops, or for accessing possibly huge sets of data — depending on your background, you may think of these either as streams (in OCaml and most languages) or as iterators (in Python, JavaScript and other dynamic languages). Oh, and for functional-minded people, don’t worry, your usual functional loops are available on enumerations, too. You can fold, map or unfold at will.

So, what does iter do? Well, if asked, OCaml will tell you that it has type ('a -> unit) -> 'a Enum.t -> unit. In other words, for any type 'a, this function takes as first argument a function (let’s call it f), as second argument an enumeration of elements of type 'a and returns nothing. Function f itself should take as argument an element of type 'a and return nothing. In other words, iter takes a function which works on one element of type 'a and turns it into a function which works on a whole enumeration of elements of type 'a. Yep, it’s called a loop.

Before looking at the definition of the function, let’s take a look at the enumeration passed to iter: args (). Well, if we look at the documentation (for instance by using our on-line help), we may read

args(): An enumeration of the arguments passed to this program through the command line.

So, args () is your usual pair argc argv (if you come from C) or args [] (if you come from Java). By opposition to Java, you don’t have to always put them in your program, if you don’t use them, and by opposition to both, it’s an enumeration, which makes more sense than an array, since you don’t need to modify them and since you always move forward among the arguments. Still, if you need it as an array, it’s available in a package. No more on this for the moment.

What’s left? Oh, yes, the function. As the code indicates, fun x -> copy (open_in x) stdout) is an anonymous function (also known as a “lambda” in a few languages). This function takes an argument x, the name of a file. Function copy, defined in module IO, takes two arguments, an input (a source of data) and an output (a sink of data), and copies the whole contents of the input into the output. The first argument here is open_in x, that is the result of applying function open_in to argument x. Function open_in, defined in module File, opens a file for reading. The result open_in x is therefore an input which lets us read the contents of file x. The second argument of copy is stdout, that is the standard output, that is the screen. In other words, fun x -> copy (open_in x) stdout) is a function which takes as argument the name of a file, opens that file and prints its contents on the screen. Note that everything is done lazily, so the contents is never completely present in memory. In other words, this works on files of theoretically unlimited length.

Bottom line: this utility reads all the files whose names are given on the command line and prints their content on the screen. In three lines of code.

Note that we could equally have written our utility


open System, IO, File

iter f (args ()) where f x = copy (open_in x) stdout


open System, IO, File

let f x = copy (open_in x) stdout;;
iter f (args ())

or, in one line,


iter (fun x -> System.IO.copy (System.File.open_in x) stdout) (args ())

or a number of other variants.

Without Batteries Included, the same code, in OCaml, would have looked like


for i = 1 to Array.length Sys.argv - 1 do
  let file = open_in Sys.args.(i) in
  let ended = ref false in
    while not !ended
      try print_endline (input_line file)
      with End_of_file -> ended := true
    do
done

Not much longer but definitely more complicated. Oh, and for fun, here’s the Java version:


import java.io.*;

public class Demo {
  public static final void main(String args[]) {
    try {
      for(String x : args) {
        BufferedReader reader = new BufferedReader(new FileReader(x));
        for(String read = reader.readLine();
                       read != null;
                       read = reader.readLine())
            System.out.println(read);
      }
    } catch (IOException e) {
    }
  }
}

Note quite as short, note quite as simple and, in my humble opinion, not quite as nice.

OCaml — now with on-line help

yoric — Thu, 06 Nov 2008 11:34:42 +0000

I’m happy to tell you that the on-line help system for the OCaml toplevel has just been committed to our new git repository.

So what does this mean?

Let’s take a look at a sample session:

$ rlwrap ocamlfind batteries/ocaml

        Objective Caml version 3.10.2

----------------------------------------------
|                                            |
|     This is OCaml, Batteries Included.     |
|                                            |
|                                            |
|      If you need help, type '#help;;'      |
|                                            |
----------------------------------------------

        Camlp4 Parsing version 3.10.2

We can type “#help“? Well, that’s new. Let’s type “#help;;“.

# #help;;
Welcome to OCaml, Batteries Included.

You are currently using the Toplevel of OCaml. You can think of it as
a powerful graphics calculator, or as a command-line comparable to
that of your operating system. It is also a nice way of learning OCaml
and of testing your algorithms and programs.

Notice character '#', at the start of some lines? That's called the
"prompt", it means that the Toplevel is waiting for your
orders.

Orders can take the form of value definitions, type definitions,
evaluations or directives. Value definitions let you teach OCaml new
tricks. Type definitions let you teach OCaml how to understand new
kinds of values informations and how to check that you're not making
mistakes when using your definitions. Evaluations let you actually do
something with your definitions, such as computing the result of your
algorithm, but also doing more complex things such as opening
applications, modifying files, or even creating websites. Any program
is composed of value definitions, type definitions and evaluations.

Finally, directives are not part of any program. They just let you
interact with the Toplevel, for instance by loading other programs,
reading the documentation, asking the Toplevel for information on
existing programs, etc. Directives are easy to spot, they also start
with character '#'. So, when you wrote "#help;;", that was a directive.

At this point, you may be interested in the following directives:
 #quit;;                   (*Use this to quit OCaml.            *)
 #use "some_file.ml";;     (*Use this to load another program.  *)
 #require "some_package";; (*Use this to load a library installed
                             on your computer.                  *)
 #help;;                   (*Well, you just used that one.      *)
 #man "some subject";;     (*Read the manual on some subject.   *)

Oh, don't forget: each order you give to the Toplevel should end with
";;".  That's how the Toplevel understands that you're done
writing. Otherwise, no matter how much you write, the Toplevel assumes
that you still intend to write more.

If you are new to programming, now is time to write
 #man "the tutorial";;

If you already know some other programming language, you should probably write
 #man "for programmers";;

Let’s take a look at this tutorial.

# #man "the tutorial";;
Opening help on language construction "the tutorial" (OCaml Batteries Included)

My web browser opens the tutorial at http://www.cs.jhu.edu/~scott/pl/lectures/caml-intro.html.

# #man "print_endline";;
Opening help on value "print_endline" (OCaml Batteries Included)

My web browser now displays the ocamldoc-generated help of Pervasives.print_endline.

# #man "input";;
Several definitions exist for "input".
This item exists as a method. For more information on "input" as a method, you may use
  #man_method "input";;

Item "input" also exists as a type. For more information on "input" as a type, you may use
  #man_type "input";;

Item "input" also exists as a value, with several possibilities. For more information on "input" as a value, you may use
  #man_value "Util.Digest.input";;
  #man_value "System.IO.input";;
  #man_value "Standard.input";;
  #man_value "Meta.Marshal.input";;

etc.

Nice, isn’t it?
We hope that this feature, together with the extended documentation provided with OCaml Batteries Included will prove useful for beginners and veterans alike.