OCaml Batteries Included 1.0.0

January 23, 2010 § Leave a comment

I am happy to inform you of the release of OCaml Batteries Included 1.0.0. After I had to leave the project, it has been lead by Edgar Friendly, who seems to have done a very good job.

Now, I’ll return to my current Batteries Included project :)

OCaml Batteries Included Beta 1

April 6, 2009 § 9 Comments

Note This post is written on the 64th day of University strikes in France. During these 64 days, the government has rejected any negociation on the core reasons for the strike, has attempted to discredit the contestation (if I understand correctly, I am both « improductive » and a « mask-wearing commando ») and has used police intimidation and repression. The strike continues. Quite possibly, there will be no university exams this year and no baccalauréat. If repression continues increasing, no one can tell for sure what will happen. Nothing good, for sure.

After days and nights of coding, debugging and fighting over naming conventions, the OCaml Batteries Included team is proud to announce OCaml Batteries Included Beta 1. You can find the binaries here,  read the API documentation, the platform documentation, the release notes and the ChangeLog or the list of individual commits. A GODI package and a Debian/testing package are also available.

« Read the rest of this entry »

On-line interpreter for Batteries

March 25, 2009 § 6 Comments

Note This entry is written on the 51st day of nationwide strike in French Universities and Research laboratories. Still no sign of negociation from the government. Liquidation of the system continues.

I am happy to announce that the repository version of OCaml Batteries Included now has a full-featured (and working) interpreter.

« Read the rest of this entry »

OCaml Batteries Included, alpha 3

February 6, 2009 § Leave a comment

Dear programmers.

I am happy to inform you that, despite the in-progress liquidation of French Universities, OCaml Batteries Included Alpha 3 has landed. Barring any accident, this should be the final Alpha version, with a mostly stable API and module structure. You may now download it from the Forge. A GODI package is also available and a Debian package should follow soon. You may also read the documentation on-line.

So, what’s new with Alpha 3? Plenty of things, as you’ll see.

« Read the rest of this entry »

Let’s do it with Batteries

January 27, 2009 § 6 Comments

Or, OCaml is a scripting language, too.

Note: These extracts use the latest version of Batteries, currently available from the git. Barring any accident, this version should be made public within the next few days.

A few days ago, when writing some code for OCaml Batteries Included, I realized that, to properly embed Camomile’s Unicode transcoding module, I would need to manually write 500+ boring lines, all of them looking like:

| `ascii -> Encoding.of_name "ASCII"

The idea behind that pattern matching was to define a type-safe phantom type for text encodings. Upon installation, Camomile generates a directory containing about 540 files, one per text encoding, and it seemed like a good idea to rely upon something less fragile than a string name.

Of course, writing this pattern-matching manually was out of the question: it was boring, error-prone, and while Batteries deserves sacrifices, it doesn’t quite deserve that level of mind-numbing activities. The alternative was to generate both the list of constructors and the pattern-matching code from the contents of the directory. I could have done it with some scripting language but that sounded like a good chance to test-drive the numerous new functions of the String module of Batteries (73 for 28 in the Base Library).

The main program

The structure of the program is easy: read the contents of a directory. For each file, do some treatment on the file name and print the result:

open Shell
foreach (files_of argv.(1)) do_something

Here, foreach is the same function as iter but with its arguments reversed. It’s sometimes much more readable. Instead of reading the contents of a directory with Shell.files_of, we could just as well have traversed the command-line arguments with args, or read the lines of standard input using IO.lines_of stdin.

Actually, we could just as well generalize to a (possibly empty) set of directories. For this purpose, we just need to map our function files_of to the enumeration of command-line arguments. This yields an enumeration of enumerations, which we turn into a flat enumeration with flatten. In my mind, that’s somewhat nicer and more readable than nested loops.

Our main program now looks like:

open Shell, Enum
foreach (flatten (map files_of (args ()))) do_something

Or, for those of us who prefer operators to parenthesis:

open Shell, Enum
(foreach **> flatten **> map files_of **> args ()) do_something

String manipulation

It’s now time to take a file name and turn it into

  1. a nice constructor name
  2. a file name without extension,

That second point is the easiest, so let’s start with it. We have a function Filename.chop_extension just for this purpose. So, if we were interested only in printing the list of files without their extension, we could define

let do_something x = print_endline (Filename.chop_extension x)

The first point is slightly trickier, as we need to

  1. remove the extension from the file name (done)
  2. prepend character ` (trivial)
  3. replace any illicit character with _ (slightly more annoying, I know that the list of illicit characters which may actually appear in my list of files contains :, -, (, ) and whitespaces but I’d rather not go and check manually  which other characters may turn out problematic)
  4. prepend something before names which start with a digit, as digits cannot appear as the first character of an OCaml constructor (a tad annoying, too)
  5. make everything lowercase, just because it’s nicer (trivial).

Let’s deal with the third item, it’s bound to be central. Let’s see, replacing characters could be done with regular expressions, something I dislike, or with function String.map. It’s nicer, type-safer, and it has a counterpart Rope.map for Unicode, if we ever need one. Now, functions Char.is_letter and Char.is_digit will help us determine which names are safe. Using them together, we obtain the following function:

open Char
let replace s = String.map (fun c -> if is_letter c || is_digit c then c else '_') s

Let’s solve the fourth item on our list. We need to check the first character of a string and to determine whether it’s a digit. Well, we already know how to do this. Let’s call our prefix p:

let clean_digit p s = if is_digit s.[0] then p^s else s

If we chain up everything, we obtain

let constructor p s = "`" ^ (if is_digit r.[0] then p^r else r)
    where         r = lowercase (String.map (fun c -> if is_letter c || is_digit c then c else '_') s)

I like this where syntax.

Format

Now that we have both our strings, we just need to be able to combine and print them. For this purpose, Printf is probably the most concise tool. Here, we can just write

let print s1 s2 = Printf.printf " | %s -> %S\n" s1 s2

We could parameterize upon the format used by printf and we’re bound to do this sooner or later, but let’s keep it simple for now.

The complete program

open Shell, Enum

foreach (flatten **> map files_of **> args ()) do_something
  where do_something s =
   let name = Filename.chop_extension s in Printf.printf " | %s -> %S\n" c name
     where c = "`" ^ (if Char.is_digit r.[0] then "codemap_"^r else r)
     where r = lowercase (String.map (fun c -> if Char.is_letter c || Char.is_digit c then c else '_') name)

I don’t know about you but I find this pretty nice, for a type-safe language. I’m sure it would have been possible to make something shorter in Perl or awk, and suggestions are welcome regarding how to improve this but I’m rather happy. And, once again, we’re not trying to beat Python, Perl or awk in concision, just to do something comparably good, because we already beat them by far in speed and safety.

So, what do you think?

OCaml Batteries Included: The Hierarchy, reloaded

November 20, 2008 § Leave a comment

Well, my previous post on the Hierarchy of OCaml Batteries Included certainly triggered reactions. Essentially, judging from these, the OCaml community doesn’t seem to want of a module hierarchy. So here’s a reworked version of the library layout, without hierarchy. Again, feedback is appreciated and should go to the OCaml mailing-list.

Batteries

  1. Standard (automatically opened)
  2. Legacy
    1. Arg
    2. Array
  3. Future
    1. Lexers
      1. C
      2. OCaml

    I. Control

  4. Exceptions
  5. Return
  6. Monad Interfaces for monadic operations

    I.1. Concurrency

  7. Concurrency Interfaces for concurrency operations

    I.1.i. Built-in threads

  8. Condition
  9. Event
  10. Mutex
  11. RMutex
  12. Thread
  13. Threads A module containing aliases to Condition, Event

    I.1.ii. coThreads

  14. CoCondition
  15. CoEvent
  16. CoMutex
  17. CoRMutex
  18. CoThread
  19. CoThreads as Threads but with implementations coming from coThreads

    I.1.iii. Shared memory

  20. Shm_* Placeholders

    II. IO

  21. IO
    1. BigEndian
  22. GZip
  23. Bz2
  24. Zip
  25. Transcode

    III. Mutable containers

  26. Array
    1. Cap
      1. ExceptionLess
      2. Labels
    2. ExceptionLess
    3. Labels
  27. Bigarray
    1. Array1
    2. Array2
    3. Array3
  28. Dllist
  29. Dynarray
  30. Enum
    1. ExceptionLess
    2. Labels
  31. Global
  32. Hashtbl
    1. Make
      1. ExceptionLess
      2. Labels

    IV. Persistent containers

  33. Lazy
  34. List
    1. ExceptionLess
    2. Labels
  35. Map
    1. Make
      1. ExceptionLess
      2. Labels
  36. Option
    1. Labels
  37. PMap
  38. PSet
  39. RefList
    1. Index
  40. Queue
  41. Ref
  42. Set
    1. Make
      1. ExceptionLess
      2. Labels
  43. Stack
  44. Stream

    V. Data

  45. Unit

    V.1. Logical

  46. Bool
  47. BitSet

    V.2. Numeric

  48. Numeric Interfaces for number-related stuff
  49. Big_int
  50. Common
  51. Complex
  52. Float
  53. Int
  54. Int32
  55. Int64
  56. Native_int
  57. Num
  58. Safe_float placeholder
  59. Safe_int

    V.3. Textual

  60. Text Definition of text-related interfaces
  61. Buffer
  62. Char
  63. UTF8
  64. Rope
  65. UChar
  66. String
  67. StringText A module containing aliases to String and Char1
  68. RopeText As StringText but with implementations from Rope and UChar
  69. UTF8Text As StringText but with implementations from UTF8 and UChar
    1. Labels

    VI. Distribution-related stuff

  70. Packages
  71. Compilers

    VII. Internals

  72. Gc
  73. Modules
  74. Oo
    1. Private
  75. Weak
    1. Make

    VIII. Network (placeholders)

  76. URL
  77. Netencoding
    1. Base64
    2. QuotedPrintable
    3. Q
    4. URL
    5. Html

    VIII.1. Http

  78. Http
  79. Http_client
  80. Cgi_*
  81. Httpd_*
  82. MIME

    VIII.2. Ftp

  83. Ftp_client

    VIII.3. Mail

  84. Netmail
  85. Pop
  86. Sendmail
  87. Smtp

    VIII.4. Generic server

  88. Netplex_*

    VIII.5. RPC

  89. Rpc_*

    VIII.6. Languages

  90. Genlex
  91. Lexing
  92. CharParser
  93. UCharParser
  94. ParserCo
    1. Source
  95. Parsing
  96. Format
  97. Printf
  98. Str
  99. PCRE place-holder
  100. Scanf
    1. Scanning
  101. SExpr

    IX. System

  102. Arg
  103. File
  104. OptParse
    1. Opt
    2. OptParser
    3. StdOpt
  105. Path
  106. Shell
  107. Unix
    1. Labels
  108. Equeue

    X. Unclassified

  109. Digest
  110. Random
    1. State
  111. Date placeholder


1
Actually a slightly modified version of Char to match signatures for Latin-1 and Unicode

Batteries: reworking the hierarchy (feedback wanted)

November 18, 2008 § Leave a comment

As readers interested in OCaml may know, we’ve been working for several months of OCaml Batteries Included. Early in the development, it appeared to us that, with the large number of modules involved, we would need a hierarchy of modules.

For instance, for the moment, we have a module System containing among other submodules IO (definition of i/o operations), File (definition of operations on files), Sys (the usual OCaml Sys module, soon to be expanded), etc.  Therefore, before one may open and manipulate files, one has to do

open System.IO;;
open System.File;;

or, with the syntax extension we developed to alleviate this,

open System, IO, File;;

the syntax extension does a few other things which we’re not going to detail here — for one thing, it allows local opening of modules).

Now that we’ve reached Alpha 2 and are well on our way towards Alpha 3, we’ve decided that it’s time to rework our current hierarchy and make it shorter and more consistent. Before we proceed, we’d like some feedback from the community. Discussion will take place mostly on the OCaml mailing-list and on irc (server: Freenode, channel: #ocaml) but I preferred posting the hierarchy here for easier reading.

At the moment, our hierarchy looks roughly as follows:

Batteries (automatically opened)

  1. Control
    1. Concurrency
      1. Common
      2. Threads
        1. Condition
        2. Event
        3. Mutex
        4. Thread
    2. Exceptions
    3. Return
    4. Monad
  2. Data
    1. Containers (common interfaces)
    2. Mutable
      1. Array
        1. Cap
          1. Labels
          2. ExceptionLess
        2. Labels
        3. ExceptionLess
      2. Bigarray
        1. Array1
        2. Array2
        3. Array3
        4. Genarray
      3. Dllist
      4. Dynarray
      5. Enum
        1. ExceptionLess
      6. Global
      7. Hashtbl
        1. Make
          1. Labels
          2. ExceptionLess
      8. RefList
        1. Index
      9. Queue
      10. Ref
      11. RefList
      12. Stack
      13. Stream
    3. Logical
      1. Bool
      2. BitSet
    4. Numeric
      1. Big_int
      2. Common
      3. Complex
      4. Float
      5. Int
      6. Int32
      7. Int64
      8. Native_int
      9. Num
      10. Safe_int
      11. Unit
      12. Common
    5. Persistent
      1. Lazy
      2. List
        1. ExceptionLess
        2. Labels
      3. Map
        1. Make
          1. ExceptionLess
          2. Labels
      4. Option
        1. Labels
      5. PMap
      6. PSet
      7. Set
        1. Make
          1. ExceptionLess
          2. Labels
    6. Text
      1. Buffer
      2. Char
      3. UTF8
      4. Rope
      5. UChar
      6. String
        1. ExceptionLess
        2. Labels
  3. Languages
    1. CharParser
    2. Genlex
      1. Languages
        1. Library
          1. C
          2. OCaml
    3. Format
    4. Lexing
    5. ParserCo
      1. Source
    6. Parsing
    7. Printf
    8. Scanf
      1. Scanning
    9. SExpr
    10. Str
  4. Meta
    1. Callback
    2. CamlinternalMod
    3. CamlinternalOo
    4. Gc
    5. Obj
    6. Oo
    7. Weak
      1. Make
  5. Standard (automatically opened)
  6. System
    1. Arg
    2. Compress
      1. GZip
    3. File
    4. Filename
    5. IO
      1. BigEndian
    6. Network (placeholder)
    7. OptParse
      1. Opt
      2. OptParser
      3. StdOpt
    8. Unix
      1. Labels
    9. Sys
  7. Toolchain
    1. Execute
    2. Findlib
  8. Util
    1. Digest
    2. Random
      1. State
  9. Legacy

One possible replacement has been drafted:

  1. Control
    1. Concurrency
      1. Common
      2. Threads
        1. Condition
        2. Event
        3. Mutex
        4. Thread
      3. Exceptions
      4. Return
      5. Monad
  2. Data
    1. Containers
      1. Common
      2. Array
        1. Cap
          1. ExceptionLess
          2. Labels
        2. ExceptionLess
        3. Labels
      3. Bigarray
        1. Array1
        2. Array2
        3. Array3
        4. Genarray
      4. Dllist
      5. Dynarray
      6. Enum
        1. ExceptionLess
        2. Labels
      7. Global
      8. Hashtbl
        1. Make
          1. ExceptionLess
          2. Labels
      9. Lazy
      10. List
        1. ExceptionLess
        2. Labels
      11. Map
        1. Make
          1. ExceptionLess
          2. Labels
      12. Option
        1. Labels
      13. PMap
      14. PSet
      15. RefList
        1. Index
      16. Queue
      17. Ref
      18. RefList
      19. Set
      20. Make
        1. ExceptionLess
        2. Labels
      21. Stack
      22. Stream
    2. Logical
      1. Bool
      2. BitSet
      3. Unit
    3. Numeric
      1. Interfaces (formerly Common)
      2. Big_int
      3. Common
      4. Complex
      5. Float
      6. Int
      7. Int32
      8. Int64
      9. Native_int
      10. Num
      11. Safe_int
    4. Text
      1. Buffer
      2. Char
      3. UTF8
      4. Rope
      5. UChar
      6. String
        1. Labels
  3. Distro
    1. Packages (formerly Findlib)
    2. Tools (formerly Execute)
  4. Internals
    1. Callback
    2. Gc
    3. Mod (formerly CamlinternalMod)
    4. Obj
    5. Oo
      1. Private (formerly CamlinternalOo)
    6. Weak
      1. Make
  5. IO
    1. BigEndian
    2. Gzip
    3. Bz2
    4. Zip
    5. Transcode (placeholder)
  6. Network (placeholder)
  7. Text (formerly Languages)
    1. Lex
      1. Genlex
        1. Languages
          1. C
          2. OCaml
      2. Lexing
    2. Parse
      1. CharParser
      2. UCharParser
      3. ParserCo
        1. Source
      4. Parsing
    3. Pretty
      1. Format
    4. Printf
    5. Regexp
      1. Str
      2. PCRE (in the future)
    6. Scanf
      1. Scanning
    7. SExpr
    8. XML (placeholder)
  8. Standard (automatically opened)
  9. Sys
    1. Arg
    2. File
    3. OptParse
      1. Opt
      2. OptParser
      3. StdOpt
    4. Path (formerly Filename)
    5. Shell (formerly Sys)
    6. Unix
      1. Labels
  10. Util
    1. Digest
    2. Random
      1. State
  11. Legacy

The overall goal, with this new version, is to make for

  • a shallower hierarchy
  • shorter module names
  • more consistent groups of modules.

What do you think of this replacement? Is it any better?

Where Am I?

You are currently browsing the batteries included category at Il y a du thé renversé au bord de la table.

Follow

Get every new post delivered to your Inbox.

Join 32 other followers