Module Maki

module Maki: sig .. end

Maki: Persistent Incremental Computations

Maki is a system for memoizing costly OCaml functions using the disk. It requires the functions to be pure, that is, to always return the same result given that the set of dependencies declared by the function doesn't change.

status: experimental

This module is not thread-safe.

type 'a or_error = ('a, exn) Result.result

type 'a lwt_or_error = 'a or_error Lwt.t

Basic types

type path = string

type program = string

type time = float

Controlling Parallelism

module Limit: sig .. end

Values

Maki deals with functions that take and return values of any type that provides a 'a Value.t implementation; that is, values that we can serialize, unserialize, and hash. The reason is that values are stored on disk for memoization purposes.

module Value: sig .. end

On-Disk storage

We use a generic interface for on-disk storage, in the form of a dictionary string -> string. The default storage just uses one file per pair.

module Storage: Maki_storage

Time Utils

module Time: sig .. end

Memoized Functions

This is the heart of the library: a wrapper around pure functions from Maki.Value arguments to a Maki.Value-aware type. Such functions are supposed to always return the same value given the same arguments and dependencies (a typical dependency is an external program that might have several version).

The Maki.call function is used to actually evaluate a wrapped function, or return its memoized result if the computation was done already.

We need to name functions because, from one execution to another, the results of a function call must be stored on disk. Names are used to map function calls to their result. If two different functions share the same name (even across programs), the results will be unpredictable.

type lifetime = [ `CanDrop | `Keep | `KeepFor of time | `KeepUntil of time ]

lifetime for a cached value

val call : ?bypass:bool ->
       ?storage:Storage.t ->
       ?lifetime:lifetime ->
       ?limit:Limit.t ->
       ?tags:string list ->
       name:string ->
       deps:Value.t list ->
       op:'res Value.ops -> (unit -> 'res Lwt.t) -> 'res or_error Lwt.t

Call the function iff its result has not been cached yet

bypass : if true, then cache is disabled

storage : the storage used for caching values (default Storage.get_default ())

lifetime : how long to keep the cached value (defautl: CanDrop)

limit : if given, call will acquire a handle from limit before calling the (potentially costly) function

name : the name of the function, should be unique!

deps : dependencies of the computation; if they change, the function will have to be re-computed. In other words, the function is supposed to be pure on the list of dependencies (same dependencies => same result)

val call_exn : ?bypass:bool ->
       ?storage:Storage.t ->
       ?lifetime:lifetime ->
       ?limit:Limit.t ->
       ?tags:string list ->
       name:string ->
       deps:Value.t list ->
       op:'res Value.ops -> (unit -> 'res Lwt.t) -> 'res Lwt.t

Same as Maki.call but raises the exception instead of wrapping it in Error

On-Storage Values

type cache_value

val cache_value_of_bencode : Bencode.t -> cache_value or_error

val bencode_of_cache_value : cache_value -> Bencode.t

val cache_value_lifetime : cache_value -> lifetime

val cache_value_fun_name : cache_value -> string

function used to compute

val cache_value_deps : cache_value -> string list

dependencies

val cache_value_data : cache_value -> string

the value itself

val cache_value_tags : cache_value -> string list

tags attached to this value

GC

Garbage Collection for the stored values. It needs to be called explicitely

type gc_stats = {

	`gc_kept : int;`
	`gc_removed : int;`

}

val string_of_gc_stats : gc_stats -> string

val gc_storage : ?remove_file:bool -> Storage.t -> gc_stats or_error Lwt.t

gc_storage s removes uneeded values and uneeded dependencies, and returns some statistics. It might take a long time.

remove_file : if true, when a path result is removed, the file it corresponds to is also deleted from the file system

Utils

val last_mtime : path -> time or_error

Last modification time of the file

val sha1 : path -> Sha1.t Lwt.t

sha1 f hashes the file f

val sha1_of_string : string -> Sha1.t

hash the given string

val abspath : path -> path

Make the path absolute

val shell : ?timeout:float -> ?stdin:string -> string -> (string * string * int) Lwt.t

shell cmd runs the command cmd and returns stdout, sterr, errcode.

stdin : optional input to the sub-process

val shellf : ?timeout:float ->
       ?stdin:string ->
       ('a, Format.formatter, unit, (string * string * int) Lwt.t)
       Pervasives.format4 -> 'a

Same as Maki.shell but with a format string. Careful with escaping!