Module Unmark.Benchmarks

module Benchmarks: sig .. end

Running and elimination of whole benchmarks suites.

type name = string list * string

Fully qualified names.

Filtering

type query = string list

Operations over benchmark suites support filtering, to restrict them to a subset of benchmarks.

Queries contain query components. Components are strings with syntax [NAME1[/NAME2...]].

Each benchmark is located in a tree of groups, and can be assigned a qualified name like group1/group2/bench. A query component is the full path to a benchmark or a group. As a special case, an empty NAME between slashes selects all groups at that level.

For example:

"a" selects the top-level benchmark named a, or all the benchmarks in the top-evel group a.
"a/b/x" selects the benchmark (or group) named x in group b in group a.
"a//x" selects the benchmark (or group) named x in any second-level subgroup of a.
"" selects everything.

Query selects all benchmarks selected by any of its components. Empty lists selects everything.

val matches : ?group:bool -> q:query -> name -> bool

matches ?group ~g name checks if name matches the query q in the sense above.

When group is true, the match is strictly more permissive: name is also selected by query components more specific than it, making target's parent groups selectable. Default false.

Running

type run = {

`suite : string;`	`(*`	Name of the benchmark suite.	`*)`
`note : string;`	`(*`	A note for this run.	`*)`
`time : float;`	`(*`	Run timestamp, UNIX time.	`*)`
`counters : (string * string option) list;`	`(*`	Counters and their units.	`*)`
`benchmarks : (name * Unmark.Attr.t * Unmark.Measurement.sample list) list;`	`(*`	Benchmark names, attributes, and their sample series.	`*)`

}

Raw results of running a benchmark suite.

val run : ?probe:Unmark.Measurement.Probe.probe ->
       ?min_t:float ->
       ?min_s:int ->
       ?q:query ->
       ?note:string -> suite:string -> Unmark.bench list -> run

run ?probe ?min_t ?min_s ?q ?note ~suite benchmarks runs the benchmark suite benchmarks named suite. Individual benchmarks are passed to measure.

q - Filtering query.

note - Text snippet attached to this particular run.

probe, min_t, min_s - Passed to Unmark.Measurement.measure.

module Run: sig .. end

Operations over runs.

Inspection

type 'a thunk

Bracketed thunks.

An 'a thunk is unit -> 'a which uses additional external resources. Thunks are internalization of Unmark.group_f.

val fold : bench:(name ->
              Unmark.Attr.t -> Unmark.Measurement.runnable -> 'r) ->
       group:(name -> 'r list thunk -> 'r) ->
       suite:('r list -> 'p) -> Unmark.bench list -> 'p

Benchmark eliminator.

val eval : 'a thunk -> 'a

eval t runs t. This can cause acquisition and release of associated resources.

val (%%) : ('a -> 'b) -> 'a thunk -> 'b thunk

f %% t extends t by applying f within the resource bracket.