Interface EvalConfig

Configuration for an evaluation run.

interface EvalConfig {
    concurrency?: number;
    timeoutMs?: number;
    retries?: number;
    continueOnError?: boolean;
    thresholds?: {
        pass?: number;
        warn?: number;
    };
    customScorers?: Record<string, ScorerFunction>;
}

Properties

concurrency? timeoutMs? retries? continueOnError? thresholds? customScorers?

Properties

Optional concurrency

concurrency?: number

Maximum concurrent evaluations

Optional timeoutMs

timeoutMs?: number

Timeout per test case (ms)

Optional retries

retries?: number

Number of retries on failure

Optional continueOnError

continueOnError?: boolean

Whether to continue on error

Optional thresholds

thresholds?: {
    pass?: number;
    warn?: number;
}

Scoring thresholds

Type declaration

  • Optional pass?: number
  • Optional warn?: number

Optional customScorers

customScorers?: Record<string, ScorerFunction>

Custom scorers

Settings

Member Visibility

Theme

On This Page

concurrencytimeoutMsretriescontinueOnErrorthresholdscustomScorers