Create Evaluation

POST 
/evaluations

Creates a new comparison table document Raises: HTTPException: description Returns: description

Request

application/json

Body

required

app_id App Id (string)required
revisions_ids string[]required
evaluators_configs string[]required
testset_id Testset Id (string)required

rate_limit

objectrequired

batch_size Batch Size (integer)required

max_retries Max Retries (integer)required

retry_delay Retry Delay (integer)required

delay_between_batches Delay Between Batches (integer)required
correct_answer_columnobject

anyOf

MOD1

string

Responses

200

Successful Response

application/json

Schema

Schema

• Array [

id Id (string)required
app_id App Id (string)required
project_id Project Id (string)required
variant_ids string[]required
variant_names string[]required
variant_revision_ids string[]required
revisions string[]required
testset_idobject

anyOf

MOD1

string

testset_nameobject

anyOf

MOD1

string

status

object

required

type Type (string)required

valueobject

anyOf

errorobject

anyOf

Error

message Message (string)required

stacktrace

object

anyOf

MOD1

string

aggregated_results

object[]

required

Array [

evaluator_configobject

required

anyOf

EvaluatorConfig

id Id (string)required

name Name (string)required

project_id Project Id (string)required

evaluator_key Evaluator Key (string)required

settings_values

object

anyOf

created_at Created At (string)required

updated_at Updated At (string)required

result

object

required

type Type (string)required

value

object

anyOf

error

object

anyOf

Error

message Message (string)required

stacktrace

object

anyOf

MOD1

string

]

average_costobject

anyOf

Result

type Type (string)required

value

object

anyOf

error

object

anyOf

Error

message Message (string)required

stacktrace

object

anyOf

MOD1

string

total_costobject

anyOf

Result

type Type (string)required

value

object

anyOf

error

object

anyOf

Error

message Message (string)required

stacktrace

object

anyOf

MOD1

string

average_latencyobject

anyOf

Result

type Type (string)required

value

object

anyOf

error

object

anyOf

Error

message Message (string)required

stacktrace

object

anyOf

MOD1

string

created_at date-timerequired
updated_at date-timerequired

• ]

Example (from schema)

[
  {
    "id": "string",
    "app_id": "string",
    "project_id": "string",
    "variant_ids": [
      "string"
    ],
    "variant_names": [
      "string"
    ],
    "variant_revision_ids": [
      "string"
    ],
    "revisions": [
      "string"
    ],
    "testset_id": "string",
    "testset_name": "string",
    "status": {
      "type": "string",
      "error": {}
    },
    "aggregated_results": [
      {
        "evaluator_config": {},
        "result": {
          "type": "string",
          "error": {}
        }
      }
    ],
    "average_cost": {},
    "total_cost": {},
    "average_latency": {},
    "created_at": "2025-05-15T13:01:54.097Z",
    "updated_at": "2025-05-15T13:01:54.097Z"
  }
]

422

Validation Error

application/json

Schema

Schema

detail

object[]

Array [

loc

object[]

required

Array [

anyOf

MOD1

string

MOD2

integer

]

msg Message (string)required

type Error Type (string)required

]

Example (from schema)

{
  "detail": [
    {
      "loc": [
        "string",
        0
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

Loading...