Create Evaluation

POST 
/evaluations

Creates a new comparison table document Raises: HTTPException: description Returns: description

Request

application/json

Body

required

app_id App Id (string)required
variant_ids string[]required
evaluators_configs string[]required
testset_id Testset Id (string)required
rate_limitobjectrequired
batch_size Batch Size (integer)required
max_retries Max Retries (integer)required
retry_delay Retry Delay (integer)required
delay_between_batches Delay Between Batches (integer)required
lm_providers_keysobject
anyOf
correct_answer_columnobject
anyOf
MOD1
string

Responses

200

Successful Response

application/json

Schema

Schema

• Array [

id Id (string)required
app_id App Id (string)required
project_id Project Id (string)required
variant_ids string[]required
variant_names string[]required
variant_revision_ids string[]required
revisions string[]required
testset_idobject
anyOf
MOD1
string
testset_nameobject
anyOf
MOD1
string
statusobjectrequired
type Type (string)required
valueobject
anyOf
errorobject
anyOf
Error
message Message (string)required
stacktraceobject
anyOf
MOD1
string
aggregated_resultsobject[]required

Array [

evaluator_configobject

required

anyOf

EvaluatorConfig

id Id (string)required

name Name (string)required

project_id Project Id (string)required

evaluator_key Evaluator Key (string)required

settings_valuesobject

anyOf

created_at Created At (string)required

updated_at Updated At (string)required

resultobjectrequired

type Type (string)required

valueobject

anyOf

errorobject

anyOf

Error

message Message (string)required

stacktraceobject

anyOf

MOD1

string

]

average_costobject

anyOf

Result

type Type (string)required

valueobject

anyOf

errorobject

anyOf

Error

message Message (string)required

stacktraceobject

anyOf

MOD1

string

total_costobject

anyOf

Result

type Type (string)required

valueobject

anyOf

errorobject

anyOf

Error

message Message (string)required

stacktraceobject

anyOf

MOD1

string

average_latencyobject

anyOf

Result

type Type (string)required

valueobject

anyOf

errorobject

anyOf

Error

message Message (string)required

stacktraceobject

anyOf

MOD1

string

created_at date-timerequired
updated_at date-timerequired

• ]

Example (from schema)

[
  {
    "id": "string",
    "app_id": "string",
    "project_id": "string",
    "variant_ids": [
      "string"
    ],
    "variant_names": [
      "string"
    ],
    "variant_revision_ids": [
      "string"
    ],
    "revisions": [
      "string"
    ],
    "testset_id": "string",
    "testset_name": "string",
    "status": {
      "type": "string",
      "error": {}
    },
    "aggregated_results": [
      {
        "evaluator_config": {},
        "result": {
          "type": "string",
          "error": {}
        }
      }
    ],
    "average_cost": {},
    "total_cost": {},
    "average_latency": {},
    "created_at": "2024-11-22T12:54:54.291Z",
    "updated_at": "2024-11-22T12:54:54.291Z"
  }
]

422

Validation Error

application/json

Schema

Schema

detailobject[]

Array [

locobject[]required

Array [

anyOf

MOD1

string

MOD2

integer

]

msg Message (string)required

type Error Type (string)required

]

Example (from schema)

{
  "detail": [
    {
      "loc": [
        "string",
        0
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

Loading...