Evaluators can access all columns
Evaluators now can access all columns in the test set. Previously, you were limited to using only the correct_answer column for the ground truth / reference answer in evaluation.
Now you can configure your evaluator to use any column in the test set as the ground truth. To do that, open the collapsable Advanced Settings when configuring the evaluator, and define the Expected Answer Column to the name of the columns containing the reference answer you want to use.
In addition to this:
- We've upgraded the SDK to pydantic v2.
- We have improved by 10x the speed for the get config endpoint
- We have add documentation for observability