What is Agenta?

Agenta is an open-source end-to-end platform that helps developers and product teams build and maintain robust AI applications powered with Large Language Models (LLMs). .

With agenta, you can:

1. Quickly experiment and compare prompts, configurations and complex pipelines (chains, Retrieval Augmented Generation (RAG), agents…)
2. Quickly create test sets and golden datasets for evaluation
3. Evaluate your applications with pre-existing or custom Evaluators
4. Annotate and A/B test your applications with human feedback
5. Collaborate with product teams for prompt engineering and evaluation
6. Deploy your application in one-click in the UI, through CLI, or through github workflows.

Agenta focuses on increasing the speed of the development cycle of LLM applications by increasing the speed of iteration.

Agenta integrates with all frameworks and model providers in the ecosystem, such as Langchain, LlamaIndex, OpenAI, Cohere, Mistral, Huggingface, and self-hosted open source LLMs such as the one served using vLLM

​

Overview of agenta

​

Why use agenta?

• If you need to collaborate with domain experts and want their feedback on your LLM apps, as well as their help experimenting with prompts and parameters without having to modify your code.
• If you want the flexibility of using code for writing LLM app, without being restricted by libraries, models, or frameworks.
• If you need to save, version, and compare different variants of your LLM apps on your own data.
• If you need a systematic way to programmatically evaluate your LLM apps.
• If you care about your data privacy and do not want to be proxied through third-party services.

​

Features

• Parameter Playground: Define your app’s parameters within your code and experiment with them through a user-friendly web platform.
• Test Sets: Build test sets using the UI, by uploading CSVs, or by connecting to your own data via our API.
• Evaluation: Evaluate your app on your test sets using different strategies (e.g., exact match, AI Critic, human evaluation, etc.).
• Deployment: Deploy your app as an API in just one click.
• Collaboration: Share your app with collaborators and receive feedback on it.

​

Getting Started

​

Getting Help

If you have questions or need support, here’s how you can reach us. We’d also ❤️ your support.