Documentation

Quickstart

Get started with RAGDebugger in under 5 minutes. This guide walks you through setting up your first project and running your first evaluation.

1. Install the SDK

Install the RAGDebugger Python SDK using pip:

pip install ragdebugger

2. Initialize the Client

Create a client instance with your API key:

from ragdebugger import RAGDebugger

client = RAGDebugger(api_key="your-api-key")
project = client.project("my-rag-app")

3. Log Your First Evaluation

Send a query-response pair for analysis:

result = project.evaluate(
    query="What is the return policy?",
    retrieved_chunks=[
        {"id": "doc1", "text": "Returns accepted within 30 days...", "score": 0.92},
        {"id": "doc2", "text": "Refunds processed in 5-7 days...", "score": 0.87}
    ],
    response="Our return policy allows returns within 30 days of purchase.",
    ground_truth="Returns are accepted within 30 days for full refund."
)

print(f"Retrieval Score: {result.retrieval_score}")
print(f"Groundedness: {result.groundedness_score}")

Installation

RAGDebugger supports Python 3.8+ and provides SDKs for common frameworks.

Python SDK

pip install ragdebugger

JavaScript/TypeScript SDK

npm install @ragdebugger/sdk

Docker (Self-Hosted)

docker pull ragdebugger/server:latest
docker run -p 8080:8080 ragdebugger/server:latest

Configuration

Configure RAGDebugger using environment variables or a configuration file.

Environment Variables

export RAGDEBUGGER_API_KEY="your-api-key"
export RAGDEBUGGER_PROJECT="my-project"
export RAGDEBUGGER_ENDPOINT="https://api.ragdebugger.tech"

Configuration File

# ragdebugger.yaml
api_key: ${RAGDEBUGGER_API_KEY}
project: my-project
defaults:
  retention_days: 90
  auto_label: true
  webhook_url: https://your-app.com/webhooks/ragdebugger

Importing Data

Import existing query logs and evaluation data for analysis.

Batch Import

from ragdebugger import RAGDebugger

client = RAGDebugger()
project = client.project("my-rag-app")

# Import from JSON file
project.import_evaluations("evaluations.json")

# Import from DataFrame
import pandas as pd
df = pd.read_csv("query_logs.csv")
project.import_from_dataframe(df)

Authentication

All API requests require authentication using an API key.

API Key Authentication

curl -X POST https://api.ragdebugger.tech/v1/evaluate \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "...", "chunks": [...], "response": "..."}'

Evaluations API

The Evaluations API allows you to submit query-response pairs for analysis.

POST /v1/evaluate

{
  "project": "my-project",
  "query": "What is the return policy?",
  "retrieved_chunks": [
    {"id": "doc1", "text": "...", "score": 0.92}
  ],
  "response": "Returns are accepted within 30 days.",
  "metadata": {
    "user_id": "user123",
    "session_id": "sess456"
  }
}

Vector Database Integrations

RAGDebugger integrates with popular vector databases for automatic retrieval logging.

Supported Databases

• Pinecone
• Weaviate
• Milvus
• Qdrant
• Chroma
• pgvector

Pinecone Integration

from ragdebugger.integrations import PineconeIntegration

integration = PineconeIntegration(
    api_key="pinecone-api-key",
    environment="us-west1-gcp"
)

project.add_integration(integration)