AIDLC Evaluation Framework
Evaluation-driven Loop in Agent/LLM Development Process — Comparison of SWE-bench Verified, METR, Ragas, DeepEval, LangSmith, Braintrust, AWS Labs aidlc-evaluator
6 docs tagged with "evaluation"
View all tagsGovernance · Evaluation · Compliance
Comprehensive governance documentation covering quality evaluation, operational playbooks, AI Gateway guardrails, compliance, and domain customization
Monitoring Strategy for LLM Serving Optimization
Connect cache efficiency, KV capacity, latency, and routing across seven observability layers, with explicit delivery, evaluation coverage, and quality gates.
Prefix Cache Tuning and Accuracy Correlation Validation
Define cached-token, turn-gap, and preemption data contracts, correlation limits, and non-inferiority quality gates for prefix cache tuning.
Ragas RAG Evaluation Framework
RAG pipeline quality evaluation and continuous improvement using Ragas
Trace → Dataset Materializer
Export approved, redacted Langfuse generations as Parquet candidates and score them using explicit evaluator contracts. Conversion to GRPO/DPO inputs is a separate stage.