arxiv:2410.07520
🔄 In a Training Loop
Tarun Jain
lucifertrj
AI & ML interests
Deep Learning, FPGA, and ML
Recent Activity
posted an update 1 day ago
You can now automate EDD (eval-driven development) with Coding Harness Agents
> build a baseline LLM-based application
> score every change with judge evals
> keep what improves, reject what regresses
I made a tutorial on what EDD is, how it works, and how to use eval scores across experiments to improve an LLM app. It builds on Jeffrey's (Confident AI) article on EDD and Eugene Yan's write-up on product evals.
> Setup: a baseline RAG app using Qdrant and Gemini that every experiment starts from
> Step 1: a binary-labelled dataset with critiques
> Step 2: aligning the LLM-as-a-judge evaluator with Opik evals
> Step 3: a harness loop that runs each experiment and scores it against the baseline.
Tracing and experiment comparison then show what improved, what regressed, and what to tweak next.
Source code is open source.
Full guide (source code linked in the description): https://www.youtube.com/watch?v=e6akw_fKWPk reacted to theirpost with 🔥 13 days ago
Published a guide to TurboQuant quantization: how the algorithm works and what Qdrant adds on top of it.
It also includes a benchmark comparing float32, scalar, binary and TurboQuant across BEIR's SciFact, ArguAna and NFCorpus, measured with recall@10, precision@10 and nDCG@10.
🔗 HF article: https://huggingface.co/blog/lucifertrj/turboquant-quantization-explained posted an update 15 days ago
Published a guide to TurboQuant quantization: how the algorithm works and what Qdrant adds on top of it.
It also includes a benchmark comparing float32, scalar, binary and TurboQuant across BEIR's SciFact, ArguAna and NFCorpus, measured with recall@10, precision@10 and nDCG@10.
🔗 HF article: https://huggingface.co/blog/lucifertrj/turboquant-quantization-explainedOrganizations
