RAG from First Principles
20 partsBuild a working RAG system by hand, one concept at a time, with runnable code and interactive figures.
57 essays · 57 also in Türkçe
Essays
Framework-free courses that build RAG, agents and evals by hand, and essays on the systems around them.
Courses
-
-
Agents from First Principles
17 partsGrow one agent from a bare loop into a durable, observable, secured production system.
-
Evals from First Principles
15 partsBuild the measurement stack by hand, one concept at a time, so you can tell a real 2% gain from noise.
RAG from First Principles
- Part 1
Why RAG Exists
19 min read Available in EN · TR
Even a powerful LLM answers confidently and wrong about your own documents and yesterday's events. Part 1 of a from-scratch series on Retrieval-Augmented Generation: no code, just the four problems RAG solves and one durable mental model.
- #RAG
- #LLM
- #Retrieval
- Part 2
Embeddings, Truly Understood
28 min read Available in EN · TR
RAG retrieves the 'relevant' information, but how does a computer decide what counts as relevant? Part 2 of a from-scratch series on Retrieval-Augmented Generation: how we turn meaning into numbers, why similar meanings land close together, and the quiet geometric trick that makes search by meaning possible.
- #RAG
- #Embeddings
- #Vectors
- Part 3
Measuring Similarity
19 min read Available in EN · TR
Relevant just means close in embedding space, but how do you turn 'close' into a single number you can rank by? Part 3 of a from-scratch series on Retrieval-Augmented Generation: Euclidean distance, the dot product, and why cosine similarity, which measures direction and ignores length, is the default for scoring chunks in RAG.
- #RAG
- #Embeddings
- #Vector Search
- Part 4
Vector Databases and Indexing
30 min read Available in EN · TR
You can score one chunk against a query, but doing it for every chunk is exact, brute-force k-NN: perfectly accurate and painfully O(n). Part 4 of a from-scratch series on Retrieval-Augmented Generation: why ordinary database indexes break in high dimensions, the speed-versus-recall trade-off behind approximate nearest-neighbor (ANN) search, the intuition for HNSW and IVF, and what a vector database actually stores and does.
- #RAG
- #Vector Database
- #ANN
- Part 5
Documents and Chunking
18 min read Available in EN · TR
We have a retrieval engine, but it rests on one quiet assumption: that documents arrive as tidy chunks. Part 5 of a from-scratch series on Retrieval-Augmented Generation: getting clean text out of messy formats, why we chunk at all, the too-small versus too-large tension, the main splitting strategies (fixed-size, recursive, structure-aware, semantic), and the two dials that quietly decide retrieval quality, chunk size and overlap. Bad chunks poison everything downstream.
- #RAG
- #Chunking
- #Document Processing
- Part 6
Build Your First RAG
17 min read Available in EN · TR
Five parts of theory, now one running program. Part 6 of a from-scratch series on Retrieval-Augmented Generation: build a complete chat-with-your-documents app by hand in Python, no framework hiding the mechanics. Embed with a local model, store vectors in plain NumPy, score by cosine similarity, retrieve top-k, ground the prompt, and generate, then swap in a real vector database. Every line ties back to a concept you already learned.
- #RAG
- #Python
- #Embeddings
- Part 7
Retrieval Deep Dive
28 min read Available in EN · TR
Our Part 6 app works, but it retrieves naively: pure semantic search with a fixed top-k. Part 7 of a from-scratch series on Retrieval-Augmented Generation: why dense retrieval whiffs on exact codes and names, the sparse (keyword) retrieval that nails them, TF-IDF and BM25 explained by intuition, how hybrid search fuses the two (weighted sum and Reciprocal Rank Fusion), and why top-k is a real knob with a lost-in-the-middle trap. Dense and sparse fail in opposite directions; combine them.
- #RAG
- #Retrieval
- #BM25
- Part 8
Making Retrieval Smarter
27 min read Available in EN · TR
Part 8 of a from-scratch series on Retrieval-Augmented Generation. First-pass retrieval is fast but only roughly right: the best chunk can sit at rank six. Sharpen it with three levers, in pipeline order. Before retrieval, transform the query (multi-query, HyDE, step-back, decomposition). During retrieval, filter by metadata. After retrieval, rerank a wide candidate set with a cross-encoder and keep the best few. Includes a focused code addition that adds reranking and a metadata filter to the app you built in Part 6.
- #RAG
- #Retrieval
- #Reranking
- Part 9
Advanced Retrieval Patterns
21 min read Available in EN · TR
Eight parts in, your pipeline retrieves well. But it still assumes one unit of text does triple duty: the thing you embed, the thing you search, and the thing you hand the model. Part 9 of a from-scratch series on Retrieval-Augmented Generation breaks that assumption. The big idea is decoupling: the best unit to search on (small, sharp) is rarely the best unit to generate from (large, rich). Four patterns put it to work, parent-document, sentence-window, self-querying, and contextual compression, with one focused code addition on the running app.
- #RAG
- #Retrieval
- #Chunking
- Part 10
Advanced RAG Architectures
26 min read Available in EN · TR
The leap from a fixed pipeline that runs the same way every time to a dynamic, decision-making loop that can choose whether to retrieve, judge what came back, and try again. Part 10 of a from-scratch series on Retrieval-Augmented Generation: a guided tour of Agentic RAG, Corrective RAG (CRAG), Self-RAG, GraphRAG, and Multi-Modal RAG, what control flow each one adds, and the sober cost of reaching for any of them.
- #RAG
- #Agentic RAG
- #GraphRAG
- Part 11
Evaluating RAG
33 min read Available in EN · TR
How to replace vibes with numbers. Part 11 of a from-scratch series on Retrieval-Augmented Generation: the two failure surfaces of a RAG system, the core metrics that probe each one (context precision and recall, faithfulness, answer relevance), LLM-as-a-judge and its biases, the frameworks that automate it, how to build an evaluation set, and the disciplined loop that turns guessing into engineering.
- #RAG
- #Evaluation
- #RAGAS
- Part 12
RAG in Production
34 min read Available in EN · TR
The finale. A RAG system that works in a notebook is about 20 percent of the job; the other 80 percent is making it fast, cheap, reliable, secure, and observable under real traffic. Part 12 of a from-scratch series on Retrieval-Augmented Generation: where latency and cost actually go and how to cut them, caching (including semantic caching), monitoring and tracing, failing gracefully, and the most underrated topic of all, security (prompt injection and data leakage). It closes with a capstone checklist for the whole series and a warm send-off.
- #RAG
- #Production
- #Latency
- Part 13
Late-Interaction Retrieval
25 min read Available in EN · TR
Single-vector embeddings throw away token-level signal. Late interaction keeps a vector per token and scores with MaxSim, getting cross-encoder-quality matching at bi-encoder serving cost. Part 13 of a from-scratch series on Retrieval-Augmented Generation, opening the Frontier Track: ColBERT and ColBERTv2, MaxSim by hand in numpy, the storage tradeoff, and how ColPali extends late interaction to document page images without OCR or chunking.
- #RAG
- #Retrieval
- #ColBERT
- Part 14
Context-Aware Chunking
26 min read Available in EN · TR
A chunk that reads fine in isolation can be uninterpretable once it leaves its document: 'she' no longer resolves to 'Alice', 'the policy' loses its antecedent. Part 14 of a from-scratch series on Retrieval-Augmented Generation, on the Frontier Track: two training-free fixes, late chunking (pool token spans after the transformer) and Anthropic's Contextual Retrieval (prepend an LLM-written situating sentence before embedding), built by hand and compared.
- #RAG
- #Chunking
- #Late Chunking
- Part 15
Adaptive RAG
23 min read Available in EN · TR
Not every query needs the same machinery: a greeting needs no retrieval, a fact needs one lookup, a comparison needs several. Part 15 of a from-scratch series on Retrieval-Augmented Generation and the close of the Frontier Track: a small complexity classifier that routes each query to no-retrieval, single-step, or multi-step retrieval, unifying the pipelines built across Parts 6 to 10 into one adaptive system.
- #RAG
- #Adaptive RAG
- #Routing
- Part 16
RAG vs Long-Context vs CAG
23 min read Available in EN · TR
Part 1 asked why RAG exists. Part 16 asks the harder follow-up: when do you even need retrieval? Context windows reach about a million tokens in 2026, so sometimes you can just stuff everything in, and Cache-Augmented Generation (CAG) preloads a small, stable corpus once and reuses the cached KV state instead of retrieving. This part works out the prompt-caching economics that decide between them and gives you a clear decision matrix: massive or fast-moving or private corpus to RAG, small and stable to CAG or long-context, mid-size to long-context.
- #RAG
- #Long Context
- #CAG
- Part 17
Securing RAG
24 min read Available in EN · TR
RAG widens the attack surface in a way ordinary apps do not: its whole premise is feeding external, often untrusted, content straight into a powerful model's prompt. Part 17 of a from-scratch series on Retrieval-Augmented Generation: the threats unique to RAG (indirect prompt injection through retrieved documents, knowledge-base poisoning, cross-tenant leakage) and the layered defensive pipeline that contains them, from input redaction and provenance scoring to a delimited untrusted-context wall, decline-if-not-grounded, output filtering, and identity-scoped access control.
- #RAG
- #Security
- #Prompt Injection
- Part 18
Structured and SQL RAG
22 min read Available in EN · TR
Most enterprise knowledge does not live in documents, it lives in databases and tables, and dense passage retrieval cannot answer a question whose answer has to be computed. Part 18 of a from-scratch series on Retrieval-Augmented Generation: text-to-SQL with RAG (retrieve the schema, generate SQL, execute, answer), table retrieval and the scaling reality, and routing text-search versus SQL per query.
- #RAG
- #Text-to-SQL
- #Structured Data
- Part 19
Building a RAG Agent
18 min read Available in EN · TR
Part 19 of a from-scratch series on Retrieval-Augmented Generation: take the agentic RAG that Part 10 only toured in prose (the ReAct loop, tool use, routing, multi-hop) and build a real agent by hand, with four tools, a reason/act/observe loop, an honest step budget, and three traces you can read line by line.
- #RAG
- #Agentic RAG
- #ReAct
- Part 20
Conversational RAG
19 min read Available in EN · TR
Part 20 of a from-scratch series on Retrieval-Augmented Generation: give the one-shot agent a memory. Build multi-turn RAG by hand, where query condensation rewrites a context-dependent follow-up into a standalone question before retrieval, so 'what about damaged items?' finally finds the right chunk.
- #RAG
- #Conversational RAG
- #Multi-turn
Agents from First Principles
- Part 1
The Augmented LLM
18 min read Available in EN · TR
A bare model cannot touch your data or act. The augmented LLM (a model ringed by typed tools in the smallest loop with a stop condition) can. Part 1 names the primitive, draws the do-you-even-need-an-agent ladder, and gives tools a contract.
- #Agents
- #LLM
- #Tool Use
- Part 2
When Tools Fail
16 min read Available in EN · TR
A schema-valid tool call can still throw, time out, or return garbage at run time. The fix is a robustness layer: a failure taxonomy, bounded retries, and an idempotent refund tool.
- #Agents
- #Tool Use
- #Reliability
- Part 3
Planning the Work
14 min read Available in EN · TR
ReAct pays an LLM call per hop, so the bill balloons and the plan is re-derived every step. Making the plan a first-class artifact (plan-and-execute, ReWOO, the tool DAG) cuts LLM calls and critical-path depth.
- #Agents
- #Planning
- #ReAct
- Part 4
Surviving a Broken Plan
14 min read Available in EN · TR
An up-front plan is a bet that the world will not change. When a SKU is discontinued mid-run, a prospective critic and an error-triggered replanner that revises only the remaining steps save the run.
- #Agents
- #Planning
- #Replanning
- Part 5
Learning from Failure
13 min read Available in EN · TR
Replanning recovers within a run, but the agent repeats the same mistake across runs. In-loop self-critique fixes what it can see; Reflexion writes a verbal lesson to a buffer the next trial reads.
- #Agents
- #Reflection
- #Reflexion
- Part 6
The Four Memories
14 min read Available in EN · TR
A flat buffer cannot separate what happened from what is true from how to do a task, and read-only tools cannot update state. Four typed stores plus self-editing memory fix it.
- #Agents
- #Memory
- #MemGPT
- Part 7
Surviving the Long Haul
16 min read Available in EN · TR
A long run overflows the window and a grow-only store turns to noise. Hot/warm/cold compaction bounds the window; forgetting (decay, supersede, evict) keeps memory useful.
- #Agents
- #Memory
- #Context Window
- Part 8
Stopping the Runaway
15 min read Available in EN · TR
An agent that never gives up retries a dead search forever and burns the budget. Multi-dimensional budgets, a loop detector, and a circuit breaker trip it to a graceful partial result.
- #Agents
- #Reliability
- #Budgets
- Part 9
The Durable Agent
18 min read Available in EN · TR
A crash mid-run loses the half-issued refund and a naive restart re-charges the customer. A journal, replay, and idempotency keys make the agent effectively-once.
- #Agents
- #Durability
- #Event Sourcing
- Part 10
Pause, Approve, Resume, Steer
17 min read Available in EN · TR
Some actions need human approval and a user may want to correct the agent mid-run. A journal-backed interrupt, resume, and steer put a human in the loop, durably.
- #Agents
- #Human-in-the-Loop
- #Approvals
- Part 11
Shipping It
19 min read Available in EN · TR
A durable agent you cannot see into is undebuggable. Fold the same journal into OTel-shaped spans for trajectory, cost, latency, and cost-per-success.
- #Agents
- #Observability
- #OpenTelemetry
- Part 12
Tools as a Protocol
19 min read Available in EN · TR
Tools are a hardcoded dict no other host can discover. A minimal MCP server and host by hand turn tool use into a JSON-RPC wire protocol with discovery and multiplexing.
- #Agents
- #MCP
- #Protocols
- Part 13
The Code-Running Tool
18 min read Available in EN · TR
Agents need to run code and act on a screen, not just call JSON tools, and that power is dangerous. A sandbox permission boundary contains it. The toy is illustrative.
- #Agents
- #Code Execution
- #Computer Use
- Part 14
The Supervisor and the Handoff
19 min read Available in EN · TR
One agent with one context cannot parallelize breadth-first work and is overkill for a specialist. Orchestrator-worker fan-out, handoff-as-a-tool, honest economics.
- #Agents
- #Multi-Agent
- #Orchestration
- Part 15
Agent to Agent
16 min read Available in EN · TR
A handoff stops at the process boundary; a billing agent owned by another team is unreachable. A2A Agent Cards, JSON-RPC delegation, and a trust allowlist cross it.
- #Agents
- #A2A
- #Interop
- Part 16
Securing the Agent
19 min read Available in EN · TR
An agent with tools, memory, code, and untrusted content is an attack surface. The lethal trifecta and indirect prompt injection, beaten by breaking a leg.
- #Agents
- #Security
- #Prompt Injection
- Part 17
Grading the Agent
21 min read Available in EN · TR
Eyeballing checks only the answer, so a right answer via a wrong, expensive, or unsafe path passes; a three-layer eval and a regression gate catch it. The finale.
- #Agents
- #Evaluation
- #Trajectory
Evals from First Principles
- Part 1
What Is a Score?
14 min read Available in EN · TR
Before any statistic, evals need three atoms: a task, a gold label, and a metric. Part 1 builds the 2x2 confusion matrix by hand from ten graded outputs, derives accuracy, precision, recall, and F1 from its four counts, and shows on an imbalanced set why accuracy alone lies.
- #Evals
- #LLM
- #Metrics
- Part 2
Building a Golden Set
17 min read Available in EN · TR
A metric is only as good as the labeled set beneath it. Part 2 builds a trustworthy golden set by hand on 40 support tickets: stratified sampling pins a rare class at exactly 2 urgent per draw while simple random sampling swings from 0 to 4, a leakage trap inflates accuracy from an honest 0.667 up to 0.800, and a three-criteria rubric turns 'good answer?' into a countable pass-at-5 bar.
- #Evals
- #LLM
- #AI
- Part 3
Do Humans Even Agree?
14 min read Available in EN · TR
The gold labels you grade against are themselves opinions, and opinions have noise. Part 3 takes 20 support tickets graded by two humans who agree on 18 of them, a scary-high 90%, and shows by hand why most of that agreement is chance: Cohen's kappa collapses it to 0.61, Fleiss' kappa handles three raters, and Krippendorff's alpha waits as the general case.
- #Evals
- #LLM
- #AI
- Part 4
LLM-as-Judge by Hand
13 min read Available in EN · TR
When you cannot afford a human grader for every output, you hand the rubric to a model instead. Part 4 runs twelve labeled support answers through a deterministic mock judge, parses each free-text verdict into PASS, FAIL, or ABSTAIN, and scores the judge against human gold on the ten it could grade.
- #Evals
- #LLM
- #AI
- Part 5
Judging the Judge
15 min read Available in EN · TR
An LLM judge is a model, so it carries systematic, measurable biases. Part 5 baits a deterministic mock judge with a slot-A bonus and a per-word reward, measures its position bias (7 of 12 verdicts flip when you swap who goes first) and verbosity bias (the longer answer wins all 5 tied probes), then corrects the position bias by averaging both orderings and watches disagreement with the truth fall from 5/12 to 1/12.
- #Evals
- #LLM
- #AI
- Part 6
Uncertainty
13 min read Available in EN · TR
One eval run graded 30 outputs and gave accuracy 0.70, a single number that looks solid until you ask how much it would wobble on a different sample. Part 6 builds the bootstrap by hand: resample the 30 items with replacement 10000 times, read the 2.5th and 97.5th percentiles as a 95% CI of [0.53, 0.87], and watch that interval halve as the set grows.
- #Evals
- #LLM
- #AI
- Part 7
Is the Difference Real?
15 min read Available in EN · TR
Two models are graded on the same 50-item set: A gets 41/50 (82%), B gets 42/50 (84%), a headline gap of +2 points. Part 7 builds the paired 2x2 of agreements by hand, runs McNemar's test on the 9 items where the models disagree (both an exact binomial p and the chi-square approximation), finds it insignificant, and then works out that catching a true 2-point gain would need thousands of examples.
- #Evals
- #LLM
- #AI
- Part 8
pass@k
12 min read Available in EN · TR
One coding task, five sampled completions, two that pass the hidden tests: this part derives the unbiased pass@k estimator as a combinatorial identity, checks it by brute-force enumeration, and proves in exact arithmetic why the tempting plug-in formula reports too low.
- #Evals
- #LLM
- #AI
- Part 9
Arenas
16 min read Available in EN · TR
How leaderboards turn a pile of head-to-head battles into one ranking. Part 9 runs four chatbots through an 18-battle round-robin arena, ranks them two ways (Elo updated match by match, then the Bradley-Terry MLE fit to the aggregate win matrix), and shows numerically why Elo depends on match order while Bradley-Terry does not.
- #Evals
- #LLM
- #AI
- Part 10
Calibration
13 min read Available in EN · TR
A confidence is a promise: of the answers a model tags 80% sure, about 80% should be right. Part 10 scores the same ten trivia confidences twice, once for an over-confident model and once for a well-calibrated one, builds the reliability curve by hand across three bins, and reduces the whole picture to ECE and the Brier score.
- #Evals
- #LLM
- #AI
- Part 11
Regression Gates
13 min read Available in EN · TR
A score is worth nothing if it cannot say no. Part 11 turns the whole series into one CI gate that runs three checks on two candidate runs of a 20-item eval, a bootstrap-CI threshold, an n-gram contamination scan, and a PSI drift comparison, and blocks the deploy of the run whose mean looked fine but whose interval and distribution did not.
- #Evals
- #LLM
- #AI
- Part 12
The Eval Flywheel
15 min read Available in EN · TR
Your best eval set is not something you write, it is something you mine from what your model already did. Part 12 takes 12 production traces from a support assistant, ranks them by a cheap suspicion score, and spends a budget of 6 annotations to grow a golden set that catches all 4 real failures where random sampling catches only 2.
- #Evals
- #LLM
- #AI
- Part 13
Online Evals
15 min read Available in EN · TR
Offline scores are necessary, not sufficient: the final judge is live traffic. Part 13 splits one week of a support chatbot's sessions 50/50, computes the resolution-rate lift and a by-hand two-proportion z-test, and then lets a refusal-rate guardrail overrule a real, significant +8 pp win into a DO-NOT-SHIP verdict.
- #Evals
- #LLM
- #AI
- Part 14
Agent-Trajectory Evals
17 min read Available in EN · TR
Grading an agent's final answer misses how it got there. Part 14 scores the tool-call path of eight runs against an approved reference trajectory four ways (exact, order-aware, set-overlap, and step-level credit), then grades a pass/fail rubric with a mock judge and a human and chance-corrects their agreement with Cohen's kappa.
- #Evals
- #LLM
- #AI
- Part 15
A Tiny Eval Harness
New (most recent essay)14 min read Available in EN · TR
The capstone wires the whole series into one small pure-Python harness: a dataset, a swappable scorer, a bootstrap confidence interval, and a gate. We run it three ways on two tiny cases and watch swapping only the scorer flip the same ten-item QA data from HOLD to SHIP.
- #Evals
- #LLM
- #AI
Standalone essays
-
Inference Is Where the Money Goes
25 min read Available in EN · TR
Training an LLM happens once. Inference happens every time someone hits enter. These are my notes on what it actually costs to serve an open-source model: where the GPU memory goes, why the KV cache and not the weights is the thing that bites you, what happens inside a single forward pass, and how quantization buys back 3x the throughput for less than a point of accuracy. Part 1 of a walkthrough of efficient LLM serving.
- #LLM
- #Inference
- #vLLM
-
When the Co-Pilot Is an Algorithm: How Europe Plans to Make Aviation AI Trustworthy
10 min read Available in EN · TR
The EU's aviation regulator just published a 239-page concept paper on making AI safe enough to fly. From artificial narrow intelligence to learning assurance and the W-shape process, the mental model behind trustworthy aviation AI, explained for non-experts.
- #EASA
- #Aviation
- #AI Safety
-
ASELSAN TOYGUN: Platform Bütünleşik EOTS Teknik İncelemesi
22 min read Available in EN · TR
TOYGUN'un nasıl gördüğünü, hedefi nasıl tanıyıp izlediğini, lazerle nasıl ölçüp işaretlediğini ve uçağın radar izini bozmadan tüm bunları nasıl yaptığını kademe kademe, sinyal zinciri ve alt sistemler ile anlatır.
- #ASELSAN
- #TOYGUN
- #EOTS
-
Fine-tuning an SLM on UAV combat doctrine
14 min read Available in EN · TR
End-to-end notes from a $55 LoRA build on gpt-oss-20b: the data pipeline, the training run, the evaluation, and the surprise that the biggest win was teaching a reasoning model to stop reasoning.
- #LLMs
- #Fine-tuning
- #LoRA
-
The Silent Variable in Graph RAG: Why Corpus Language Matters More Than You Think
5 min read Available in EN · TR
Chunk size, embeddings, re-rankers: the usual suspects. But the language of your corpus quietly shapes every layer of the pipeline, and reasoning models make it decisive.
- #Graph RAG
- #LLMs
- #Multilingual