57 essays · 57 also in Türkçe
RAG from First Principles — runnable code & step-by-step notebooks for the 20-part series Code on GitHub ↗Even a powerful LLM answers confidently and wrong about your own documents and yesterday's events. Part 1 of a from-scratch series on Retrieval-Augmented Generation: no code, just the four problems RAG solves and one durable mental model.
RAG retrieves the 'relevant' information, but how does a computer decide what counts as relevant? Part 2 of a from-scratch series on Retrieval-Augmented Generation: how we turn meaning into numbers, why similar meanings land close together, and the quiet geometric trick that makes search by meaning possible.
Relevant just means close in embedding space, but how do you turn 'close' into a single number you can rank by? Part 3 of a from-scratch series on Retrieval-Augmented Generation: Euclidean distance, the dot product, and why cosine similarity, which measures direction and ignores length, is the default for scoring chunks in RAG.
You can score one chunk against a query, but doing it for every chunk is exact, brute-force k-NN: perfectly accurate and painfully O(n). Part 4 of a from-scratch series on Retrieval-Augmented Generation: why ordinary database indexes break in high dimensions, the speed-versus-recall trade-off behind approximate nearest-neighbor (ANN) search, the intuition for HNSW and IVF, and what a vector database actually stores and does.
We have a retrieval engine, but it rests on one quiet assumption: that documents arrive as tidy chunks. Part 5 of a from-scratch series on Retrieval-Augmented Generation: getting clean text out of messy formats, why we chunk at all, the too-small versus too-large tension, the main splitting strategies (fixed-size, recursive, structure-aware, semantic), and the two dials that quietly decide retrieval quality, chunk size and overlap. Bad chunks poison everything downstream.
Five parts of theory, now one running program. Part 6 of a from-scratch series on Retrieval-Augmented Generation: build a complete chat-with-your-documents app by hand in Python, no framework hiding the mechanics. Embed with a local model, store vectors in plain NumPy, score by cosine similarity, retrieve top-k, ground the prompt, and generate, then swap in a real vector database. Every line ties back to a concept you already learned.
Our Part 6 app works, but it retrieves naively: pure semantic search with a fixed top-k. Part 7 of a from-scratch series on Retrieval-Augmented Generation: why dense retrieval whiffs on exact codes and names, the sparse (keyword) retrieval that nails them, TF-IDF and BM25 explained by intuition, how hybrid search fuses the two (weighted sum and Reciprocal Rank Fusion), and why top-k is a real knob with a lost-in-the-middle trap. Dense and sparse fail in opposite directions; combine them.
Part 8 of a from-scratch series on Retrieval-Augmented Generation. First-pass retrieval is fast but only roughly right: the best chunk can sit at rank six. Sharpen it with three levers, in pipeline order. Before retrieval, transform the query (multi-query, HyDE, step-back, decomposition). During retrieval, filter by metadata. After retrieval, rerank a wide candidate set with a cross-encoder and keep the best few. Includes a focused code addition that adds reranking and a metadata filter to the app you built in Part 6.
Eight parts in, your pipeline retrieves well. But it still assumes one unit of text does triple duty: the thing you embed, the thing you search, and the thing you hand the model. Part 9 of a from-scratch series on Retrieval-Augmented Generation breaks that assumption. The big idea is decoupling: the best unit to search on (small, sharp) is rarely the best unit to generate from (large, rich). Four patterns put it to work, parent-document, sentence-window, self-querying, and contextual compression, with one focused code addition on the running app.
The leap from a fixed pipeline that runs the same way every time to a dynamic, decision-making loop that can choose whether to retrieve, judge what came back, and try again. Part 10 of a from-scratch series on Retrieval-Augmented Generation: a guided tour of Agentic RAG, Corrective RAG (CRAG), Self-RAG, GraphRAG, and Multi-Modal RAG, what control flow each one adds, and the sober cost of reaching for any of them.
How to replace vibes with numbers. Part 11 of a from-scratch series on Retrieval-Augmented Generation: the two failure surfaces of a RAG system, the core metrics that probe each one (context precision and recall, faithfulness, answer relevance), LLM-as-a-judge and its biases, the frameworks that automate it, how to build an evaluation set, and the disciplined loop that turns guessing into engineering.
The finale. A RAG system that works in a notebook is about 20 percent of the job; the other 80 percent is making it fast, cheap, reliable, secure, and observable under real traffic. Part 12 of a from-scratch series on Retrieval-Augmented Generation: where latency and cost actually go and how to cut them, caching (including semantic caching), monitoring and tracing, failing gracefully, and the most underrated topic of all, security (prompt injection and data leakage). It closes with a capstone checklist for the whole series and a warm send-off.
Single-vector embeddings throw away token-level signal. Late interaction keeps a vector per token and scores with MaxSim, getting cross-encoder-quality matching at bi-encoder serving cost. Part 13 of a from-scratch series on Retrieval-Augmented Generation, opening the Frontier Track: ColBERT and ColBERTv2, MaxSim by hand in numpy, the storage tradeoff, and how ColPali extends late interaction to document page images without OCR or chunking.
A chunk that reads fine in isolation can be uninterpretable once it leaves its document: 'she' no longer resolves to 'Alice', 'the policy' loses its antecedent. Part 14 of a from-scratch series on Retrieval-Augmented Generation, on the Frontier Track: two training-free fixes, late chunking (pool token spans after the transformer) and Anthropic's Contextual Retrieval (prepend an LLM-written situating sentence before embedding), built by hand and compared.
Not every query needs the same machinery: a greeting needs no retrieval, a fact needs one lookup, a comparison needs several. Part 15 of a from-scratch series on Retrieval-Augmented Generation and the close of the Frontier Track: a small complexity classifier that routes each query to no-retrieval, single-step, or multi-step retrieval, unifying the pipelines built across Parts 6 to 10 into one adaptive system.
Part 1 asked why RAG exists. Part 16 asks the harder follow-up: when do you even need retrieval? Context windows reach about a million tokens in 2026, so sometimes you can just stuff everything in, and Cache-Augmented Generation (CAG) preloads a small, stable corpus once and reuses the cached KV state instead of retrieving. This part works out the prompt-caching economics that decide between them and gives you a clear decision matrix: massive or fast-moving or private corpus to RAG, small and stable to CAG or long-context, mid-size to long-context.
RAG widens the attack surface in a way ordinary apps do not: its whole premise is feeding external, often untrusted, content straight into a powerful model's prompt. Part 17 of a from-scratch series on Retrieval-Augmented Generation: the threats unique to RAG (indirect prompt injection through retrieved documents, knowledge-base poisoning, cross-tenant leakage) and the layered defensive pipeline that contains them, from input redaction and provenance scoring to a delimited untrusted-context wall, decline-if-not-grounded, output filtering, and identity-scoped access control.
Most enterprise knowledge does not live in documents, it lives in databases and tables, and dense passage retrieval cannot answer a question whose answer has to be computed. Part 18 of a from-scratch series on Retrieval-Augmented Generation: text-to-SQL with RAG (retrieve the schema, generate SQL, execute, answer), table retrieval and the scaling reality, and routing text-search versus SQL per query.
Part 19 of a from-scratch series on Retrieval-Augmented Generation: take the agentic RAG that Part 10 only toured in prose (the ReAct loop, tool use, routing, multi-hop) and build a real agent by hand, with four tools, a reason/act/observe loop, an honest step budget, and three traces you can read line by line.
Part 20 of a from-scratch series on Retrieval-Augmented Generation: give the one-shot agent a memory. Build multi-turn RAG by hand, where query condensation rewrites a context-dependent follow-up into a standalone question before retrieval, so 'what about damaged items?' finally finds the right chunk.
A bare model cannot touch your data or act. The augmented LLM (a model ringed by typed tools in the smallest loop with a stop condition) can. Part 1 names the primitive, draws the do-you-even-need-an-agent ladder, and gives tools a contract.
A schema-valid tool call can still throw, time out, or return garbage at run time. The fix is a robustness layer: a failure taxonomy, bounded retries, and an idempotent refund tool.
ReAct pays an LLM call per hop, so the bill balloons and the plan is re-derived every step. Making the plan a first-class artifact (plan-and-execute, ReWOO, the tool DAG) cuts LLM calls and critical-path depth.
An up-front plan is a bet that the world will not change. When a SKU is discontinued mid-run, a prospective critic and an error-triggered replanner that revises only the remaining steps save the run.
Replanning recovers within a run, but the agent repeats the same mistake across runs. In-loop self-critique fixes what it can see; Reflexion writes a verbal lesson to a buffer the next trial reads.
A flat buffer cannot separate what happened from what is true from how to do a task, and read-only tools cannot update state. Four typed stores plus self-editing memory fix it.
A long run overflows the window and a grow-only store turns to noise. Hot/warm/cold compaction bounds the window; forgetting (decay, supersede, evict) keeps memory useful.
An agent that never gives up retries a dead search forever and burns the budget. Multi-dimensional budgets, a loop detector, and a circuit breaker trip it to a graceful partial result.
A crash mid-run loses the half-issued refund and a naive restart re-charges the customer. A journal, replay, and idempotency keys make the agent effectively-once.
Some actions need human approval and a user may want to correct the agent mid-run. A journal-backed interrupt, resume, and steer put a human in the loop, durably.
A durable agent you cannot see into is undebuggable. Fold the same journal into OTel-shaped spans for trajectory, cost, latency, and cost-per-success.
Tools are a hardcoded dict no other host can discover. A minimal MCP server and host by hand turn tool use into a JSON-RPC wire protocol with discovery and multiplexing.
Agents need to run code and act on a screen, not just call JSON tools, and that power is dangerous. A sandbox permission boundary contains it. The toy is illustrative.
One agent with one context cannot parallelize breadth-first work and is overkill for a specialist. Orchestrator-worker fan-out, handoff-as-a-tool, honest economics.
A handoff stops at the process boundary; a billing agent owned by another team is unreachable. A2A Agent Cards, JSON-RPC delegation, and a trust allowlist cross it.
An agent with tools, memory, code, and untrusted content is an attack surface. The lethal trifecta and indirect prompt injection, beaten by breaking a leg.
Eyeballing checks only the answer, so a right answer via a wrong, expensive, or unsafe path passes; a three-layer eval and a regression gate catch it. The finale.
Before any statistic, evals need three atoms: a task, a gold label, and a metric. Part 1 builds the 2x2 confusion matrix by hand from ten graded outputs, derives accuracy, precision, recall, and F1 from its four counts, and shows on an imbalanced set why accuracy alone lies.
A metric is only as good as the labeled set beneath it. Part 2 builds a trustworthy golden set by hand on 40 support tickets: stratified sampling pins a rare class at exactly 2 urgent per draw while simple random sampling swings from 0 to 4, a leakage trap inflates accuracy from an honest 0.667 up to 0.800, and a three-criteria rubric turns 'good answer?' into a countable pass-at-5 bar.
The gold labels you grade against are themselves opinions, and opinions have noise. Part 3 takes 20 support tickets graded by two humans who agree on 18 of them, a scary-high 90%, and shows by hand why most of that agreement is chance: Cohen's kappa collapses it to 0.61, Fleiss' kappa handles three raters, and Krippendorff's alpha waits as the general case.
When you cannot afford a human grader for every output, you hand the rubric to a model instead. Part 4 runs twelve labeled support answers through a deterministic mock judge, parses each free-text verdict into PASS, FAIL, or ABSTAIN, and scores the judge against human gold on the ten it could grade.
An LLM judge is a model, so it carries systematic, measurable biases. Part 5 baits a deterministic mock judge with a slot-A bonus and a per-word reward, measures its position bias (7 of 12 verdicts flip when you swap who goes first) and verbosity bias (the longer answer wins all 5 tied probes), then corrects the position bias by averaging both orderings and watches disagreement with the truth fall from 5/12 to 1/12.
One eval run graded 30 outputs and gave accuracy 0.70, a single number that looks solid until you ask how much it would wobble on a different sample. Part 6 builds the bootstrap by hand: resample the 30 items with replacement 10000 times, read the 2.5th and 97.5th percentiles as a 95% CI of [0.53, 0.87], and watch that interval halve as the set grows.
Two models are graded on the same 50-item set: A gets 41/50 (82%), B gets 42/50 (84%), a headline gap of +2 points. Part 7 builds the paired 2x2 of agreements by hand, runs McNemar's test on the 9 items where the models disagree (both an exact binomial p and the chi-square approximation), finds it insignificant, and then works out that catching a true 2-point gain would need thousands of examples.
One coding task, five sampled completions, two that pass the hidden tests: this part derives the unbiased pass@k estimator as a combinatorial identity, checks it by brute-force enumeration, and proves in exact arithmetic why the tempting plug-in formula reports too low.
How leaderboards turn a pile of head-to-head battles into one ranking. Part 9 runs four chatbots through an 18-battle round-robin arena, ranks them two ways (Elo updated match by match, then the Bradley-Terry MLE fit to the aggregate win matrix), and shows numerically why Elo depends on match order while Bradley-Terry does not.
A confidence is a promise: of the answers a model tags 80% sure, about 80% should be right. Part 10 scores the same ten trivia confidences twice, once for an over-confident model and once for a well-calibrated one, builds the reliability curve by hand across three bins, and reduces the whole picture to ECE and the Brier score.
A score is worth nothing if it cannot say no. Part 11 turns the whole series into one CI gate that runs three checks on two candidate runs of a 20-item eval, a bootstrap-CI threshold, an n-gram contamination scan, and a PSI drift comparison, and blocks the deploy of the run whose mean looked fine but whose interval and distribution did not.
Your best eval set is not something you write, it is something you mine from what your model already did. Part 12 takes 12 production traces from a support assistant, ranks them by a cheap suspicion score, and spends a budget of 6 annotations to grow a golden set that catches all 4 real failures where random sampling catches only 2.
Offline scores are necessary, not sufficient: the final judge is live traffic. Part 13 splits one week of a support chatbot's sessions 50/50, computes the resolution-rate lift and a by-hand two-proportion z-test, and then lets a refusal-rate guardrail overrule a real, significant +8 pp win into a DO-NOT-SHIP verdict.
Grading an agent's final answer misses how it got there. Part 14 scores the tool-call path of eight runs against an approved reference trajectory four ways (exact, order-aware, set-overlap, and step-level credit), then grades a pass/fail rubric with a mock judge and a human and chance-corrects their agreement with Cohen's kappa.
The capstone wires the whole series into one small pure-Python harness: a dataset, a swappable scorer, a bootstrap confidence interval, and a gate. We run it three ways on two tiny cases and watch swapping only the scorer flip the same ten-item QA data from HOLD to SHIP.
Training an LLM happens once. Inference happens every time someone hits enter. These are my notes on what it actually costs to serve an open-source model: where the GPU memory goes, why the KV cache and not the weights is the thing that bites you, what happens inside a single forward pass, and how quantization buys back 3x the throughput for less than a point of accuracy. Part 1 of a walkthrough of efficient LLM serving.
The EU's aviation regulator just published a 239-page concept paper on making AI safe enough to fly. From artificial narrow intelligence to learning assurance and the W-shape process, the mental model behind trustworthy aviation AI, explained for non-experts.
TOYGUN'un nasıl gördüğünü, hedefi nasıl tanıyıp izlediğini, lazerle nasıl ölçüp işaretlediğini ve uçağın radar izini bozmadan tüm bunları nasıl yaptığını kademe kademe, sinyal zinciri ve alt sistemler ile anlatır.
End-to-end notes from a $55 LoRA build on gpt-oss-20b: the data pipeline, the training run, the evaluation, and the surprise that the biggest win was teaching a reasoning model to stop reasoning.
Chunk size, embeddings, re-rankers: the usual suspects. But the language of your corpus quietly shapes every layer of the pipeline, and reasoning models make it decisive.