Engineering notes
Platform Engineering · notes
Original deep-dives on platforms, Kubernetes, cloud, SRE, security and FinOps.
-
Atomic static site deploys where there is neither git nor transactions
An atomic static site deploy without git on the server: the receiving end offers only scp and ssh. No transaction can be built from that, but a deploy that survives a drop and repeats without consequence can.
-
Crossplane vs Terraform: a control plane instead of a state file
Terraform keeps a snapshot of state in a file and reconciles it on demand; Crossplane turns the Kubernetes API into a control plane and reconciles continuously. What version two changed, where continuous reconciliation wins and what it costs.
-
Argo Rollouts: canary and blue-green with metric-driven auto-rollback
A standard RollingUpdate declares success once pods pass the readiness probe. Argo Rollouts turns the release into a testable hypothesis: stepped canary traffic, blue-green with preview, PromQL-driven AnalysisTemplates and SLO burn rate as an automatic rollback.
-
Multi-cluster ArgoCD: hub-and-spoke vs per-cluster and where both need pull-mode
Hub-and-spoke is the default, but it breaks against blast radius, inbound networking and compliance. When to pick per-cluster local, when pull-mode, and what a hybrid without ideology looks like.
-
Policy as Code in the AI-code era: the speed-versus-control paradox
Production-code velocity now outruns human review. Without deterministic guardrails at CI and admission, an AI agent turns into an amplifier of mistakes — not throughput.
-
The golden path as a product: why the thinnest viable platform beats the five-year build
Build the platform thin — one path at a time — and standardize the process before you automate it. Otherwise you ship a monument no one uses.
-
Internal Developer Platform: Six Layers and Three Adoption Drivers
An IDP isn't "yet another portal" — it's a product for developers. Why platform engineering took off now, the six layers a working Kubernetes platform is built from, and the four mistakes that kill a portal in six months.
-
Production Dockerfile: distroless, build cache and secrets
A checklist of about ten lines: what ships to production and what stays in the builder stage. Measured base images, BuildKit cache mounts, secrets that never reach a layer and attestations that arrive on their own.
-
Istio Ambient Mesh: sidecarless and the end of the double hop
Ambient took Envoy out of every pod. Where that removes the double hop and the overhead — and where the sidecar is still justified.
-
Cilium and eBPF: replacing kube-proxy and removing iptables from Kubernetes networking
Everyone sees eBPF as a speedup, but the real value is a decision rule: where replacing kube-proxy pays off, and where iptables still wins.
-
Ingress NGINX reaches EOL: migrating to Gateway API without panic
Why the end of kubernetes/ingress-nginx is about risk, not roadmap, and how to move to Gateway API in stages.
-
OOMKilled forensics: from pmap to cgroups memory.stat
Exit code 137 leaves no stack trace and no final log line, while the dashboard swears there was plenty of memory. The tools that answer «where did the memory go» — while the pod is still alive.
-
Kubernetes 1.36 (Haru): What Actually Changes in Production
Mutating webhooks have started dying, Ingress NGINX is retired, HPA scale-to-zero is still alpha. A pragmatic 1.36 triage for the platform team — without the blogosphere hype.
-
A Kubernetes Debugging Agent: Query Templates or Scripts?
Zinchenko hands the LLM MetricsQL templates; my VM skill feeds the agent a finished aggregate. I dissect the flexibility-versus-reproducibility axis and why read-only by blacklist is weaker than an allowlist.
-
Drift detection in 2026: from the alert to remediation as code
Plan says “no changes” while a security group sits open to the internet. The four blind spots of single-layer detection, and where automatic remediation should stop.
-
OpenTofu vs Terraform in 2026: when the fork is already worth it
The question is no longer whether the fork is safe, but when it beats the original. Features, lock-in, and migration cost in 2026.
-
OIDC → AWS STS: CI/CD Without Long-Lived Keys
Federated identity replaces AWS_ACCESS_KEY_ID in CI/CD: one pattern for GitHub, GitLab and Atlantis — no rotation, real CloudTrail attribution.
-
HTTP/2 for Internal Traffic: A Well-Known Class of Pitfalls
One connection for every request, a shared flow control window, and a pool that never evicts the dead: why HTTP/2 services hang "without errors" and what Go and gRPC did about it.
-
Observability 2.0: eBPF, OpenTelemetry, and AI that finds the root cause
Instrumentation moved into the kernel, telemetry into the OTel standard, root cause analysis toward the machine. Three shifts and the order in which they pay off.
-
An incident closed twice: severity, ICS roles and three gates
Why «recovered» is the most expensive word in an incident, and the three checks that must pass before you say it.
-
Four golden signals: what they actually catch and why the stack is VictoriaMetrics + Loki
What each of the four signals really catches, and three traps where «we have monitoring» turns out to be green checkmarks above a broken service.
-
Error Budget as a Stop Button: SLOs Without Panic
Error budget turns reliability into a resource you can spend — and multi-burn-rate alerts turn it into a page that's actually worth waking up for.
-
CSP behind a consent gate: why a green header proves nothing
The evaluator flagged two errors and both turned out to be dead entries. The real defect surfaced only once the check ran against browser behaviour instead of the header.
-
External Secrets Operator + Workload Identity: secrets access without a single long-lived key
An external secret store answers where secrets live — and creates a new secret: the credential for the store itself. How ESO and workload identity close the loop with no long-lived keys.
-
Cyber Resilience Act: why you need an SBOM by September 2026, not 2027
Everyone is watching December 2027, but vulnerability reporting starts a year earlier — and you cannot report what your inventory cannot list.
-
SLSA L0–L3 and provenance: a chain of trust from commit to admission
An SBOM tells you «what's inside», a signature tells you «who signed». Between them sits provenance — «how it was built». After SolarWinds that's the question admission has to ask.
-
DevSecOps in five stages: from secret-scan to admission policy
Five CI stages with exit-code 1 plus a cluster-side admission gate — the only pattern under which DevSecOps actually blocks production instead of running as a green-checkmark ritual.
-
OpenCost: Kubernetes cost allocation by namespace and pod
The cloud bill knows the price of an instance, not the price of a service. OpenCost (CNCF Incubating) splits cluster spend by namespace and pod: the engine and its pitfalls, OpenCost vs Kubecost vs AWS SCAD, idle cost and the path to unit economics.
-
Three levers against Kubernetes overspend: idle, scale-to-zero, right-sizing
The average cluster runs at 20–30% utilization. You pay for the rest. Three levers that recover the money without rewriting your architecture.
-
FinOps Is Rationality, Not Cost-Cutting
Cloud overspend isn’t negligence — it’s the sum of locally rational decisions. Change the information environment, not the people.
-
Native versus normalized tokens: why your cost formula diverges from the OpenRouter bill
Three systems reported three sums for one window. Only one of them was an independent measurement, and the tokens in the response were not the tokens on the bill.
-
The agent loop under a durable orchestrator: where the line runs
An agent graph and a durable execution engine are both orchestrators. A month on a bench showed where to draw the line between them, and what it costs.
-
A Staff Engineer Made of Sub-Agents: What I Adopt and What I Skip
The Senior Staff Engineer pattern for Claude Code, read by an engineer running his own four-role sub-agent teams: context isolation and the verification gate — yes; the 1% Rule and the Delete Rule — no.
-
MCP for DevOps: start with a read-only agent in your cluster
By 2026 there are 10,000+ MCP servers. The first one to give an agent in your cluster is read-only — here is what it buys and how to grant it.
-
AI rewrites DevOps: from syntax to judgment
What AI already does better than an engineer, where the human’s value remains, and how the senior DevOps role shifts by 2026.
-
A Local LLM for Coding: Not «Which» but «Where»
A review of the practical method for picking a local coding model by hardware — and where local models have a real niche while only the cloud holds the line on precision.
-
The 2026 Terminal Stack Through a Linux User's Eyes: What's Portable, What's MacBook Optics
The "ultimate 2026 stack" guide is written for macOS. I review it from a Linux/Wayland chair: Ghostty versus Konsole, a strong backend, and the gaps you must fill.
-
Agentic AI Security: Porting Enterprise Patterns Down to a Solo Harness
Biswas's architecture handles the identity layer well via OAuth and token exchange — but the real danger lives on the intent layer, which the article never reaches.
-
Saving Tokens in Agents: What the Survey Gets Right and What It Over-sells
A map of four token-cost techniques — caching, lazy tools, routing, compaction. It buries the cache's real consequence in a footnote, while two counter-intuitive facts are worth the whole map.
-
The Short Memory File Collapses at Scale: Tokens Versus Rule Precedence
Keeping the memory file tiny is right for a solo repository. But compliance rules from several organizations cannot be pinned to paths — they must be routed.
-
An Obsidian Second Brain Is a Layer, Not the Agent's Whole Memory
Monteiro's zoned wiki is the right mechanism, but agent memory is a taxonomy of layers; and any layer that ingests the outside world must be scanned for injection before synthesis.
-
CLAUDE.md: Contract or Layers? Where the Viral File Is Right and Where It Stops
The viral CLAUDE.md is right for the median: start flat. But compliance cannot be asked for — it must be mechanized. On what to write in the file and what to lift into the harness.
-
The Harness Is the Cheap Part: Notes on Rebuilding Claude Code
A commentary on Fareed Khan's from-scratch rebuild of Claude Code — and why its central conclusion should be flipped.
-
Claude Code Is a Skill: Notes on Leo Godin's Argument
Agreeing with the "LLMs are a skill" thesis and adding the experience the original left out.
-
Should you still learn to code in 2026 — notes on Marina Wyss
A walk-through of her Medium piece: where she is right, where she over-simplifies, and what I would add.