Home / Blog

Notes from real builds.

33 articles on agents, evaluation, Web3, mobile and shipping software. Read them here and share the ones you like.

Showing 33 articles

AI 24 Sep, 2026 5 min

AI for Learning and People Teams: Six Workflows to Automate First

Learning and people teams in large organisations have a particular kind of workload. A small number of specialists support thousands of employees, trainees, and managers…

Read note →
AI 24 Sep, 2026 5 min

Microsoft 365 Copilot Rollouts Fail on Permissions, Not the Model

Buying Microsoft 365 Copilot licences takes an afternoon. Getting a pilot group to trust what Copilot tells them takes months, and most of that time has nothing to do…

Read note →
Evaluation 12 Aug, 2026 6 min

Policy-Violation Testing: Will Your Agent Refuse a Bad Instruction?

An agent that scores well on your task benchmark can still delete the wrong file, close a ticket it should have escalated, or push a change it was told not to push, the…

Read note →
Evaluation 12 Aug, 2026 6 min

Signing Off an MCP Agent Before Production: The Pre-Deployment Checklist

Most teams ship an MCP agent to production the same way they ship a feature: it passed the demo, the tests are green, and someone says "looks good." That is not a…

Read note →
Evaluation 12 Aug, 2026 6 min

Choosing a Model for Your Agent With Evidence, Not a Leaderboard

The model at the top of the leaderboard is not necessarily the model you should ship. Leaderboards rank one axis, task score, on someone else's harness, on tasks your…

Read note →
Evaluation 11 Aug, 2026 5 min

MCP Tool Selection Evaluation: Test Your Own Tools

Public MCP benchmarks cannot tell you whether your agent picks the right tool from your MCP servers. They test generic, fixed toolsets under generic policy. To know if…

Read note →
Agents 10 Aug, 2026 5 min

Agent Trajectory Evaluation: Score the Path, Not Just the Answer

Scoring only the final answer of an agent is dangerous. An agent can return the correct result while calling the wrong tool, retrying five times, and reading a record it…

Read note →
AI 08 Aug, 2026 5 min

Benchmark Contamination: Why LLM Leaderboards Lie

Benchmark contamination is when benchmark data leaks into a model's training data, so the model has effectively seen the test before taking it. High scores then reflect…

Read note →
AI 07 Aug, 2026 5 min

There Is No Best LLM: How to Choose the Right Model per Task

There is no best LLM. There is only the best model for a specific job, measured on your task, through your harness, against your budget. So to choose the best LLM for…

Read note →
AI 06 Aug, 2026 5 min

Agent Harness vs Model: Why the Harness Moves Your Scores

Yes, the harness affects LLM performance, and it affects it a lot. You do not ship a model. You ship a system: a prompt, an agent loop, tools, retrieval, retries, and…

Read note →
AI 05 Aug, 2026 5 min

Is LLM-as-a-Judge Reliable? Biases and How to Fix Them

LLM-as-a-judge is reliable, but only when you control for its known failure modes. Used as a raw scorer it inherits three documented biases that quietly corrupt your…

Read note →
Agents 04 Aug, 2026 6 min

pass@k vs pass^k: The Honest Agent Reliability Metric

pass@k is the probability that an agent succeeds on at least one of k tries. pass^k is the probability that it succeeds on all k tries. That one operator flip is the…

Read note →
Agents 03 Aug, 2026 5 min

How Reliable Are AI Agents in Production? What to Measure

A demo tells you an agent can do a task once. Production asks whether it can do the task a thousand times unattended without a costly mistake. Those are different…

Read note →
AI 02 Aug, 2026 6 min

Private Eval vs Public Benchmark: When You Need Each

Use a public benchmark to survey the field and shortlist models. Use a private eval to decide what you ship. A benchmark is a shared public corpus that everyone runs. An…

Read note →
AI 01 Aug, 2026 5 min

Contamination-Free LLM Evaluation: A Practical Guide

Contamination-free LLM evaluation means testing a model on data it could not have seen during training. If your evaluation set overlaps with the model's training corpus…

Read note →
AI 26 Apr, 2026 7 min

Building an AI Workforce for Your Company

The pitch your team has heard ten times this year: "We can replace 30% of your ops headcount with AI." The pitch usually comes attached to a chatbot that hallucinates on…

Read note →
AI 22 Apr, 2026 4 min

Shipping AI Agents That Hold Real Money

Most agent demos you'll see are toys. A model wrapped in a loop, given a few tools, and pointed at a sandbox. The interesting question, the one we keep getting asked by…

Read note →
Web3 22 Apr, 2026 6 min

LMSR vs Orderbook: Picking a Prediction Market Engine

A prediction market needs an engine that does one job: turn an order to buy or sell into a price. There are two answers in production today, LMSR (a logarithmic…

Read note →
AI 15 Apr, 2026 5 min

Production RAG: From Demo to a System That Doesn't Hallucinate

The first version of any RAG system works on a curated demo set. The user asks a question, the system returns relevant chunks, the model answers, everyone is impressed…

Read note →
Web3 08 Apr, 2026 5 min

The Pre-Audit Checklist Every Protocol Team Should Run

A smart contract audit is one of the more expensive engineering invoices most protocol teams ever sign. A serious firm bills $50–250k for a meaningful engagement. The…

Read note →
AI 01 Apr, 2026 5 min

Cutting AI Costs by 10x Without Cutting Quality

Most AI products we audit are paying somewhere between 5× and 10× what they need to be paying. The reasons are remarkably consistent across companies, same handful of…

Read note →
Process 25 Mar, 2026 5 min

How We Scope a Product in Five Days

Most agency engagements get into trouble in the first two weeks, before a single line of code has been written. The team is excited, the founder is excited, the contract…

Read note →
AI 18 Mar, 2026 5 min

AI for Enterprise Ops: Where the ROI Actually Lives

Most enterprise AI projects don't fail because the technology doesn't work. They fail because the team picked a workflow where AI doesn't compound. The model produces…

Read note →
Web3 11 Mar, 2026 5 min

Building a Perpetual DEX: Production Architecture

Perpetual DEXes look simple from the outside. Users post collateral, take leveraged positions, the protocol manages the book, and someone, LPs, a vault, a counterparty…

Read note →
AI 04 Mar, 2026 5 min

AI Evals: Beyond Vibes-Based QA

The most common AI quality-assurance setup we see in production is one engineer running a few prompts through the new model version, eyeballing the outputs, and…

Read note →
Web3 25 Feb, 2026 5 min

Building a Polymarket-Style Prediction Market

Prediction markets are simple to describe and unforgiving to build. Users buy shares in outcomes ("Will X happen by Y date?"), prices move toward the market's collective…

Read note →
Mobile 18 Feb, 2026 5 min

React Native vs Native: A 2026 Decision Framework

The "React Native vs native" debate has been running for so long that most of the takes you'll find online are stale. The state of both ecosystems has shifted enough in…

Read note →
Web3 11 Feb, 2026 4 min

Cross-Chain Bridges That Don't Get Drained

Cross-chain bridges have lost more money than any other category of DeFi protocol. The cumulative number across the major incidents, Ronin, Wormhole, Nomad, Multichain…

Read note →
Engineering 30 Jan, 2026 5 min

From MVP to Scale: When to Refactor and When to Rewrite

There's a moment, usually nine to eighteen months into a successful product, when the team starts having the same conversation in slightly different forms. Things are…

Read note →
Web3 22 Jan, 2026 5 min

Stablecoin Payment Rails for B2B Products

Stablecoin volume in 2025 surpassed Visa's transaction volume. Most of that is on-chain trading, not real-economy payments, but the share that's real-economy payments is…

Read note →
Web3 08 Jan, 2026 5 min

Account Abstraction in Production: ERC-4337 for Mainstream Users

The biggest reason consumer Web3 hasn't gone mainstream isn't the technology. It's that every onboarding flow asks the user to understand seed phrases, signing, gas…

Read note →
Web3 18 Dec, 2025 5 min

RWA Tokenization: A Pragmatic Architecture for Institutional Assets

Real-world asset tokenization is one of the categories most-talked-about and least-shipped in Web3. The 2025 wave finally produced some real products, BlackRock's BUIDL…

Read note →
Web3 04 Dec, 2025 4 min

NFT Marketplaces with Enforced Creator Royalties

The race to zero on NFT royalties between 2022 and 2024 nearly killed creator economics on-chain. Marketplaces optimized for trader volume over creator earnings…

Read note →
Ready when you are

Building something like this?

A free 30-minute call with the engineers who wrote these.