AI for Learning and People Teams: Six Workflows to Automate First
Learning and people teams in large organisations have a particular kind of workload. A small number of specialists support thousands of employees, trainees, and managers…
33 articles on agents, evaluation, Web3, mobile and shipping software. Read them here and share the ones you like.
Showing 33 articles
Learning and people teams in large organisations have a particular kind of workload. A small number of specialists support thousands of employees, trainees, and managers…
Buying Microsoft 365 Copilot licences takes an afternoon. Getting a pilot group to trust what Copilot tells them takes months, and most of that time has nothing to do…
An agent that scores well on your task benchmark can still delete the wrong file, close a ticket it should have escalated, or push a change it was told not to push, the…
Most teams ship an MCP agent to production the same way they ship a feature: it passed the demo, the tests are green, and someone says "looks good." That is not a…
The model at the top of the leaderboard is not necessarily the model you should ship. Leaderboards rank one axis, task score, on someone else's harness, on tasks your…
Public MCP benchmarks cannot tell you whether your agent picks the right tool from your MCP servers. They test generic, fixed toolsets under generic policy. To know if…
Scoring only the final answer of an agent is dangerous. An agent can return the correct result while calling the wrong tool, retrying five times, and reading a record it…
Benchmark contamination is when benchmark data leaks into a model's training data, so the model has effectively seen the test before taking it. High scores then reflect…
There is no best LLM. There is only the best model for a specific job, measured on your task, through your harness, against your budget. So to choose the best LLM for…
Yes, the harness affects LLM performance, and it affects it a lot. You do not ship a model. You ship a system: a prompt, an agent loop, tools, retrieval, retries, and…
LLM-as-a-judge is reliable, but only when you control for its known failure modes. Used as a raw scorer it inherits three documented biases that quietly corrupt your…
pass@k is the probability that an agent succeeds on at least one of k tries. pass^k is the probability that it succeeds on all k tries. That one operator flip is the…
A demo tells you an agent can do a task once. Production asks whether it can do the task a thousand times unattended without a costly mistake. Those are different…
Use a public benchmark to survey the field and shortlist models. Use a private eval to decide what you ship. A benchmark is a shared public corpus that everyone runs. An…
Contamination-free LLM evaluation means testing a model on data it could not have seen during training. If your evaluation set overlaps with the model's training corpus…
The pitch your team has heard ten times this year: "We can replace 30% of your ops headcount with AI." The pitch usually comes attached to a chatbot that hallucinates on…
Most agent demos you'll see are toys. A model wrapped in a loop, given a few tools, and pointed at a sandbox. The interesting question, the one we keep getting asked by…
A prediction market needs an engine that does one job: turn an order to buy or sell into a price. There are two answers in production today, LMSR (a logarithmic…
The first version of any RAG system works on a curated demo set. The user asks a question, the system returns relevant chunks, the model answers, everyone is impressed…
A smart contract audit is one of the more expensive engineering invoices most protocol teams ever sign. A serious firm bills $50–250k for a meaningful engagement. The…
Most AI products we audit are paying somewhere between 5× and 10× what they need to be paying. The reasons are remarkably consistent across companies, same handful of…
Most agency engagements get into trouble in the first two weeks, before a single line of code has been written. The team is excited, the founder is excited, the contract…
Most enterprise AI projects don't fail because the technology doesn't work. They fail because the team picked a workflow where AI doesn't compound. The model produces…
Perpetual DEXes look simple from the outside. Users post collateral, take leveraged positions, the protocol manages the book, and someone, LPs, a vault, a counterparty…
The most common AI quality-assurance setup we see in production is one engineer running a few prompts through the new model version, eyeballing the outputs, and…
Prediction markets are simple to describe and unforgiving to build. Users buy shares in outcomes ("Will X happen by Y date?"), prices move toward the market's collective…
The "React Native vs native" debate has been running for so long that most of the takes you'll find online are stale. The state of both ecosystems has shifted enough in…
Cross-chain bridges have lost more money than any other category of DeFi protocol. The cumulative number across the major incidents, Ronin, Wormhole, Nomad, Multichain…
There's a moment, usually nine to eighteen months into a successful product, when the team starts having the same conversation in slightly different forms. Things are…
Stablecoin volume in 2025 surpassed Visa's transaction volume. Most of that is on-chain trading, not real-economy payments, but the share that's real-economy payments is…
The biggest reason consumer Web3 hasn't gone mainstream isn't the technology. It's that every onboarding flow asks the user to understand seed phrases, signing, gas…
Real-world asset tokenization is one of the categories most-talked-about and least-shipped in Web3. The 2025 wave finally produced some real products, BlackRock's BUIDL…
The race to zero on NFT royalties between 2022 and 2024 nearly killed creator economics on-chain. Marketplaces optimized for trader volume over creator earnings…
A free 30-minute call with the engineers who wrote these.