You can run a rigorous LLM evaluation without a single row of production data leaving your network. That is what in-VPC LLM evaluation means: the harness, the datasets, and the scoring all execute inside your own cloud account or data center, and nothing sensitive is shipped to a vendor.
For a fintech, a bank, or a health platform, this is not a nice-to-have. It is the difference between a project that clears security review and one that dies in it.
TL;DR
- In-VPC LLM evaluation runs the entire eval pipeline inside your perimeter, so production data and PII never transit to a third party.
- Data residency is a hard requirement under DPDP, GDPR, and sector rules for BFSI and health, not a preference.
- The eval harness deploys into your VPC or on-prem; judge models are called with your own API keys or your self-hosted endpoints.
- PII is redacted at the boundary before any external judge call, and redaction is logged and auditable.
- This is what unlocks regulated buyers who cannot use a hosted eval SaaS.
What is in-VPC LLM evaluation?
In-VPC LLM evaluation is a deployment model where the evaluation software runs inside the network boundary you already control. Your test cases, your logged production traces, and the model outputs all stay in your VPC, subnet, or on-prem cluster.
Compare that to a hosted eval SaaS. There, you upload prompts, traces, and outputs to a vendor’s cloud, and the scoring happens on their infrastructure. That flow moves your data across a trust boundary. For a regulated buyer, every byte that crosses that boundary is a control you now have to justify.
The in-VPC model inverts the flow. The vendor ships code and configuration to you. Your data does not move.
Why does data residency matter for regulated buyers?
Because the law and your regulators treat the movement of data as a distinct, auditable event. Where your data physically sits, and who can technically access it, are questions you must answer with evidence.
Under India’s DPDP Act, you are accountable for how personal data is processed and where it flows. Under GDPR, cross-border transfers carry their own legal basis and documentation burden. In BFSI, model validation and third-party risk are governed directly: SR 11-7 for US banks, the EU AI Act for high-risk systems, ISO 42001 for AI management systems, and the RBI FREE-AI framework published in 2025 for Indian financial institutions.
None of these frameworks forbid evaluation. They forbid unaccounted data movement. In-VPC evaluation removes the movement, which removes the hardest part of the compliance conversation. We go deeper on the banking case in our note on independent model validation for banks.
How does the evaluation run inside your perimeter?
The harness is deployed into your environment as a container or a set of jobs you run. It reads your datasets from storage you control and writes results back to storage you control.
Judge models are the one place an external call can happen, and it stays under your keys. When the eval uses an LLM as a judge, that judge is called with your own provider credentials or your self-hosted endpoint. Groow Labs never sees the traffic. You hold the keys, you set the allowlist, and you see every call in your own logs.
This matters technically as well as legally, because judges are not neutral. LLM judges show position bias and verbosity bias (Zheng 2023, Saito 2023), and self-preference bias where a model favors its own outputs (Panickssery 2024). A jury of several judges is more reliable than a single one (Verga 2024). Running that jury under your own keys means you get the reliability without exporting the data. More on judge design in our piece on contamination-free evaluation.
How is PII handled at the boundary?
PII is redacted before any call leaves the perimeter, and the redaction itself is logged.
Here is the boundary model.
| Stage | Location | Sees raw PII? |
|---|---|---|
| Test data and traces | Your storage in-VPC | Yes, and it stays there |
| Harness and scoring | Your compute in-VPC | Yes, inside your boundary |
| Redaction layer | Your VPC egress boundary | Strips PII before egress |
| Judge model call | Your keys or self-hosted | No, redacted payload only |
| Results and report | Your storage in-VPC | Aggregated scores, no raw PII |
The redaction layer runs before the judge call, not after. That ordering is the whole point. If you use a self-hosted judge, no redaction is needed at all, because nothing leaves. If you use a hosted judge under your keys, the payload is stripped and the redaction event is recorded for audit.
What will your security review ask?
A security or risk team will not take “it runs in your VPC” at face value. They will ask for evidence. The common questions, and the in-VPC answer to each:
- Where does our data go? Nowhere. It stays in storage and compute you own.
- What egress happens, and to where? Only redacted judge calls, to endpoints on your allowlist, under your keys.
- Who at the vendor can access our environment? No one by default. You run the jobs; the vendor ships code.
- How is PII handled before any external call? Redacted at the boundary, with the redaction logged.
- Can we audit every model call? Yes, in your own provider logs and network logs.
- What is left behind after the eval? Aggregated scores and a report in your storage, no raw data with the vendor.
If your evaluation cannot answer these cleanly, it will stall. In-VPC design answers all of them by construction rather than by policy promise.
How does in-VPC evaluation unlock regulated buyers?
It removes the objection that kills most eval deals: “we can’t send production data to a vendor.” Once evaluation runs inside the perimeter, the buyer’s security team is reviewing a deployment they control, not a data export they have to defend.
That is why the market has converged here. Serious teams are running their most sensitive evals privately rather than on public leaderboards, a pattern visible across the industry through 2026. Public benchmarks still matter for triage, but they carry contamination risk: Llama 2 showed over 16% contamination on MMLU, roughly 29% of audited benchmarks showed some contamination, and contamination detection is itself unreliable. You cannot trust a public score for a production decision, and you cannot put production data on a public board. In-VPC private evaluation resolves both problems at once. See how we structure that engagement on our method page.
FAQ
Do judge models still need internet access? Only if you choose a hosted judge. Even then, the call goes out under your own keys to an endpoint you allowlist, carrying a redacted payload. A self-hosted judge needs no egress at all.
Does in-VPC evaluation change the rigor of the results? No. The same harness, datasets, jury-of-judges scoring, and error bars apply. Location of compute does not weaken the statistics. Note that error bars matter regardless: a 95% per-step pass rate compounds to roughly 36% over a 20-step agent trajectory.
Is this only for cloud VPCs, or does on-prem work too? Both. The harness deploys into a cloud VPC or an on-prem cluster. The design goal is the same: your data stays behind your boundary.
How do we prove to auditors that no data left? Through your own network and provider logs, plus the redaction log at the boundary. Because the vendor never receives your data, the evidence lives entirely in systems you already audit.
In-VPC LLM evaluation gives you the rigor of an independent eval without the data-export risk that stops regulated projects cold. If you want to see it against your own stack, book a free eval diagnostic and we will map the deployment to your perimeter.