HubSpot's Vector-as-a-Service Hits 20 Billion Vectors, and Kubernetes Operators Retired the Helm Charts

From Proof of Concept to Internal Platform

Vector search has quietly become critical infrastructure inside large software companies, and HubSpot has now shown what it looks like at serious scale. In an engineering account surfaced on July 7, the company described how a semantic search proof of concept grew into Vector as a Service, an internal platform that today manages more than twenty billion vectors on behalf of over thirty-eight teams. VaaS is not a database. It is a platform layer that sits in front of Qdrant and adds the things an internal customer actually needs: access control, embeddings generation, data versioning, and feedback collection.

The framing matters for any organization standing up its own retrieval infrastructure. HubSpot did not simply hand teams a vector database and wish them luck. It built an opinionated service that hides the operational complexity of running Qdrant at scale, so that dozens of product teams can consume vector search without each one becoming an expert in shard placement or memory management. That is the same platform engineering pattern that turned raw Kubernetes into internal developer platforms, applied to the retrieval layer that now underpins search, recommendations, and a growing roster of AI agents.

The Numbers Behind the Platform

The scale metrics are the part that should make infrastructure leaders sit up. VaaS spans more than two hundred indexes across upward of one hundred and forty clusters, deployed in five regions and two environments, and it absorbs peak write traffic of one hundred thousand requests per second. Some individual collections contain billions of points. These are not the numbers of a science experiment; they are the numbers of a production dependency that many teams cannot function without.

At that size, the failure modes are subtle. HubSpot's engineers noted that even one imbalanced shard can force an entire cluster to scale before it genuinely needs to, wasting capacity and money. When a single index holds billions of points, the difference between an evenly distributed workload and a skewed one is the difference between a cluster that runs comfortably and one that is perpetually one hot spot away from an incident. Managing that distribution, rather than raw query speed, becomes the dominant engineering challenge as the platform grows.

Why Helm Gave Way to a Kubernetes Operator

The most instructive part of HubSpot's story is an operational migration, not a modeling breakthrough. The team moved from managing clusters with Helm to an internal Kubernetes Operator framework built around a sixty-second reconciliation cycle. The results were dramatic: cluster spin-up time fell from hours to minutes, and the company was able to eliminate the standby clusters it previously kept warm as insurance against slow provisioning. The same reconciliation model now handles horizontal scaling, shard rebalancing, and replication recovery.

This is a textbook illustration of why the operator pattern won in the Kubernetes ecosystem. Helm is excellent at installing software, but it is a templating tool, not a control loop. Encoding the desired state of a fleet and continuously reconciling toward it is what lets HubSpot treat scaling and recovery as automated, self-correcting behaviors rather than manual runbooks. Retiring standby clusters is the kind of unglamorous win that pays for itself quickly, because idle capacity held purely to hedge against slow startup is pure waste at one hundred and forty clusters.

Balancing Shards and Fighting Memory Skew

Underneath the platform, HubSpot's hardest problems are about placement and memory. Balancing shard distribution and avoiding memory skew are the levers that determine whether a cluster runs efficiently or scales prematurely. The team leaned on automation to manage this complexity, because the alternative, having engineers manually rebalance billions of points across clusters, does not survive contact with growth. To hold down cost, they also applied quantization and on-disk storage, trading a controlled amount of precision and latency for a substantially cheaper footprint.

On the query side, VaaS supports hybrid search with named vectors, multi-stage querying, and weighted reranking. These are not academic features. As more of HubSpot's internal consumers are AI agents rather than human-facing search boxes, retrieval quality and latency directly shape how well those agents perform. A reranking stage that surfaces the right context is the difference between an agent that answers correctly and one that hallucinates from irrelevant matches. The retrieval layer, in other words, has become part of the AI quality problem, not a solved commodity beneath it.

Serving Agents, RAG and Deduplication

VaaS was originally about semantic search, but its workload has broadened in a way that mirrors the wider industry. The platform now backs retrieval-augmented generation, powers a growing set of agents, and handles contact deduplication, the unglamorous but valuable task of recognizing that two records describe the same person. That range is instructive: the same vector infrastructure that improves a search experience turns out to be the substrate for agentic features and for data-quality work that has nothing to do with generative AI.

The rise in agent usage is precisely what raised the stakes on quality and latency. Human users forgive a mediocre search result and reformulate their query. An autonomous agent takes whatever the retrieval layer returns and acts on it, so a weak match propagates into a wrong action. HubSpot's investment in reranking, hybrid search, and low-latency serving is a recognition that when agents are the primary consumer, the vector platform is no longer a nice-to-have behind the product. It is load-bearing infrastructure whose quality bounds the quality of everything built on top of it.

The Lesson: Manual Operations Don't Survive Growth

HubSpot engineers Oleg Tereshin and Xin Liu distilled the entire effort into one line worth pinning to a wall: "Manual operations don't survive growth." It is a blunt summary of what changes when a retrieval prototype becomes a platform serving twenty billion vectors. The techniques that got the proof of concept working, hand-tuned clusters and manual interventions, become liabilities the moment dozens of teams depend on the service around the clock across five regions.

For CTOs weighing how to build their own AI retrieval stack, the takeaway is not the specific choice of Qdrant or the exact reconciliation interval. It is the discipline. HubSpot invested in an operator-driven control plane, automated shard balancing, cost controls through quantization, and a platform abstraction that let internal teams consume vector search without operating it. As retrieval moves from feature to foundation, the organizations that treat it with that level of operational seriousness will be the ones whose agents and RAG systems stay fast and accurate as demand climbs.