What is a Vector Database?

Vector Databases: Where They Actually Earn Their Keep

You can bolt a vector approach onto your relational database as you implement AI use cases in your organizational environments.

With a new database type showing up every few years, it has become common for half of any conference session on that new type to be spent redefining the term for a new audience. Right now vector databases are having their time in the sun, and everybody seems to need one for their RAG pilot or their AI-assisted service in the enterprise. Below the level of the slides and the conference talks, though, vector databases are not terribly complicated. An embedding model turns input data — text, images, support tickets — into a vector of numbers. Those vectors get stored. At query time, you embed the query the same way and search the stored vectors for the ones closest in meaning.

What it actually is

So first off, let me define it. A vector database is a database where you store vectors in a high-dimensional space — think 50, 100, 1,000 dimensions or more — and where you can query for the vectors most similar to a given vector in that space. There are broadly two strategies for getting data into that space: single-shot approaches, where an entire item is mapped to one vector, and chunking approaches, where the data is split into pieces and each piece is embedded separately.

In terms of suitability, vector databases fit retrieval problems where many items in a dataset are all somewhat like one given item. They are a poor fit for problems where you need to find the single row matching an exact set of criteria. That second category is what relational databases have always been good at. Postgres is very well suited to finding one specific piece of data given criteria that match on every field — think where user_id = 4471. That is in sharp contrast to what a vector database does, which is to hand you back the twenty items most similar to a given item.

Both kinds of system sit on top of storage engineered for a high volume of lookups, so in that sense they are cousins. The difference is in how the lookup happens. A relational database evaluates a set of criteria against the fields of each candidate row to determine a match. A vector database compares a query vector against the stored vectors — all of them, or more often a subset — using a distance metric, and returns the closest ones. Which metric you use depends on the nature of the data, but cosine similarity and Euclidean distance are the usual suspects.

Most vector databases use approximate nearest neighbor (ANN) indexes, such as HNSW or IVF, to make the search efficient. The primary trade-off with these indexes is precision against performance: the index may return vectors that are slightly less similar than the true nearest neighbors in exchange for returning them much faster. At very large scale — hundreds of millions or billions of vectors — even a small amount of additional retrieval time makes a real difference, so giving up a little accuracy is often an acceptable price for a real-time interactive application.

Where the engineering complexity actually lives

I think the biggest misperception about the engineering required for vector databases is that the hard part is the storage. It isn't. It's the indexing. There are a number of approaches, each with its own trade-offs, and a great deal of engineering goes into wringing performance out of them.

HNSW (Hierarchical Navigable Small World). Builds a multi-layer graph in which each vector is connected to its neighbors. A search starts in the coarse upper layers and descends into progressively finer ones. It is very fast and, for many applications, very accurate — which is why it is the default index in a lot of vector databases handling real-time interactive workloads.

IVF (Inverted File Index). Organizes all the vectors into partitions — buckets or clusters — centered on a set of centroids. At query time, you identify the clusters of interest and only traverse the vectors inside them. You give up some precision for a given recall target in exchange for speed. At very large scale (100M to 1B+ vectors), narrowing to a subset before running the similarity search is frequently the only remaining lever you have. IVF variants differ in how they structure partitions and how they select centroids, which is what lets the approach cover a lot of different use cases.

Product quantization. Compresses the stored vectors to reduce memory footprint and to cut the number of operations needed for each distance computation. It is very often paired with IVF — IVF-PQ — as the retrieval strategy for huge datasets.

Flat, brute-force scan. The simplest method of all: no index, just compare against everything. Worth remembering that brute force is exact — it never returns a wrong answer; it just gets slow. That makes it the right call more often than people assume, for small collections or anywhere exact results genuinely matter more than latency.

None of these is better than the others in any absolute sense. Each represents an engineering trade-off between latency, recall, and memory for a specific use case. Saying HNSW is the best index for every problem is about as sound as saying every workload belongs on the fastest disk tier.

Where retrieval quality actually comes from

Here's the core of the problem, and I'll defend it: the index is not what's failing your RAG system. The chunking and embedding strategy is.

The quality of the index depends entirely on the quality of what went into it — chunks of a document, embedded into a vector space by an embedding model. At retrieval time, the same model has to embed the query so the two can be compared. If the model that indexed the corpus and the model that embeds the queries are not the same model, you are comparing coordinates from two different spaces that happen to have the same number of dimensions. The similarity scores that come back will look perfectly reasonable and mean absolutely nothing. Swapping embedding models mid-project causes a great deal of pain for exactly this reason.

Two things I've seen bite teams repeatedly:

• Naive token splitting underperforms semantic splitting. Cutting every N tokens regardless of structure produces chunks that straddle two unrelated ideas. Split on paragraphs, headings, or logical sections instead.
• Embed consistently. Use the same model for your corpus and your queries. Embedding models are defined in terms of a coordinate space, and two models with the same dimensionality are not defined in the same coordinate space. Swapping one for the other is switching to an entirely different space, and retrieval breaks completely — silently.

It is also very useful to store metadata alongside the vectors: source, date, tags, owner, department. That metadata is what lets you filter candidates out before the similarity search runs, or filter results after it returns. A lot of teams realize too late that retrieval is the part that matters, and that a nice demo can hide an enormous number of problems.

Security, or why retrieval carries a false sense of safety

This is the part of the architecture everybody wants to skip, because storing derived numbers feels safer than storing sensitive text. It isn't. As I alluded to above, it is often possible to reconstruct or infer meaningful information about the original input from the embedding itself, particularly when the model is known.

So treat the vector store with the same rigor you would apply to any production database holding regulated data. That means access controls, tenant isolation, encryption at rest and in transit, and deliberate logging discipline around both query vectors and retrieved results — the queries themselves are a sensitive signal, not just the documents. Enforce metadata filtering before retrieval rather than bolting it on afterward. And in a regulated environment, keeping the whole retrieval layer inside your own VPC is usually the difference between a straightforward compliance conversation and a painful one.

Where this shows up in practice

The dominant use case at the moment is supporting a generation model: embed a large corpus, store the vectors, retrieve the relevant chunks for each query, and pass them to the model as additional context. But it doesn't stop at semantic search — the same mechanics power recommendation, anomaly detection, and deduplication. Here is how this tends to land in three domains I get asked about often enough to be worth spelling out.

Higher education. For a university's AI advising assistant to be useful to students and staff, it has to retrieve across financial aid policy, degree requirements, and course catalogs from every department. Financial aid policy changes year over year and by cohort, and it varies with in-state or out-of-state status, scholarships, and professional programs. So chunk that content by policy section and effective date rather than by PDF page break, or you will confidently serve a 2019 refund policy in answer to a 2026 question. The access control point from the previous section is not theoretical here either: a student must not have a path, even an indirect one, to another student’s advising notes or FERPA-protected records simply because their embeddings landed near each other in the vector space. Filtering by student ID and role before the similarity search runs is what keeps this a retrieval system rather than an incident report.

Finance. Research desks and trading floors accumulate documents continuously — vendor research, internal risk memos, filings — over long periods, with vendors added and dropped along the way. Two failure modes show up here reliably. The first is the model swap: a collection is built with one embedding model, someone later substitutes a comparable one, and answer quality quietly collapses while the similarity scores continue to look flawless. The second is staleness. Without collection-level metadata — when a document entered the corpus, how many revisions it has been through, whether it has been superseded — an analyst will eventually get handed a three-year-old note with no indication that it no longer reflects the house view. Filtering on that metadata before retrieval is the fix. And logging discipline matters twice over in this domain, because query vectors over trading strategy documents and client portfolio data are themselves sensitive.

Healthcare. Consider semantic search over clinical notes and discharge summaries for a clinician-facing assistant. Clinical narrative is exactly the kind of narrow, structured, sensitive text where reconstruction risk against embeddings is highest, so PHI-bearing vectors belong behind the same authentication and audit controls as the EHR itself — not a lighter-weight index sitting alongside it. On the retrieval side, chunking by clinical encounter and tagging each note with department and note type (history and physical, progress note, consult) is what keeps a nurse's query about a medication history from surfacing that patient's psychiatric notes. That is as much a chunking and metadata design decision as it is a policy one.

The Takeaway

A vector database is more than a search index bolted on top of your stack. It is part of your data layer, and it deserves the same rigor you would give any production system holding data your organization is accountable for. The indexing algorithm is the interesting part, but the chunking strategy is the part that decides whether retrieval is any good, and the access controls on the retrieval layer matter every bit as much as the ones on the rest of your data layer. Get the interesting part right and skip the other two, and you will have built something fast, confident, and wrong.

Dr. Sam Kurien

The Quietest Thing an Attacker Does After Root

Privilege escalation gets all the attention. It's the dramatic beat in every write-up — the moment the shell prompt flips from $ to # and the attacker owns the box. But in my experience, escalation is rarely the point. It's the ticket. What the attacker does with that ticket in the next ninety seconds tells you far more about their intent, and it's usually the thing that leaves the evidence you'll actually use later.

So let me pick the one action I find most instructive to reason about: installing a persistence mechanism. Specifically, a malicious systemd service on Linux. It's unglamorous, it's common, and — this is the part I want to sit with — it's a control the defender can win, if they understand what they're really watching.

What the attacker gains

The honest answer is time. Everything else follows from that.

An exploit is fragile. It depends on a vulnerable version, a reachable service, a particular condition holding true. Patch the service and the door closes. This is exactly what I saw in this week's lab work chaining CVE-2025-32433 — the exploit is a beautiful thing until the target gets updated, and then it's a museum piece. An attacker who has done any real work to get to root is not going to leave their continued access hostage to a CVE that could be remediated on Tuesday.

Persistence decouples access from the vulnerability. A systemd service that calls back on a schedule survives reboots, survives the patch, and survives the analyst who thinks closing the original hole ended the incident. That last one matters more than people admit. I have watched remediation efforts declare victory on the entry vector while the persistence quietly sat one directory over, waiting. The attacker's goal isn't a shell today; it's a shell next month, on their terms, after everyone has stopped looking.

Systemd in particular is an attractive home because it's legitimate infrastructure. A rogue .service unit dropped into /etc/systemd/system/ with an innocuous name blends into a directory full of things administrators are conditioned to trust. Enable it with systemctl enable, and the OS itself becomes the thing that maintains the attacker's foothold. That's the uncomfortable elegance of it — you're not fighting malware anymore, you're fighting the service manager doing its job.

What it leaves behind

Here's my core claim, and I'll defend it: persistence is loud if you know where to listen, and the attacker's need for reliability is what makes it loud.

Reliability requires artifacts. A callback that has to survive a reboot must be registered somewhere durable. That durability is the tell. You cannot have persistence that is both reboot-surviving and evidence-free, because the mechanism that survives the reboot is, by definition, written down somewhere the system reads on boot.

For a malicious systemd service, that "somewhere" is a short list:

• A new or modified unit file, typically under /etc/systemd/system/ or /usr/lib/systemd/system/, with a creation or modification timestamp that rarely matches any change window you can account for.
• The payload itself — an executable or script, often parked in a world-writable path like /tmp, /var/tmp, or /dev/shm, or masquerading in /usr/local/bin.
• Journald entries showing the unit being enabled and started, which is the service manager narrating the attacker's actions in its own logs.
• The behavioral signature at runtime: a process spawned by systemd (PID 1 lineage) that promptly opens an outbound socket to a host that has no business being contacted from that server.

None of these is subtle once you've decided to look. The problem is almost never that the evidence is absent. The problem is that nobody correlated the file with the process with the connection.

How a defender detects and investigates

If I could give a defender one piece of advice here, it would be this: stop watching for "a bad file" and start watching for "a change to how this host schedules and runs code." That reframe is what makes the detection durable against an attacker who has read the same hardening guides you have.

Concretely, in order of what I'd reach for first:

File integrity monitoring on the systemd unit directories is table stakes, but on its own it only tells you a file appeared. Pair it with auditd rules that watch those same directories and record execve calls, so you catch not just the unit file being written but the payload actually running. FIM tells you the trap was set; auditd tells you it sprang.

Then correlate on lineage and behavior. A process whose parent traces back to systemd, launched from/tmp, opening an outbound connection, is a specific and huntable pattern. I care much more about that causal chain — scheduler spawns unusual binary, binary calls home — than about any single indicator in isolation. It's the combination that turns a maybe into a finding without waiting on reverse engineering.

For the investigation itself, the questions I'd want answered are the boring, useful ones. When was the unit file created, and by which subject’s session? Correlate that timestamp back to the authentication and process records around it — the same logon-session pivot I'd use for account-creation cases applies here. Has the service fired more than once? Journald and execve records give you every invocation, which bounds how long the persistence has actually been live and therefore how far back your scope has to reach. And critically: does the entry vector's remediation actually touch this? If not, you haven't closed the incident. You've closed a door in a house with an open window.

The takeaway

Privilege escalation is the headline, but persistence is where the investigation is usually won or lost. The attacker installs it because they need reliability, and that same need for reliability forces them to write something durable to disk and register it with the system — which is precisely the evidence a prepared defender is watching for. The mechanism that keeps them in is the mechanism that gives them away.

Watch the scheduler, not just the file. Correlate the artifact to the execution to the connection. And never let anyone declare an incident closed on the strength of a patch alone.

Privilege, Not Passwords: Where the Real Risk Lives

I wrote this for a class post and thought it was worth capturing here as well, both to share and for my own future documentation of where I landed on this.

I've been following a debate that keeps resurfacing — in our own classroom, in vendor briefings, and in boardrooms alike: which is the bigger security problem, weak passwords or weak privilege management? Most people settle firmly into one of two camps. Having spent more than one season of my career building and contributing to an identity management product, I'd like to try bridging the gap — though I'll confess up front that I've already picked a side. Privilege is the bigger governance and management problem, and it deserves the stronger focus. Let me explain.

Password vulnerability is a real problem, but the initial breach of an organization's systems through a login is, at this point, close to inevitable. Most large breaches begin with adversary-in-the-middle (AitM) phishing that bypasses MFA, or with infostealer malware quietly harvesting credentials by the thousands. And service accounts, by their nature, cannot use MFA at all. So under an assume-breach mindset, the decisive question is not whether an account will be compromised, but what that account can then access. A typical user account can be contained as an incident. An over-privileged identity is something else entirely: an attacker can escalate from it, disable security tooling, move laterally across the network, and establish the command-and-control foothold that turns a phishing click into a ransomware event. The privilege granted to the initially compromised identity determines the blast radius of the incident.

Meanwhile, the password problem is slowly solving itself, for the most part. NIST's recently finalized SP 800-63B-4 (National Institute of Standards and Technology [NIST], 2025) decisively shifts password guidance toward length, breached-credential screening, and phishing-resistant authentication. I've written before about my enthusiasm for moving toward passwordless login using passkeys and methods such as FIDO2 — eliminating the shared secret that attackers phish, spray, and stuff. But notice what a passwordless world leaves behind. Entitlements will still pile up on individual accounts. Accounts belonging to former employees will still be abandoned rather than deprovisioned. Admin rights needed for a two-week project will still be found, years later, quietly attached to someone's identity. A password policy, once set correctly, largely holds its value. A privilege model erodes daily unless it is actively managed and governed.

That is why I consider identity management and zero trust not to be two separate security solutions, but two components of a complete, layered defense. Zscaler seems to have solved a good deal of this (I have nothing to do with Zscaler, and I am not promoting them here). In a zero trust architecture, no user, device, or network location is implicitly trusted, and every request must be verified (Rose et al., 2020) — meaning that when another control fails, identity is the layer that bears the brunt of the damage. In practice, I am a strong believer in combining Role-Based Access Control (RBAC) and Attribute-Based Access Control (ABAC). RBAC is the right starting point: map users to the roles their jobs actually require, and audit regularly to ensure every entitlement is accounted for. But roles are static by nature. ABAC adds the dynamic layer — factoring in the posture of the user's device, the location they're connecting from, and the classification of the data they're trying to reach. Combined, the two produce an access model that is both auditable and adaptive: what a user can access changes with circumstances, rather than sitting as a standing entitlement waiting to be abused.

One note I feel strongly about: the road to passwordless runs through strong passwords. Until the transition is complete, the credentials we still depend on must be hardened — going passwordless doesn't excuse weak passwords in the meantime; it depends on surviving the meantime.

A stolen credential gets an intruder into the lobby. Unmanaged privilege is what escorts that identity down to the server room or the boardroom, where everything that matters is kept. So harden the entrance — then govern everything beyond it.

References

National Institute of Standards and Technology. (2025). Digital identity guidelines: Authentication and authenticator management (Special Publication 800-63B-4). https://doi.org/10.6028/NIST.SP.800-63B-4

Rose, S., Borchert, O., Mitchell, S., & Connelly, S. (2020). Zero trust architecture (NIST Special Publication 800-207). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-207

Your University's AI Strategy Shouldn't Look Like Anyone Else's

 I've been deep in the architecture of my next book — an AI playbook for higher education — and I keep bumping into a problem that most of the "AI will transform the university" content quietly ignores: which university?

Because here's the thing nobody at the conference keynote wants to say out loud. A flagship R1 with a medical school, a regional state comprehensive, a 900-student faith-based college, and a system office are not four sizes of the same institution. They're four different organisms wearing the same word, "university," and handing them the same AI strategy is like handing a powerlifter, a marathoner, and a guy recovering from knee surgery the same workout plan. Technically, it's all exercise.

So one of the chapters I'm building is devoted entirely to this, and the more I work on it, the more I think it's the load-bearing wall of the whole book. The premise: institutional type isn't a demographic footnote. It determines your core AI problem — the thing that will actually eat you if you ignore it.

At an R1, the core problem is coordination. AI is already everywhere on that campus — the engineering college has three pilots, the med school signed something nobody in central IT has seen, and forty faculty have quietly wired models into their research workflows. The flagship doesn't need an AI adoption strategy; it needs an AI governance strategy, and a federated one at that, because the colleges of medicine, engineering, and law will run their own portfolios no matter what the provost's council decides. Central's real job is setting the floor — data protection, disclosure, human decision rights — and auditing against it. Not approving projects. That ship sailed before the committee formed.

At a regional comprehensive, the problem flips. It's capacity. These are the institutions that need AI's efficiency gains the most — advising caseloads north of 300:1, a financial aid office running on caffeine and heroics — and they have the least staff to implement anything. The cruel joke of this moment is that the institutions with the strongest per-dollar case for AI are the ones least equipped to deploy it. (This, incidentally, is the thesis of the whole book: higher ed needs AI most at exactly the moment it's least ready for it.)

At a small private, the problem is readiness. The data spine — the SIS, the CRM, the warehouse — is often held together with spreadsheets and institutional memory named Deborah. The upside is that a small private's cabinet can move in one board cycle, no faculty senate marathon required. The downside is identical: it can err in one board cycle. Speed without a data foundation just gets you to the wrong place faster.

And at a system office, the problem is authority: what gets decided centrally, what stays on campus, and who pays. Anyone who's watched a system-wide ERP project knows this fight. AI just runs it at higher velocity.

Two wrinkles from the research so far that deserve their own paragraphs, because almost nobody in the sector is talking about them.

First: if you're a public institution, your AI policy isn't entirely yours to write. A growing number of states have issued executive orders or legislation governing AI use in public agencies, and public universities are frequently in scope. Your regional comprehensive may already have obligations flowing downhill from the governor's office that your cabinet has never read.

Second, and this is the one that made me sit back from the keyboard: sunshine laws reach algorithms. A public university's AI-influenced admissions or financial aid decisions are potentially discoverable — public records requests, litigation — in ways a private institution's never will be. If a model is nudging who gets admitted or how aid is packaged at a state school, assume that someday a reporter or a plaintiff's attorney will get to look under the hood. That should change how you write vendor contracts and how you build audit trails, starting now, not after the request lands.

The chapter ends with a diagnostic — place your institution on the grid, then every subsequent chapter of the book closes with a "how this plays for you" note keyed to your archetype. Because a playbook that pretends the whole sector runs the same offense isn't a playbook. It's a poster.

More as the book develops. If you lead technology at one of these four kinds of institutions and your AI reality doesn't match my grid, I genuinely want to hear about it — the grid gets better when it gets argued with.  I think it would be a good idea to run a summary of all my chapters here and get genuine discussion going to reflect on and validate my ideas. 

Dr. Sam Kurien