The One Concept That Helps You Understand AI
If you’ve worked on AI projects, you’ve seen this. The proof of concept looks solid. Then, as more people get involved, problems start to show up. A customer gets confused with a prospect. Answers come from the wrong source. Your metrics become unreliable.
The instinct is to blame the model. Sure, there’s always a shinier new version of ChatGPT. But will it actually solve your problem?
This article gives you one mental model you can use on every AI project. AI is built on the relationships that words, phrases, sentences, and even paragraphs have to each other. A simple formula called the dot product is how AI measures those relationships and finds your data. Once you see how it works, you’ll know exactly what to fix to build more reliable AI.
AI Runs on Relationships
Traditional software is deterministic. In other words, you write the code, and the same input always produces the same output. AI isn’t. It measures how closely your input matches patterns it has learned, then predicts what comes next.
To do that, AI splits text into tokens (words or pieces of words) and turns each token into a vector or a list of numbers. Each number places the token along one dimension of meaning.
Think about the Chicago Bears and the Green Bay Packers. Both are NFL football teams. Both play in the NFC North, both are in the Midwest, and they share one of the oldest rivalries in sports. On those dimensions, they’re nearly identical. But search for “sports teams in Illinois,” and the Packers disappear. Now the Bears sit next to the Cubs, White Sox, Bulls, and college teams like Illinois and Northwestern.
Real AI models use thousands of dimensions like these. Which things count as “close” depends on the relationships the model can see, and on what you ask. So how does it measure closeness?
The Dot Product in 60 Seconds
Here is the formula for two vectors, a and b, each with two dimensions:
a · b = a₁ × b₁ + a₂ × b₂
Multiply the first numbers together, then multiply the second numbers together, and finally add everything together. Many numbers go in, and one number comes out. Real models do the same thing across thousands of dimensions. These calculations measure how similar two vectors are.
a · b = (3 × 2) + (1 × 2) = 8
That number tells you how much two vectors point the same way. These numbers can be very large or very small, which makes them hard to compare. To solve this, search systems usually normalize every vector to the same length (1). This version is called cosine similarity, and it always lands between −1 and +1:
• Close to +1: the vectors point the same way. The meanings are similar.
• Close to 0: the vectors are at right angles. The meanings are unrelated.
• Close to −1: the vectors point in opposite directions.
That’s all the math you need. The same calculation drives attention, the mechanism at the heart of every modern language model, and vector search, which finds the right documents in a RAG system.
The model does the math. You decide what it’s comparing.
Now, let’s put all the pieces together. You can’t change how a foundation model was trained. However, what you can change is what goes into those comparisons. You control how the data is structured. The rest of this article covers six ways to make those relationships clearer.
1. Name the Same Thing the Same Way
This one should be the most obvious. Name your columns the same way in every table, every time.
Table | Before (inconsistent) | After (consistent) |
orders | cust_no | customer_id |
invoices | client_key | customer_id |
support_tickets | CustomerID | customer_id |
orders | ord_dt | order_date |
invoices | InvDate | invoice_date |
In the “before” version, the model has to guess that three spellings mean one thing. In the “after” version, the relationship is obvious. Consistency creates patterns, and AI is built on patterns.
The relationship: One concept, one name, so the AI knows customer_id in one table means the same thing in another.
2. Define What Things Mean
Consider a CRM with client, customer, and prospect tables. Your team knows how they relate, but the model doesn’t. Worse, a lead and a contact are often the same person. The key difference is where they are in the sales lifecycle. A lead is unqualified, while a contact has passed a few checks and is closer to becoming a client.
If that isn’t written down, the model sees two near-identical things and guesses. Put the definition where AI will read it, in your catalog’s table descriptions:
-- Describe each table in plain language for people and AI
COMMENT ON TABLE crm.lead IS
'Person or company that has shown interest but is NOT yet qualified.
Becomes a contact after passing sales qualification.';
COMMENT ON TABLE crm.contact IS
'Qualified person. Same individual as their original lead (lead_id).
Becomes a client when the first contract is signed.';
The relationship: A written definition tells the AI how similar values like a lead and a contact differ, so it doesn’t treat them as the same thing.
3. Define Each Metric Once
Ask three departments what “revenue” means and you may get three answers. It could mean gross bookings for sales, recognized revenue for finance, or paid invoices for operations. If an AI assistant has to work that out from raw tables, it will pick one, and it may not be the correct one.
Define important metrics once, in a semantic layer such as Databricks metric views or a Power BI semantic model. Use plain-language descriptions of how each is calculated. Then every report, dashboard, and AI answer uses the same relationship between the word and the number. Skills can then point the AI to the right metric, but the calculation itself should live with the data.
Here is how each department’s version might be defined in the semantic layer, each under its own clear name:
-- FinanceRecognized revenue = SUM(quantity × price)
for delivered orders, minus refunds
-- SalesGross bookings = SUM(contract_value)
signed orders, before delivery or refunds
-- OperationsCollected revenue = SUM(payment_amount)
for invoices marked as paid
The relationship: Each metric gets one name and one calculation, so “revenue” never means three different things.
4. Connect Your Tables
Databricks Unity Catalog tracks lineage and relationships across your tables, including primary and foreign keys. That metadata helps tools like Genie, Databricks’ natural-language analytics assistant, learn the relationship between tables instead of guessing from column names.
Defining table relationships was a key point at the recent Databricks Data + AI World Tour. Declaring a relationship takes one statement:
-- Make the relationship explicit instead of implied
ALTER TABLE sales.orders
ADD CONSTRAINT orders_customer_fk
FOREIGN KEY (customer_id) REFERENCES sales.customers (customer_id);
This is also where graph databases sometimes beat relational ones. A graph database stores the relationships between records directly, so the path is stored, not guessed. Either way, the goal is the shortest, most accurate path to the correct result.
The relationship: Keys tell the AI exactly how tables link, so it follows the path instead of guessing it.
5. Keep Ideas Together When You Chunk
In a RAG system, the tempting shortcut is to slice documents into fixed 2,000-token chunks. That breaks apart sentences, paragraphs, and the relationships between them.
Imagine a chunk that ends with “I went to the bank.” Is that a financial institution or the edge of a river? Without other clues, the model can’t tell. The chunk gets an ambiguous vector, gets retrieved for the wrong questions, and produces a confident but wrong answer.
Instead, chunk by idea: a section, a chapter, a policy clause. Look for logical breaks like headings and topic changes. If you overlap chunks, follow the document’s structure, not a token count. Overlapping into the last paragraph of a different chapter doesn’t make sense, because those are unrelated thoughts.
The relationship: Keeping context together lets each chunk land near the meaning it actually belongs to.
6. Give Messy Documents One Structure
Databricks recently introduced the ai_parse_document function, which turns PDFs, Word files, and images into structured JSON. Better categories create better relationships.
I once did something similar with meeting records from the nursing boards of all 50 states. Each board meets monthly, and I collected two years of data. For each meeting, I captured when and where it was held, who attended, and the topics, sorted into three categories: nursing regulations, disciplinary cases, and new educational programs.
The original API pull was a mix of PDFs, Word files, and plain text, all in different formats. Converting them all to JSON helped normalize the results.
{
"state": "TX",
"meeting_date": "2025-03-14",
"location": "Austin",
"attendees": ["..."],
"topics": {
"nursing_regulations": ["..."],
"disciplinary_cases": ["..."],
"new_educational_programs": ["..."]
}
}
Fifty formats went in, and one structure came out. All 1,200 meetings went into a few tables that fed Power BI for easy analysis.
The relationship: The same tags every time put the same kind of information in the same place, no matter where it came from.
A Relationship Checklist for Your Next AI Project
Here is a key checklist to ensure the highest level of accuracy.
• Naming: Does the same concept have the same name in every table and document?
• Definitions: Are look-alikes (lead vs. contact, client vs. customer) clearly defined?
• Metrics: Is each key metric defined once, in one place?
• Chunking: Are documents split at logical boundaries rather than fixed token counts?
• Structure: Do messy inputs land in one consistent format every time?
• Connections: Are your table links defined with keys, and does each table have a plain-language description?
• • •
Conclusion: Make the Right Things Close
Behind every AI answer is a simple question. How close are these two things? The dot product answers it in milliseconds, billions of times a day. But it can only compare what you give it.
That’s why the best AI investment usually isn’t a newer model. In fact, you can often switch to a cheaper model if you get the fundamentals right. Consider all of these data relationships when building your data models:
• Consistent names
• Clear definitions
• Single metric definitions
• Connected tables
• Meaningful chunks
• Structured documents
That’s data governance and smart data engineering, and it’s what turns a promising demo into a system people trust.
Make the right things close, and nearly every model gets better.
If this helped you think differently about your AI projects, follow for more on data engineering and practical AI.
Comments