5 Takeaways from the Databricks World Tour in Chicago

On October 29, 2026, the Databricks World Tour stormed Chicago. The event showcased more than 20 innovative sponsors, several breakout sessions, and a keynote that unveiled major upgrades to existing offerings. To set the stage, the keynote opened with the song “The Greatest Show.” Here are my five biggest takeaways.

1. Lakebase and LTAP: one home for transactions and analytics

If you have ever built a change data capture (CDC) pipeline, you know the feeling. Records go missing. Rows show up twice. Things can go awry quickly and turn into hours of frustrating work. The keynote speaker called CDC “change data corruption” to a crowd of nodding heads.

Databricks' answer to those CDC headaches is the newly enhanced Lakebase. It combines two worlds that have traditionally lived in separate databases:

•         OLAP (online analytical processing), the engine behind data warehouses, optimized for column aggregations.

•         OLTP (online transactional processing), the engine behind applications, built for fast row inserts and lookups.

The beauty is that it consolidates your data in one place designed to serve both new transactions and near-real-time reporting. Databricks calls this new paradigm LTAP, or Lake Transactional/Analytical Processing.

New applications run on Lakebase’s Postgres engine, and the same data is readable in open formats like Delta and Iceberg. Data flows in smoothly, and analysts don’t have to make a ton of changes to build reports. Under the hood there are still two engines, each tuned to store and index data differently. But because data is no longer copied from system to system, you avoid the delays, missing records, and frustrating error messages. For simpler applications, your integration works seamlessly out of the box.

2. Unity Gateway: putting a budget on AI

I was also impressed by the sleek new Unity Gateway. In the demo it looked clean and easy to navigate, unlike the clunky and often confusing setups of competing tools. Below are just a few of the new features that impressed me.

Cost control

•         Token limits. Cap total tokens or tokens per minute, and go as granular as individual users.

•         Unified Trace Table. One view of AI usage that makes cost far easier to manage.

•         A built-in Genie for savings. It hunts for cost-cutting opportunities. In one example, Genie flagged a frontier model being used to retrieve Slack messages and suggested a more affordable model.

Security and governance

•         MCP control. Decide exactly which MCP servers are allowed.

•         Guardrails. Add data masking for personally identifiable information (PII).

•         AI asset registry. Keep track of every model you are using in one organized place.

3. Lineage: a road map for your AI

Because all of your data lives in Unity Catalog, and lineage is already built into Databricks, your AI gets a boost for free. Since your model is only as good as your ontology, it makes sense to govern everything in one place while organizing it into tables connected by common relationships.

That organization helps a model find the right columns and rows. Once it does, it can traverse your lineage layers like following roads on a map, tracing a number back to where it came from instead of guessing. Less wandering means fewer tokens, lower cost, and more trustworthy answers.

Picture an agent asked for last quarter's readmission rate. Without lineage, it scans dozens of tables with similar names and picks the one that looks right. With lineage, it sees which table feeds the official metric, how it was transformed, and where it came from. It follows the path instead of guessing.

One speaker compared it to Google’s PageRank. Google judges a web page partly by which other pages link to it. In the same way, how your tables connect to each other tells AI which data to trust.

4. ai_parse_document: skip the chunking pipeline

One of the most practical functions they showed was ai_parse_document. Instead of building a LangChain pipeline to chunk and store your documents, you call one SQL function. It reads PDFs, images, Word files, and PowerPoint decks, and returns structured output that labels titles, section headers, text, tables, and more. Here is a sample query.

SELECT  path,  ai_parse_document(content) AS parsed

FROM READ_FILES('/Volumes/main/docs/contracts/', format => 'binaryFile');

Why does that matter? Picture a folder of vendor contracts. A typical chunking pipeline slices each one into fixed-size blocks of text, so a pricing table gets split across two chunks and a page footer lands in the middle of a clause. Your model then retrieves fragments and fills in the gaps itself. With ai_parse_document, the pricing table stays a table, headers and footers are labeled so you can drop them, and each clause keeps its section heading. The model gets cleaner context, which means fewer tokens and better answers.

It also lives where the rest of your data lives. The parsed output lands in a governed table in Unity Catalog, with lineage, instead of in a side pipeline someone has to maintain. For teams sitting on piles of PDFs, contracts, invoices, or scanned forms, it is well worth checking out.

5. Genie in healthcare: lessons from Vizient

Genie is not a new offering, but Vizient provided an excellent example of how to use it properly. They explained its limitations and shared solid ways to improve the results.

The challenge

Vizient's reports were too limited, and its analytics often lagged. But clinical data is massive, sprawling, and sensitive. More than 400 tables sit behind their reporting application, and more than 1,700 clients submit data. Furthermore, HIPAA sets strict rules for protected health information (PHI). With so many tables, an AI agent gets lost in the joins, and results can come back as wild hallucinations.

What made it work

Vizient narrowed the problem. They built one agent per domain on Unity Catalog, using a few clean, documented tables and metric views instead of all 400. They defined clinical jargon and KPI logic, wrote SQL expressions for measures like length-of-stay (LOS) index and readmissions, and added example SQL for common questions. Complex queries were pre-built, providing a strong ground truth. Since AI tends to fill in gaps when information is missing, the pre-built queries plugged the holes.

Then they tested. Subject-matter experts asked real clinical questions and gave a thumbs-up or thumbs-down to each response. The rollout moved in phases: SMEs first, then a pilot group, then everyone. Here is what’s most interesting. About 60% of the pre-rollout effort went into validation, proof that humans are still a necessary part of a successful AI rollout.

The work didn’t stop at launch. The team monitored accuracy, errors, and drift against defined benchmarks, and simple thumbs-up and thumbs-down feedback helped consistently improve accuracy.

Here is where it all pays off. End users can reach more data and ask better questions. Vizient didn't throw away its dashboards, but Genie gave them another way to explore their data and became one more tool to help management make smarter decisions.

The bottom line

Three things I plan to act on:

1.       Build new applications in Lakebase. New development should be built directly in Lakebase where it fits. This saves time, simplifies the process, and ultimately saves money.

2.       Unity Gateway enables AI governance and cost controls. Token limits and a trace table are far easier to put in place early than to bolt on after the bill arrives.

3.       Good development standards are still key to AI success. Your AI is only as good as how you build it. However, Databricks can help with automatic lineage and functions like ai_parse_document.

I'm also keeping some healthy skepticism. LTAP looks like a game changer, but it will not replace every process. Data from your ERP or CRM systems will still need to be remodeled from complex normalized structures into more denormalized designs for reporting. Genie has improved but still requires a lot of testing and fine-tuning. I am excited to see how these tools evolve, but a few pre-recorded demos are not enough to declare victory.

Still, the thread running through all five takeaways is hard to dispute. Developers will now move less data, and development may become simpler. The tools around AI have also improved dramatically since the introduction of the original DBRX and early versions of Genie. If your team is still babysitting CDC jobs and wondering where its AI budget went, now is a good time to take a closer look at Lakebase and Unity Gateway. In Chicago, Databricks truly put on “The Greatest Show.”

Further reading: LTAP overview (Databricks docs) · Unified trace table (Databricks docs) · ai_parse_document (Databricks docs)