Can large language models play a role in data analytics? Absolutely. But in 2026, that question feels almost quaint. We’re well past debating whether LLMs belong in the analytics stack. The real question now is how they fit, where they fall short, and what happens when you pair them with the right predictive engine.
Because it turns out, LLMs are spectacularly good at some things. And genuinely terrible at others. The companies seeing real results have figured out which is which.
The 2026 AI landscape looks nothing like 2024
Let’s set the scene. Enterprise spending on generative AI hit $37 billion in 2025, a staggering 3.2x jump from the year prior. McKinsey’s mid-2025 survey found that 88% of organizations now use AI in at least one business function, and 72% have adopted generative AI specifically. Gartner reported a 1,445% surge in inquiries about multi-agent AI systems between early 2024 and mid-2025, and expects 40% of enterprise applications to feature task-specific AI agents by end of this year.
Those numbers are big. Impressive, even.
And yet only about 6% of companies qualify as AI “high performers” by McKinsey’s definition, meaning organizations where AI contributes more than 5% of earnings. Nearly two-thirds of enterprises haven’t even begun scaling AI beyond isolated experiments. The gap between having AI and getting value from AI is enormous.
So what’s going wrong? In many cases, it’s a mismatch between tool and task. Companies are trying to use LLMs for things LLMs were never built to do, like making accurate predictions from structured business data. And that’s where a lot of budget goes to waste.
LLMs don’t predict. They orchestrate.
This is probably the single most important thing to understand about the current AI landscape, and it’s something we’ve written about extensively in our piece on why LLMs alone won’t solve your business’s predictive needs.
LLMs were trained on language. They’re extraordinary at understanding questions, generating text, summarizing documents, writing code, and reasoning through complex problems in natural language. What they weren’t trained to do is look at 50,000 rows of transactional data and tell you which customers are about to churn.
The research backs this up convincingly. A rigorous 2025 study published in Nature Scientific Reports compared GPT-4 against XGBoost (a widely used gradient-boosted tree model) on a structured prediction task. XGBoost achieved an F1 score of 0.87. GPT-4 in zero-shot mode? Just 0.43. That’s roughly half the performance, on the same dataset, for the same task. Another study found that simply changing variable names could swing an LLM’s prediction error by up to 82%, which is a problem that traditional ML models don’t have at all.
A landmark NeurIPS paper also confirmed across 45 different datasets that tree-based models remain state-of-the-art for medium-sized tabular data. And Google’s TabPFN-v2, published in Nature in January 2025, showed that a purpose-built tabular model could match what ensemble baselines need four hours of tuning to achieve, in under three seconds.
The pattern is clear. For structured, tabular business data (the kind that powers churn prediction, lead scoring, demand forecasting, and customer lifetime value modeling), purpose-built ML models dramatically outperform LLMs. We dig deeper into this distinction in our guide to generative vs. predictive AI.
So if LLMs aren’t great at prediction, what are they good for in analytics?
Quite a lot, actually.
Where LLMs genuinely shine in the analytics workflow
LLMs have become the connective tissue of modern analytics. They’re not doing the predicting, but they’re making the entire predictive workflow more accessible, faster, and more useful to business teams.
Natural language interfaces. Instead of writing SQL queries or navigating complex dashboards, teams can ask questions in plain English. “Which customer segments had the highest churn last quarter?” or “What’s driving the drop in our lead conversion rate?” Platforms like Databricks (with AI/BI Genie) and ThoughtSpot (with their new agentic analytics suite) are building entire product lines around this capability. At Pecan, our Predictive AI Agent uses LLMs to let business teams ask predictive questions directly, then routes those questions to validated ML models under the hood.
Data preparation and feature engineering. One of the most time-consuming parts of any predictive project is cleaning, joining, and preparing data. LLMs can automate much of this work, identifying relevant columns, suggesting data transformations, and flagging quality issues. This alone can cut weeks off a project timeline.
Explaining results. A prediction score is only useful if people understand it and act on it. LLMs translate model outputs into clear, contextualized explanations that business users can actually work with. Instead of seeing “churn probability: 0.83,” a marketing ops manager sees a plain-language summary of why a particular customer is at risk, and what they might do about it.
Unstructured data enrichment. Customer reviews, support tickets, social media mentions, email threads: there’s a goldmine of signal in unstructured text data. LLMs can extract sentiment, topics, and intent from these sources and feed that enriched data into predictive models as additional features. Our post on what you can do with an LLM covers this territory in more depth.
The takeaway: LLMs and predictive ML aren’t competing technologies. They’re collaborators. LLMs make analytics accessible and understandable. Predictive ML makes analytics accurate.
The benchmarks, side by side
We think it’s worth laying this out clearly, because the “just use ChatGPT for everything” temptation is real, and it’s costing companies time and money. (For a deeper dive into those risks, check out our piece on the risky reality of relying on ChatGPT for predictive modeling.)
Here’s how different approaches actually perform across key business prediction tasks:
Churn prediction: XGBoost and gradient-boosted ensemble models consistently achieve AUC-ROC scores above 0.93 on standard benchmarks, with accuracy and F1 scores around 0.84 or higher. In e-commerce settings, recent research showed XGBoost hitting 97.8% accuracy. No published study has shown LLMs matching these results on structured churn data.
Lead scoring: Companies using ML-powered predictive lead scoring report around 40% improvement in lead-to-purchase conversion rates, with models predicting conversion at 85% accuracy before a single sales call. These models primarily use logistic regression, random forests, and gradient boosting. Not LLMs. (We cover the mechanics of this in our guide to predictive lead scoring with AI.)
Demand forecasting: ML-driven forecasting reduces errors by 30-50% compared to traditional statistical methods, leading to a 65% reduction in lost sales from stockouts. Random forests with feature differentiation have outperformed ARIMA, SARIMA, and TBATS across multiple simulation scenarios.
Small datasets (under 100 samples): This is the one area where LLMs and tabular foundation models can outperform classical ML, leveraging their pretrained knowledge to compensate for limited data. TabPFN-v2 is particularly strong here.
Text-heavy classification: When features require natural language understanding (say, predicting customer satisfaction from support tickets), LLMs outperform. This is their home turf.
The smart play? Use LLMs where they excel (language, interfaces, explanations, unstructured data) and purpose-built ML where it excels (structured predictions on your proprietary business data). That hybrid approach consistently delivers the best business outcomes.
Welcome to the agentic era (and why predictions need ML more than ever)
If there’s one buzzword dominating 2026, it’s “agentic AI.” And unlike most buzzwords, this one reflects a genuine shift in how businesses deploy AI.
Agentic AI refers to autonomous systems that don’t just answer questions; they take actions. They make decisions, execute workflows, and complete tasks with minimal human intervention. Salesforce launched Agentforce. Microsoft built Agent 365. Google released Gemini Enterprise with pre-built data science agents. AWS introduced SageMaker Data Agent. ThoughtSpot rolled out four specialized analytics agents. The Open Semantic Interchange consortium (Snowflake, Salesforce, ThoughtSpot) formed to standardize how these agents communicate.
Deloitte’s Tech Trends 2026 report describes organizations now managing AI agents as, essentially, “a silicon-based workforce.” Forrester predicts enterprise applications will shift from enabling employees to accommodating a digital workforce of AI agents. The agentic AI market is projected to grow from $7.8 billion to over $52 billion by 2030.
This is where things get interesting for predictive analytics. Because AI agents use LLMs for reasoning, planning, and communication. That’s their brain. But when an agent needs to predict something, like which customers will churn next month, which leads are most likely to convert, or what demand will look like for a particular product, it needs to call a reliable, validated ML model. The LLM doesn’t have the answer. It knows how to ask the question and interpret the response, but the actual prediction has to come from a purpose-built engine.
Think of it this way: an agentic workflow might use an LLM to understand a business user’s question, prepare and route the data, select the right model, and then explain the results in plain English. Every step except the actual prediction runs on language models. The prediction itself? That’s ML.
This is exactly the architecture Pecan built its Predictive AI Agent around: LLMs handling the conversational interface and workflow orchestration, with Pecan’s validated predictive engine doing the actual modeling. We covered the thinking behind this approach in our recent launch announcement, where our CEO Zohar Bronfman put it simply: every business has its own unique data fingerprint, and LLMs alone can’t make sense of it.
Real results: what companies are actually achieving
Theory is nice. Numbers are better. Here’s what real companies have accomplished by combining LLMs with purpose-built predictive models:
Clearwave Fiber deployed churn prediction models that reduced churn by 20x in their highest-risk customer segment, with live predictions running within two months of starting the project.
Hydrant, a DTC wellness brand, used predictive churn and winback models to achieve a 2.6x higher conversion rate and 3.1x higher revenue per customer on targeted campaigns. The model was built in two weeks and delivered 83% accuracy.
A major social casino publisher used customer lifetime value predictions to customize offers, increasing average revenue per user by 30%.
PlaySimple leveraged predictive user acquisition models to achieve 3x more installs at one-third the cost per install.
Travis Perkins, a building materials company, saw a 54% reduction in customer churn, 86% growth in database value, and a 34% increase in customer lifetime value within 12 months using predictive analytics.
And the broader industry data tells a similar story. Companies using predictive analytics are 2.5x more likely to experience significant revenue growth. Customer churn prediction delivers an average ROI of 775% in the first year. AI-powered demand forecasting reduces supply chain errors by 30-50%. Early adopters of automated ML platforms report $3.70 in value for every $1 invested, and top performers see returns exceeding $10 for every dollar.
The common thread? These results come from actual predictive ML models built on proprietary business data, not from asking a chatbot to guess.
See Pecan’s Predictive AI Agent in action
Curious how this all works in practice? Watch our product tour to see how Pecan’s Predictive AI Agent turns a plain-English business question into a validated, production-ready prediction:
Getting started: practical implementation steps
If you’re ready to move beyond LLM-only experimentation and start getting real predictive value from your data, here’s a practical path forward:
1. Identify your highest-value prediction. Start with one clear business question. “Which customers are likely to churn next quarter?” is better than “help us use AI.” Churn prediction, lead scoring, and demand forecasting are the most common starting points because they have clear, measurable ROI. Our complete guide to predictive modeling can help you scope this.
2. Audit your data readiness. You don’t need perfectly clean data (nobody has that), but you do need historical outcome data. For churn, that means records of which customers actually left. For lead scoring, which leads converted. The good news is that most CRM and marketing platforms already store this. Pecan’s agent handles data prep, joins, and quality checks automatically, so the bar is lower than you might expect.
3. Choose the right tool for each job. Use LLMs for natural language interfaces, data exploration, and explaining results. Use purpose-built ML for the actual predictions. Don’t try to force a chatbot to do regression analysis on 100,000 rows of transaction data. (We’ve seen companies waste months on this. It doesn’t end well.)
4. Prioritize integration. A prediction that lives in a standalone dashboard is a prediction that gets ignored. Make sure your predictions flow into the systems where your team actually works: Salesforce, HubSpot, your data warehouse, Slack, wherever decisions happen. This is consistently the factor that separates companies that get value from predictive analytics and companies that don’t.
5. Start small, measure, expand. Deploy your first model on a single use case. Measure the impact over 30-60 days. Use those results to build the business case for expanding to additional predictions. Pecan customers typically go from first question to production-ready model in days or weeks, not months, so the feedback loop is fast.
6. Treat predictions as living products. Business conditions change. Customer behavior shifts. Models need monitoring and occasional retraining. Look for platforms that track model performance automatically and flag when retraining is needed, rather than treating deployment as a finish line.
The bottom line
The companies getting the most from AI in 2026 aren’t the ones spending the most. They’re the ones that understand what each type of AI is actually good at.
LLMs are remarkable at language, reasoning, and making complex systems accessible. Predictive ML models are remarkable at making accurate, reliable forecasts from your structured business data. Together, they create something neither can achieve alone: predictions that are accurate, accessible, and actionable.
That’s what Pecan’s Predictive AI Agent delivers. You ask a question in plain English. Pecan’s AI agents handle the data preparation, feature engineering, model building, and validation. And you get back a trusted, explainable prediction, right where you need it, in minutes or days instead of months.
No data science team required. No months of waiting. Just your data, working for you.
Team Pecan is what happens when you put a bunch of data geeks in a room and tell them to make machine learning suck less. We’ve built models, broken models, fixed models, and occasionally questioned our life choices at 2am debugging feature pipelines. Now we write about it so you don’t have to learn the hard way. Think of us as your slightly unhinged data science friends who actually want you to succeed.
