How domain expertise changes AI analysis: Shopify vs. HubSpot vs. Google Search Console
The same AI model produces fundamentally different output when it has domain-specific knowledge. Here is what that looks like across three real data sources.
TL;DR
The same foundation model — same weights, same training — produces fundamentally different analysis when given domain-specific knowledge about a data source. This post shows the concrete difference across three providers: Shopify, HubSpot, and Google Search Console. The model is the same. The domain knowledge changes everything. Domain expertise is layer two of the semantic layer.
Every AI analytics product uses the same handful of foundation models. Claude, GPT-4, Gemini — they are all capable of reading a chart and producing a paragraph. The output sounds professional. It uses the right vocabulary. It is often wrong in ways that are hard to detect.
The problem is not the model. The problem is that analyzing Shopify data correctly requires knowing things about Shopify that the model was not trained to prioritize. The same applies to HubSpot, Google Search Console, and every other data source.
This is what domain expertise does in an AI analytics context: it gives the model structured knowledge about the specific data source it is analyzing. Not a generic prompt. Not "be a good analyst." Actual per-provider knowledge about metric definitions, common misinterpretations, benchmark ranges, and analysis patterns.
Here is what the difference looks like across three real sources.
Shopify: revenue tells a story, but often the wrong one
Generic analysis
Revenue increased 14% week-over-week, reaching $87,400. This represents strong growth and suggests the business is trending positively. Top-selling products included the Heritage Tote and the Classic Hoodie.
This is accurate. It is also useless. A team reading this learns nothing they couldn't see by glancing at the chart for two seconds.
Domain-informed analysis
Revenue increased 14% WoW to $87,400, but order count is flat at 1,240 (up 0.3%). The entire revenue increase is AOV-driven — AOV jumped from $61.80 to $70.48. This pattern typically means fewer small transactions replaced by fewer-but-larger ones, not genuine demand growth.
Breakdown: Net Sales (excluding tax, shipping, and returns) grew 11%, while Total Sales grew 14%. The 3-point gap suggests higher-priced items carry proportionally more tax and shipping. Conversion rate held at 2.1%, but traffic from Meta Ads converted at 1.4% vs. organic at 3.8% — the ad traffic is diluting the blended rate.
Watch: if AOV-driven growth continues without order volume recovery, the revenue line is fragile. One fewer high-value order per day erases the trend.
Same data. Same model. The difference is the domain knowledge layered in:
- AOV masking: the model knows that revenue growth without order growth is AOV-driven, and that AOV-driven growth is structurally different from volume-driven growth.
- Net Sales vs. Total Sales: the model knows Shopify reports both, that they differ by tax/shipping/returns, and that the gap between them carries diagnostic information.
- Conversion rate by source: the model knows that blended conversion rate hides traffic-source effects, and that ad-driven traffic typically converts lower than organic.
None of this is secret knowledge. Any experienced e-commerce analyst knows it. The point is that the model doesn't apply it unless it is explicitly encoded in the analysis context.
HubSpot: pipeline metrics that mislead
Generic analysis
Pipeline value increased 22% this month, with 47 new deals created. The average deal size is $18,200. Sales activity is healthy with 312 calls logged and 89 meetings booked.
Again, accurate. And almost entirely uninformative. What does "healthy" mean for this team? Is 22% pipeline growth good or expected? Are those 47 deals progressing or stuck?
Domain-informed analysis
Pipeline value grew 22% to $856K, but stage-conversion analysis shows the growth is concentrated in Stage 1 (Discovery). Deals in Stage 2 (Qualification) grew only 4%, and Stage 3 (Proposal) declined 8%. This is a top-of-funnel surge that has not yet converted to real pipeline — expect the headline number to correct within 4-6 weeks as Stage 1 deals either advance or die.
Deal velocity tells a different story than deal count. Median days-in-stage for Stage 2 deals increased from 11 to 18 days this month. When qualification velocity slows by more than 30%, it historically correlates with a drop in close rates 6-8 weeks later. Worth investigating whether the new deals are lower-quality or whether the sales process has a bottleneck.
Activity-to-outcome ratios: 312 calls and 89 meetings produced 47 new deals — a 15.1% meeting-to-deal rate, down from 19.3% last month. The team is doing more activity with less conversion. Common causes: targeting shift, messaging drift, or seasonal buyer hesitation.
The domain knowledge at work:
| Dimension | Generic output | Domain-informed output |
|---|---|---|
| Pipeline growth | "Increased 22%" | "Concentrated in Stage 1 — not yet real pipeline" |
| Deal velocity | Not mentioned | "Qualification velocity slowed 63% — historically predicts close-rate decline" |
| Activity ratios | "312 calls, 89 meetings" | "Meeting-to-deal rate dropped from 19.3% to 15.1% — activity up, conversion down" |
| Forecast implication | "Sales activity is healthy" | "Expect pipeline correction in 4-6 weeks" |
The generic version would leave a sales leader feeling fine. The domain-informed version would have them on the phone with their team by lunch.
Google Search Console: the data source most models get wrong
Google Search Console data is uniquely challenging for AI analysis because the metrics interact in non-obvious ways. Position, clicks, impressions, and CTR are interdependent, and most generic analysis treats them as independent.
Generic analysis
Total clicks increased 8% this month. Average position improved from 14.2 to 12.8. Impressions grew 12%. Top queries include "project management software" and "task tracking tool."
Domain-informed analysis
Total clicks grew 8%, but the growth came from a different mechanism than the position improvement suggests.
Page movers: Three pages drove 73% of the click increase:
/features/gantt-charts— new to page 1 for "gantt chart software" (position 18→6), adding ~420 clicks/blog/project-management-tips— impressions up 40% from "project management" broad match expansion, position stable at 11/pricing— CTR improved from 2.1% to 3.8% after the title tag update on May 3, no position changeQuery clustering: The "project management" cluster (47 queries) gained 15% impressions but lost 3% clicks — average position improved from 15→13, but this moves queries from the bottom of page 2 to the top of page 2, which has negligible CTR impact. Meaningful click growth requires breaking into positions 1-5.
Position distribution: The average position improvement (14.2→12.8) is misleading. The improvement is driven by new long-tail queries entering at positions 8-15, which dilute the average upward. Core head terms are positionally flat. The "improvement" is a compositional effect, not a ranking gain.
This is where domain expertise matters most. The generic model read four numbers and described them. The domain-informed model applied three analytical frameworks that are standard practice for SEO analysts:
- Page movers: disaggregating total performance into page-level contributors. The model knows that aggregate GSC metrics almost always mask page-level stories.
- Query clustering: grouping related queries and analyzing the cluster as a unit. The model knows that individual query data is noisy but cluster-level trends are meaningful.
- Position distribution vs. averages: the model knows that average position is one of the most commonly misinterpreted metrics in SEO, because compositional changes (new queries entering the index) look like ranking improvements.
The pattern across all three
Every data source has its own version of the same problem: surface-level metrics that look straightforward but carry hidden structure. Revenue masks AOV effects. Pipeline value masks stage distribution. Average position masks compositional shifts.
A generic model reads the surface. A domain-informed model reads the structure.
This is not about making the model "smarter." The same model handles all three examples above. The difference is a structured knowledge layer — per provider — that encodes the analysis patterns, benchmark ranges, and common misinterpretations specific to that data source.
In the five-layer semantic layer model, domain expertise is layer two. It sits above metric definitions (layer one: what the numbers are) and below business context (layer three: what the numbers mean for your business specifically). Together, the three layers are the difference between AI that describes charts and AI that interprets them.
What this means in practice
If you are evaluating AI analytics tools, the question to ask is not "which model do they use." The question is: what does the tool know about the specific data source it is analyzing?
Ask it to analyze a Shopify revenue chart. If it doesn't mention the distinction between Net Sales and Total Sales, or notice when AOV is masking order trends, it doesn't have Shopify domain expertise. It has a prompt.
Ask it to analyze a HubSpot pipeline. If it doesn't look at stage-conversion rates or deal velocity, it is doing arithmetic, not analysis.
Ask it to analyze Google Search Console data. If it reports average position as a straightforward metric without questioning the composition, it doesn't understand how GSC data works.
The model is table stakes. The knowledge layer is the product.
