AI & Networks

Why software has not learned from every customer

Shared software rarely turns every customer's data into better decisions for everyone else. Privacy, comparability, decay, applicability, and causality get in the way.

Effective
Last updated
Reading time
6 min

Enterprise software has promised the same thing for years: if many companies use the same platform, the platform should learn from all of them and make each one smarter.

Salesforce Einstein made that promise in 2016. Adobe Sensei made a similar promise. Microsoft, Workday, HubSpot, and many AI platforms have made some version of it too.

The pitch is attractive: your data improves our model, our model improves your decisions, and every customer benefits from every other customer.

In practice, the flywheel rarely works. MIT's 2025 State of AI in Business report, covered by Fortune, found that 95% of corporate generative AI pilots produced no measurable P&L impact. Gartner moved generative AI into the Trough of Disillusionment in 2024. The issue is not only model quality. The issue is that cross-company learning runs into five walls.

Wall 1: Privacy

Most companies do not let vendors train shared models on their operating data.

Contracts say it. Regulations push toward it. Enterprise buyers demand it. Microsoft, OpenAI, Salesforce, and others now emphasize that business data is not used to train shared models by default.

That makes sense. A brand does not want its customer data, pricing logic, support history, or growth strategy helping a competitor.

Cross-company learning has to start by respecting that boundary.

Wall 2: Comparability

Even if every company opted in, the data would not automatically be comparable.

A $2,000 ACV PLG company and a $500,000 ACV enterprise sales company both sell SaaS. Their CAC, payback, retention, and channel economics are not the same kind of number.

The same is true in ecommerce. A subscription supplement brand, a fashion brand, a high-ticket furniture brand, and a marketplace may all run Meta ads. Their creative fatigue, refund rates, repeat purchase curves, shipping costs, and margin profiles differ.

Benchmarks need cohort context, or they mislead.

That context has to be more specific than industry. In e-commerce, two brands can both sell supplements and still have very different economics. One sells a $39 first order with heavy subscription upside. Another sells a $120 bundle with lower repeat purchase. One wins through Meta creative volume. Another wins through affiliates and email. A shared average can be mathematically correct and operationally useless.

Wall 3: Decay

Growth lessons go stale.

Meta attribution changed after iOS 14.5. Google updates change SEO. Creative formats change. AI-generated content changes search and social feeds. Competitors copy faster. A tactic that worked last quarter may not work this quarter.

Annual benchmarks cannot keep up with quarterly behavior change.

The decay rate is different by domain. A finance policy may stay valid for years. A paid social creative rule can expire in weeks. A landing-page pattern may survive until competitors copy it. A tracking workaround may disappear after a browser or platform change. A useful shared-learning system has to know the shelf life of the lesson it is recommending.

Wall 4: Applicability

Even a clean insight from one company may not apply to another.

"Brands like you should use offer X" is only useful if "like you" is defined. Stage, AOV, margin, audience, category, channel mix, geography, and operations all matter.

Without applicability rules, a recommendation that worked for one brand becomes a polished guess for another.

Wall 5: Cause

Most cross-company insights are correlations.

Companies that did X also had better retention. Brands that used Y also had higher ROAS. Teams with Z process shipped faster.

Maybe X caused the result. Maybe the better companies were already better and also happened to do X. Maybe a hidden factor caused both. Without tests, holdouts, or careful design, the vendor is telling a story, not proving cause.

What this means for AI products

A useful AI system does not need to claim a magic data network effect on day one.

It first needs to make one company's own work legible:

  • what was tested
  • what was launched
  • what happened
  • what was decided
  • what evidence supported the decision
  • what can be reused
  • what should not be recommended because the data is too weak

Only then can cross-company learning be considered. If the internal data is messy, incomparable, stale, and unproven, pooling it with other messy data does not create intelligence. It creates a larger average nobody should trust.

The path to useful shared learning

The honest path is slower and more valuable than the usual pitch.

First, make one company's work measurable and reusable. The team should know what was launched, which audience saw it, what changed in revenue or margin, what decision was made, and whether the lesson is still current.

Second, make internal comparisons reliable. A brand should be able to compare one offer against another, one landing page against another, one channel against another, and one cohort against another without changing metric definitions every time.

Third, allow carefully scoped external comparisons. "Subscription beauty brands with $50-$90 AOV, U.S. paid social above 40% of revenue, repeat purchase above 25%, measured in the last 90 days" is a useful comparison. "E-commerce brands" is not.

Fourth, separate correlation from proof. If many similar brands saw better payback after adding bundles, that is a useful lead. It is not proof that a specific brand should add bundles. The recommendation should say what kind of evidence it has.

That is the difference between a real learning network and a benchmark dressed up as AI.

What Lyberty does

Lyberty starts with the per-company loop.

For each company, it ties work to evidence: campaigns, offers, pages, spend, revenue, approvals, decisions, and learnings. Metrics carry definitions. Recommendations carry scope and confidence. Missing inputs produce abstention instead of fake certainty.

Lyberty does not claim a cross-company data network effect today. That would be premature. The honest prerequisite is a clean internal record of work and outcomes. Cross-company benchmarks or shared learning should only ship when privacy, comparability, decay, applicability, and causal limits are handled explicitly.

What to ask vendors

  1. Does our data train a shared model?
  2. What cohort produced this benchmark?
  3. How fresh is the underlying data?
  4. Under what conditions does this recommendation apply?
  5. Is the claim causal or correlational?

If the vendor cannot answer all five, treat the "AI learned from everyone" story as marketing until proven otherwise.

Sources

  1. Salesforce. "Salesforce Introduces Salesforce Einstein." September 19, 2016. https://www.salesforce.com/news/press-releases/2016/09/19/salesforce-introduces-salesforce-einstein-artificial-intelligence-for-everyone/
  2. Fortune. "Adobe Says Sensei AI Makes Its Software Smarter." November 2, 2016. https://fortune.com/2016/11/02/adobe-sensei-ai/
  3. Oliv AI. "Salesforce Einstein Features." 2024. https://www.oliv.ai/blog/salesforce-einstein-features
  4. HubSpot. "2024 State of Marketing Report." https://www.hubspot.com/state-of-marketing
  5. Fortune. "MIT report: 95% of generative AI pilots at companies are failing." August 18, 2025. https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/
  6. Gartner. "2024 Hype Cycle for Emerging Technologies." August 21, 2024. https://www.gartner.com/en/newsroom/press-releases/2024-08-21-gartner-2024-hype-cycle-for-emerging-technologies-highlights-developer-productivity-total-experience-ai-and-security
  7. McKinsey & Company. "The state of AI in early 2024." https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-2024
  8. Gartner. "30% of Generative AI Projects Will Be Abandoned." July 29, 2024. https://www.gartner.com/en/newsroom/press-releases/2024-07-29-gartner-predicts-30-percent-of-generative-ai-projects-will-be-abandoned-after-proof-of-concept-by-end-of-2025
  9. Andreessen Horowitz. "The Empty Promise of Data Moats." May 9, 2019. https://a16z.com/the-empty-promise-of-data-moats/
  10. Open Science Collaboration. "Estimating the reproducibility of psychological science." Science, 2015. https://www.science.org/doi/10.1126/science.aac4716
  11. Simmons, Nelson, and Simonsohn. "False-Positive Psychology." Psychological Science, 2011. https://journals.sagepub.com/doi/10.1177/0956797611417632
  12. Microsoft. "Enterprise data protection in Microsoft 365 Copilot." https://learn.microsoft.com/en-us/microsoft-365/copilot/enterprise-data-protection
  13. OpenAI. "Business data privacy, security, and compliance." https://openai.com/business-data/
  14. Search Engine Journal. "Google Algorithm Updates & Changes." https://www.searchenginejournal.com/google-algorithm-history/
  15. DOJO AI. "Meta Ads Attribution in 2026." https://www.dojoai.com/blog/meta-ads-attribution-2026-changes-fixes
  16. Kohavi, Tang, and Xu. Trustworthy Online Controlled Experiments. Cambridge University Press, 2020. https://www.cambridge.org/core/books/trustworthy-online-controlled-experiments/D97B26382EB0EB2DC2019A7A7B518F59