Conclusion

This study analysed 12,000 LLM recommendation queries across three software categories. What the data shows clearly is that LLM recommendations do not follow a single pattern. The underlying behaviour changes depending on the category, the model, and how established the market appears to be.

What is less certain — and more interesting — is why these patterns exist. Based on the data, and combined with our interpretation, a few themes stand out:

1. Search popularity is not a consistent driver — and likely depends on category dynamics
   
   The data shows that search volume does not uniformly influence LLM rankings.

   In email marketing, there is no meaningful correlation. Smaller brands consistently outrank much larger ones. In project management, the opposite pattern appears — higher search volume brands tend to rank better across all models. Video editing sits somewhere in between, with mixed behaviour depending on the model.

   One possible explanation is category competitiveness. Project management is a more mature and crowded space with clearer market leaders, which may make it easier for LLMs to reflect real-world brand hierarchy. Email, on the other hand, appears more fragmented, which may push models to rely on other signals like content quality or documentation.

   This is not something the data directly proves — but it is the most consistent explanation across what we observed.
   
   

2. Each model shows a distinct pattern — likely tied to how it retrieves and ranks information
   
   The differences between models are very clear in the data, but the reasons behind them require interpretation.
   • Claude consistently produces stable, repeatable rankings.
   • OpenAI appears to maintain a fixed pool of tools, while varying their order.
   • Gemini generates highly diverse outputs, often producing completely different lists.
   • Perplexity surfaces a noticeably different set of tools, often aligned with review and aggregator sites.
   
   These patterns likely reflect differences in training data, retrieval layers, and ranking logic. For example, Perplexity’s behaviour suggests a stronger reliance on external sources, while Claude appears more “opinionated” or internally consistent.
   
   While we cannot directly observe the internal systems, the consistency of these patterns across all runs suggests that each model operates with its own stable logic.
   
   

3. The “locked top positions” pattern suggests a strong positional advantage — but the cause is less clear
   
   Across all models and categories, the top positions behave very differently from the rest. Positions 1–3 are significantly more stable than positions further down the list. In some cases, these positions do not change at all across 1,000 runs. Most variation happens outside this top tier.
   
   What this suggests is a form of “lock-in” at the top. Once a brand reaches these positions, it tends to remain there.
   
   Why this happens is harder to pin down. It could be driven by stronger training signals, clearer consensus in source data, or reinforcement effects where frequently recommended tools continue to appear. Most likely, it is a combination of these factors.
   
   What we can say with confidence is the outcome, the top of the list behaves very differently from the rest, even if the exact mechanism remains uncertain.