Why we don't pilot with the most senior team

The conventional consulting playbook says start your AI pilot with the most senior team. The argument is straightforward: senior people set tone, executive buy-in cascades, the rest of the organization follows. It has been right often enough that many rollouts rely on it.

Three findings, as we read them, argue it’s wrong for AI specifically.

The gain is at the bottom, not the top. Brynjolfsson, Li, and Raymond’s 2025 study of a staggered rollout (n=5,172, Quarterly Journal of Economics) found AI raised productivity by 15% on average, with a 30% gain for less skilled and less experienced workers, little effect for the most skilled, and a small decline in the quality of their conversations.⁷ The study measured skill in customer support, not rank; we read senior teams as closer to the most skilled end. A pilot scoped to the senior team starts with the cohort where the productivity story is hardest to demonstrate, and there’s no clean before-and-after to take to the board.

We expect the visibility tax to be highest at the top. Slack’s Fall 2024 Workforce Index found 48% of desk workers are uncomfortable telling their manager they use AI.¹⁴ The Duke PNAS work adds a twist: evaluators who use AI often themselves penalize AI users less, so a manager who uses AI is a different audience from one who doesn’t.¹² The figures above are not broken out by rank, so this is our reading: senior employees have the most to lose from being seen as AI-dependent, because the role identity is “I make the judgment call”. So they may hide use, perform non-use, or partially adopt and partially revert. That makes pilot data noisy.

Self-report and telemetry measure different things. In a three-week randomized trial at Microsoft, developers given GitHub Copilot rated it more useful and more enjoyable, while telemetry showed no statistically significant change in code changes, pull requests or development time.⁶ People are good at narrating their own productivity. That doesn’t make them good measurement instruments for the rollout. It is one more reason we agree one number up front and compare work samples before and after, rather than rely only on how people say it went.

What we do instead. We brief leaders and managers first, so they know what is being asked and using AI is safe. Then we sequence pilots to the cohort that has the most room to grow and the lowest visibility cost. New hires, recent rotators, people who started the role inside the last 18 months. Their gain is easier to see. Their relationship to manager-visibility is less fraught. Whether their adoption holds through the 30, 60 and 90-day check-ins is the data that justifies the next phase of the rollout. The senior cohort’s training comes second, with a different curriculum aimed at the judgment layer where automation degrades output, and it’s anchored against the evidence the first cohort produced.

In the engagements we’ve run, pilots that start at the top tend to stall. Pilots that start one layer down tend to hold, and produce the evidence that funds the senior-cohort program. Buy-in comes from outcomes.