A survey that asks respondents to rate their own organization's security or governance maturity is not measuring an independently verified fact. It is measuring what a respondent — usually someone with a professional stake in the answer — believes and is willing to report, which is a different and less reliable thing.Why Self-Reported Data Skews OptimisticA security leader rating their own programme has some incentive, conscious or not, to describe it favourably: it reflects on their own performance, and an anonymous survey rarely carries any verification step. The result is a well-documented pattern across self-reported data generally — respondents tend to rate themselves as better than average, which is not statistically possible for the population as a whole.Reading Self-Reported Findings More CarefullyDiscount the absolute numbers, trust the relative trend: if a survey shows more respondents reporting a capability this year than last, the direction is more reliable than the exact percentage.Look for behavioural questions, not opinion questions: 'do you test your incident response plan annually' is more verifiable, and less prone to inflation, than 'how would you rate your incident response maturity.'Check who was surveyed and how they were reached: a survey distributed through a vendor's existing customer list, or at a conference for a paid audience, is not a neutral sample of the market.Treat a survey as a hypothesis generator, not a conclusion: a finding worth taking seriously is one worth checking against your own organization's actual, verifiable state — not one worth citing on its own authority.“Ask a room of security leaders to rate themselves, and more than half will describe themselves as above average. That alone should tell you what the number is really measuring.”Why It Is Still Worth ReadingNone of this makes industry surveys worthless — the direction of change, the topics gaining or losing attention, and the questions an industry is starting to ask itself are all genuinely informative. What they are not is a substitute for measuring your own organization directly, and the value of a survey is highest when it is read as a prompt to check something, not as the check itself.Share this articleX (Twitter)LinkedInFacebookCopy Link
Industry SurveyJul 24, 2026What a Board Survey Cannot Tell You About Your Own RiskAn industry survey describes what other boards say they worry about. It says nothing about what your board should actually be worried about, and the two get conflated more often than either side notices.
Benchmark StudyMay 27, 2026Benchmarking Without a Peer Group You Can TrustMost organizations cannot name, with confidence, who their real security peer group is. Benchmarking against an assumed one is common, and it is one of the quieter ways a comparison misleads.
Regulatory OutlookJul 10, 2026Reading a Regulatory Outlook Without Overreacting to ItA regulatory outlook describes what could change. Treated as a description of what has already changed, it produces either premature spending or, worse, a false sense that a real change is 'just proposed' long after it has taken effect.
