Ten quality criteria to watch out for...

1. Data collection Many market reports and studies are based on figures extrapolated from surveys of professionals vested with responsibility for cyber security management, strategy and governance. Largely, such reports make clear the given practitioner profiles of its survey sample – total number of respondents, job roles, size of employer organisation, its given vertical sector, geographic location and other pertinent contextual information. Good reports also provide information about survey methodologies, and explain any interpretation or filtering that’s been applied to the data – e.g., for weighting or bias correction – and state the reasons for doing so.
   

2. Focus / scope The more focused market intelligence is, the more intelligible its findings are likely to be. A desire to be as comprehensive as possible is understandable, but can diminish a report’s insight quotient by stretching its scope. Setting a high number of survey questions doesn’t necessarily result in more informative survey results.
   

3. Contents breakdown Some reports seem very keen to impart/share their findings and dive straight into the stats and percentages without offering an overview or summary of its discoveries. A prefatory section headed ‘Executive Briefing’ is a throwback term to the times when executives were deemed indisposed to getting their heads round ‘deep dive’ analysis, but is now often an introduction by another name. As more senior executives have acquired greater responsibility for cyber governance, the notion of the ‘executive briefing’ is somewhat outdated; they too need to read intelligence in full.
   

4. Expression Setting the appropriate editorial ‘voice’ for the main part of a report can prove a thorny challenges for its producers. Achieving the right tone on the scale between accessibly first-person conversational style and more objective third-person gravitas, calls into play some key considerations. In some instances it’s arguable that the field of cyber intelligence can be of profound economic and societal importance, and should not sacrifice gravitas in order to make a report more laid-back and ‘accessible’. Adopting a sensationalist or alarmist tone to draw attention to trends, incidents and worst-case scenarios, is another pitfall for the hapless editorial voice.
   

5. Size / structure Intelligence reports and studies can range in size from 10 pages to upwards of 50 pages. The longer the piece of work, the more vital a well thought-out structure is to ensuring reader engagement and understanding. Good structure serves as a framework that holds the report together, and can also support an underlying narrative flavour to run through the report.
   

6. Presentation Effective page layout and graphical design are, of course, highly important to ensuring that intelligence – figures and commentary – are presented clearly and intelligibly. Yet intelligence publications and studies regularly err toward eye-catching and amusing imagery that adds nothing to their intelligence quotient. Decorating a report’s pages with quirky illustrations and ‘safe’ stock imagery (think photo library models posed round a conference table) may please graphic designers but can be seen as distractive to target audiences with a technological information requirement. Effective layout and design also incorporates navigational features as seamlessly as possible, especially in larger pieces of intelligence.
   

7. Data visualisation Graphs, diagrams and visual elements that display categorical or numerical data – e.g., bar charts, pie charts, infographics, ‘big number’ percentages – are a staple of most market reports and studies. These often convey complex data sets, so it’s imperative that they are designed to be easy to understand and are clearly readable. Designs conceits such as reduced type sizes and white text on coloured backgrounds might look cool on a designer’s screen, but can prove hard to discern on a PDF or print document.
   

8. Gobbledygook In English-speaking markets at least, the global ICT sector has a long-established over-reliance on industry buzzwords, clichés, slogans and other hackneyed phrases, and it is not too surprising these are repeated in sector outputs like reports, studies and briefing documents and even white papers. In so far as these buzzwords serve as a kind of lingua franca between professional cohorts – well, they’re harmless if a tad tedious. A wider problem, arguably, is that buzzwords have also become a kind of indicator that a piece of work may not be of good quality.
   

9. Shortcuts Reports can always benefit from the inclusion of an accurate and comprehensive index: these are useful for first-glance readers who want to check if a specific topic, technology or subject heading is covered. (Sure, the search function of a PDF can serve the same purpose, but they do have their idiosyncrasies). In fact, indexes seem comparatively unusual components in many of the market intelligence reports that Cyber Risk Review editors have viewed over the last few years. Indexes are also navigational aids, and can serve as entry points into reports that work as effectively as the main contents listing.
   

10. Sources Similarly, footnote citations and references are especially useful for enlarging on technological terms and providing links to primary sources and further information. They also function to flag up the fact that an intelligence report’s authors and researchers have done supplementary groundwork and fact-sourcing and checking, and have attempted to produce work of market insight that is not altogether proprietary in its perspective and conclusions.

Image credit: Moon / Unsplash