A member rings about Table 4
The quarterly statistics went out yesterday. This morning a member's commercial director calls. Table 4 shows production in one region for one product group, and there are only two firms in that region making that product. Their competitor can subtract its own figure and see theirs. They are not happy, and they want to know how it happened.
Nobody meant it to. The analyst built the tables in Excel as always, checked the big cells, and missed that one breakdown had become sparse since a firm closed its plant. Your confidentiality undertaking to members says this should not happen. The trust that makes firms share data at all is the thing at risk.
Why sparse cells slip through
Disclosure rules are usually simple to state and tedious to apply. Applying them by eye across dozens of tables, every period, is where they fail.
- Rules exist in a policy document, not in the spreadsheets that build the tables.
- Membership changes (a closure, a merger, a new entrant) change which cells are sparse without anyone noticing.
- Suppressing one cell is not enough when row and column totals let a reader work it out, so secondary suppression is needed and easy to forget.
- Ad hoc requests from government or press get built quickly, outside the usual checks.
- There is no record of what was checked for each table, so nobody can show the rules were applied.
Your rules, and whether they meet any legal or competition-law requirements, are for your board and advisers. What we fix is making sure the rules you have actually run on every table.
What a disclosure costs
Members stop sending data, or send less. A statistics series that took years to build loses coverage. Staff spend days on apologies, investigations and reassurance. And your team becomes nervous about publishing detail at all, which makes the statistics less useful to the members who do contribute.
Aggregation with the rules built in
We move table production into a small, repeatable pipeline that knows your confidentiality rules and applies them before any figure leaves the building.
- Clean member returns sit in one dataset with firm identity attached, restricted to the analysts who need it.
- Each published table is defined once: which measure, which breakdowns, which period.
- For every cell, the pipeline counts contributing firms and each firm's share, and tests them against your rules, such as a minimum number of firms or a maximum share for the largest contributor.
- Cells that fail are suppressed, and the pipeline then finds any other cells that would let a reader recover them through totals, and suppresses those as well, or merges categories if your rules prefer that.
- The analyst reviews a preview with suppressed cells marked and the reason for each, then approves.
- Approved tables are written out to Excel, CSV or your publication template, and a disclosure log records the rule results for every table and period.
| Check | What it catches | Result in the published table |
|---|---|---|
| Minimum contributors | Cells with too few firms | Cell suppressed or merged |
| Dominance | One firm providing most of a cell | Cell suppressed |
| Secondary suppression | Totals that reveal a suppressed cell | Extra cells suppressed |
| Period comparison | A cell that became sparse since last period | Analyst warned before publishing |
| Ad hoc request | One-off tables outside the usual set | Same checks, same log |
The pipeline can be written in Python or R, run on your own machine or in Azure or AWS, and your analysts can read and change it. We avoid building anything only we can maintain.
Publication day, afterwards
The analyst runs the tables, reviews the suppressions, and publishes. When a journalist or a department asks for a special breakdown, it goes through the same checks instead of being built by hand under pressure. If a member ever asks how their data is protected, you can show them the rules and the log for the table they are worried about.
Warning signs in your statistics process
- Your confidentiality rules are written down but applied by eye.
- Tables are built by copying formulas from last period's workbook.
- Ad hoc requests skip the normal checks because they are urgent.
- Nobody checks whether totals reveal a suppressed cell.
- You could not show a member which checks were applied to a given table.