Making Compliance Audits Actionable at Scale

Role

Lead Designer

Team

Sole designer · PM · Engineer

Duration

3 months

All data shown is for illustration purposes only.

Summary

A compliance team of 17 audits classifications across roughly 600,000 tables every quarter, scanning manually over several days. I designed a flow that narrows that down to only the columns worth reviewing. After the first audit using it, unassigned columns dropped by about 30% and columns with two classifications by about 40%.

What I owned

Reframed the brief from “export a file” to “narrow to the affected columns”.
Designed three directions.
Split the work into phases so the narrowing could ship before the deadline.

The stakes

One unreviewed column can set the classification of the whole application.

Diagram showing how one column's classification rolls up to the table, database and application

Who this was for

The Tech and Data Compliance (TDC) team with 17 analysts. Each analyst owns a set of domains and is accountable for the classification accuracy of every table in them.

The workflow looked something like this for each analyst.

Four-step audit workflow: filter applications, open tables, spot mismatches, fix each column

The request from the users

The users asked for a file to work on, and the product team suggested an import and export option on the UI.

The users asked for a file to work on, and product suggested an import and export option on the UI.

Comic of users asking for an Excel export of every domain, and the team offering export and upload instead

But it did not address the audit itself…

The analyst still had to scan thousands of columns to find the few mismatched columns.

So I reframed the goal.

“Get the user as close to the affected columns as possible, and let them act there.”

Three directions

The three options differ on where the analyst does the work and how much narrowing the platform does before they start.

The three options differ on where the analyst does the work, and how much narrowing the platform does before they start.

1. Requested audit report

The analyst specifies domains, sensitivity, and the purpose of the report — mismatches, unassigned columns, or a specific sensitivity review. The platform returns a pre-filtered list with only the columns matching that purpose.

2. Classification and mismatch filters.

3. Classification wizard

1. Requested audit report

The constraint and the decision

The team preferred option 3, but engineering could not build it inside the deadline, which was driven by a compliance commitment.

What shipped

This is how the Phase 1 design looked.

This is the final design, where the user can either upload or edit all the data.

Final audit design where analysts can upload or edit classifications for flagged columns

Results

~30%

Unassigned columns classified

~40%

Columns with two classifications resolved

Compliance team satisfied with the result

Positive reception of the pace and accuracy of what shipped in this phase.

Positive reception on the pace and accuracy of what shipped in this phase.

Still open

The in-UI edit flow from Phase 2 hasn’t shipped yet.

What I took from it

Pushing back on a direction works better when you bring alternatives with you. Rejecting the export-only approach was only useful because I came back with three flows and a reason for each one.
When a constraint rules out the best solution, it is worth treating it as a sequencing problem rather than dropping the solution. Engineering’s timeline meant option 3 could not ship in that release, but splitting the narrowing from the editing meant that the most useful part of it still shipped first.
We only showed the compliance team the design at the end, once it was resolved. That was late enough that they could give feedback on the design itself, but not on which of the three directions we picked, because that decision was already made.

Pushing back on a direction works better when you bring alternatives with you. Rejecting the export-only approach was only useful because I came back with three flows and a reason for each one.
When a constraint rules out the best solution, it is worth treating it as a sequencing problem rather than dropping the solution. Engineering's timeline meant option 3 could not ship in that release, but splitting the narrowing from the editing meant the most useful part of it still shipped first.
We only showed the compliance team the design at the end, once it was resolved. That was late enough that they could give feedback on the design itself, but not on which of the three directions we picked, because that decision was already made.

Other projects

Designing AI Search and Trust Signals for Millions of Data Assets

3 mins read

Explaining Automated Classification Decisions with AI

3 mins read

Nikhil Dasari

Designed in Arkansas, mentally relocated to Big Sur. Photo by George Cox on Unsplash