Classification mismatch and unassigned classifications in compliance audits

Classification mismatch and unassigned classifications in compliance audits

Classification mismatch and unassigned classifications in compliance audits

Role

Role

Lead Designer

Lead Designer

Team

Team

PM, Engineer and Designer

PM, Engineer and Designer

Duration

Duration

2 weeks for design

2 weeks for design

Summary

The compliance team’s monthly audit meant manually scanning thousands of columns to find the few with missing or conflicting classifications. I designed a flow that narrows the list down to only the columns needing attention before the analyst sees it. In the first two to three months, about 30% of unassigned columns were classified and about 40% of conflicts were resolved.
The compliance team’s monthly audit meant manually scanning thousands of columns to find the few with missing or conflicting classifications. I designed a flow that narrows the list down to only the columns needing attention before the analyst sees it. In the first two to three months, about 30% of unassigned columns were classified and about 40% of conflicts were resolved.

The stakes

Every column in the enterprise data catalog carries a classification that determines who can access it and how it must be protected.
Compliance runs a monthly audit to catch such issues, and the audit was mostly manual.
Every column in the enterprise data catalog carries a classification that determines who can access it and how it must be protected.
Compliance runs a monthly audit to catch such issues, and the audit was mostly manual.

Two classification systems on one column

Classifications reached a column through two independent paths. One was the legacy classification system and the other was the updated enterprise classification process. When metadata was harvested, some columns came through with both. Both were valid sources, and neither had authority over the other.
So a column could hold two classifications at the same time, and the compliance team had to look at most columns manually to sort them out.
Classifications reached a column through two independent paths. One was the legacy classification system and the other was the updated enterprise classification process. When metadata was harvested, some columns came through with both. Both were valid sources, and neither had authority over the other.
So a column could hold two classifications at the same time, and the compliance team had to look at most columns manually to sort them out.

Who this was for

The Tech and Data Compliance (TDC) team with 17 analysts. Each analyst owns a set of domains and is accountable for the classification accuracy of every table in them.

The workflow looked something like this for each analyst.

What the users asked for

The compliance team came to us asking for a spreadsheet. They wanted every column for their domains exported to Excel, so they could mark it up and send it back for us to upload.

There were two problems…

The back-and-forth

The back-and-forth put the engineering and product teams into a monthly compliance process they had no reason to be in.

Sensitive data out there

It also put sensitive classification data into email attachments. The compliance team acknowledged this, but they wanted something built quickly so they could start fixing classifications.

The direction I was given

Product's direction was to give users a self-serve export of the full table list, let them work in Excel, and let them upload it back.

But it did not address the audit itself…

The analyst still had to scan thousands of columns to find the few mismatched columns.
The analyst still had to scan thousands of columns to find the few mismatched columns.

So I reframed the goal.

“Get the user as close to the affected columns as possible, and let them act there.”
“Get the user as close to the affected columns as possible, and let them act there.”

Three directions

The three options differ on where the analyst does the work, and how much narrowing the platform does before they start.
The three options differ on where the analyst does the work, and how much narrowing the platform does before they start.

1. Requested audit report

The analyst specifies domains, sensitivity, and the purpose of the report — mismatches, unassigned columns, or a specific sensitivity review. The platform returns a pre-filtered list with only the columns matching that purpose.

The analyst specifies domains, sensitivity, and the purpose of the report — mismatches, unassigned columns, or a specific sensitivity review. The platform returns a pre-filtered list with only the columns matching that purpose.

2. Classification and mismatch filters.

3. Classification wizard

1. Requested audit report

The analyst specifies domains, sensitivity, and the purpose of the report — mismatches, unassigned columns, or a specific sensitivity review. The platform returns a pre-filtered list with only the columns matching that purpose.

The constraint and the decision

The team preferred option 3, but engineering could not build it inside the deadline, which was driven by a compliance commitment.

What shipped

This is how the Phase 1 design looked.

This is the final design, where the user can either upload or edit all the data.

Early Results

What I took from it


  1. Pushing back on a direction works better when you bring alternatives with you. Rejecting the export-only approach was only useful because I came back with three flows and a reason for each one.
  2. When a constraint rules out the best solution, it is worth treating it as a sequencing problem rather than dropping the solution. Engineering's timeline meant option 3 could not ship in that release, but splitting the narrowing from the editing meant the most useful part of it still shipped first.
  3. We only showed the compliance team the design at the end, once it was resolved. That was late enough that they could give feedback on the design itself, but not on which of the three directions we picked, because that decision was already made.

Other projects

Other projects

Other projects