Unstructured Data Discovery: Finding PII in Files (2026)

Unstructured data discovery is the process of locating personal and sensitive information inside files, documents, emails, and collaboration sites where no fixed schema tells you what a column contains. It combines pattern matching, contextual analysis, and entity recognition to find where personal data lives, so that classification and protection decisions can follow from evidence rather than assumptions.

Working on data governance at the Department of Veterans Affairs taught me that the most consequential personal data often sits outside the systems anyone thinks to audit. Structured databases have owners, schemas, and access reviews. The shared drive with eleven years of scanned forms, the mailbox export someone saved to their desktop, and the project folder full of spreadsheets do not. This article is for privacy, security, and compliance teams that need to find that data, decide what it is, and reduce the exposure it creates without stalling the business.

I’ll cover why unstructured content is where PII problems hide, where personal data actually accumulates across file shares, documents, and email, and how pattern matching, context, and named entity recognition work together in a scan. From there, we’ll look at how discovery differs from classification and from data loss prevention, what to evaluate in PII scanning tools, how to triage findings into decisions, and a workflow compliance teams can repeat. The final sections address governing discovery at scale and connecting findings back to your catalog. Where a topic has a deeper treatment elsewhere on this site, I’ll point you to it rather than repeat it.

Table of contents

• Why Unstructured Data Is the Hidden PII Problem
• Where Personal Data Hides in File Shares, Documents, and Email
• How to Find PII in Documents: Patterns, Context, and Named Entities
• Data Discovery vs Data Classification and DLP
• Tools for Unstructured Data Discovery: What to Evaluate
• Triage: Turning Scan Results into Classification Decisions
• Unstructured Data Discovery Workflow for Compliance Teams
• Governing Unstructured Data Management and Governance at Scale
• Connecting Unstructured Findings to Your Catalog and Lineage
• The Bottom Line on Unstructured Data Discovery
• Frequently Asked Questions About Unstructured Data Discovery

Why Unstructured Data Is the Hidden PII Problem

Most personal data governance programs started with structured systems for good reason. A customer table has named columns, documented lineage, and an owner who can answer questions. Unstructured content does not offer any of that. A Word document may contain a Social Security number in a footnote, and nothing about the file’s name, location, or metadata will tell you so.

The volume alone makes this difficult. Organizations accumulate documents faster than they retire them, and most file stores were never designed with retention or classification in mind. When I’ve reviewed shared storage in large public and private organizations, the pattern is consistent: a small share of the content is actively used, and a large share is stale, duplicated, or orphaned. Stale content is often where personal data sits longest, because nobody is responsible for it and nobody reads it.

Regulation has made the gap harder to ignore. The General Data Protection Regulation (GDPR) requires organizations to know what personal data they hold and to demonstrate appropriate safeguards, and the California Consumer Privacy Act as amended by the California Privacy Rights Act (CCPA/CPRA) gives consumers rights to access and deletion that are very hard to honor if you cannot find their data. Those obligations do not distinguish between a row in a database and a scanned intake form. An access request covers both.

There is also a security argument that privacy teams sometimes underplay. Attackers who gain access to a file share rarely look for a specific table. They look for credentials, identity documents, financial records, and anything that can be monetized or used for further access. Unstructured stores are attractive precisely because they are under-inventoried. If the security team does not know a folder contains identity documents, the access controls on that folder are probably wrong.

In my experience, the first useful conversation with a business unit almost never starts with “we have a PII problem.” It starts with “where are our contracts, and who can open them?” Discovery gives you the factual basis for that conversation. It moves the discussion from theoretical risk to a specific folder, a specific mailbox, and a specific count of records that should not be there.

The reason this is a “hidden” problem is not secrecy. It is that unstructured data falls between the responsibilities of several teams. Records management owns retention schedules. IT owns storage. Security owns access controls. Privacy owns the policy. Data governance owns the catalog. Without discovery, none of them can see the whole picture, and each assumes another team has it covered.

Discovery closes that gap by producing an inventory of what exists, where it lives, and what it appears to contain. That inventory is not a finished classification. It is the input to classification, and the quality of everything downstream depends on how honest that input is.

Where Personal Data Hides in File Shares, Documents, and Email

Before you choose a method, you need to understand the places personal data accumulates. Each environment has different characteristics that affect how you scan it, how often, and with what expectations about accuracy.

File Shares and Network Storage

Network file shares are the oldest and often the largest source of unstructured personal data. They accumulate through decades of project work, departmental practice, and migrations that copied folders forward without review. A single share may contain HR exports, customer correspondence, medical forms, and engineering drawings with embedded contact details, all in one directory tree.

The characteristic risk here is permission sprawl. Folders inherit permissions, get broken out for specific teams, and are rarely re-reviewed. When discovery finds personal data in a share, the first question is usually not “what is this?” but “who can open it?” Those two answers together determine priority.

When scoping a file share scan, I start with the largest and most access-broad shares, not the most obviously sensitive ones. Sensitive data discovery in file shares tends to reveal the most risk in shares that nobody considers sensitive, such as a general departmental drive that quietly became the place where intake forms were saved.

Documents and Collaboration Sites

Document libraries in collaboration platforms behave differently from file shares. They are often connected to sync clients, shared with external parties, and linked into workflows. A document may move from a personal library to a team site to an external link within a few clicks, and its effective exposure changes at each step.

Versioning matters too. A contract that was clean in version five may contain a signatory’s personal phone number in version three that still exists in the version history. Scanning only the current version misses this. Depending on your platform and policy, you may need to decide whether version history is in scope for discovery, and document that decision.

Scanned documents and images are a separate category. Optical character recognition (OCR) is required to read them, and OCR accuracy varies with scan quality, language, and layout. Anything the scanner cannot read is invisible to pattern-based discovery, which is a gap you should record rather than ignore.

Email and Attachments

Email is where personal data shows up in the most informal and least controlled form. People forward records to colleagues to get something done, paste account numbers into messages to resolve a problem, and attach spreadsheets that contain far more than the recipient needs. Much of this happens inside the organization, which means email is often less exposed to outside parties than file shares, but the volume of copies is enormous.

Email discovery raises practical questions that file scans do not. Mailboxes are large, personal data is distributed across many items, and a single message may be copied to dozens of recipients. Most organizations scan email at the mailbox or tenant level and then decide whether to act on individual items, retain them, or remove them.

If you are asking how to find PII in emails, the answer begins with scope. Decide which mailboxes are in scope (shared mailboxes, departmental mailboxes, and former employees’ mailboxes are common priorities), which folders and attachments are included, and what your legal hold and retention obligations require before anyone deletes anything. Scanning is the easy part; the governance decisions about what happens next are where programs stall.

Archives, Backups, and Shadow Copies

Archives and backups deserve explicit attention because they hold personal data that is supposed to be gone. A retention policy that deletes content from production but leaves it in a backup set is a retention policy with a hole in it. Discovery in archives is slower and often limited to metadata or sampling, but identifying which archives hold personal data is itself valuable.

Shadow copies are the informal cousins of archives: local downloads, exported reports, and copies made to personal devices or removable media. These are the hardest to find through central scanning and often require endpoint tooling, policy, and user education. I treat shadow copies as a known blind spot to be managed, not a problem a single scan will solve.

Mapping the Landscape Before You Scan

A practical first step is to build a simple map of unstructured repositories: name, platform, approximate size, owner if known, access breadth, and retention status. This map does not need to be perfect. Its purpose is to let you prioritize scanning and to make visible the repositories that nobody owns. Those unowned repositories are usually the first thing an auditor asks about.

How to Find PII in Documents: Patterns, Context, and Named Entities

The technical core of unstructured data discovery is a set of detection techniques that operate on text extracted from files. Most products combine three approaches, and understanding each one helps you judge what a tool is actually doing when it reports a finding.

Pattern Matching and Validation

Pattern matching is the oldest and most predictable technique. A regular expression looks for text that has the shape of a particular identifier: a Social Security number has a three-two-four digit structure, a passport number follows a country-specific format, and a payment card number has a length and prefix pattern. Patterns are fast, transparent, and easy to explain to an auditor.

Their weakness is false positives. Any nine-digit number matches a loose Social Security number pattern, including product codes, invoice numbers, and timestamps. Good detection adds validation on top of the pattern. For payment card numbers, the Luhn checksum is a widely used algorithm that rejects many random digit strings, which substantially reduces noise. Validation logic like this is one of the first things I check when evaluating a tool, because it determines whether the output is usable without weeks of manual review.

A practical pattern library usually includes identifiers relevant to your jurisdictions and industry: national identification numbers, tax identifiers, bank account formats, health plan identifiers where applicable, and passport numbers. The list should come from your policy, not from a vendor default. If you operate in multiple countries, each jurisdiction needs its own patterns, and you should test them against representative samples before trusting the results.

Context and Proximity Rules

Context separates a phone number from a random sequence of digits. A detection rule that requires words such as “SSN,” “social security,” or “date of birth” within a set distance of a candidate value produces far fewer false positives than the pattern alone. The trade-off is recall: a document that writes the identifier without a label may be missed.

I have found that the best rules combine validation, context, and a confidence threshold. For example, a card number that passes checksum validation and appears near the words “card” or “expiry” is high confidence. A nine-digit number near no contextual terms is low confidence and should be sampled rather than acted on automatically. Writing these rules down, and tuning them against labeled samples, is the work that makes discovery defensible. It is also the work most often skipped.

Named Entity Recognition for Names, Addresses, and Free Text

Patterns cannot find a person’s name, a home address, or a medical condition described in a sentence. This is where named entity recognition (NER) comes in. NER models identify spans of text as people, organizations, locations, dates, and similar categories. They are useful for identifying names and addresses, which are personal data under most privacy regimes even though they have no fixed format.

NER has its own failure modes. Models trained on general text can mistake product names for people, miss names in unusual scripts, or mislabel place names. Accuracy depends heavily on language and domain. A model tuned on news articles may perform poorly on clinical notes or on the abbreviated style of case files. Before relying on entity recognition, test it against documents that resemble your own content, and measure precision and recall on a labeled sample.

Entity recognition also raises a conceptual issue. A name on its own may or may not be personal data in a specific context, depending on what else the document reveals. A name in a public press release is different from the same name in a case file. This is why discovery output should carry context, such as the surrounding text or the document type, and why classification decisions cannot be automated from entity labels alone.

Metadata, Structure, and Document Types

Not all discovery is text analysis. File metadata reveals authorship, creation dates, and sometimes embedded comments or tracked changes that contain personal data. Document structure matters too: a spreadsheet is not a paragraph, and a column of email addresses is different from a sentence that mentions one.

Document type detection is a useful pre-processing step. Knowing that a file is an intake form, a payroll export, or a meeting transcript changes which detection rules are likely to matter and how findings should be interpreted. Some teams maintain a short list of document types that are high-risk by definition, such as identity verification records, and treat any file matching those types as sensitive regardless of what the automated rules find.

Combining Techniques in a Single Scan

In a practical scan, these techniques run in sequence. Text is extracted, metadata is read, document types are assigned where possible, patterns are matched and validated, contextual rules are applied, and entity recognition runs on the remaining text. Each finding is recorded with its technique, confidence level, location, and a sample of the surrounding text where policy allows.

The output of this process is a set of candidate findings, not a verdict. A scan that reports ten thousand matches has produced a queue for triage, not a list of ten thousand violations. Treating the output correctly is the difference between a discovery program that informs decisions and one that generates reports nobody can act on.

Data Discovery vs Data Classification and DLP

Three terms appear constantly in these conversations and are often used interchangeably. They are related but distinct, and confusing them leads to tools being bought for the wrong job and teams working at cross-purposes.

Discovery Finds, Classification Labels

Data discovery answers the question “what do we have, and where is it?” It produces an inventory of data assets and their characteristics, including the presence of personal data. Discovery is observational. It describes what exists without necessarily deciding what to do about it.

Data classification answers the question “how sensitive is this, and how should it be handled?” It assigns a category, such as public, internal, confidential, or restricted, based on policy. Classification is prescriptive. It connects a finding to a handling rule.

The distinction matters for sequencing. You cannot classify what you have not discovered, and discovery without classification leaves you with a list of personal data locations and no agreed treatment for them. In practice, the two processes feed each other: discovery findings inform the classification scheme, and the scheme determines which findings matter most.

Here is how the three disciplines compare in practice:

What DLP Does Differently

Data loss prevention is the term that causes the most confusion, because DLP tools also inspect content for sensitive patterns. The difference is in timing and purpose. DLP monitors data in motion or at a boundary: an email being sent externally, a file being uploaded to a personal cloud service, a document being printed. Its job is to stop or flag a specific action according to a policy.

Discovery looks at data at rest across stores that already exist. It answers retrospective questions about what has accumulated. DLP answers prospective questions about what is about to leave. Both use content inspection, which is why the confusion persists, but the decisions they support are different.

If you ask whether DLP vs data discovery is an either-or choice, the answer is no. A mature program uses discovery to understand the estate and DLP to enforce boundaries. Discovery tells you that a folder contains customer identifiers and should be restricted. DLP tells you that someone has just tried to email that folder’s contents outside the company. Without discovery, DLP policies are written against guesses. Without DLP, discovery findings can sit in a report while the same data continues to leave the building.

Microsoft Purview is one example of a platform that spans several of these capabilities, and I recommend reading what Microsoft Purview does to understand how its information protection and data governance components relate. Whatever platform you use, check whether discovery, classification, and DLP are integrated or separate products that must be reconciled manually.

Where Discovery Ends and Governance Begins

Discovery is a technical function, but its outputs have governance consequences. Someone must decide what a finding means, who owns the repository, what the retention requirement is, and whether the data should be moved, restricted, or deleted. Those decisions belong to governance, guided by policy.

This is also where the broader concept of discovery connects to analytics. Data discovery in analytics is about helping people find and understand datasets for use. Unstructured data discovery is about finding sensitive content for protection. The two share vocabulary but serve different audiences. For a fuller comparison of the analytics meaning, see what data discovery means in analytics.

Tools for Unstructured Data Discovery: What to Evaluate

Vendors in this space include discovery features within broader data security, privacy, and governance platforms, as well as specialist tools focused on file and content scanning. I will not rank products here, because the market changes quickly and I cannot verify current feature sets, licensing, or pricing from memory. Instead, I’ll describe what to evaluate, and you should confirm every product-specific detail with the vendor and against a proof of concept.

Anyone searching for PII scanning tools or PII discovery tools is usually trying to shortlist a few options. The evaluation criteria below will help you narrow the list without relying on marketing claims.

Testing With Your Own Content

The most useful evaluation I have run is a bake-off on a representative sample of our own files, with a labeled answer key prepared by people who knew the content. Each tool scans the same set. Results are compared for precision, recall, and the time required to review the output. This exercise exposes differences that no demonstration will show, such as how a tool handles scanned PDFs, mixed-language documents, or unusual identifier formats.

Build the answer key carefully. A small team with domain knowledge can label a few hundred documents in a few days, and that effort pays for itself many times over. Include deliberately tricky cases: near-miss numbers, names that are also place names, and documents that mention personal data in the abstract without containing any.

Questions to Put to Vendors

Ask whether detection logic is transparent enough for an auditor to understand, and whether you can export the rules. Ask how the product handles versioned content, encrypted files, and password-protected archives. Ask what happens to scan results, where they are stored, and who can access them. These questions matter because the discovery output is itself personal data in many cases. A scan that copies sensitive snippets into a central reporting database has created a new store that needs protection.

Ask about licensing in terms of what you will actually scan, not the headline list price. Unit definitions vary across vendors and can change scope in ways that surprise budget owners. Confirm current licensing terms in writing. [VERIFY: current licensing models for each shortlisted product].

Build Versus Buy and Hybrid Approaches

Some organizations build their own discovery pipelines using open-source text extraction, regular expression libraries, and general-purpose NER models. This can work well for a narrow scope with strong engineering capacity. It becomes expensive when you need connectors to many platforms, audit trails, and ongoing model maintenance.

Many teams land on a hybrid: a commercial scanner for broad coverage and a small set of custom rules for identifiers specific to their jurisdiction or business. If you go this route, keep the custom rules under version control and test them against the same labeled sample used for vendor evaluation. Custom detection is only as good as its testing.

If you are still deciding how discovery fits into a wider governance platform, the platform selector can help you think through the trade-offs before you commit to a product category.

Triage: Turning Scan Results into Classification Decisions

A scan produces findings. Triage converts those findings into decisions. This is the least glamorous part of unstructured data discovery and the part most likely to determine whether the program succeeds. Without a triage process, you accumulate a backlog that grows faster than anyone can clear it, and the organization learns to ignore the reports.

Start With Risk, Not Volume

The natural instinct is to work through findings in order of count. That is usually wrong. Prioritize by a combination of data sensitivity, access breadth, and retention status. A single file containing identity documents, open to all staff, outranks ten thousand low-risk marketing contacts behind a restricted permission set.

I use a simple scoring model in practice: the type of personal data found, the number of individuals likely affected, who can access the location, whether the location is shared externally, and how long the content has been there. The model does not need to be sophisticated. It needs to be consistent and written down, so that two triage analysts reach similar conclusions on similar findings.

Sampling Before Acting

Before any finding triggers action, verify a sample. Automated detection is probabilistic, and acting on unverified results causes two problems: legitimate content gets restricted or deleted, and trust in the program erodes when business users are wrongly accused of mishandling data. Sample findings at each confidence level, measure how many are true positives, and adjust rules accordingly.

Sampling also reveals patterns. A cluster of false positives from one document template suggests a rule that needs refinement. A cluster of true positives in one folder suggests a process problem worth addressing directly with the team that owns it.

Decision Categories

Every finding should end in a decision from a small, defined set. A typical set includes: retain and classify, restrict access, move to a controlled location, apply retention and delete, redact, or accept as a documented exception. Each decision needs an owner and a record. Accepting a risk should be a deliberate act with a named approver and a review date, not a default.

The classification itself should follow the policy. If your organization uses a data classification policy template or has written its own, the triage decisions should map directly to its categories and handling rules. Discovery that produces findings without a policy to classify against will generate endless debate about labels.

Working With Data Owners

Triage fails when it happens entirely within the governance or security team. Data owners know what their content is for, whether a record must be kept, and what the business impact of restricting access would be. Build a routine for sending findings to owners in a form they can act on: the location, the type of personal data, a short sample, a recommended action, and a deadline.

Give owners a clear way to dispute findings. In my experience, owners are quick to identify false positives in their own areas and slow to act on true positives they consider inconvenient. The dispute process protects both accuracy and cooperation, because owners who see their feedback improve the rules are more willing to act on the results that remain.

Documenting Decisions for Auditors

Every triage decision should leave a trail: what was found, who reviewed it, what was decided, and why. Auditors and regulators will ask how you knew about personal data and what you did about it. A well-kept decision log answers those questions without a scramble. Keep the log in a system with access controls, because it will reference sensitive locations and sometimes sample content.

Unstructured Data Discovery Workflow for Compliance Teams

A repeatable workflow turns discovery from a one-time project into a standing capability. The steps below describe a sequence I have seen work across regulated environments. The exact tools and owners will vary, but the order and the checkpoints should hold.

1. Define scope and purpose. Document which repositories are in scope, what personal data categories matter for your jurisdictions, and what the discovery is for, whether that is an access request response, a risk assessment, a retention review, or a regulatory inquiry. Scope drives every later decision, and a scan without a stated purpose tends to produce results nobody uses.

2. Inventory and prioritize repositories. Build the repository map described earlier and rank sources by risk and by how likely they are to contain personal data. Flag unowned repositories for immediate attention. Agree on who will own the outcome for each one before scanning begins.

3. Establish detection rules and a test set. Write or select the identifier patterns, context rules, and entity types that apply to your scope. Build a labeled sample from your own content and measure current detection performance. Do not skip this step, even under time pressure, because untested rules produce confident-looking but unreliable results.

4. Run a pilot scan on a bounded area. Choose one repository, ideally one with known content, and run the full process end to end. Review the results with the data owner, adjust rules, and document what worked. The pilot is where most workflow problems surface, and it is far cheaper to find them there than across the whole estate.

5. Scale the scan with appropriate controls. Schedule full and incremental scans, confirm that scan activity does not degrade production systems, and ensure scan results are stored with access controls matching the sensitivity of their contents. Record the scan configuration so you can reproduce results later.

6. Triage and assign decisions. Apply the risk-based prioritization and sampling described earlier. Route findings to owners with clear actions and deadlines. Track decisions in a log that can support audit and legal hold requirements.

7. Remediate and verify. Carry out the agreed actions, whether that means restricting permissions, moving content, applying retention, or deleting it where the retention schedule and any legal holds allow. Then verify that the change took effect by rescanning the affected location. Verification is the step most often skipped, and without it you cannot demonstrate that the risk was reduced.

8. Feed results back into classification and the catalog. Update the classification of affected repositories and record the discovery results as metadata where your catalog supports it. This connects the unstructured findings to the rest of your governance inventory, which I return to below.

9. Report and review. Summarize outcomes for the privacy, security, and governance committees. Include the number of repositories scanned, the proportion of findings that were true positives, decisions taken, and open exceptions. Review the rules and the workflow on a set schedule, not only after an incident.

Roles and Responsibilities

Compliance teams often assume discovery is a tool problem and discover that it is a role problem. At minimum you need someone who owns the program, someone who maintains detection rules and tests them, analysts who perform triage, data owners who make decisions about their content, and IT or security staff who execute permission and deletion changes. Map these roles explicitly. A RACI matrix is a practical way to do this, and the data governance RACI matrix template can serve as a starting point.

Handling Legal Holds and Subject Access Requests

Discovery intersects with legal holds and subject access requests in ways that require care. A legal hold may prohibit deletion of content that discovery flags for removal. A subject access request may require you to find every instance of an individual’s data, which means your detection rules need to support searches by identifier as well as broad scanning. Confirm with legal counsel how these obligations interact with your workflow before a live request arrives, and test the search capability in advance.

Measuring Whether the Workflow Works

Choose a small set of measures and track them consistently. Useful examples include the proportion of in-scope repositories scanned in the last cycle, the share of high-risk findings with a recorded decision, the average time from finding to remediation, the false positive rate from sampling, and the number of repositories with no owner. Trend lines matter more than single values. A program that is getting faster and more accurate is working, even if the absolute numbers look poor at the start.

Governing Unstructured Data Management and Governance at Scale

A pilot proves the method. Scale requires governance: standards, ownership, funding, and a way to keep discovery current as the estate changes. This is where unstructured data governance becomes an operational discipline rather than a project, and where unstructured data management stops being a set of ad hoc cleanups and becomes a lifecycle.

Policy Foundations

Discovery needs a policy to be measured against. The policy should define what counts as personal data for your organization, which categories are restricted, how long each category may be retained, where it may be stored, and who may access it. Without that foundation, every finding becomes a debate. The DAMA-DMBOK (the Data Management Body of Knowledge published by DAMA International) organizes data governance into knowledge areas, including data governance, metadata management, and data security, and treating unstructured data as part of each of those areas keeps the scope from fragmenting. [VERIFY: current DAMA-DMBOK edition reference before citing in publication].

The policy also needs to say who decides when discovery conflicts with business need. A rule that says “personal data must not be stored on shared drives” will be broken the day a team needs one, unless there is a documented exception path. Build the exception path into the policy rather than leaving it to informal negotiation.

Ownership of Unstructured Repositories

Every repository in scope needs an accountable owner. This sounds obvious and is rarely true in practice. Many file shares were created by people who have since left, and their contents now belong to nobody. Assigning ownership is often the most valuable outcome of an early discovery program, because an owner is the person who can decide whether content should exist at all.

Ownership should be recorded in a system that outlives individuals, such as the data catalog or a configuration management database, and reviewed on a regular cycle. A repository with no owner after the review should be escalated to the business unit that benefits from it, or scheduled for archival.

Keeping Discovery Current

Unstructured estates change constantly. New content arrives every day, permissions shift, and teams reorganize. A one-time scan is out of date within weeks. Plan for recurring discovery at a frequency matched to the risk of each repository. High-risk, high-churn locations may warrant frequent incremental scans, while archival stores may need only periodic review.

Incremental scanning, where only changed or new content is evaluated, is essential at scale. Confirm that your chosen tooling supports it and that it catches changes to permissions as well as to content, because a file that becomes accessible to a wider audience is a change in risk even if its text is unchanged.

Retention and Deletion at Scale

Discovery is only valuable if it leads to retention and deletion where the policy requires it. Many organizations find more personal data than they are permitted to keep, and the remediation backlog is substantial. Prioritize deletion of content with no business purpose and no legal hold, and make sure deletion is recorded so that you can demonstrate compliance.

Be careful with automated deletion. Rules that delete based on a single detection can destroy records that must be kept. I recommend a two-stage approach: automated quarantine or restriction first, followed by owner confirmation and then deletion after a defined waiting period. Quarantine gives the business a way to recover content that was wrongly flagged.

Training and Culture

Technology reduces the number of places personal data accumulates, but people still create it. Staff need to understand what counts as personal data, where it should not be stored, and what to do when they find it in the wrong place. Short, role-specific guidance works better than a single annual module. Recognize teams that clean up their own repositories, because compliance that is only enforced through findings creates resentment rather than habits.

Reporting to Leadership

Leadership cares about exposure, trend, and accountability. Report the number of high-risk locations, the time to remediate, and the status of repositories with no owner. Avoid presenting raw finding counts as a measure of success; a rising count can mean the scanner is finding more, not that the organization is worse. Explain trends in context and tie them to decisions leadership can make, such as funding for remediation or a policy change for a particular data type.

Connecting Unstructured Findings to Your Catalog and Lineage

Unstructured discovery produces its best value when its results join the rest of your governance metadata. A finding that says “this folder contains identity documents” is useful on its own. The same finding connected to the business process that generated the documents, the systems that consume them, and the policy that governs them is far more actionable.

Recording Findings as Metadata

Most data catalogs can hold information about data assets, including sensitivity labels, owners, and classifications. Unstructured repositories can be cataloged in the same way, with discovery results recorded as properties. The benefit is that analysts searching the catalog for personal data sources will find unstructured locations alongside structured ones, rather than discovering them years later in a separate report.

For a grounding in what a catalog does and how its metadata is structured, read What Is a Data Catalog? The Complete Guide for 2026. The key design decision is granularity: cataloging every file is rarely practical, but cataloging repositories, folders, and document types with aggregated discovery results usually is.

Tracing Where Personal Data Flows

Discovery tells you where personal data sits. Lineage tells you how it got there and where it goes next. When a discovered file share turns out to be a staging area for a reporting process, the lineage makes that visible and tells you which downstream systems may hold copies. Understanding these flows is essential for responding to access requests and deletion obligations, because you need to know every location where a person’s data resides.

If you are new to tracing these relationships, data lineage fundamentals covers the concepts in more depth. Combining lineage with discovery is one of the more productive ways to close the gap between what is stored and what is actually used.

Profiling and Its Limits

Data profiling examines structured datasets for patterns, distributions, and quality issues. It does not work directly on documents, but the techniques transfer conceptually: understanding the distribution of identifier types across a repository, or the proportion of files that contain personal data by document type, gives you a statistical view that complements individual findings. For how profiling works on structured data and where it fits in a governance program, see data profiling techniques.

Be honest about the difference, though. Profiling a table can tell you that a column has a high null rate. Profiling a file share tells you that a sample of documents contains identifiers at a certain rate, with a confidence interval that depends on your sampling method. Present such numbers as estimates tied to the sample, not as facts about the whole estate.

Keeping the Catalog Honest

A catalog that records discovery results must be updated when those results change. Stale metadata that says a repository is clean after its contents have changed is worse than no metadata at all, because it creates false assurance. Tie catalog updates to the scan cycle and make the last-scanned date visible to anyone reading the record.

The Bottom Line on Unstructured Data Discovery

Unstructured data discovery is not a single tool or a one-time cleanup. It is a capability that lets you see the personal data your organization holds outside its structured systems, decide what that data is, and act on the decision. The organizations I have seen do this well treat discovery as the start of a governance cycle rather than a report to file away.

What matters most in my experience is discipline in three places. First, define your purpose and scope before scanning, because a scan without a question produces findings without decisions. Second, test your detection rules against your own content, because vendor accuracy claims rarely hold for your formats, languages, and document habits. Third, make triage and ownership routine, because a backlog of findings with no owner is the most common way these programs fail. Tools will keep changing. The need to know where personal data lives, and to be able to show an auditor that you know, will not.

If you are building this capability now, start with a bounded pilot in the repository most likely to surprise you, assign an owner to every location you scan, and measure how long it takes to move a finding from detection to decision. Those three numbers will tell you more about your readiness than any feature comparison.

Frequently Asked Questions About Unstructured Data Discovery

What is unstructured data discovery?

Unstructured data discovery is the process of finding personal and sensitive information inside files, documents, emails, and collaboration sites that lack a fixed schema. It uses pattern matching, contextual rules, and named entity recognition to locate content, producing an inventory that informs classification and protection decisions. The output is a set of findings for triage, not a final verdict.

How do I find PII in documents?

To find PII in documents, extract the text (using OCR for scanned files), then apply identifier patterns validated with checks such as the Luhn algorithm for card numbers. Add contextual rules and named entity recognition for names and addresses. Measure accuracy on a labeled sample from your own content, and sample results before acting on them.

How do I find PII in emails?

To find PII in emails, first define the scope: which mailboxes, folders, and attachments are included, and what legal hold and retention rules apply. Then run pattern and entity scanning at the mailbox or tenant level. Triage findings by sensitivity and access breadth, and confirm any deletion against your retention policy before acting.

What is the difference between data discovery and data classification?

Data discovery identifies what data exists and where it lives, producing an inventory. Data classification assigns sensitivity categories, such as confidential or restricted, and links each category to handling rules. Discovery is observational, while classification is prescriptive. You generally discover first, then classify against a written policy.

What is the difference between DLP and data discovery?

Data loss prevention (DLP) monitors data in motion and enforces policies on actions such as sending or uploading sensitive content. Data discovery examines data at rest to show what has already accumulated. Both inspect content, but DLP is preventive and real-time, while discovery is retrospective. Mature programs use discovery to inform the DLP policies they enforce.

What are the best PII scanning tools?

There is no universal best option, because fit depends on your repositories, identifier formats, languages, and governance platform. Evaluate candidates on source coverage, measured accuracy against your labeled sample, tuning controls, scan performance, and integration with your catalog. Confirm current features and licensing directly with each vendor before shortlisting.

How does sensitive data discovery in file shares work?

Sensitive data discovery in file shares connects to the share, reads files, extracts text and metadata, and applies detection rules to each file. Incremental scans then check only new or changed content. Teams prioritize broadly accessible shares and unowned folders first, and they track permissions as well as content, since exposure changes when access widens.

Is unstructured data classification different from structured data classification?

Yes. Structured classification can often rely on column names, data types, and table-level ownership. Unstructured classification must infer sensitivity from the content of each file, which makes it probabilistic and dependent on detection accuracy. Unstructured classification also depends more on document context and access patterns, so it typically needs sampling and human review to remain reliable.

How often should unstructured data scans run?

Run scans at a frequency matched to each repository’s risk and churn. High-risk, frequently changing locations may need incremental scans on a short cycle, while archival stores may need periodic review. Set the schedule in your governance policy, record it in the catalog, and rescan after any remediation to verify the change took effect.