Types of Data Tokenization: Methods, Examples and How to Choose the Right One

Data tokenization comes in several types, from vault-based and vaultless to random, deterministic and format-preserving tokens. Each one changes whether you can restore values, match records or keep legacy systems running.

The main types of data tokenization are:

1. Vault-based tokenization: Stores the encrypted original in a secure vault and gives your systems a token that maps back to it.
2. Vaultless tokenization: Computes the token from the value with a secret key, so no vault or lookup table exists.
3. Random tokenization: Creates a new, unrelated token every time, even for the same value.
4. Deterministic tokenization: Returns the same token whenever the same value appears, so records still match.
5. Format-preserving tokenization: Keeps the length and character pattern of the original, so legacy systems accept the token.
6. Reversible and irreversible tokenization: Decides whether the original can come back through a vault or key, or never at all.
7. Single-use, multi-use and expiring tokens: Control how many times a token works and how long it lives.

What Is Data Tokenization and Why Does the Type You Choose Matter?

Data tokenization replaces sensitive values with non-sensitive tokens. The type you choose decides whether those tokens can be restored, matched across systems, accepted by legacy software or retired after a set time.

Picture two organizations that both tokenize SSNs. One is a payroll provider that must restore each SSN for tax filings and link employee records across HR and benefits systems. The other is a research sponsor that wants participant identities kept away from analysts for good. Both are tokenizing, yet they need very different tokens.

Every token type answers one of four design questions:

1. Storage: Where does the original value live, in a vault or nowhere at all?
2. Matching: Should the same value always produce the same token?
3. Format: Does the token need to keep the shape of the original?
4. Reversibility and lifespan: Can the original ever come back, and for how long should the token work?

Most real deployments combine answers from all four, which is why it helps to understand each type before you choose.

Types of Data Tokenization at a Glance: How Each Method Compares

The main types of data tokenization differ in how tokens are generated, stored, matched and retired. Vault-based, random and format-preserving tokens suit payments, while deterministic tokens suit analytics and record matching.

Vault vs Vaultless Tokenization: Comparing the Two Core Token Architectures

Vault-based tokenization stores the encrypted original in a secure vault and maps each token to it, while vaultless tokenization computes tokens from the value with a secret key, so no lookup store exists at all.

This is the first decision most teams make, because it shapes everything that follows: how tokens are generated, how quickly they return, how you delete data and what an attacker would need to reverse them.

How Vault-Based Tokenization Works

1. A form, API call or phone keypad captures the sensitive value.
2. The vault encrypts the value and stores it.
3. Your application receives a token and saves it in place of the original.
4. An approved process sends the token back when it needs the real value, and the vault returns it or forwards it to a trusted endpoint such as a payment gateway.
5. Deleting the vault record retires the value everywhere at once.

How Vaultless Tokenization Works

Vaultless systems skip the lookup store entirely. A secret key and a format-preserving encryption algorithm, such as the FF1 mode described in NIST SP 800-38G, turn each value into a token of the same length and character set. Reversing the token means running the same algorithm backward with the same key. Because nothing is stored, the system scales easily, yet the key becomes the single thing protecting every token.

Choose vault-based tokenization when:

• You must restore values for specific people or processes and log every restore.
• Your retention or erasure rules require deleting one person’s data on request.
• Card numbers, SSNs or member IDs need random tokens that reveal nothing even if someone learns how the system works.
• Different teams need different restore rights over the same data.

Choose vaultless tokenization when:

• Your pipeline handles very high volumes where a lookup per record adds noticeable latency.
• Data rarely needs restoring, and per-record deletion isn’t a requirement.
• Strong key management, including rotation and tight access to the key, is something your team can run.

Some organizations run both: a vault for regulated identifiers people act on, and vaultless tokens for analytics streams that never return to plaintext. For a deeper look at latency, key risk and erasure trade-offs, read our full comparison of vault vs vaultless tokenization.

Random vs Deterministic Tokenization: Choosing Between Privacy and Matching

Random tokenization gives the same value a different token every time, which keeps records unlinkable, whereas deterministic tokenization always returns the same token, so you can join, search and deduplicate safely.

Here is what each approach does when the same SSN arrives twice:

Random tokenization
  Request 1:  SSN 523-41-8890   →  tkn_Qz81PmW0aLr2
  Request 2:  SSN 523-41-8890   →  tkn_7HvNc3KsYe5d

Deterministic tokenization
  Request 1:  SSN 523-41-8890   →  tkn_Lm4XbR9wTq07
  Request 2:  SSN 523-41-8890   →  tkn_Lm4XbR9wTq07

Random tokens suit a law firm representing the same client in two separate matters. Nobody comparing the case files can tell they belong to one person, which keeps the matters cleanly apart.

Deterministic tokens suit a health insurer whose member IDs are copied across claims, eligibility and member service systems. With one consistent token per member, analysts can follow claims history, catch duplicate submissions and run exact-match searches without anyone seeing the real ID.

Deterministic tokens carry one risk worth planning for. On fields with few possible values, such as gender or state, identical tokens repeat often enough to reveal the underlying values through frequency analysis. Reserve deterministic tokens for high-variety identifiers, and scope them by tenant or purpose so a single token doesn’t become a universal identifier across your business.

Format-Preserving Tokenization: Keeping Legacy Systems Working With Tokens

Format-preserving tokenization creates tokens that keep the length and character pattern of the original value. Legacy systems that validate card numbers, SSNs or phone numbers accept the token without any code changes.

Many older systems check every field before saving it. A property management system might reject anything that isn’t a 16-digit card number, and a payroll tool might expect exactly nine digits in the SSN field. Random tokens fail those checks. Format-preserving tokens pass them.

Hotels feel this most at check-in, where the property management system validates card numbers before saving a reservation. Format-preserving tokens let front desk staff see the card brand and last four digits while the system stores only tokens, which is why format support ranks high on most shortlists of data tokenization software for the hotel industry.

There’s one rule to respect with payment data. The PCI SSC Tokenization Product Security Guidelines state that a tokenization product should include a way to distinguish tokens from real card numbers, for example by issuing tokens from a specific BIN range or labeling them. A token that looks exactly like a live card, with nothing to tell them apart, creates confusion during audits.

Reversible vs Irreversible Tokens and How the PCI SSC Classifies Them

The PCI SSC splits tokens into reversible and irreversible types. Reversible tokens can become the original value again through a vault or key, while irreversible tokens can never be converted back to the original.

The same PCI SSC guidelines give the payment industry’s most formal way to classify tokens. Although they focus on card numbers, the four categories apply to any sensitive field.

Reversible Cryptographic Tokens

These tokens are generated from the original value using strong cryptography. The original isn’t stored anywhere; only the key is. Vaultless tokenization built on format-preserving encryption falls into this category, so the key deserves the same protection as the data itself.

Reversible Non-Cryptographic Tokens

• Relationship to the value: None. The token has no mathematical link to the original.
• Recovery path: A lookup in a secure vault, available only to approved requests.
• Typical platforms: Most vault-based tokenization services, including card vaults and PII vaults.

Irreversible Authenticatable Tokens

A one-way function, such as a keyed hash, produces these tokens. You can’t recover the original, yet you can confirm whether a given value matches. A contact center verifying callers might use one like this:

1. Callers provide their card number to verify an account.
2. Your system runs the number through the same one-way function.
3. Matching results confirm the caller’s identity.
4. Nothing about the full card number is stored for this check.

Irreversible Non-Authenticatable Tokens

These tokens can’t be traced to a specific value at all, although they may still be linked to an account or customer. The PCI SSC describes them as carrying little to no risk of exposing the original, which makes them useful for reporting datasets where nobody will ever need the value again.

Reversible tokens, in any form, remain pseudonymised data under Article 4(5) of the GDPR, so they are still PII for anyone who can restore them.

Single-Use, Multi-Use and Expiring Tokens: How Token Lifespan Changes Risk

Single-use tokens work for one transaction, multi-use tokens stay valid across many, and expiring tokens delete themselves after a set time. The shorter a token lives, the less damage it can do if it ever leaks.

A single-use token works once and then stops, a multi-use token stays tied to the same record for repeat transactions, and an expiring token disappears from the vault on a set date. Most businesses need all three, often within the same customer relationship.

Follow one private aviation charter from booking to wheels-up:

1. Booking: The client’s card is tokenized at capture and kept as a multi-use token, because the operator charges a deposit now and the balance after the flight.
2. Manifest: The passport number is stored as a token, and dispatch restores it once to file the international manifest.
3. Ground transport: The concierge team shares the client’s name and phone number with a car service through a one-time key that self-destructs after the driver opens it.
4. After the trip: The passport record expires on the date the operator’s retention policy sets, while the card token stays active for the client’s next charter.

Each step uses the shortest lifespan the task allows. That habit limits what a leaked token can unlock, and it keeps the vault from holding data longer than your business needs.

Batch vs Real-Time Tokenization: Why the Moment You Tokenize Matters

Real-time tokenization replaces sensitive values the moment they are captured, before any system stores them, while batch tokenization converts large volumes of existing records in bulk during migrations or cleanups.

Real-Time Tokenization at Capture

Tokenizing at capture means the sensitive value travels straight from the form, API call or phone keypad into the vault, and your systems receive a token from the first moment.

Collections agencies show why timing matters. When collectors take card or bank details on recorded calls, plaintext lands in the audio, on the agent’s screen and in CRM notes. Routing payment entry through a secure IVR flow tokenizes the card before any of those systems see it, and the resulting token supports the consumer’s payment plan next month.

Batch Tokenization for Existing Data

Real-time capture protects new data, yet most organizations also carry years of plaintext in older systems. A staffing firm whose recruiters have copied workers’ tax IDs and bank details into onboarding spreadsheets might clean up like this:

1. Export the spreadsheets and implementation trackers that hold SSNs and bank accounts.
2. Send the values to the tokenization service in large batches.
3. Write the returned tokens back into each tracker.
4. Confirm record counts match, then delete the plaintext exports.

From that point, real-time capture handles every new hire, and the trackers keep their status columns with tokens in place of the sensitive values.

Data Tokenization Types by the Kind of Sensitive Data You Protect

Different kinds of sensitive data call for different token types. Payment cards often use network or format-preserving tokens, PII and PHI fields use vault tokens, and AI prompts rely on deterministic tokens.

Payment cards deserve a closer look because two token systems often run side by side. Merchant tokens from a card vault replace stored card numbers for card on file and recurring billing. Network tokens come from the card networks under the EMVCo framework, keep the card’s format and add a one-time cryptogram to every transaction. Tokens that can start a payment on their own are often called high-value tokens, and systems that hold them may stay in PCI scope. Buyers evaluating payment tokenization software should check which of these a vendor supports.

AI prompts are the newest category. Deterministic tokens keep each person and account distinct for the model, so a reply can address the right customer once the application restores real values, the approach behind PII tokenization for AI.

Data Tokenization Use Cases: Which Token Types Fit Each Industry Workflow

Travel agencies, hotels, clinical research teams, health insurers and managed service providers all tokenize sensitive data, yet each one leans on a different mix of token types to match how its teams use that data.

These scenarios come from the workflows where sensitive data most often leaks: inboxes, supplier portals, spreadsheets, tickets and shared drives.

Travel Agencies Paying Airline, Hotel and Cruise Suppliers

Cards reach agents by phone, email and chat, then get re-entered into airline, hotel, cruise and tour portals that the agency can’t replace. A vault keeps that workflow intact. The traveler enters the card once through a secure intake link, the booking record stores a multi-use token, and the agent either reveals the card in an approved, logged session or the vault injects it into an authorized supplier payment through a proxy. Repeat travelers never have to resend a card, and the CRM never holds one. Agencies weighing vendors for this can review our picks for data tokenization software for travel agencies.

Hotels Handling Third-Party Card Authorizations and Group Deposits

• Exposure today: Corporate travel teams, assistants and event planners send card authorization PDFs and card images by email.
• Token approach: A branded intake link tokenizes the card as it’s entered, captures the cardholder’s signature and expires after a set time.
• Group sales: Event deposits link the sales record to card tokens, so charges run through an approved gateway path and sales tools stay out of card handling.
• Result: Every authorization becomes a traceable record in place of an attachment sitting in a shared inbox.

Clinical Research Teams Separating Identity From Study Data

Research workflows need identity and study data kept apart, with re-identification limited to a few authorized people:

1. Direct identifiers are tokenized during participant pre-screening, before broader review.
2. The identity key stays in the vault, controlled by a small group.
3. Each study works with its own study-specific tokens, so data from different studies can’t be linked by token alone.
4. Analysts work on coded data, while re-identification authority stays separate from routine analysis.

Health Insurers and TPAs Tracking Members Across Systems

Managed Service Providers Storing Client Credentials

Managed service providers and outsourcing firms handle client passwords and API keys every day, and those secrets tend to land in tickets and team chat. Storing the secret in the vault and the token in the ticket changes the picture:

Ticket #48213  |  Reset SFTP access for client Northwind
Credential:    tkn_V8rJ2nQxL5aW   (restore requires on-call engineer role)

Staff see a reference and finish the task, only the approved role retrieves the credential, and each retrieval is logged with who asked and when. Pairing random tokens with short expiry means a token copied into the wrong channel stops working quickly.

How to Choose the Right Type of Data Tokenization for Each Workflow

Choose a token type by working backward from the workflow. Ask whether you need the value back, whether records must match, whether legacy systems check formats and how long the token should live, then combine the answers.

Run each data flow through these questions and combine the answers:

A travel agency’s card on file, for instance, answers yes to the first, third and fourth questions, which points to vault-based, format-preserving, multi-use tokens. Once you know the combinations you need, our guide to the best data tokenization tools helps you match them to vendors.

Common Mistakes Teams Make When Mixing Different Tokenization Types

Most tokenization problems come from using the wrong type for the job. Deterministic tokens on low-variety fields, format-preserving tokens that look like real cards and irreversible tokens where values must return cause the most trouble.

1. Deterministic tokens on low-variety fields: Tokenizing a gender or state column deterministically leaves only a handful of distinct tokens, and their frequencies give the values away.
2. Format-preserving tokens that pass as real cards: If nothing distinguishes a token from a live card number, auditors and downstream teams can’t tell them apart, which complicates PCI scope reviews.
3. Vaultless tokens for data subject to erasure: Without a vault record to delete, honoring one person’s erasure request can mean re-tokenizing entire datasets.
4. Irreversible tokens where values must return: Hashing SSNs in a payroll system looks safe until tax season, when nobody can recover the originals.
5. Multi-use tokens with no expiry: Card tokens kept for years after a customer leaves expand your exposure with no business reason behind them.

How Enigma Vault Helps You Use the Right Token Type for Your Data

At Enigma Vault, every product is vault-based, so you get per-record deletion and one place to control who can restore a value. Within that vault, you choose the token behavior each workflow needs:

• Random tokens: Data Vault generates 128-bit tokens from a cryptographically secure random generator, with nothing reversible embedded in them.
• Deterministic tokens: Available in Data Vault and Card Vault for joins, lookups and deduplication, and used by NoPII to keep each person consistent in AI prompts.
• Format-preserving tokens: Card Vault offers optional tokens that keep the first six and last four digits, and it can issue them from your own BIN range.
• Expiring and single-use access: Records in Data Vault can expire anywhere from 60 seconds to 7 years, and one-time keys let you share a secret once before it self-destructs.
• Batch tokenization: Up to 5,000 values per request for migrations and enrollment surges.

Everything runs on PCI DSS Level 1 and SOC 2 Type II certified infrastructure. Request a demo to match token types to your data flows.