Tokenization Terms and Definitions

Hello Everyone -

I just finished a very interesting panel on internal tokenization – the event was hosted by BrightTalk and you can catch a replay of it here. The panel included Brandon Dunlop from Brightfly, John Kindervag from Forrester, and Ult Mattsson from Protegrity. At the end of the panel I mentioned the Intel product, Intel(R) Expressway Tokenization Broker that can help Enterprises with internal tokenization.

The event was very apt due to the recent headlines that Sony has made with their playstation breach. While it is unclear if internal tokenization controls would have prevented the breach, the data stolen highlights the need for an internal tokenizaton strategy that covers both personally identifiable information AND credit card information.

We discussed many topics but I wanted to take a moment to offer some clarification on some of the terms and definitions that were used in the panel I feel that we need good, clear definitions of some of the terminology used in tokenization discussions. For those of you who listened to my part, I still feel that encryption and tokenization are not mutually exclusive options and both should be employed for a complete security strategy. Here are some of the definitions I use when describing tokenization:

Token: This one is actually important. In my view a token in this context is a pointer or replacement for the original data that is not mathematically connected to the input. Note that this contrasts the VISA tokenization guidelines which allow for tokens to be mathematically connected to the input data. If we go with the VISA definition, we have a situation where encryption, and format-preserving encryption are considered tokenization strategies when they have very different key management profiles and underlying security properties.

Single Use Token: This is a token expected to be used once for a transaction and then thrown away. Put another way, the token may be recycled for a future transaction without worrying about a collision. I see these more useful in applications with limited storage or tracking requirements. Overall I believe that single use tokens aren’t quite as cool or useful as multi-use tokens. Expressway Tokenization Broker supports single-use tokens.

Multi-Use Token: This is a token that persists over time. Put another way, the same credit card number will map to the same token for a given time interval, where the interval can be configurable but is expected to be very long. These tokens are expected to be used in loyalty tracking, data warehousing and other applications that require repeated payments. Expressway Tokenization Broker supports multi-use tokens with a configurable time interval.

Format Preserving Token: Using my initial definition of token, this is a token that preserves the structure and properties (or anti-properties) of the original data. In the case of a credit card number a format-preserving token is generally 16 digits. By anti-properties it may be useful to ensure that generated tokens are not LUHN valid. This prevents the corner case where a random token is generated that collides with a yet-to-be issued credit card number. Without this additional check, generated tokens my accidentally map to real credit card numbers with very small probability. On the other hand, some business applications are enforcing LUHN checks on tokens – in this case the reverse needs to happen – e.g. tokens must be LUHN valid. In this case, the Enterprise may have to endure the small risk of accidentally generating a real card number for some future-issued 16 digit PAN. Expressway Tokenization Broker supports these pattern checks with a configurable token generation policy.

Format Preserving Encryption: A mode of encryption where the output is of fixed length. This is not tokenization, but a way of increasing the usability of encryption by removing one of the drawbacks where the size of the ciphertext is generally bigger than the plaintext due to padding.

Secure Vault: An abstraction for a data-store that stores the relationship between a token (random number) and a real credit card number, where the credit card number is almost certainly stored in encrypted form. This forms the “look-up” table for retrieving the original card number based on the token or the token based on the card number. The vault may be physically centralized or distributed. Expressway Tokenization Broker ships with a sample vault and supports Oracle, Microsoft, and mySQL as vault options.

I’d be interested in any comments on the panel. While I feel sorry for Sony, it does make an excellent example of why internal tokenization could help reduce risks and keep your company out of the headlines.


Blake Dournaee

About Blake Dournaee

Passionate, energetic product manager that lives at the intersection of business, innovation and technology. I currently work at Intel in the Datacenter Software Division working on products and technologies relating to mobile, APIs, cloud services and big data.

Comments are closed.