Transcript More Symmetric Encryption Thomas Ristenpart University of Wisconsin Outline • • • • Tweakable block ciphers OCB Format-preserving encryption Format-transforming encryption.

More Symmetric Encryption
Thomas Ristenpart
University of Wisconsin
Outline
•
•
•
•
Tweakable block ciphers
OCB
Format-preserving encryption
Format-transforming encryption
3
Pseudorandom Functions (PRFs)
Let
be viewed as a “keyed” function family
Pseudorandom Permutations (PRPs)
Let
be viewed as a “keyed” function family
Tweakable block ciphers (TBCs)
• Add public “spice” or “tweak” to block ciphers
– block cipher should appear to behave independently
for each tweak
• First such block cipher is Hasty Pudding Cipher by
[Schroeppel 1998]
• First formalized by [Liskov, Rivest, Wagner 2003]
Tweak space (some finite set)
6
TBCs: definitions
A tweakable PRP is a map
Inverse
algorithm
Must be efficiently computable in both directions
7
TBCs : Why?
• TBCs offer powerful security properties while
having significant flexibility due to tweaks
• We will motivate with two target applications:
– Nonce-based symmetric encryption
– Format-preserving encryption
TBC Application 1: Fast Encryption
Recall that ECB mode does not provide confidentiality
M1
M2
M3
C1
C2
C3
Encrypted with ECB
Images courtesy
of Wikipedia
9
TBC Application 1: Fast Encryption
Tweakable block ciphers with nonce can fix this!
Let N be an n-bit nonce.
The CPA core of OCB mode (more on this later):
M1
M2
M3
C1
C2
C3
Uses a TBC
10
TBC Application 1: Fast Encryption
Tweakable block ciphers with nonce can fix this!
Let N be an n-bit nonce.
The CPA core of OCB mode (more on this later):
M1
M2
M3
C1
C2
C3
Encrypted with OCB core
Images courtesy
of Wikipedia
11
TBC Application 2:
Disk-sector encryption
Say we want to use an n-bit block cipher to encrypt n-bit disk sectors
Plaintext = n bits
(In practice disk sectors are, e.g., 512 bytes.
Ciphertext = n bits
We’ll deal with it later)
Sector 1
n bits
n bits
10101
10110
Sector 2
11100
00001
Sector 3
10101
10110
Woops,
repeat
blocks
detectable
Length-preservation means we can’t use randomized encryption
12
TBC Application 2:
Disk-sector encryption
Say we want to use an n-bit block cipher to encrypt n-bit disk sectors
Plaintext = n bits
(In practice disk sectors are, e.g., 512 bytes.
Ciphertext = n bits
We’ll deal with it later)
Sector 1
n bits
n bits
10101
00110
Sector 2
11100
10111
Sector 3
10101
10100
Using independent keys with block cipher gives
independent random permutations
Repeats
not
detectable,
but now
too many
keys!
13
TBC Application 2:
Disk-sector encryption
Say we want to use an n-bit block cipher to encrypt n-bit disk sectors
Plaintext = n bits
(In practice disk sectors are, e.g., 512 bytes.
Ciphertext = n bits
We’ll deal with it later)
Sector 1
n bits
n bits
10101
00110
Sector 2
11100
10111
Sector 3
10101
10100
Repeats
not
detectable,
only one
key still!
Using distinct tweaks + good tweakable block cipher
gives same effect as distinct keys. Tweaks can be public.
14
Consider the set
the “family” of all families of permutations over n-bit strings
Sampling an element of
00…00
111010110…110101
00…01
10000010…100111
00…10
00000010…011111
…
11…10
11…11
1011111111…100111
00…00
101101101…000011
00…01
1111111010…100111
00…10
001011010…011111
010101110…100111
…
11…10
11…11
1011001111…101011
0111101100…111011
…
Security of TBCs:
Tweakable Pseudorandom Permutations (TPRPs)
Security of TBCs:
Tweakable Pseudorandom Permutations (TPRPs)
An equivalent 2-game formulation
Fast PRP to TPRP constructions
Question: Given secure PRP how do we build secure TPRP?
(We will start with simple tweak spaces, such as {0,1}n )
Example:
AES
???
Fast TPRPs: Attempt 1
Given a good PRP (e.g., AES) how do we build a tweakable PRP?
Assume
M
T
M 1
T 1
Can we break TPRP security?
Yup
=
1
=
1 / 2n
19
Fast TPRPs: Attempt 2
Given a good PRP (e.g., AES) how do we build a tweakable PRP?
Assume
M
T
Can we break TPRP security?
Nope
20
Fast TPRPs: Attempt 2
Given a good PRP (e.g., AES) how do we build a tweakable PRP?
Assume
M
Why does this work?
Intuition: For all M,T and M’,T’
T
is tiny.
Can we break TPRP security?
Nope
21
Fast TPRPs: Construction 1
Let’s sketch the proof.
Intuition:
Adversary can’t force input
to second E call to ever collide
M
M
T
T
Actually this construction
is equivalent to CBC-MAC!
CBC-MAC[E](K,M||T)
Claim:
Why?
We next sketch proof that:
22
Proof sketch
Proving CBC-MAC[E] is secure PRF
M
T
Intuition:
Adversary can’t force input
to second E call to ever collide
with prior E inputs
1) Replace E with random permutation. Use E’s PRP security
24
Proof sketch
Proving CBC-MAC[E] is secure PRF
M
T
Intuition:
Adversary can’t force input
to second E call to ever collide
with prior E inputs
1) Replace E with random permutation. Use E’s PRP security
2) Replace random permutation with random func. Birthday bound
25
Proof sketch
Proving CBC-MAC[E] is secure PRF
M
T
Intuition:
Adversary can’t force input
to second E call to ever collide
with prior E inputs
Let Coll be event that
for some
1) Replace E with random permutation. Use E’s PRP security
2) Replace random permutation with random func. Birthday bound
3) Argue inputs to second random func “fresh”
26
Fast TPRPs: Construction 1
Let’s sketch the proof.
Intuition:
Adversary can’t force input
to second E call to ever collide
M
M
T
T
Actually this construction
is equivalent to CBC-MAC!
CBC-MAC[E](K,M||T)
Claim:
We next sketch proof that:
P
27
Fast TPRPs: Construction 1
Given a good PRP (e.g., AES) how do we build a tweakable PRP?
Assume
M
M
T
T
Can we switch ordering
of M and T ?
CBC-MAC[E](K,M||T)
28
Fast TPRPs: Construction 1
Given a good PRP (e.g., AES) how do we build a tweakable PRP?
Assume
T
T
M
M
Can we switch ordering
of M and T ?
CBC-MAC[E](K,T||M)
We can! Same proof.
This will be faster in some cases. When?
29
Fast TPRPs: Construction 1
Given a good PRP (e.g., AES) how do we build a tweakable PRP?
Assume
T
M
Speed-up for fixed T,
changing M
M
T
Speed-up for fixed M,
changing T
Can we get best of both worlds? Yes, for special cases.30
Recall a motivating application
For many applications TBC with specialized tweak space
suffices. Here
M1
M2
M3
C1
C2
C3
Encrypted with OCB core
Images courtesy
of Wikipedia
31
Towards faster TPRPs
Given a good PRP (e.g., AES) how do we build a tweakable PRP?
Assume
M
Big idea:
change this to multiplying by 2i
T
i
T
M
Ideal to have something like this, where
i can be an integer
But this is insecure!
32
Essentials on GF(2n)
• GF(2n) is Finite (Galios) Field over n-bit strings
• Represent point a equivalently as:
– Polynomial
– Integer
– Bit string
33
Essentials on GF(2n)
• GF(2n) is Finite (Galios) Field over n-bit strings
• Addition = bitwise xor of points (as bit strings)
• Multiplication:
– fix primitive poly. For n = 128 use
– Multiple points a*b by treating as polynomials,
multiplying formal polys, and take remainder mod
fixed prim poly
• Benefit: x = 2 (a = 012610) generates
multiplicative subgroup of GF(2n)
34
Essentials on GF(2n)
Multiplication by 2 is fast!
Divide by primitive polynomial and take remainder
If an-1 = 1 then:
left-shift a by 1 and add prim poly
If an-1 = 0 then:
left-shift a by 1
One shift and one conditional xor!
35
Fast TPRPs: XE construction
[Rogaway 2004]
Given a good PRP (e.g., AES) how do we build a tweakable PRP?
Assume
2i
T
M
Multiplication in GF(2n) using primitive
polynomial. This means that
2Y, 4Y, 8Y, …
are all distinct! (Think Y = EK(T) )
Proof by extension of previous arguments
But is this fast? Yes in an amortized sense:
Computing 2Y given Y is 1 shift and 1 conditional xor!
36
Fast TPRPs: XE construction
Using XE in a mode in many settings is fast. Compute EK(N) once
and then only have to do doublings
M1
M2
M3
C1
C2
C3
Cost for m block message:
m+1 block cipher calls
m shifts
m conditional xor’s
37
Nonce-based SE
A nonce-based encryption scheme is a triple of algorithms
Key-generation
algorithm
Encryption
algorithm
Decryption
algorithm
(See Rogaway’s
Nonce-Based Encryption Paper)
What do we need for
secure symmetric encryption?
• IND-CPA (confidentiality)
• INT-CTXT (integrity)
IND-CPA in the nonce-based setting
Restrictions:
1.
2. All nonce values queried unique
“Nonces” are meant to be used once.
An adversary that never repeats a nonce is called “nonce-respecting”
IND-CPA of OCB CPA-core
Reduction to switch from real TBC to family of random
permutations
As long as nonces never repeat, then tweak inputs are
unique to every use, selecting distinct random
permutations
Outputs are uniformly random
41
IND-CPA of OCB CPA-core
M1
M2
M3
C1
C2
C3
Reduce to
adversary B
M1
M2
M3
C1
C2
C3
Each random perm used on at most one point
C1, C2, C3 all independent uniform points 42
What do we need for
secure symmetric encryption?
• IND-CPA (confidentiality)
• INT-CTXT (integrity)
P
Nonce-respecting CTXT
How to get CTXT?
Does OCB CPA enjoy CTXT?
No! Insecure as every 3n-bit string is a valid ciphertext
M1
M2
M3
C1
C2
C3
Ideas for getting CTXT?
45
OCB mode
- Add integrity checksum of message blocks
- Tweak N,z used for checksum call must be distinct from others
- M1 + M2 + M3 is XOR of message blocks
Note:
M1
M2
M3
M1 + M2 + M3
C1
C2
C3
T
This is simpler
version only
handling strings
of length a
multiple of n.
See [Rogaway 04]
46
Tweakable Strong PRPs (TSPRPs)
Fast TSPRPs: XEX[Rogaway
construction
2004]
Given a good PRP (e.g., AES) how do we build a tweakable PRP?
Assume
2i
T
M
Does this work?
Not quite: can’t use tweak (T,0) for any T
48
Fast TSPRPs: XEX[Rogaway
construction
2004]
Given a good PRP (e.g., AES) how do we build a tweakable PRP?
Assume
Just don’t allow 0
n
20 N = EK(0 )
Does this work?
Not quite: can’t use tweak (T,0) for any T
0n
0n
49
Fast TSPRPs: XEX[Rogaway
construction
2004]
Given a good PRP (e.g., AES) how do we build a tweakable PRP?
Assume
Just don’t allow 0
n
20 N = EK(0 )
Does this work?
Not quite: can’t use tweak (T,0) for any T
0n
0n
50
CTXT Security of OCB
M1
M2
M3
M1 + M2 + M3
C1
C2
C3
T
CTXT Security of OCB
M1
M2
M3
M1 + M2 + M3
New N means:
inverse of T random
New C means:
one of M1, M2, M3, inverse of
T random
C1
C2
C3
T
Either way, low prob. of hitting
a correct T
Recap of OCB
• OCB gives fast, single pass nonce-respecting
scheme
• OCB doesn’t give repetition-resp. security
• OCB remains fastest AES-based AE scheme
Cycles/byte
From [Krovetz, Rogaway 2011]:
Message length (bytes)
53
Outline
•
•
•
•
Tweakable block ciphers
OCB
Format-preserving encryption
Format-transforming encryption
54
Example #1: Disk sector encryption
Say we want to do full-disk encryption sector-by-sector
Plaintext = 4096 bytes
How to build TBC on 4096 bytes
Ciphertext = 4096 bytes
given block cipher with n = 128?
4096 bits
4096 bits
Sector 1
10101
00110
Sector 2
11100
10111
Sector 3
10101
10100
55
Example #1: Disk sector encryption
Idea one: Split 4096 byte sector across multiple invocations of XEX
M1
M2
M3
What’s wrong with OCB core?
(say N is tweak = sector #)
Each ciphertext block not affected
by all plaintext blocks
C1
C2
C3
M1’
M2
M3’
What does this leak?
Multiple ciphertexts leak n-bit diffs
Practitioners seem to think this is ok:
XTS mode standardizes variant of this
XTS used in many products
C1’
C2
C3’
We can do better but
need more tools
56
Example #2: Data field encryption
Jane Doe
1343-1321-1231-2310
Thomas Ristenpart
9541-3156-1320-2139
John Jones
5616-2341-2341-1210
Eve Judas
2321-4232-1340-1410
AESK : {0,1}128
{0,1}128
Ciphertexts are too big for
replacing plaintext within
database!
Database schemas
and software require
<= 16 decimal digits
and valid Luhn
checksum. Last 4 left
in clear
M = 2321-4232-1340
AESK
128 bits
C
57
Example #2: Data field encryption
Jane Doe
1343-1321-1231-2310
Thomas Ristenpart
9541-3156-1320-2139
John Jones
5616-2341-2341-1210
Eve Judas
2321-4232-1340-1410
AESK : {0,1}128
{0,1}128
Ciphertexts are too big for
replacing plaintext within
database!
Instead:
- Want a TBC that works on
domain 12-decimal digits
- Tweak using last 4 digits
Database schemas
and software require
<= 16 decimal digits
and valid Luhn
checksum. Last 4 left
in clear
M = 2321-4232-1340
T = 1410
C = 1924-4327-0914
58
Towards building arbitrary-domain TBCs
Question (informal): Given PRF how do we build PRP?
FK’
Non-invertible
Example: CBC-MAC[E]
We need a
security and
efficiency-preserving
transform
Family of permutations;
invertible
Feistel networks used in design of DES
Luby-Rackoff give first theoretical treatment
of PRF to PRP using Feistel. Easy to add tweaks.
Towards building arbitrary-domain TBCs
Question (informal): Given PRF how do we build tweakable PRP?
FK’
Non-invertible
Example: CBC-MAC[E]
We need a
security and
efficiency-preserving
transform
Tweakable block ciphers
Feistel networks used in design of DES
Luby-Rackoff give first theoretical treatment
of PRF to PRP using Feistel. Easy to add tweaks.
01010101 01010110
Feistel permutation:
• Split input in half
• Apply PRF to right half
• XOR with left half
• Drop right half down to left
+
10101010
11101101
FK2
+
10101011
10101111
…
…
Feistle network is just a sequence of
Feistel permutations
FK1
Let Fe[f1,…,fr] be an r-round Feistel network using functions f1,…,fr
Example: Fe[FK1,FK2] is two-round Feistel using PRF F with two keys
61
Is 1-round Feistel a PRP?
Let f be random function
L0
Answer: No!
R0
f
Left output is equal
to right input!
+
L1
R1
Is 2-round Feistel a PRP?
Let f1,f2 be random functions
L0
Answer: No!
R0
f1
+
L1
R1
f2
+
L2
R2
L0
Is 3-round Feistel a PRP?
R0
f1
+
R0
L1
f2
+
How can we prove it?
R2
L2
f2
If L1 and L2 never collide across all queries,
then R3 and L3 are random outputs
+
R3
L3
L0
Is 3-round Feistel a PRP?
R0
f1
+
R0
L1
f2
+
How good is this bound?
R2
n = 128
L2
f2
q = 232 gives advantage 1
+
How can we do better?
R3
L3
Time to shuffle!
66
Time to shuffle!
Consider a maximally unbalanced
Feistel network.
1 bit
n-1 bits
b
x
FK1
This is the same as the so-called
Thorpe shuffle [Morris, Rogaway, Stegers 09]
x
d
Flip random coin.
Heads put ace down first, tails 5. Say it is heads
67
Time to shuffle!
Consider a maximally unbalanced
Feistel network.
1 bit
n-1 bits
b
x
FK1
This is the same as the so-called
Thorpe shuffle [Morris, Rogaway, Stegers 09]
x
d
Now look at next two in each pile.
Flip a coin, and put them down in appropriate order
68
Time to shuffle!
Consider a maximally unbalanced
Feistel network.
1 bit
n-1 bits
b
x
FK1
This is the same as the so-called
Thorpe shuffle [Morris, Rogaway, Stegers 09]
x
d
Now look at next two in each pile.
Flip a coin, and put them down in appropriate order
69
Time to shuffle!
1 bit
Consider a maximally unbalanced
Feistel network.
n-1 bits
b
x
FK1
This is the same as the so-called
Thorpe shuffle [Morris, Rogaway, Stegers 09]
x
0
1
2
3
4
5
6
d
7
One round of Thorp shuffle
Now look at next two in each pile.
Flip a coin, and put them down in appropriate order
0
1
2
3
4
5
6
7
Now think of position in deck
as a numerical value:
5 gets mapped to 2
1 gets mapped to 3
…
This defines a permutation on
3 bits!
70
Time to shuffle!
1 bit
Consider a maximally unbalanced
Feistel network.
n-1 bits
b
x
FK1
This is the same as the so-called
Thorpe shuffle [Morris, Rogaway, Stegers 09]
x
000 001 010 011
100 101 110
d
111
One round of Thorp shuffle
Now look at next two in each pile.
Flip a coin, and put them down in appropriate order
000 001 010 011
100 101 110
111
Now think of position in deck
as a numerical value:
5 gets mapped to 2
1 gets mapped to 3
…
This defines a permutation on
3 bits!
71
Time to shuffle!
1 bit
Consider a maximally unbalanced
Feistel network.
n-1 bits
1
00
FK1
This is the same as the so-called
Thorpe shuffle [Morris, Rogaway, Stegers 09]
00
x
000 001 010 011
100 101 110
d
1
111
One round of Thorp shuffle
Now look at next two in each pile.
Flip a coin, and put them down in appropriate order
000 001 010 011
100 101 110
111
Now think of position in deck
as a numerical value:
5 gets mapped to 2
1 gets mapped to 3
…
This defines a permutation on
3 bits!
72
Balanced Feistel as Shuffle
n = 4 bits
Split deck into 2n/2 piles of size 2n/2
Take top card from each pile
Place them down in random order
Take next card from each pile
Place them down in random order
…
n/2 bits
L
n/2 bits
R
FK1
R
L’
73
Do all card shuffles define
block ciphers?
Sadly no, the shuffle must be oblivious
Idea goes back to Naor
(Counter) Example:
The traditional “Riffle” shuffle
is not oblivious
Can trace position of a card without
knowing position of most cards in deck,
just need to know random coins related to this card
74
Provable results about shuffles
N = 2n
Approach
Efficiency
(# PRF calls)
Provable PRP
Security
Balanced Feistel [LR 88]
3
q ≈ N1/4
Granboulin-Pornin [GP 07]
O(log3 N)
q=N
Thorp [MRS 09]
O(log N)
q ≈ N(1 – ε)
Thorp [M 09]
O(log3 N)
q=N
Balanced Feistel [HR 10]
O( 6 / ε )
q ≈ N(1 – ε)/2
Swap-or-Not [HMR 12]
O(log N)
q ≈ (1 - ε)N
Stefanov-Shi [SS 13]
O(N1/2)
q=N
Mix-and-Cut [RY 13]
O(log2 N)
q=N
Sometimes-Recurse [MR 13]
Expected
O(log N)
q=N
When q = N get strong PRP security
75
Mix-and-Cut card shuffle
on 8 cards
mix
cut
76
Mix-and-Cut card shuffle
on 8 cards
log N PRF calls
using Swap-or-Not
mix
Tree has depth
log N
Recursively shuffle each
of the two halves
(independently)
cut
mix
mix
cut
mix
cut
cut
mix
mix
cut
cut
mix
cut
Gather
77
Quick summary of where we’re at
• Tweakable block ciphers
– We can build fast ones for domain {0,1}n where n
is AES block size
– We can use shuffling constructions to build ones
for domains of size {0,1}n for any n. Provably
secure ones require at least O(log N) calls
78
Disk sector encryption
Say we want to do full-disk encryption sector-by-sector
Plaintext = 4096 bytes
How to build TBC on 4096 bytes
Ciphertext = 4096 bytes
given BC with n = 128 bits?
4096 bytes
4096 bytes
Sector 1
10101
00110
Sector 2
11100
10111
Sector 3
10101
10100
79
Disk sector encryption
Simple solution is 4 rounds of a type of unbalanced Feistel
4096 bytes
L
R
Repeat twice more and
tweak appropriately (via F and G)
FK1
Build F, G from AES
L’
R
Security is birthday-bound in
|L| (e.g., up to q = ~2n/2)
GK2
L’
R’
Efficiency is not great:
~4 AES calls per n-bit input
80
Disk sector encryption
Faster modes: EME (~2 block ciphers per n-bit input)
SP = PPP2 + PPP3
SC = CCC2 + CCC3
M = MP + MC
Provably secure up to ~2n/2
blocks encrypted in total
Proof is super complex
Recast using TBC and get simpler
proof?
Diagram from [Halevi , Rogaway 2003]
81
Data-field encryption
Jane Doe
1343-1321-1231-2310
Thomas Ristenpart
9541-3156-1320-2139
John Jones
5616-2341-2341-1210
Eve Judas
2321-4232-1340-1410
- Want a TBC that works on domain
12-decimal digits
- Tweak using last 4 digits
Are tools we have so far
going to work here?
Database schemas
and software require
<= 16 decimal digits
and valid Luhn
checksum. Last 4 left
in clear
M = 2321-4232-1340
T = 1410
C = 1924-4327-0914
82
Encryption for arbitrary integers
Required domain with |X| = 1012
• Used to operating on bit strings, but N often not power-of-2
• Many shuffling constructions work for arbitrary N
• Mix-and-cut, Swap-or-Not, Sometimes-Recurse, etc.
• A bit slow for practical deployment
• In practice: FFX mode based on some radix (e.g., r = 10)
d digits
L
d digits
This works for N
a power of some
radix. What about
more general
domain sizes?
R
T
FK1
Now use addition
modulo rd
R
L’
FFX specification heuristically suggests
10 rounds of Feistel. (Fast)
83
Unbalanced Feistel-based FPEs
Domain is Zab for some integers a < b
Split input M ∈ Zab into L0 ∈ Za and R0 ∈ Zb
Apply R rounds of an unbalanced Feistel network
FE1
What if domain size is prime?
Addition
mod a
[Hoang, Rogaway 2010] show security up to ~N/a queries for enough rounds.
Probably can get by with fewer rounds than they suggest
84
Use cycle-walking [Black, Rogaway 2002]
to go from Zab for some integers a < b to arbitrary integer domain ZL
X
To encrypt message X ∈ ZL
Choose a,b so that ab > L
Y = FE1(K,T,X)
Y
Is Y ∈ ZL? Yes, stop and output Y
Worst-case running time:
O(L)
Expected running time:
O(ab/L)
Y’ = FE1(K,(a,b),T, Y)
Y’
Is Y’ ∈ ZL? Yes, stop and output Y’
…
…
85
TBC constructions so far:
Zs
E(K,T,  )
Zs
If s = 2m for some m then use Feistel over bit strings
If s ≠ 2m for some m then use Feistel over integers + cycle walk
What about for domains like:
Credit card numbers with
valid checksum
Bit strings with ASCII text
“HTTP” embedded
Format-Preserving Encryption (FPE)
Domain X is some finite set
Ē(K, T,  )
X
Task:
Specifying
fast ranking
functions
s = |X|
Rank(  )
Zs
X
Ē(K,T,  ) is
permutation
on X
for all K,T
Unrank(  )
E(K,T,  )
Zs
This is the Rank-Encipher-Unrank
construction [BRRS 09]
Ē inherits security of E. Proof is straightforward
87
Rank-then-Encipher: Fast ranking schemes
Specialized constructions
X = { X ∈ {0,1,…,9}12 }
Rank( D1 … D12 )
1) Compute number X that is
base-10 represented by D1 … D12
2) Return X
Z1012
Unrank( X )
1) Compute base-10
representation of X as D1 … D12
2) Return D1 … D12
X
88
Rank-then-Encipher: Fast ranking schemes
General constructions for regular languages
0
q1
q3
Regular languages naturally describe many
formatting constraints!
0,1
1
0
q2
X = L for any regular language L
q2
1
[BRRS09] gives polytime
Rank(  ), Unrank(  )
following [Goldberg, Sipser ’85]
Based on dynamic programming
Requires DFA
representation of L.
Starting from regex is
PSPACE-complete
Q = number of DFA states
Σ = the alphabet
Nmax = length of longest string
N = length of string to be ranked
Precomputation:
cNmax |Σ|Q time, cNmax Q space
rank: O(N)
unrank: O(N |Σ|)
89
Ranking & formatted ciphertexts
proves useful in surprising applications
OpenNet Initiative (ONI), Reporters Without Borders (via wikipedia; updated Jan 6, 2014)
Protocol identification via
deep-packet inspection (DPI)
Client of
protocol X
DPI system
Server of
protocol X
Check packet contents against regular expressions
/^(HTTP/(0.9|1.0|1.1) [1-5][0-9][0-9] [\x09-\x0d -˜]* …/
Free translation: Does packet include HTTP headers?
DPI users want to identify protocol X
X = TLS or Tor then throttle connection
X = HTTP then leave it alone
X = ??? then throttle traffic
TLS 1.1 …
A43FB89CD213F31456
DPI systems can classify
protocols
Encryption doesn’t look like
plaintext protocols
Iran deploys filters for
Tor handshakes
Tor rolls out new code
that avoids this filter
Example scenario:
DPI system only allows HTTP traffic unfettered
Tor client
DPI system
Tor proxy
“I think this is HTTP traffic!”
Want way to force DPI to classify traffic incorrectly as HTTP
So-called “misclassification attacks” against DPI
Surveying modern DPI systems
System
Look at
ports?
TCP stream
reassembly
Uses regex’s
Use’s C/C++
AppID
Yes
No
Yes
No
L7-filter
Yes
No
Yes
No
Yaf
Yes
Yes
Yes
No
Bro
Yes
Yes
Yes
Yes
nProbe
No
Yes
Not explicitly
Yes
Proprietary*
Yes
Yes
?
?
* Hint: it’s a serious product (~$10k) and similar ones seem to be used in Iran.
Can we build encryption schemes that
fool regex-based systems?
Format-transforming encryption
[DCRS 2013]
Cryptographic
secret key K
M
K
(Conventional)
Encryption
C
C looks like
random
junk. Won’t
look like HTTP
Format-transforming encryption
[DCRS 2013]
Cryptographic
secret key K
{0,…,|L(R)| - 1 }
Unrank
Rank
M
K,R
FormatY
$ Enc(K,M)
Transforming
C
Unrank(R,Y)
Encryption
L(R)
Ranking for DFAs (1985).
But want it now for regexes:
Regex R
NFA M
Regular expression R
specifying desired
ciphertext format
C
C is guaranteed to
match against R
[Goldberg,Sipser `85]
[BRRS `09]
DFA M’
Exponential blow-up in worst case. Ok for regexes we used.
FTE using NFAs
[LDJRS 2014]
We built a complete FTE record layer and proxy system
FTE
client
FTE(K,R1, M1)
FTE(K,R2, M2)
Client of
protocol X
FTE
server
Server of
protocol X
You are protocol Y!
Want to trick DPI into thinking we’re protocol Y != X
Where do we get R1 and R2 ???
(1) Easy to manually craft
(2) Get from DPI themselves
(3) Learn from traffic samples
We build regexes for
variety of “cover” protocols:
Y = HTTP, SSH, SMB, SIP, RTSP
FTE forces protocol misidentification
FTE
client
FTE(K,R1, M1)
FTE(K,R2, M2)
FTE
server
Client of
protocol X
Server of
protocol X
Tests with gets on Alexa Top 50 sites (X = mix of HTTPS/HTTP)
R1 R2 set to HTTP, SSH, SMB, and more. When do we trick DPI ?
System
DPI-derived
regex’s
Manual
regex’s
Learned
regex’s
AppID
Always
Always
Always
L7-filter
Always
Always
Always
Yaf
Always
Always
Always
Bro
Sometimes
Always
Always
nProbe
Never
Always
Almost always
Proprietary
Always
Always
Always
Top 50 Alexa websites
42 Mbps goodput (manual/extracted regexes)
2 Mbps goodput (learned regexes)
58 Mbps goodput baseline
We added our record layer to Tor
FTE
client
FTE
server
Tor
network
Virtual private
server in China
FTE Tor proxy
in Wisconsin
We rented virtual private server in China. Setup Tor proxy in USA
Confirmed vanilla Tor blocked on this setup
FTE never blocked as we expected
http://fteproxy.org
# of users
• FTE is open source
• Now deployed with Tor
• Ongoing integration into Lantern, uProxy, others
Outline
•
•
•
•
Tweakable block ciphers
OCB
Format-preserving encryption
Format-transforming encryption
104
Take-home exercises
• Detail an efficient (just a small number of
queries) TSPRP adversary against XE when
instantiated using a secure block cipher
• Given an efficient (just a small number of
queries) SPRP adversary that distinguishes
Fe[f1,f2,f3] for random functions f1,f2,f3 from a
random permutation
– (Recall SPRP adversary has oracles for both forward
permutation and inverse)
Hash functions and Message Authentication
Game plan
• Building compression functions
• The basic iterative hash construction
• Message authentication using hash functions
108
What are cryptographic hash functions?
Message
string in
{0,1}*
Publicly computable
“Hash function”
eg., MD5, SHA-1
string in
n
{0,1}
eg., n=160
Cryptographic
“Fingerprint”
109
“Good properties” of hash function H
1. lots of mixing from input to output (“random looking”)
2. changing one bit of input affects whole output
3. easy to compute (in the forward direction!)
4. hard to invert (“one way”)
5. hard to produce inputs that collide
… many others (as we’ll see)
Collision resistance (CR)
is CR if no one can give an efficient adversary
that finds two messages with the same hash value
There is a CR attack against any H. What is it?
Pick lots of domain points randomly
See if any hash to same value
Called a “birthday attack”
Birthday attack against H
q balls are messages
Ball M thrown into bin Y = H(M)
Hash function is “ideal” if it “behaves
like” Y
Y1
2
a public random function.
So-called random oracle model [BR93]
When do we have a collision?
…
Y 2n
2 balls assigned into same bin
Suppose H assigns points to bins
uniformly at random.
Probability of bin having two balls?
This is a lower bound on birthday paradox
Collision resistance (CR)
is CR if no one can give an efficient adversary
that finds two messages with the same hash value
Birthday attack means no H with security beyond q = 2n/2 work.
We can set n largish, 160 or above should be enough
But how do we build H such that we have confidence of requiring
2n/2 work?
Building CR hashes is hard
•
•
•
•
“Near-collisions” in SHA-0 [Biham, CRYPTO’04]
Collisions in SHA-0 [Joux, rump session, CRYPTO’04]
Collisions in reduced-round SHA-1 [Biham, rump session]
Collisions in MD4, MD5, RIPEMD, HAVAL-128, SHA-0
[Wang et al., rump session]
• Collisions in SHA-1 (263 work) [Wang et al.] (Now 253 work)
• MD5 collisions in <1 minute [Klimov]
• Colliding MD5 Postscript Documents [Lucks, Daum]
• Colliding MD5 X.509 Certificates [Lenstra, Wang, Weger]
• Colliding MD5 C programs [Black, Cochran, Highland]
115
Building a hash function
arbitrary length string (1KB, 1TB, whatever…)
n-bit string
116
Just like in symmetric encryption, we start small…
Sometimes
d=n
d
f
n
n
Step 1: Design fixed-input-length (FIL) compression function f
Step 2: Design length-extension transform that turns f into
variable-input-length (VIL) hash function
117
“ECB-like” mode of operation
n
f
n
n
n
Input
blocks
f
n
n
Output blocks
...
n
f
n
n
parallelizable
only compresses by a factor of 2
requires a lot of state
118
Cascade mode of operation
Input
blocks
n
f
n
f
n
f
n
Output
block
lots of compression
requires very little state
not parallelizable
119
Input
blocks
n
f
n
f
f
n
Two questions:
1. How to make the “Input blocks” from the input string?
2. If f has some good properties, does the cascade
preserve them?
n
Input
blocks
n
f
n
f
n
f
n
How to make the “Input blocks” from the input string?
If |input| a multiple of n, straightforward what to do…
If not?
Truncate input to a multiple of n? (Why is this wrong?)
Repeat bits? (Which bits?)
Pad with a fixed string? (Which string?)
121
Input
blocks
n
f
n
f
n
f
n
Let’s try to make input blocks in a way that:
1. is simple, efficient
2. accommodates strings of arbitrary length
3. does not destroy any collision hardness of f;
in particular, it better be unambiguous!
122
[Damgaard’89]
x1
x2
n-1 bits long
x1
x2
…
d
xn 0s
0 padding
step
xn
length-appending
step
x1
x2
0||x1 1||x2
n bits long
xn
<d>
the number of
1 ||x
0s added above n 1||<d>
(rep. in binary as
an n-1 bit string)
[Damgaard’89]
0||x1 1||x2
1||xn 1||<d>
1 || x2
0 || x1
1 || xn
1 || <d>
n
0n
f
n
f
…
h2=f(h1 || 1 || x2)
f
f
hn+1=f(hn || 1 || <d>)
h1=f(0n || 0 || x1)
124
Assume X ≠ X’ collide
(Case 1: |X| = |X’|)
1 || x2
0 || x1
0t
f
1 || xn
f
1 || <d>
f
…
f
<d>=<d’> by assumption.
Assume
If these wires different, then done.
collision
1 || x’2
0 || x’1
0t
f
1 || x’k
f
…
1 || <d’>
f
f
Assume X ≠ X’ collide
(Case 1: |X| = |X’|)
1 || x2
0 || x1
0t
f
1 || xn
f
1 || <d>
f
…
f
Are these inputs different?
Then done. Otherwise keep going
1 || x’2
0 || x’1
0t
f
1 || x’k
f
…
1 || <d’>
f
f
Assume X ≠ X’ collide
(Case 2: |X| ≠ |X’|)
1 || x2
0 || x1
0t
f
1 || xn
f
1 || <d>
f
…
f
<d>=<d’>?If these wires
Assume <d>=<d’>.
not,done.
collision in Assume
f
different, Ifthen
collision
DONE.
1 || x’2
0 || x’1
0t
f
1 || x’k
f
…
1 || <d’>
f
f
Assume X ≠ X’ collide
(Case 2: |X| ≠ |X’|)
1 || x2
0 || x1
0t
f
1 || xn
f
f
…
= x’
k?
Assume xn =xnx’
k.
not, hold
collision
f.
Soinnow,
Then theseIfwires
the
same stringDONE.
(internal collision),
assume collision here
otherwise we’re done.
1 || x’2
0 || x’1
0t
f
1 || x’k
f
…
f
128
Assume X ≠ X’ collide
(Case 2: |X| ≠ |X’|)
1 || x2
0 || x1
0t
f
f
1 || x2 = 0 || x’1 ?
Impossible!
Continue this argument all the
way back to the beginning…
Thus collision in f.
DONE.
(we assumed |X| ≠ |X’| and <d> ≠ <d’>
0 || x’1
0t
f
1 || x2
0 || x1
0n
f
1 || xn
f
…
1 || <d>
f
f
padding rule
P Simple
P Handles inputs of any length
P A proof: If f is collision-resistant, then so is the iteration
Anything to complain about?
You can gain efficiency
with an extra assumption on f…
Pad with 0s to the
next full block length
x2
x1
IV
f
f
…
x’2
x’1
IV
xn || 1 || 0*
f
f
x’k || 1 || 0*
f
…
f
x2
x1
IV
f
y
Cases:
x’1
IV
f
1. x2 ≠ x’1, done
f
2. x2 = x’1, y ≠ IV, done
3. x2 = x’1, y = IV ?!
Extra assumption: hard to find a preimage of the IV.
(There is an attack if not…)
Merkle-Damgard construction
(with “length strengthening”)
[M’89, D’89]
append the length of X in bits
as an r-bit string (say r=64),
“right justified” in the last block.
xn || 1 || 0* || <X>
x2
x1
n
IV
f
c
f
…
f
P
P
PA proof: If f is collision-resistant, then so is the iteration
Simple padding rule
r
Handles inputs of length up to 2 (264 isn’t really a restriction)
Just like in symmetric crypto, we start small…
Sometimes
d=n
d
f
n
n
Step 1: Design fixed-input-length (FIL) compression function f
Step 2: Design length-extension transform that turns f into
variable-input-length (VIL) hash function
134
Mi
d = 512
512 bits
f
n
n = 160
Wt =
{
t-th word of Mi
( Wt-3 Wt-8 Wt-14  Wt-16 ) << 1
A H0i-1;
B  H1i-1;
C H2i-1;
0  t  15
16  t  79
D  H3i-1;
E  H4i-1
for t = 0 to 79 do
T  A << 5 + gt (B, C, D) + E + Kt + Wt
E D;
D C;
C B >> 2; B  A; A T
end
H0..4i-1
H0i A +H0i-1;
H3i  D + H3i-1;
H1i  B + H1i-1;
H4i  E + H4i-1
H2i C+ H2i-1;
160 bits
160 bits
136
Mi
Blockcipher
512-bit key, 160-bit data
Wt =
{
t-th word of Mi
( Wt-3 Wt-8 Wt-14  Wt-16 ) << 1
A H0i-1;
B  H1i-1;
C H2i-1;
0  t  15
16  t  79
D  H3i-1;
E  H4i-1
for t = 0 to 79 do
T  A << 5 + gt (B, C, D) + E + Kt + Wt
E D;
D C;
C B >> 2; B  A; A T
end
H0..4i-1
H0i A +H0i-1;
H3i  D + H3i-1;
H1i  B + H1i-1;
H4i  E + H4i-1
H2i C+ H2i-1;
137
Build f from a blockcipher
A blockcipher is a set of permutations
over strings of some fixed size (the blocksize),
where each permutation is named by a key.
X
K
Often, we assume it “looks” like a random
permutation when the key is random and
secret (PRP assumption)
E
Y=EK (X)
Ideally, it is a random permutation for every key
138
Why build hash function from blockciphers?
“In the beginning, there was DES”
(late 70s-early 90s): DES
– weak keys causes design difficulties
– small blocksize (64 bits): CR at most 232
(now): AES has changed the playing field
– no known weak keys
– bigger blocksize (128 bits): CR at 264
People have some sort of “trust” in blockciphers!
139
Let’s build a blockcipher-based hash!
(cipher-block-chaining hash)
[Akl83]
Is this collision-resistant? No.
01
M
0
EK(IV)M2 EK(0)
E
K
IV
IV
K
E
EK(EK(0)) = EK(EK(0))
140
Let’s try again…
[PGV93]
How about this?
IV
IV  1
M
IV1
E1(1)  IV
M2 IV
E0(0)
E
E
IV = IV
141
Blockcipher-based schemes
[PGV93] analyzed (by attack)
64 blockcipher-based constructions
[BRS02] proved upper and lower bounds
on collision resistance and preimage resistance
[Stam08] generalized the constructions and proved
bounds
142
12 provably secure compression functions
Davies-Meyer
143
Davies-Meyer compression function
Mi
512 bits
hi-1
160 bits
hi
E
Let’s say I label the inputs this way… look familiar?
160 bits
144
Collision Resistance analysis
Assume EK defines a random permutation for every K
(So-called ideal cipher model)
M1
IV = 0128
M2
E
E
H(M)
[BRS04] show that the DM compression function provides
CR security up to 2n/2
This shows structure is sound. But cryptanalysis needs to
confirm that E lives up to behaving like random permutations 145
Mi
SHA-1
Blockcipher
512-bit key, 160-bit data
Wt =
{
t-th word of Mi
( Wt-3 Wt-8 Wt-14  Wt-16 ) << 1
A H0i-1;
B  H1i-1;
C H2i-1;
0  t  15
16  t  79
D  H3i-1;
E  H4i-1
for t = 0 to 79 do
T  A << 5 + gt (B, C, D) + E + Kt + Wt
E D;
D C;
C B >> 2; B  A; A T
end
H0..4i-1
H0i A +H0i-1;
H3i  D + H3i-1;
H1i  B + H1i-1;
H4i  E + H4i-1
Davies-Meyer feedforward
H2i C+ H2i-1;
146
Recap of hash function basics
xn || 1 || 0* || <X>
x2
x1
n
IV
f
c
f
…
f
Step 1: Design fixed-input-length (FIL) compression function f
Merkle-Damgard w/ strengthening
Many other designs (we only looked at preserving CR!)
Step 2: Design length-extension transform that turns f into
variable-input-length (VIL) hash function
From existing strong block ciphers
From scratch (SHA-1)
147
Game plan
• Building compression functions
• The basic iterative hash construction
• Message authentication using hash functions
148
Recall generic composition approach for
authenticated-encryption
Encrypt-then-MAC:
P IND-CPA
P INT-CTXT
We desire an F that is:
(1) VIL
(2) A secure PRF when keyed
Can we use hash functions to build such an F?
We have VIL hash functions
xn || 1 || 0* || <X>
x2
x1
n
IV
f
f
c
How can we key them?
Prepend key: H(K || M)
IV-as-key:
Set IV = K
…
f
Can we argue security?
f(IV,K) , f(K,x) good PRFs?
f(K,x) is good PRF?
Let HK(M) be Strengthened Merkle-Damgard
with IV replaced by K
150
Key unknown to attacker
Message X known to attacker
X1
f
xn* = xn || 1 || 0* || <X>
...
X2
f
K
Given just Y and |X|,
attacker can easily
compute HK(Z)
Z = X1X2… xn* || X’
for any X’
Extension attacks
f
...
Y = HK(X)
n
10*<Z>
X’
f
Y
f
HK(Z)
But a secure PRF does not admit length extension attacks!
Given just Y = FK(X) and |X|,
no way to compute FK(Z)
Z
*
FK
n
FK(Z)
???
We have VIL hash functions
xn || 1 || 0* || <X>
x2
x1
n
IV
f
f
c
How can we key them?
Prepend key: H(K || M)
IV-as-key:
Set IV = K
…
f
Can we argue security?
f(IV,K) , f(K,x) good PRFs?
f(K,x) is good PRF?
Both suffer from length-extension attacks!
H(K || M) almost standardized as RFC 2063 for web
digest authentication
152
Key unknown to attacker
Message X known to attacker
X1
f
xn* = xn
...
X2
f
K
Given just Y and |X|,
attacker cannot easily
compute HK(Z)
Z = X1X2… xn* || X’
for any X’
f
...
n
Y = HK(X)|t
Truncate all
but first t bits
of HK(X)
10*<Z>
X’
Y||??
Resisting extension
attacks
|| 1 || 0* || <X>
f
f
HK(Z) ???
Security requires both t and n-t to be large.
If n is 160, can maximally get 80 bit security.
Can we do better?
The NMAC (nested MAC) construction
X1
f
K1
xn* = xn || 1 || 0* || <X>
X2
f
...
Wrap HK output with
separately keyed f call
f
f
K2
NMACK1K2(X)
This prevents extension attacks: hides inner chaining values
How can we prove it?
X1
f
K1
xn* = xn || 1 || 0* || <X>
X2
f
...
No collisions here by
CR of H (equiv. f)
f
f
K2
We leave giving formal specification of two
adversaries as an exercise.
NMACK1K2(X)
The NMAC (nested MAC) construction
X1
xn* = xn || 1 || 0* || <X>
X2
f
f
K1
...
Wrap HK output with
separately keyed f call
f
f
K2
NMACK1K2(X)
Limitations of NMAC?
1. Most code uses MD hash H as a black-box. IV is not exposed
2. Requires 2 keys, not one.
The HMAC (hashed MAC) construction
Let
be parameterized by block length d ≤ n
H
H
X
Kin
HMAC(K,X)
Kout
P Only uses H as-is
P One key
HMAC(K,X):
If |K| > d then K’ = H(K)
else K’ = K
K’’ = K’ || 0d-|K’|
Kin = K’’ ⊕ ipad
Kout = K’’ ⊕ opad
Return H( Kout || H(Kin || X) )
ipad ≠ opad fixed strings
What about security?
A closer look at keying in HMAC
Let
be parameterized by block length d ≤ n
HMAC(K,X):
If |K| > d then K’ = H(K)
else K’ = K
K’’ = K’ || 0d-|K’|
Kin = K’’ ⊕ ipad
Kout = K’’ ⊕ opad
Return H( Kout || H(Kin || X) )
ipad ≠ opad fixed strings
A closer look at keying in HMAC
Let
be parameterized by block length d ≤ n
1) Colliding key pairs
Any K1 ≠ K2 such that
HMAC(K1,X) = HMAC(K2,X)
Example: K2 = K1 || 0
|K1| < d
HMAC(K,X):
If |K| > d then K’ = H(K)
else K’ = K
K’’ = K’ || 0d-|K’|
Kin = K’’ ⊕ ipad
Kout = K’’ ⊕ opad
Return H( Kout || H(Kin || X) )
ipad ≠ opad fixed strings
A closer look at keying in HMAC
Let
be parameterized by block length d ≤ n
1) Colliding key pairs
Any K1 ≠ K2 such that
HMAC(K1,X) = HMAC(K2,X)
Example: K2 = K1 || 0
|K1| < d
2) Ambiguous key pairs
Any K1 ≠ K2 such that
K1in = K2out
K2in = K1out
HMAC(K,X):
If |K| > d then K’ = H(K)
else K’ = K
K’’ = K’ || 0d-|K’|
Kin = K’’ ⊕ ipad
Kout = K’’ ⊕ opad
Return H( Kout || H(Kin || X) )
ipad ≠ opad fixed strings
Example: K2 = K1 ⊕ ipad ⊕ opad |K1| = |K2| = d
For all X:
H(K1in || X ) = H( K2out || X )
H(K2in || X ) = H( K1out || X )
No inner/outer H
domain separation.
HMAC security as PRF for length k = d, uniform keys
Kin
X1
Kin
H
H
X
HMAC(K,X)
Kout
xn* = xn || 1 || 0* || <X>
X2
Add d-n bit padding here
f
IV
f
f
...
f
f
Kin
f
IV
Basically NMAC with two keys derived as shown via f
Key-derivation is secure under suitable assumption on f
Game plan
• Building compression functions
• The basic iterative hash construction
• Message authentication using hash functions
162
Take-home exercises
• Let E be a block cipher with key length n and block length n.
Give CR adversaries against the functions (‘+’ is xor):
– f1(c,x) = Ec(x)
– f2(c,x) = Ec(x+c)
– f3(c,x) = Ec+x(c)
• Write a formal theorem statement and proof showing that
the CR of MD with strengthening is implied by the CR of the
compression function
• Detail the pseudocode of the adversaries used in the proof
of NMAC.