1. Network Applications of Bloom Filters: A Survey
Andrei Broder and Michael Mitzenmacher
Presenter: Chen Qian
Slides credit: Hongkun Yang

2. Outline Bloom Filter Overview Historical Applications
Standard Bloom Filters
Counting Bloom Filters
Historical Applications
Network Applications
Distributed Caching
P2P/Overlay Networks
Resource Routing
Conclusion
First of all, we will talk about mathematics behind the bloom filter. We will consider standard bloom filters, and a extension, counting bloom filters. We will see how they work, their operations, and how to optimize them. Then I will briefly introduce historical applications of BF. BF is invented in 1970s, about 40 years old. In early time, memory is scares in computers. People use BF to dramatically reduce the usage of memory of programs. Then we will take more time on modern applications of BF in the area of computer networking. I will give examples from distributed caching, p2p networks, and resource routing. Finally we draw conclusions of this lecture

3. Overview Burton Bloom introduced it in 1970s Randomized data structure
Representing a set to support membership queries
Dramatic space savings
Allow false positives
BF is introduced by Burton Bloom in the 1970s, and is named after him. Since then they have been very popular in database applications. Recently they started receiving more interest in the networking literature. A Bloom filter is a simple space-efficient randomized data structure for representing a set in order to support membership queries. By membership queries, you are given a set and an element, and you can use BF to check whether the element is in the set. BF has dramatic space savings, but the cost is it will have a small false positive rate. So it is very useful when memory consumption is of great concern and a small false positive rate can be tolerated.

4. Standard Bloom Filters: Notations
S the set of n elements {x1, x2, …, xn}
k independent hash functions h1, …, hk with range {1, …, m}.
Assume: hash functions map each item in the universe to a random number uniformly over the range {1, …, m}
MD5
An array B of m bits, initially filled with 0s
Now let’s look at standard BF. We need to introduce some notations. Let S be a set of n elements. We use x1, x2, dot dot dot to denote its elements. We have k independent hash functions. Let us denote them h1 through hk. These hash functions will hash each element in the S to a number within the range from 1 to m. For the convenience of mathematical analysis, here we assume hash functions map each item in the universe to a random number uniform over the range {1, …, m}. This means that if you uniformly pick up an element in the universe at random, then the value of hash functions will be a random variable uniformly distributed over the range 1…m. in practice, hash functions cannot exactly follow this assumption. But the paper suggest that MD5 fits the assumption well. A BF can be denoted by an array of m bits B. initially all bits are set to 0.

5. Standard Bloom Filters: How It Works
Hash each xi in S k times. If Hj(xi) = 1, set B[=1.
To check whether y is in S, check B at H_j(y), j = 1,2,…,k
If all k values are set to 1, y is assumed to be in S,
If not, y is clearly not in S.
No False Negative
How does standard BF work? First we hash each element in S k times, using k hash functions. If a hash function maps the element x to a, then set the ath bit of B to 1. given an element y, to check whether y is in S, check k bits in B, H1(y), H2(y)…Hk(y). They are the values of k hash functions. If all k bits are set to 1, we claim that y is in S. if not all k bits are set to 1, which means that there are some 0 bits, then we claim that y is not in S. here let’s look at the cases. In the second case, it is obvious that y is not in S. otherwise, all k bits are set to 1. so this case will not have problems. That is, there is no false negative. But in the first case, it is not necessarily true. It is possible that y is not in S and the k corresponding bits are set to 1 by some (say, two or three) elements in S. then we will have a false positive rate.
Possible False Positive

6. Standard Bloom Filters: An ExampleB
Now we consider an example of how BF works. From this example, we can see how a false positive in BF happens. Assume that the BF has 6 bits. Initially they are set to 0. there are two hash functions.
INTIAL STATE

7. Standard Bloom Filters: An Example1 1 1 B
Now we insert two elements in S into the BF. X1 is mapped two times by the two hash functions and the corresponding bits are set to 1. Then x2.
INSERTION

8. Standard Bloom Filters: An Exampley1 y2 1 1 1 B
Now using the BF, we check two elements y1, y2, whether they are in set S. y1 is hashed two times, and we find that both the corresponding bits are not set to 1. so we can convincingly conclude that y1 is not in S. since BF has no false negatives. Then we check y2. We find that both the bits have been set to 1 by x1 and x2 respectively, as we have shown in the previous slide. Then we falsely claim that y2 is in the set S. So a false positive occurs.
CHECK

9. Standard Bloom Filters: False Positive Rate (1)
Pr[a given bit in B is 0]=
The probability of a false positive is
Let r be the proportion of 0 bits after all elements are inserted in the Bloom filter
Conditioned on r, the probability of a false positive is
Now we calculate the false positive rate of the BF. Under the assumption that hash functions are uniformly random, we can see that the probability of a given bit in B is still zero after inserting n elements in S is p prime. This is not hard to see. Each element in S is hashed k times and there are n elements. We can consider that when an element is hashed once, one bit is chosen randomly and set to 1, and this will be done in kn times. 1-1/m is the probability that a bit is not chosen. So after kn times, a given bit is still 0 with probability p’. We usually use the approximation p in place of the exact value p’ because it is a very accurate approximation and it can make things simpler when we do the math. Then the false positive rate is the probability that an element not in S is accepted by the BF. This will happen if and only if all k corresponding bits are set to 1. So the false positive rate is f’, and we can use f as an approximation. Here the false positive rate is in an average sense. By average I mean we randomly choose the set S of size n in the universe, then consider the false positive rate. Now let us consider a conditional false positive rate. Let rho be the proportion of 0 bits after all elements are inserted in the BF. If the set S is selected at random, then we have that the expected value of rho is p prime. If we condition on rho, i.e. rho is already known, then an element not in S is hashed to a 1 bit with probability 1-rho. So conditioned on rho, the false positive rate is 1-rho over k. The question is why do we need conditional false positive rate? If rho has a large variation, then the false positive rate also has a large variation. This means the false positive rate heavily depends on the set S inserted in the BF. Then the average false positive rate will not give us too much insight.

10. Standard Bloom Filters: False Positive Rate (2)
The fraction of 0 bits is extremely concentrated around its expectation
Therefore, with high probability,
However, theoretical results show that rho is close to its mean value with high probability. So we can have this approximation. This is true with high probability.

11. Standard Bloom Filters: Optimal Number of Hash Functions (1)
Two competing forces:
More hash functions gives more chances to find a 0 bit for an element that is not a member of S
Fewer hash functions increases the fraction of 0 bits in the array
Now we consider the optimal number of hash functions. Actually there are two competing forces. Firstly, if we have more hash functions, we can have more more chances to find a 0 bit for an element that is not a member of S, since BF accepts an element if only all corresponding values are set to 1. secondly fewer hash functions can increase the fraction of 0 bits in the array since we may set fewer bit to 1 when inserting an element. So there should be an optimal value.

12. Standard Bloom Filters: Optimal Number of Hash Functions (2)
This is a numerical example. M is the length of the BF, n is the size of the set S. From the graph, we can see that at some point, the false positive rate is minimized. In this case, the optimal k is about 5.5

13. Standard Bloom Filters: Optimal Number of Hash Functions (3)
Note that
Let g=kln(1-e-kn/m) , solve
Rewrite g as
where p
Using symmetry, g is minimal when p = ½
Then
We can solve the optimal number of hash functions using calculus. We consider the false positive rate f. we can get rid of the exponential and consider g instead. Since the exponential function is increasing, so once g is minimized, f is minimized. Then we take the derivative of g with respect to k and solve this equation for k. it can be shown that k is the optimal nuber of hash functions. But we can avoid doing calculus by exploiting the symmetry of g. g can be rewrite as follows where p is the probability that a specific bit is still 0. according to symmetry, we can guess that g is minimized when p equals 1 over 2. this is true actually. At this time k=ln2 times m over n.

14. Standard Bloom Filters: Space Efficiency
A lower bound
Let e be the false positive ratio, then
The optimal case
The false posive rate for the optimal Bloom filter is
Let f>e
Now we consider how many bits a BF need if we give a threshold of false positive rate.
Using some tricks, we can get a lower bound. We can consider a BF as a mapping from a set of size n to an m bits binary string.
If we consider the optimal BF. We can see this is the false positive rate f. let f be less the epsilon, then we can have this equation. So the optimal BF is within a factor of 1.44 of the lower bound.

15. Standard Bloom Filters: Operations (1)
Union
Build a Bloom filter representing the union of A and B by taking the OR of BF(A) and BF(B)
Shrinking a Bloom filter
Halving the size by taking the OR of the first and the second half of the Bloom filter
Increase false positive rate
The intersection of two sets
Let us look at the operations of BF. Some operations can be easily done in BF. If we have two BF which representing set A and B respectively. Then the bloom filter representing the union of A and B is the OR of the corresponding two Bloom filters. And if you want to reduce the size of the BF, you can take the OR of the first and the second half of the BF. Then you can have a half-sized BF, but the false positive rate will be increased. It seems that there is no simple ways to generate a BF representing the intersection of two sets from their BFs. Note that simply taking the AND of the two BFs does not make the BF that represents the intersection set. But we still find ways to estimate the size of their intersection set.

16. Standard Bloom Filters: Operations (2)
The intersection of S1 and S2
The average number of 1 bits in the AND of BF(S1) and BF(S2)
Z1 the number of 0 bits in BF(S1), Z2 BF(S2), Z12 the AND of BF(S1) and BF(S2)
So we can take the AND of the two BFs and count the number of 1 bits. Let this number be equal to the average number. And we can solve the equation for the size of the intersection set. Then we can have the estimation of the size.

17. Counting Bloom Filters: Motivation
Standard Bloom filters
Easy to insert elements
Cannot perform deletion operations
Counting Bloom filters
Each entry is not a single bit but a small counter
Insert an element: increment the corresponding counters
Delete an element: decrement the corresponding counters
So far, we have talked about standard BF. Standard BF is easy to insert elements. You just set the corresponding bits to one. But if the set of elements is changing over time, We may need to delete old elements. for example, the content in a web cache. Unpopular items are deleted and popular items are inserted. But we cannot perform deletion in Standard BF. Once you set a bit to 0, it is possible that some other elements are also hashed to this bit. Then the BF is not correct. To address this problem, the idea of counting BF is introduced. We will talk about the applications of counting BF later. Now let us look at how it works. In a counting BF,

18. Counting Bloom Filters: An ExampleB
Now we consider an example of how BF works. From this example, we can see how a false positive in BF happens. Assume that the BF has 6 bits. Initially they are set to 0. there are two hash functions.
INTIAL STATE

19. Counting Bloom Filters: An Example1 2 1 1 B
Now we insert two elements in S into the BF. X1 is mapped two times by the two hash functions and the corresponding bits are set to 1. Then x2.
INSERTION

20. Counting Bloom Filters: An Example1 1 2 1 B
Now using the BF, we check two elements y1, y2, whether they are in set S. y1 is hashed two times, and we find that both the corresponding bits are not set to 1. so we can convincingly conclude that y1 is not in S. since BF has no false negatives. Then we check y2. We find that both the bits have been set to 1 by x1 and x2 respectively, as we have shown in the previous slide. Then we falsely claim that y2 is in the set S. So a false positive occurs.
DELETION

21. Countering Bloom Filters: How Large Counters Do We Need? (1)
n elements, k hash functions, m counters, and c(i) the count associated with the ith counter
The tail probability is bounded by
Then use the union bound again
Then one will ask, how large counters do we need? If the counter is too small, then it will overflow after inserting some elements. If it is two large, then it will be a waste of resource. The value of the counter can be considered as the binomial random variable. The probability can be written as follows.

22. Countering Bloom Filters: How Large Counters Do We Need? (2)
4 bits per counter is enough
The maximum counter value is O(log m) with high probability, and hence O(loglog m) bits are sufficient
Let j = 3ln m/ lnln m
If each counter has 4 bits, then the maximal value of counter is 15. because the value of the counter should be so the counting BF overflow when the maximal value of these counters is larger than 15. actually we can prove that the maximal counter value is O(log m) with high probability.

23. Historical Applications
Dictionaries
Hyphenation programs
UNIX spell-checkers
Dictionary of unsuitable passwords
Databases
Semi-join operations
Differential files
Now let us look at the applications of BFs. In early days, memory is a scarce resource in computers. BFs are succinct representations of sets of items. So programm at that times used BF to reduce the usage of memory. In the dictionaries applications, BF are used to represent a dictionary of words. For example, in spell-checkers, BF is used to store all correct words. Memory is saved, but the implication of false positive is that the spell-checker is not 100% accurate. It is possible that you misspell a word, but it is accepted by the checker. BF are also popular in database applications. They are used to compute set intersection and set difference. Some peer2peer applications also use this idea and we will elaborate them later.

24. Distributed Caching: Scenario
Let us look at more recent applications. First is distributed caching. This work is from university of Wisconsin-Madison. It appeared in sigcomm in the paper consider the following scenario. When clients send a url request to the web, web proxies will check whether web cache has the desired web page. If so, web proxies will return the clients the desired web page rather than make a request to WEB. Using web caches can reduce latencies and reduce the workload of web

25. Distributed Caching: Summary Cache
Motivation
Sharing of caches among Web proxies to reduce Web traffic and alleviate network bottlenecks
Directly sharing lists of URLs has too much overhead
Solution
Use Bloom filters to reduce network traffic
Use a counting Bloom filter to track cache contents
Broadcast the corresponding standard Bloom filter to other proxies
We can further improve the performance by sharing caches among web proxies. This can further reduce web traffic and alleviate network bottlenecks. Since one web proxy can check whether other proxies has the desired web page rather than directly sending the request to web. However, simply sharing lists of URLs will cause large communication overhead. The solution proposed by the paper is using BF to reduce network traffic. Since content in web caches are changing over time, so it is not suitable to use standard BF because it cannot perform deletion. Then the paper introduce counting BF and use it to track the content of cache. And broadcast the corresponding standard BF to other proxies, because others only need to know whether a particular item is in this web cache.

26. P2P/Overlay Networks: Content Delivery
Problem
Peer A has a set of items SA, peer B has SB, B wants useful items from A (SA-SB)
Solution
B sends A its Bloom filter BF(B)
A sends B its items that is not in SB according to BF(B)
Implications of false positives
Not all elements in SA-SB will be sent
Redundant items (e.g. erasure coding)
A large fraction of SA-SB is sufficient (not necessarily the entire set)
Then we look at some applications in peer2peer networks. In content delivery, the problem can be formulated as follows.

27. P2P/Overlay Networks: Efficient P2P Keyword Searching (1)
Problem
Peer A has a set of items SA, peer B has SB, A wants to determine
Solution
A sends B its Bloom filter BF(A)
B sends A its items that appears to be in SA according to BF(A)
B eliminates false positives and determines exactly
Fewer bits transmitted than A sending the entire set SA
Let us look at another application, efficient peer2peer keyword searching. In this application, documents have several keywords. Each peer node is attributed to one keyword. Documents with this keyword will be stored stores documents which have a same keyword. Clients will use for example, two keywords to search documents. Then to find documents with both keywords, we need to compute set intersection.

28. P2P/Overlay Networks: Efficient P2P Keyword Searching (2)
Server A
Server B
(2) BF(A)
3 4 6
1 2
3 4
3 4
5 6 SA SB
Let us look at these procedure.
A and B are servers storing the document ID lists for the keywords kA and kB. A and B are the sets of document IDs matching the keywords kA and kB. F(A) is a Bloom filter representation of A.
Bloom filters help reduce the bandwidth requirement of "AND" queries. The gray box represents the Bloom filter F(A) of the set A. Note the false positive in the set B ^ F(A) that server sB sends back to server sA, which sA eliminates in order to send A ^ B to the client.
3 4
(1) request
Client

29. Resource Routing (1) Network is in the form of a rooted tree
Nodes hold resources
Each node keeps Bloom filters representing
A unified list of resources that it holds or reachable through one of its children
Individual lists of resources for it and each child.
When receiving a request for a resource
Check the unified list to see whether the node or its descendants hold the resource
Yes: check the individual lists
No: forward the request up the tree toward the root

30. Resources Routing (2)
Let us see the animation. The red node asks for a resource in the blue node. Then first he will deliver the request to his father.

31. Conclusion
Simple space-efficient representation of a set or a list that can handle membership queries
Applications in numerous networking problem
Bloom filter principle

32. THANK YOU!