1. Persistent Bloom Filter: Membership Testing for the Entire History
Yanqing Peng1 Jinwei Guo2 Feifei Li1 Weining Qian2 Aoying Zhou2
1 University of Utah East China Normal University
Problem Definition
PBF-2
Has IP address ever visited my web server between 9:30am and 9:40am?
Maintain a binary tree logically, merge nodes in the same level in practice.
Number of BFs = Number of Levels
In PBF-1: operation on element x at the i-th node of level j
In PBF-2: operation on (x, i) at the j-th node
Temporal Membership Testing
Given a temporal set 𝓐= { 𝒙 𝟏 , 𝒕 𝟏 , 𝒙 𝟐 , 𝒕 𝟐 , …}, an element 𝒙, and a time range [𝑠, 𝑒]
Test if exists 𝑡∈[𝑠,𝑒] such that 𝒙, 𝒕 ∈𝓐
High Level Idea
Binary decomposition
Reduce #accesses to log(query length)
Ex: Binary decomposition of [3, 8]
Bit Allocation
𝒅 𝒊 :Number of elements in the i-th BF
𝒇 𝒊 :Number of accesses to the i-th BF
𝒎 𝒊 :Number of bits allocated for the i-th BF
𝝀 𝟏 :A parameter needs to be solved
𝒎 : Total number of bits (budget)
PBF-1
Split time range into epochs, learn parameters for current epoch from previous epochs.
A binary tree of Bloom Filters
Each node maintains a BF of elements appears in time [𝑠,𝑒]
Insert x at time t: Insert x into all nodes the path from root to the leaf [𝑡,𝑡]
Query x in time [𝑠,𝑒] : Query x to all nodes in the decomposition of [𝑠,𝑒]
O(T) #BFs. T: Time upper bound
Evaluation
About 50x space saving than original dataset
Baseline: Standard BF, pair elements with timestamps
Slightly slower insertion, magnitudes better query efficiency compared to baseline
False positive rate: PBF1-offline > PBF2-offline = PBF2-online > PBF1-online >>> baseline