Mining High Utility Itemset in Big Data Ying Chun Lin, Cheng-Wei Wu, Vincent S. Tseng Department of Computer Science and Information Engineering National Cheng Kung University, Taiwan Good morning Ladies and gentlemen Today I am going to talk about “Mining HUI in Big data” My name is Ying Chun Lin. I am from department of CS and information engineering of national cheng kung university. The co-author of this work is chengwei wu and my advisor Vincent Tseng Intelligent Database System Lab

Outline Introduction Definition PHUI-Growth Experiment Conclusion This is the outline of the presentation. In the part of introduction I will talk about what the high utility itemset mining is, why should we study high utility itemset mining in big data Second, I will briefly introduce the definition The third part is PHUI-Growth, which is the main framework of our work And, I will show the performance of PHUI-Growth in the Experiment part Last, close the presentation with brief conclusion.

Introduction Let’s start with introduction

Introduction What is High utility itemset (HUI) mining? It is one of the most important tasks of frequent pattern mining, which can be used to discover sets of items carrying high utilities (e.g., high profits) However, in the era of Big Data, there is no parallel solution for high utility itemset mining currently. 念第一點 Like you go to a shopping mall, and buy some of products which are called items also each product has profits we want to find out which set of items are the most profitable ones 念第二點

Major Problems for High Utility Mining in Big Data Large amount of transactions and varied items in big data High computational complexity Large search space Combination explosion Scalability issue Data cannot be held or processed in a single machine An parallel algorithm is needed What’s the major problems for HUI when we face with big data (1) the size of transactions is large and items vary across dataset (1-1) first high computational complexity It would face the large search space and the combination explosion problem. This leads the mining task to suffer from very expensive computational costs in practical. (1-2) second data cannot be processed in a single machine it may take months or years to solve a problem of big data in a single machine (2)As a result, a well-designed algorithm incorporated with parallel programming architecture is needed.

A New Framework for High Utility Mining Parallel mining High Utility Itemsets by pattern-Growth (PHUI-Growth) Implemented on Hadoop platform Store large dataset separately on HDFS Design a new pruning strategy, Discarding local unpromising items in MapReduce framework (DLU-MR) We proposed a new framework parallel mining HUIs by pattern Growth, PHUI-Growth (1) We implemented on Hadoop platform, thus it inherits several nice properties from Hadoop, such as easy deployment in high level language, fault tolerance, low communication overheads and high scalability on commodity hardware. (2) The dataset is distributed among HDFS (3) A new pruning strategy, discarding local unpromising items in MapReduce framework, DLU-MR, is proposed. In traditional itemset mining, pattern explosion is a common problem. And we propose a strategy for pruning unpromising pattern efficiently and in parallel

Definition Then, it the definition part

Transactional Database Problem Definition Unit Profit A 2 B 3 C 1 D E 4 F G 8 Transactional Database Total 𝑇 1 A(4), B(2), C(8), D(2) 28 𝑇 2 A(4), B(2), C(8) 22 𝑇 3 C(4), D(2), E(2), F(2) 26 𝑇 4 E(2), F(2), G(1) 24 Internal Utility Utility of the transaction Transaction Let’s introduce some definition by this example First, we have items A, B, C, E, F, and G. (press) These items are in the set of items And the right column shows their unit profit, like the unit profit of A is 2 dollars. (press) This is external utility. Then we move on to our transactional database. Take t4 as example. (press) This is an transaction. In t4, we know that it contains item E, F, and G and have the quantities 2, 2, and 1 respectively. (press) We can called those quantities internal utility How about the utility of t4. We simply multiply the quantities with the unit profits and sum them up. We can get 24. (press) This is the utility of the transaction. Last come to our main goal. (press) Which itemsets, or combination of items, are the high utility itemsets? External Utility Set of items Which itemsets are the high utility itemsets?

High Utility Itemset If the utility of an itemset 𝑋 is no less than a user-specified minimum utility threshold 𝜃, we call the itemset high utility itemset. 𝑢 𝑋 ≥𝜃→𝑋 𝑖𝑠 ℎ𝑖𝑔ℎ 𝑢𝑡𝑖𝑙𝑖𝑡𝑦 𝑖𝑡𝑒𝑚𝑠𝑒𝑡 What is the high utility itemset? 念投影片上的字 (press) The following is the formal definition of the high utility itemset

High Utility Itemset Mining Unit Profit A 2 B 3 C 1 D E 4 F G 8 Transactional Database Total 𝑇 1 A(4), B(2), C(8), D(2) 28 𝑇 2 A(4), B(2), C(8) 22 𝑇 3 C(4), D(2), E(2), F(2) 26 𝑇 4 E(2), F(2), G(1) 24 𝑚𝑖𝑛𝑖𝑚𝑢𝑚 𝑢𝑡𝑖𝑙𝑖𝑡𝑦 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 𝜃=30 After the definition. Let’s do utility mining manually First, (press) we will set the minimum utility, like 30 in this example Then, I want to know (press) 念跳出來的字 So, we scan the transactional database to find which transactions contains both A and C. We can find it in t1 (point A and C in t1) and t2 t1(point A and C in t2) And we can calculate the transaction utility in t1(press) and t2 (press) Because the utility of A and C in the transactional database is higher than the minimum utility, 30, it can called high utility itemset in this transactional database. Is {A, C} a high utility itemset? 𝑢 𝐴,𝐶 𝑖𝑛 𝑡ℎ𝑒 𝑡𝑟𝑎𝑛𝑠𝑎𝑐𝑡𝑖𝑜𝑛𝑎𝑙 𝑑𝑎𝑡𝑎𝑏𝑎𝑠𝑒 = 4×2+8×1 + 4×2+8×1 =32 >𝜃→𝐻𝑈𝐼 𝑇 1 𝑇 2

Basic Pruning Strategy – TWU Downward Closure Property Transaction-Weighted Utilization(TWU) of an itemset X E.g. 𝑇𝑊𝑈 𝐸,𝐹 =26+24=50 High TWU itemset E.g. 𝑇𝑊𝑈 𝐸,𝐹 ≥(𝜃=30) TWU downward closure property The TWU downward closure property states that any superset of a low TWU itemset is not high utility itemset [Y. Liu. et. al., UBDM, 2005] Transactional Database Total 𝑇 1 A(4), B(2), C(8), D(2) 28 𝑇 2 A(4), B(2), C(8) 22 𝑇 3 C(4), D(2), E(2), F(2) 26 𝑇 4 E(2), F(2), G(1) 24 Like frequent itemset mining, we can enumerate all the possible combination of items and calculate the utility of each itemset. Then, determine whether the itemset is HUI or not. But it may face the problem pattern explosion, especially with big data, so we need an efficient pruning strategy. Let’s define something before we go into our pruning strategy. First, transaction weighted utilization of an itemset X. When the itemset is contained in a transaction, the TWU of the itemset in the transaction is the utility of the transaction. Take E and F for example. They shows up in t3 (point E and F) and t4 (point E and F). So, the TWU of E and F in the transactional database is 26 (point t3) plus 24 (point t4) and equals 50. Second, what is high TWU itemset? the same, we take itemset E and F as example. The TWU of E and F is greater than the minimum utility threshold, 30, as usual. Consequently, they are high TWU itemset. Last, we can have the feeling what the TWU downward closure property is. As we know the TWU of an itemset is the maximum possible utility a itemset could get in the transactional database. As a result, low TWU itemset can be pruned, because they are impossible to become HUIs in the transactional database

Proposed method PHUI-Growth After the definition. Let’s talk about our method

Mining High Utility Itemset in Big Data PHUI-Growth Counting Phase Apply DCP to prune low TWU 1-itemsets Reorganize each transaction Mining Phase HUIs k-HUIs & conditional u-transactions Mapper 1 Reducer 1 Mapper 1 Reducer 1 Mapper 2 Reducer 2 Mapper 2 Reducer 2 Distributed Database Mapper 3 Reducer 3 Mapper 3 Reducer 3 This is the overview of our framework., (press) PHUI-Growth. First, during the counting phase, we can know the TWU of each item. Then, we prune the low TWU items, (press) according the TWU downward closure property and the database is transformed for the mining phase. In the mining phase, the high utility itemsets are discovered in each iteration and local unpromising items are prunes in reducer by (press) DLU-MR, which is introduced later. Mapper n Reducer m Mapper n Reducer m Iterative MapReduce Basic Pruning Strategy DLU-MR

Counting Phase Calculate the TWU of all items Key-value pair Key is the item in a transaction Value is the TWU of the key item Reducer – 1 Key Value Output A <A, 28>,<A, 22> <A, 50> B <B, 28>,<B, 22> <B, 50> Mapper – 1 ABCD <A, 28>,<B, 28> <C, 28>,<D, 28> Distributed Database T1 A(8) B(6) C(8) D(6) Mapper – 2 ABC <A, 22>,<B, 22> <C, 22> Reducer – 2 Key Value Output C <C, 28>, <C, 22>, <C, 26> <C, 76> D <D, 28>, <D, 26> <D, 54> Let’s look at the counting phase. The distributed transactional database is same as previous example, and the utilities of each item in each transaction have been calculated. The minimum utility threshold is 30. In the mapper phase, we take item as the key, and the TWU of the item in the transaction as value. Like t1, separate the transaction into key-value pair as <A, 28>, <B, 28>, <C, 28> and <D, 28> In the reduce phase, all the key-value pairs with the same key go to the same reducer and the TWU of items in the transactional database is counted. Like B is in t1 and t2 (point t1 and t2). And they goes to the mapper phase become <B, 28> and <B,22> (point to the pairs). Then both two pairs go to the same reducer (point to the reducer B). Simply add the TWU of B and the TWU of B is 50 in the transactional database. T2 A(8) B(6) C(8) T3 C(4) D(6) E(8) F(8) Mapper – 3 CDEF <C, 26>,<D, 26> <E, 26>,<F, 26> T4 E(8) F(8) G(8) Reducer – 3 Key Value Output E <E, 26>, <E, 24> <E, 50> F <F, 26>, <F, 24> <F, 50> G <G, 24> Mapper – 4 EFG <E, 24>,<F, 24>, <G, 24>

Database Transformation TWU A 50 B C 76 D 54 E F G 24 Original Database A(8), B(6), C(8), D(6) A(8), B(6), C(8) C(4), D(6), E(8),F(8) E(8), F(8), G(1) Transform Database A(8), B(6), D(6), C(8) A(8) B(6) C(8) E(8) F(8) D(6) C(4) E(8) F(8) Then the database is transformed for the mining phase. According to the result of counting phase, the TWUs of each items are obtained (point to the table). (press) Prune the low TWU items based on TWU downward closure property. In this example, G is a low TWU item. Last, (press) the transactional database is sorted in the TWU increasing order Prune the low TWU items Sort the items in TWU-increasing order 𝑚𝑖𝑛𝑖𝑚𝑢𝑚 𝑢𝑡𝑖𝑙𝑖𝑡𝑦 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 𝜃=30

Mining Phase All combination of items in a transactional database Pattern explosion problem Pruning strategy Discarding local unpromising items in MapReduce framework (DLU-MR) The utility of an itemsets in the transactional database With the help of conditional u-transactions, the utility of an itemset is calculated easily In the mining phase, if we solve the following two tasks in an efficiently way, the HUIs in the transactional database can be found easily.

Combination of Items Mapper of mining phase A transactions become conditional u-transactions Key contains an itemset Value contains the utility of the key itemset and the possible items for being combined into new itemset Combination of items is done in parallel Conditional u-transaction Mapper A(8), B(6), C(8) <{A}, 8, {B(6), C(8)}> <{B}, 6, {C(8)}> <{C}, 8, {ϕ}> Let’s look at the first task in the mining phase (press) the transaction is called u-transaction, which containing the utility of each item in the transaction. First we need to define the key-value pairs in the map phase. We separate the u-transaction into several key value pairs. Let’s look at an example directly. Like the u-transaction, it is sent to a mapper (point the mapper). Then, A, B and C is separated into three key value pairs. The key of the first pair is A and the value part is the utility of A in the transaction as well as the following items of the transaction. Also the key of second pair is B, and the value is the utility of B and the rest part of u-transaction. And the key of last pair is C, and there is no following item, as the result the value part only contains the utility of C and a empty set. And the combination of length-2 itemset is as following. We put B and C into key part and the utility of the key parts are updated and rest is the following part of the transaction. Then, we can solve the task of combination of items. And this task can be done in parallel However, this may face with pattern explosion problem. So efficient pruning strategy will be proposed latter. A(8) B(6) C(8) Mapper <{A}, 8, {B(6), C(8)}> <{A, B}, 14, {C(8)}> <{A, C}, 16, {ϕ}> <{A}, 8, {B(6), C(8)}>

Utility of An Itemset Reducer in the mining phase The utility in each transaction of an itemset goes to the same reducer The utility of key itemset is calculated by adding up the utility of key itemset Each itemset goes to different reducer, as a result the calculation of utility is done in parallel Before pruning strategy, let’s look at the second task first. Calculate the utility of an itemset with the help of conditional u-transaction(press). All the utility of an itemset in each transaction will go to the same reducer. Like the example, because all utilities in each transaction of A is in the reducer. We can easily calculate the utility of A simply adding these utilities up. So the utility of A in the transactional database is 16. Due to the structure, the calculation of the utility is also done in parallel Reducer Key Value A <{A}, 8, {B(6), D(6), C(8)}> <{A}, 8, {B(6), C(8)}>

Pruning Strategy Discarding local unpromising items in MapReduce framework (DLU-MR) The strategy states that any local superset of a low local TWU itemset is low utility itemset The pruning task is done in parallel Local TWU B 50 C D 28 In the mining phase of PHUI-Growth, the pruning strategy, DLU-MR is proposed for discarding local unpromising items in the framework. Take the conditional u-transactions of key A as example. First, the TWU of each items in the reducer is calculated . And, (press)those items with low TWU are pruned from the conditional u-transactions of the key. The TWU of each items will gradually decrease when there is an item being pruned in the conditional u-transaction. So the pattern explosion can be solved in parallel and efficiently Reducer Key Value A <{A}, 8, {B(6), D(6), C(8)}> <{A}, 8, {B(6), C(8)}> Prune local unpromising item

Experiment Then, we can evaluate the performance of our method

Experiment Settings Environment settings Dataset 5-node Hadoop Cluster CPU 2.6 GHz and 4 GB memory Dataset Dataset # Trans. # Items Average Trans. Length Maximum Trans. Length Retail 88,162 16,470 10 76 Chainstore 1,112,949 46,086 7 170 T10I4N10K|D| 2,00K 2,000,000 10,000 33 Chainstore x 5 5,564,745 All experiments were conducted on a five-node Hadoop Cluster. Each node is equipped with 2.60GHz CPU and 4 GB memory. Retail dataset was obtained from FIMI Repository. Chainstore is a real-life dataset. A synthetic dataset T10I4N10K|D|2M was generated from the IBM data generator. Then, the chain store of 5 times is for checking the scalability of our method.

Performance on Small Dataset Comparing algorithm HUI-Miner [M. Liu et. al., CIKM, 2012] PHUI-Growth (Baseline) PHUI-Growth(DLU-MR) In this section, we compare the performance of PHUI-Growth with HUI-Miner [7], a state-of-the-art non-parallel type of HUI mining algorithms. To evaluate the effectiveness of the DLU-MR strategy, we prepared two versions of PHUI-Growth, respectively called PHUI-Growth(Baseline) and PHUI-Growth(DLU-MR). First let’s look at figure of execution time. When the minimum utility is higher than 0.02, we do spend more time than HUI-Miner. That’s because when the size of data is small, the communication overhead will dominate the execution time. However when the minimum utility is 0.01, HUI-miner takes about 4429 seconds. On the other hand, our method takes 556 seconds. How about number of candidates? When the threshold decreases, the number of HUIs dramatically increases and HUI-Miner need to produce a large amount of intermediate itemsets. However, the number of candidates produced by PHUI-Growth(DLU-MR) is up to two orders of magnitude smaller than that produced by HUI-Miner. Execution Time Number of Candidate

Performance on Large Datasets PHUI-Growth(Baseline) and Growth(DLU-MR) outperform HUI-Miner significantly Mining HUI in parallel greatly improve the performance About the execution time of large dataset Results show that PHUI-Growth(Baseline) and Growth(DLU-MR) outperform HUI-Miner significantly. The reason why they perform so well is that they effectively use nodes of a cluster to parallel process HUIs across multiple machines, while HUI-Miner is executed on non-parallel single machine. Chainstore T10I4N10K|D|2,000K Chainstore x 5

Conclusion

Conclusion A new parallel framework, PHUI-Growth, for mining high utility itemsets in big data Parallel discover HUIs from distributed data across multiple computers. DLU-MR is proposed to prune the search space in parallel and greatly improve the performance for mining HUIs Empirical evaluations show that PHUI-Growth has good scalability on large datasets we propose a new framework, PHUI-Growth, for mining high utility itemsets in big data. The proposed algorithm is for efficiently parallel mining high utility itemsets from distributed data across multiple commodity computers. A novel strategy called DLU-MR is proposed to effectively prune the search space and greatly improve the performance for mining HUIs. Empirical evaluations of different types of real and synthetic datasets show that PHUI-Growth has good scalability on large datasets and outperforms the state-of-the-art algorithms.

Q & A