Optimizing Sorting With Genetic Algorithms Xiaoming Li, María Jesús Garzarán, and David Padua University of Illinois at Urbana-Champaign

ESSL on Power3

ESSL on Power4

Outline  Our Solution  Primitives & Selection mechanisms  Genetic Algorithm  Performance results  Classifier System  Conclusion

Motivation  No universally best sorting algorithm  Can we automatically GENERATE and tune sorting algorithms for each platform (such as FFTW and Spiral)? –Performance of sorting on the platform and on the input characteristics.  The algorithm selection may not be enough.

Algorithm Selection (CGO’04)  Select the best algorithm from Quicksort, Multiway Merge Sort and CC-radix.  Relevant input characteristics: number of keys, entropy vector.

Algorithm Selection (CGO’0

Proposed Solution  We need different algorithms for different partitions  The best sorting algorithm should be the result of the composition of the these different best algorithms.  Build Composite Sorting algorithms –Identify primitives from the sorting algorithms –Design a general method to select an appropriate sorting primitive at runtime –Design a mechanism to combine the primitives and the selection methods to generate the composite sorting algorithm

Outline  Our Solution  Primitives & Selection mechanisms  Genetic Algorithm  Performance results  Classifier System  Conclusion

Sorting Primitives  Divide-by-Value –A step in Quicksort –Select one or multiple pivots and sort the input array around these pivots –Parameter: number of pivots  Divide-by-Position (DP) –Divide input into same-size sub-partitions –Use heap to merge the multiple sorted sub-partitions –Parameters: size of sub-partitions, fan-out and size of the heap

Sorting Primitives  Divide-by-Radix (DR) –Non-comparison based sorting algorithm –Parameter: radix (r bits) –Step 1: Scan the input to get distribution array, which records how many elements in each of the 2 r sub-partitions. –Step 2: Compute the accumulative distribution array, which is used as the indexes when copying the input to the destination array. –Step 3: Copy the input to the 2 r sub-partitions counter accum.dest src

Sorting Primitives  Divide-by-radix-assuming-uniform-distribution (DU) –Step 1 and Step 2 in DR are expensive. –If the input elements are distributed among 2 r sub-partitions near evenly, the input can be copied into the destination array directly assuming every partition have the same number of elements. –Overhead: partition overflow –Parameter: radix (r bits) accum.dest.src

Selection Primitives Branch-by-Size Branch-by-Entropy –Parameter: number of branches, threshold vector of the branches

Leaf Primitives  When the size of a partition is small, we stick to one algorithm to sort the partition fully.  Two methods are used in the cleanup operation –Quicksort –CC-Radix

Composite Sorting Algorithms The composite sorting algorithms are built from these primitives. The algorithms have shapes of tree.

Outline  Our Solution  Primitives & Selection mechanisms  Genetic Algorithm  Performance results  Classifier System  Conclusion

Search Strategy  Search the best tree  Search the best parameter values of the primitives –Good solutions for small size problem should be retained to use in the solution for larger problem.  Genetic algorithms are a natural solution that satisfy the requirements: –Preserve good sub-trees –Give good sub-trees more chances to propagate

Composite Sorting Algorithms Search the best parameter values to adapt –To the architectural features –To the input characteristics

Search Strategy  Search for the best tree  Search for the best parameter values of the primitives –Good solutions for small size problem should be retained to use in the solution for larger problem.  Genetic algorithms are a natural solution that satisfy the requirements: –Preserve good sub-trees –Give good sub-trees more chances to propagate

Genetic Algorithm Mutation –Mutate the structure of the algorithm. –Change the parameter values of primitives.

Crossover Propagate good sub-trees

Fitness Function  A fitness function measures the relative performance of the genomes in a population.  The average performance of a genome on the training inputs is the base for the fitness of the genome.  A genome which performs well across inputs is preferred –fitness is penalized when performance varies across the test inputs

Library Generation  Installation phase: Use genetic algorithm to search for the sorting genome. –Set of genomes in initial population –Test the genomes in a set of inputs with different characteristics

Outline  Our Solution  Primitives & Selection mechanisms  Genetic Algorithm  Performance results  Classifier System  Conclusion

Platforms  AMD Athlon MP  Sun UltraSparcIII  SGI R12000  IBM Power3  IBM Power4  Intel Itanium2  Intel Xeon

AMD Athlon MP

Power3

Multiple-peak Performance

Outline  Our Solution  Primitives & Selection mechanisms  Genetic Algorithm  Performance results  Classifier System  Conclusion

The best genomes in different regions

Problems of Genetic Adaptation  Fitness function is the average performance of the genome on the test inputs.  Fitness function in our genetic algorithm prefers genomes with stable performance  The genetic algorithm is not powerful enough to evolve into the complex genome which chooses the best genome in each small region

Using Classifier System  Search the best genomes for different regions of the input characteristics. –Selects the regions –Selects the best algorithm for each region  Nice feature: The fitness of a genomes in a region will not be affected by its fitness in other regions

Map sorting composition into a classifier system  The input characteristics (number of keys and entropy vector) are encoded into bit strings.  A rule in the classifier system has two parts –Condition: A string consisting of ‘0’, ‘1’, and ‘*’. Condition string will be used to match the encoded input characteristics. –Action: Sorting genomes without branch primitives

Example for Classifier Sorting Example: –For inputs of up-to 16M keys –Encode number of keys with 4 bits. 0000: 0~1M, 0001: 1~2M… Number of keys = 10.5M. Encoded into “1100” ConditionActionFitnessAccuracy (dr 5 (lq 1 16)) …… (dp 4 2 ( lr 5 16)) …… …… ** *(dv 2 ( lr 6 16))

Performance of Classifier Sorting Power3

Power4

Conclusions  Replace the complexity of finding an efficient algorithm with the task of defining a set of generic primitives.  Design methods to search in the space of the composition of the primitives. Genetic algorithms Classifier system