C# and LINQ Yuan Yu Microsoft Research Silicon Valley

Collections and Iterators 2 IEnumerable Elements of type T Iterator (current element) Very easy to use: foreach (string name in persons) { Console.WriteLine(name); }

More on IEnumerable IEnumerable is a generic collection – C# generics is very similar to C++ template – T is a type parameter representing its element type IEnumerable.NET collections implement IEnumerable – T[], List, HashSet, Stack, Queue, Dictionary, …

IEnumerable Examples Example 1 Example 2 int[] numbers = { 1, 5, 2, 12, 4, 5 }; int sum = 0; foreach (int x in numbers) { sum += x; } string[] persons = { “Frank”, “Bob”, “Chandu”, “Mike”, “Dennis” } foreach (string name in persons) { Console.WriteLine(name); }

5 LINQ: Operators on Collection Collection collection; bool IsLegal(Key); string Hash(Key); var results = from c in collection where IsLegal(c.key) select new { Hash(c.key), c.value};

LINQ Operators 6 Where (filter) Select (map) GroupBy OrderBy (sort) Aggregate (fold) Join Input

Lambda Expression A nice way to represent anonymous functions Examples: – Func inc = x => x + 1; – Func sqrt = x => Math.Sqrt(x); – Func mul = (x, y) => x * y; Func represents any method that takes a argument of type T and returns a value of type R – Similar to C++ function pointer

Where Filters the elements in a collection based on a predicate Example: IEnumerable Where (IEnumerable source, Func pred) int[] a = { 1, 5, 2, 12, 4, 5 }; IEnumerable result = a.Where(x => x > 4);

Select Transforms each element of a collection into a new form Example: IEnumerable Select (IEnumerable source, Func selector) int[] a = { 1, 5, 2, 12, 4, 5 }; IEnumerable result = a.Select(x => x * x);

Composing Operators Composing computations Or simply You can use “var” to represent the type int[] a = { 1, 5, 2, 12, 4, 5 }; IEnumerable r1 = a.Where(x => x > 4); IEnumerable r2 = r1.Select(x => x *x); int[] a = { 1, 5, 2, 12, 4, 5 }; IEnumerable r2 = a.Where(x => x > 4).Select(x => x * x); int[] a = { 1, 5, 2, 12, 4, 5 }; var r2 = a.Where(x => x > 4).Select(x => x * x);

Query Comprehension If you really hate lambda expression, you can also use the alternate SQL-like syntax int[] a = { 1, 5, 2, 12, 4, 5 }; var r2 = from x in a where x > 4 select x * x;

Invoking User-Defined Functions int[] a = { 1, 5, 2, 12, 4, 5 }; var r2 = from x in a where MyPredicate(x) select Math.Sqrt(x); public static bool MyPredicate(int x) { // User code here }

SelectMany Tranforms each element of a sequence to an IEnumerable and flattens the resulting sequences into one sequence IEnumerable Example: IEnumerable SelectMany ( IEnumerable source, Func > selector) string[] lines= { “A line of words of wisdom”, “Dryad and DryadLINQ are great”}; var result = lines.SelectMany(x => x.Split(' '));

OrderBy Sorts the elements of a sequence in ascending order according to a key Example: IEnumerable OrderBy (IEnumerable source, Func keySelector) IEnumerable employees; var result = employees.OrderBy(x => x.Name);

GroupBy Groups the elements of a sequence according to a specified key selector function IGrouping represents a group of elements of type T with key K – g.Key returns the key of the group – g is IEnumerable IEnumerable > GroupBy (IEnumerable source, Func keySelector)

GroupBy Examples Example 1: Example 2: string[] items= { "carrots", "cabbage", "broccoli", "beans", "barley" }; IEnumerable > foodGroups = items.GroupBy(x => x[0]); int[] numbers = { 5, 4, 1, 3, 9, 8, 6, 7, 2, 0 }; var groups = numbers.GroupBy(x => x % 5);

Distinct Returns distinct elements from a sequence Example IEnumerable Distinct (IEnumerable source) int[] numbers = { 1, 5, 2, 11, 5, 30, 2, 2, 7 }; var result = numbers.Distinct();

Aggregate Applies an accumulator function over a sequence Example: R Aggregate (IEnumerable source, R seed, Func func) double[] doubles = { 1.7, 2.3, 1.9, 4.1, 2.9 }; double result = doubles.Aggregate(1.0, (r, n) => r * n);

Pre-defined Aggregators Useful pre-defineds – Count, LongCount, Sum, Max, Min, Average, All, Any, Contains, … Examples: IEnumerable numbers; long result = numbers.LongCount();

Exercises (1) Keep all numbers divisible by 5 var div = x.Where(v => v % 5 == 0); The average value var avg = x.Sum() / x.Count(); Normalize the numbers to have a mean value of 0 var avg = x.Sum() / x.Count(); var norm = x.Select(v => v - avg); Keep each number only once var uniq = x.GroupBy(v => v).Select(g => g.Key); var uniq = x.Distinct(); 20

Exercises (2) Flatten lists var flatten = x.SelectMany(v => v); The average of all even #s and of all odd #s var avgs = x.GroupBy(v => v % 2).Select(g => g.Sum() / g.Count()); The most frequent value var freq = x.GroupBy(v => v).OrderBy(g => g.Count()).Select(g => g.Key).First() The number of distinct positive values var pos = x.Where(v => v >= 0).Distinct().Count(); 21

Putting them together: Histogram 22 public static IEnumerable Histogram( IEnumerable input, int k) { var words = input.SelectMany(x => x.line.Split(' ')); var groups = words.GroupBy(x => x); var counts = groups.Select(x => new Pair(x.Key, x.Count())); var ordered = counts.OrderByDescending(x => x.count); var top = ordered.Take(k); return top; } “A line of words of wisdom” [“A”, “line”, “of”, “words”, “of”, “wisdom”] [[“A”], [“line”], [“of”, “of”], [“words”], [“wisdom”]] [ {“A”, 1}, {“line”, 1}, {“of”, 2}, {“words”, 1}, {“wisdom”, 1}] [{“of”, 2}, {“A”, 1}, {“line”, 1}, {“words”, 1}, {“wisdom”, 1}] [{“of”, 2}, {“A”, 1}, {“line”, 1}]

MapReduce in LINQ 23 public static IEnumerable MapReduce ( IEnumerable input, Func > mapper, Func keySelector, Func, IEnumerable > reducer) { var map = input.SelectMany(mapper); var group = map.GroupBy(keySelector); var result = group.SelectMany(reducer); return result; }