1. Evolution of.NET Collections (C#) Chen Dong 2010.6.24

2. Roadmap of C# & Collections C#1.0 C#3.0 C#4.0 C#2.0 Dynamic Programming – Concurrent Collections Language Integrated Query Generics Managed Code – Collection Essential

3. .NET 1.1 Collection Essential

4. .NET 1.1 Collections Namespaces –Mscorlib.dll: System.Collections –System.dll: System.Collections System.Collections.Specialized

5. .NET 1.1 Collections Fundamental Interfaces Why do we need to create various interfaces? Interface Segregation Principle!

6. .NET 1.1 Collections Fundamental Interfaces –IEnumerator - A Forward Const Iterator interface IEnumerator { object Current {get;} bool MoveNext(); void Reset(); } "Forward" means enumerator can move in only one direction - forward. "Const" means that it cannot change the underlying collection. Current property is read-only.

7. .NET 1.1 Collections Fundamental Interfaces –IEnumerable - An Enumerable Collection With Read-Only Forward-Only Access interface IEnumerable { IEnumerator GetEnumerator(); } IEnumerable is implemented by virtually all.NET collections. GetEnumerator() returns an enumerator for the collection. Any object that implements IEnumerable can be used with the foreach operator.

8. .NET 1.1 Collections Fundamental Interfaces –Iterator Pattern Single-Responsibility Principle Inside –Why does.NET introduce two interfaces to implement Iterator Pattern? To separate Enumerator

9. .NET 1.1 Collections Fundamental Interfaces –ICollection - Synchronizable Collection With Count interface ICollection : IEnumerable { int Count {get;} bool IsSynchronized {get;} object SyncRoot {get;} void CopyTo(Array array, int Index ); } SyncRoot : Gain the object used for synchronization

10. .NET 1.1 Collections Fundamental Interfaces –IList - Modifiable Collection with Integer Index interface IList : ICollection { bool IsFixedSize { get; } bool IsReadOnly { get; } object this[ int Index ] {get; set;} int Add( object value ); void Clear(); bool Contains( object value ); int IndexOf( object value ); void Insert( int Index, object value ); void Remove( object value ); void RemoveAt( int Index ); } Array implements IList, but class SortedList does not. Because of the integer index, IList behaves more like an array rather than a list..NET does not provide operation complexity guarantees for the methods of IList. In particular, it is not formally guaranteed that all IList implementation will perform indexed access in constant time, or even in amortized constant time.

11. .NET 1.1 Collections Fundamental Interfaces –IDictionary - Associative Container interface IDictionary : ICollection { bool IsFixedSize { get; } bool IsReadOnly { get; } object this[ object key ] {get; set;} ICollection Keys{ get; } ICollection Values{ get; } void Add( object key, object value ); void Clear(); bool Contains( object value ); IDictionaryEnumerator GetEnumerator(); void Remove( object key ); } ICollection.GetEnumerator returning IEnumerator and IDictionary.GetEnumerator returning IDictionaryEnumerator are two different, unrelated methods. IDictionary defines an associative container that stores key and value pairs. The interface is relatively straightforward. IDictionaryEnumerator is a helper interface that derives from IEnumerator.

12. .NET 1.1 Collections Classes –System.Array : IList - Fixed Size Array This class is an abstract base class for all built-in C# arrays. It represents an array of fixed size and implements IList interface. It’s a special class. Built-in C# arrays derive from it implicitly. But user created class can not derive from it - this causes a compilation error. public abstract class Array : ICloneable, IList, ICollection, IEnumerable –Array implements IList, and IList derives from ICollection. Why does Array still explicitly announces the implementation of ICollection? –Array implements IList. Why can’t it invoke Add()? Interface Flatten Array explicitly implements IList.Add()

13. .NET 1.1 Collections Classes –ArrayList : IList - Dynamic Array ArrayList is heterogenous - it can contain any object. ArrayList can work as an adaptor for arbitrary IList. –ArrayList.Adapter(IList). ArrayList uses Array to store items internally (unless it acts as adapter). Array capacity is automatically doubled when more room is needed for elements. Array capacity is never automatically reduced when removing elements. Add() takes amortized constant time. Insert() in the beginning of the array takes linear time. Sort() uses QuickSort(), which is quadratic in the worst case.

14. .NET 1.1 Collections Classes –BitArray : ICollection - Array of Bits This class implements a fixed-sized array of Boolean values, or bits. Note that it implements merely an ICollection - not a full IList. Once constructed, the size of the bit array never changes. Besides, it does not implement IList methods such as IndexOf(), Contains(), etc. From the other hand it implements a number of additional, Boolean-specific methods such as And(), Or(), and Xor()

15. .NET 1.1 Collections Classes –HashTable : IDictionary Hash table is an unordered collection of key-value pairs. Key objects must correctly implement Equals() and GetHashCode. Hashtable allows you to specify your own hash code provider, and your own load factor.

16. .NET 1.1 Collections Classes –SortedList : IDictionary Sorted list is a sorted collection of key-value pairs. Besides implementing IDictionary interface, it also allows access to its items by integer index via GetByIndex() and SetByIndex() methods. A SortedList object internally maintains two arrays to store the elements of the list. Operations on a SortedList object tend to be slower than operations on a Hashtable object because of the sorting. However, the SortedList offers more flexibility by allowing access to the values either through the associated keys or through the indexes.

17. .NET 1.1 Collections Classes –ReadOnlyCollectionBase : ICollection Provides the abstract base class for a strongly typed non-generic read-only collection. This class makes the underlying collection available through the InnerList property, which is intended for use only by classes that are derived directly from ReadOnlyCollectionBase. The derived class must ensure that its own users cannot modify the underlying collection.

18. .NET 1.1 Collections Classes –CollectionBase : IList Provides the abstract base class for a strongly typed collection. Internally CollectionBase is a wrapper around ArrayList. Potentially type-unsafe calls such as Insert() are implemented in two steps. –First virtual method OnInsert() is called. This method should check the validity of the inserted object (e.g. whether it belongs to the collection type). –If OnInsert() does not throw, CollectionBase proceeds with calling Insert() on the underlying ArrayList.

19. .NET 1.1 Collections Classes –DictionaryBase : IDictionary Provides the abstract base class for a strongly typed collection of key/value pairs. Internally DictionaryBase relies on a HashTable. The idea is similar to CollectionBase.

20. .NET 1.1 Collections Classes –System.Collections.Specialized Classes StringCollection StringDictionary NameObjectCollectionBase NameValueCollection OrderedDictionary ListDictionary HybridDictionary BitVector32

21. .NET 1.1 Collections Collection Operations –Comparison –Sort –Search –Convert –Synchronization

22. .NET 1.1 Collections Comparison –IComparable int CompareTo(object obj); Compares the current instance with another object of the same type The role of IComparable is to provide a method of comparing two objects of a particular type –IComparer int Compare(object x, object y); Compare two given objects The role of IComparer is to provide additional comparison mechanisms

23. .NET 1.1 Collections Sort –Sortable Collections: Array, ArrayList SortedList is sorted automatically, but you can specify the comperer in constructor Sort() – –use default comparer Sort(IComparer comparer) – –use specified comparer Sort(int index, int count, IComparer comparer) – –use specified comparer and sort the elements in a range

24. .NET 1.1 Collections Search –Searchable Collections: Array, ArrayList BinarySearch(object value) – –use default comparer BinarySearch(object value, IComparer comparer) – –use specified comparer BinarySearch(int index, int count, object value, IComparer comparer) – –use specified comparer and search the elements in a range

25. .NET 1.1 Collections Convert –Convert an ArrayList into an Array void CopyTo(Array array) void CopyTo(Array array, int arrayIndex) void CopyTo(int index, Array array, int arrayIndex, int count) object[] ToArray() Array ToArray(Type type) –Add an Array to an ArrayList void AddRange(ICollection c)

26. .NET 1.1 Collections Synchronization Hashtable ht = Hashtable.Synchronized(new Hashtable()); Synchronized() creates a synchronized (thread- safe) wrapper for the Hashtable: SynchHashtable which derives from Hashtable

27. .NET 2.0 Generic Collections

28. .NET 2.0 Collections Namespaces –Mscorlib.dll: System.Collections.Generic System.Collections.ObjectModel –System.dll: System.Collections.Generic

29. .NET 2.0 Collections Generic Mapping –Collection (not abstract) – CollectionBase –ReadonlyCollection (not abstract) - ReadonlyCollectionBase –List - ArrayList –Dictionary - Hashtable –Queue,Stack,SortedList - … –Several generic collection types that do not have nongeneric counterparts LinkedList is a general-purpose linked list that provides O(1) insertion and removal operations. SortedDictionary is a sorted dictionary with O(log n) insertion and retrieval operations, making it a useful alternative to SortedList. KeyedCollection is a hybrid between a list and a dictionary, which provides a way to store objects that contain their own keys.

30. .NET 2.0 Collections Benefits of Generics –Type-safe at compile-time –Better performance, no inbox/outbox –Maximize code reuse

31. .NET 2.0 Collections Generic Interfaces

32. .NET 2.0 Collections Generic Interfaces –Why doesn’t IList inherit from IList? Against Liscov Substitution Principle (LSP)! –Why does IEnumerable inherit from IEnumerable and IEnumerator inherit from IEnumerator? Their type parameters T are only used in “output” positions (return values), so that they are co- variant –Why does IEnumerator extend IDisposable? For the purpose to release resources at the end of the enumeration

33. .NET 2.0 Collections Collection & List –List is designed for speed and for use as an internal implementation detail you shouldn’t expose List, is because it exposes too many members, many of which are not relevant in most situations. List from a property won’t be able to get notified when the collection is modified. DoNotExposeGenericLists fires when you publicly expose List via a field, method, property, or parameter –Collection is designed for extensibility. Collection provides 4 overridable methods; ClearItems, InsertItem, RemoveItem and SetItem, which allow a derived class to be notified when a collection has been modified.

34. .NET 2.0 Collections Operations –Predicate delegate that allows you to specify methods for searching the list Find/Remove/Exists/TrueForAll –Action delegate that represents methods that act on each element of the list ForEach –Converter delegate that lets you define conversions between types ConvertAll

35. .NET 2.0 Collections Iterators –An iterator is a section of code that returns an ordered sequence of values of the same type. –An iterator can be used as the body of a method, an operator, or a get accessor. –The iterator code uses the yield return statement to return each element in turn. yield break ends the iteration. For more information –Multiple iterators can be implemented on a class. Each iterator must have a unique name just like any class member. –The return type of an iterator must be IEnumerable, IEnumerator, IEnumerable, or IEnumerator

36. .NET 2.0 Collections Iterators –Enables you to support foreach iteration in a class or struct without having to implement the entire IEnumerable interface!

37. .NET 3.0/3.5 Language Integrated Query

38. .NET 3.0 Collections Collection Initializers

39. .NET 3.5 Collections LINQ

40. .NET 3.5 Collections Namespaces –System.Core.dll: System.Linq

41. .NET 3.5 Collections LINQ to Objects –If performance to the application is not so critical, you can choose LINQ.

42. .NET 3.5 Collections LINQ Query Types –Method Syntax –Query Syntax – will be translated into Method Syntax by compiler

43. .NET 3.5 Collections Language Extensions to Support LINQ

44. .NET 3.5 Collections Extension Methods –Enable you to "add" methods to existing types without creating a new derived type, recompiling, or otherwise modifying the original type. –Extension methods are a special kind of static method, but they are called as if they were instance methods on the extended type. –An extension method will never be called if it has the same signature as a method defined in the type.

45. .NET 3.5 Collections Extension Methods for LINQ –Some extension methods on IEnumerable still return IEnumerable, that’s why they can be chained, which is similar to LISP. –public static IEnumerable Select ( –this IEnumerable source, –Func selector –) –public static IEnumerable Where ( –this IEnumerable source, –Func predicate –)

46. .NET 3.5 Collections Deferred Execution –Generally, LINQ uses deferred execution. Only the query is invoked (foreach) then it will be executed. –If you want to make it execute immediately, you can use ToList<>, ToArray<>, ToDictionary<> –var linqBooks = books.Where(book => book.Title.StartsWith("L")) –.OrderBy(book => book.Price) –.Select(book => new { key = book.Title, value = book.Price } –.ToList ();

47. .NET 4.0 Concurrent Collections

48. .NET 4.0 Collections Namespaces –Mscorlib.dll System.Collections.Concurrent –System.dll: System.Collections.Concurrent

49. .NET 4.0 Collections Concurrent Collections –The.NET Framework 4 introduces the System.Collections.Concurrent namespace, which includes several collection classes that are both thread-safe and scalable. –Multiple threads can safely and efficiently add or remove items from these collections, without requiring additional synchronization in user code.

50. .NET 4.0 Collections Concurrent Collections TypeDescription BlockingCollection Provides bounding and blocking functionality for any type that implements IProducerConsumerCollection. For more information, see BlockingCollection Overview. IProducerConsumerCollection BlockingCollection Overview ConcurrentDictionary Thread-safe implementation of a dictionary of key- value pairs. ConcurrentQueue Thread-safe implementation of a FIFO (first-in, first-out) queue. ConcurrentStack Thread-safe implementation of a LIFO (last-in, first-out) stack. ConcurrentBag Thread-safe implementation of an unordered collection of elements. IProducerConsumerCollection The interface that a type must implement to be used in a BlockingCollection.

51. .NET 4.0 Collections Concurrent Collections –Some of the concurrent collection types use lightweight synchronization mechanisms such as SpinLock, SpinWait, SemaphoreSlim, and CountdownEvent. –When wait times are expected to be very short, spinning is far less computationally expensive than waiting, which involves an expensive kernel transition. –Build-In supports for Producer-Consumer scenarios

52. .NET 4.0 Collections BlockingCollection

53. .NET 4.0 Collections Co-variant Support –A covariant interface allows its methods to return more derived types than those specified in the interface. –E.g: IEnumerable

54. .NET 4.0 Collections Contra-variant Support –A contravariant interface allows its methods to accept parameters of less derived types than those specified in the interface. –E.g: IComparable

55. Selecting a Collection Class Which one is best? –No definite best one but most appropriate. –Your choose should be based on your needs. –Be sure to choose your collection class carefully. Using the wrong type can restrict your use of the collection.

56. Selecting a Collection Class Consider the following questions: –Do you need a sequential list where the element is typically discarded after its value is retrieved? ◦If yes, consider using the Queue class or the Queue generic class if you need first-in- first-out (FIFO) behavior. Consider using the Stack class or the Stack generic class if you need last-in-first-out (LIFO) behavior. ◦If not, consider using the other collections. –Do you need to access the elements in a certain order, such as FIFO, LIFO, or random? ◦The Queue class and the Queue generic class offer FIFO access. ◦The Stack class and the Stack generic class offer LIFO access. ◦The LinkedList generic class allows sequential access either from the head to the tail or from the tail to the head. ◦The rest of the collections offer random access.

57. Selecting a Collection Class Consider the following questions (cont.): –Do you need to access each element by index? ◦The ArrayList and StringCollection classes and the List generic class offer access to their elements by the zero-based index of the element. ◦The Hashtable, SortedList, ListDictionary, and StringDictionary classes, and the Dictionary and SortedDictionary generic classes offer access to their elements by the key of the element. ◦The NameObjectCollectionBase and NameValueCollection classes, and the KeyedCollection and SortedList generic classes offer access to their elements by either the zero-based index or the key of the element. –Will each element contain one value, a combination of one key and one value, or a combination of one key and multiple values? ◦One value: Use any of the collections based on the IList interface or the IList generic interface. ◦One key and one value: Use any of the collections based on the IDictionary interface or the IDictionary generic interface. ◦One value with embedded key: Use the KeyedCollection generic class. ◦One key and multiple values: Use the NameValueCollection class.

58. Selecting a Collection Class Consider the following questions (cont.): –Do you need to sort the elements differently from how they were entered? ◦The Hashtable class sorts its elements by their hash codes. ◦The SortedList class and the SortedDictionary and SortedList generic classes sort their elements by the key, based on implementations of the IComparer interface and the IComparer generic interface. ◦ArrayList provides a Sort method that takes an IComparer implementation as a parameter. Its generic counterpart, the List generic class, provides a Sort method that takes an implementation of the IComparer generic interface as a parameter. –Do you need fast searches and retrieval of information? ◦ListDictionary is faster than Hashtable for small collections (10 items or fewer). The SortedDictionary generic class provides faster lookup than the Dictionary generic class. –Do you need collections that accept only strings? ◦StringCollection (based on IList) and StringDictionary (based on IDictionary) are in the System.Collections.Specialized namespace. ◦In addition, you can use any of the generic collection classes in the System.Collections.Generic namespace as strongly typed string collections by specifying the String class for their generic type arguments.

59. Selecting a Collection Class Consider Thread-Safe and Efficiency –Recommend the concurrent collections classes in the.NET Framework 4 because they provide not only the type safety of the.NET Framework 2.0 collection classes, but also more efficient and more complete thread safety than the.NET Framework 1.0 collections provide. –If you are only reading from a shared collection, then you can use the classes in the System.Collections.Generic namespace. –Recommend that you do not use 1.0 collection classes unless you are required to target the.NET Framework 1.1 or earlier runtime.

60. Relevant Concepts Interface Segregation Principle (ISP) Single Responsibility Principle (SRP) Iterator Pattern Explicit/Implicit Interface Implementations Generic Linq –Implicitly typed local variables –Object initializers –Lambda expressions –Extension methods –Anonymous types Thread Safety (Synchronization & Concurrent) Liscov Substitution Principle (LSP) Co-variant/Contra-variant

61. The End Thank You! Chen Dong 2010.6.24