Please elaborate with your own illustrations! Sets and Counting JA Ritcey EE 416 Revised Oct 5 2009 Please elaborate with your own illustrations!

Elementary Set Theory Universe – superset containing any element of interest. This is the complement of the empty set. Subset – collection of none, some, or all elements Set operations – complement, union, intersection Cardinality – the number of elements in a set, size Cardinality can be finite, countable, or uncountable Venn diagrams are useful pictorial representations Set operations can be carried out in MATLAB!

Set Operations U = { u_1, … , u_N } with card(U) = N Count by placing elements of U in one-to-one correspondence with the natural numbers You should be able to carry out set operations by taking unions, intersections, and complements. You should be able to handle both lists and geometric regions.

Disjoint or Mutually Exclusive Sets 2 sets a disjoint or mutually exclusive if they have no common elements disjoint means the intersection is the empty set Disjoint sets are simpler than those with overlap A collection of sets is mutually disjoint if no two intersect. Together they partition the union. Partition – a collection of disjoint subsets that is complete and together yields the entirety

Partitions A set S is partitioned by a collection of disjoint subsets that together encompass all of S The days of the week are partitioned into work days and weekend The numbers {1,2,3, …} are partitioned into odd and even Partitions are used to when something is held invariant over each subset

Sensor Nodes Partitioned by Radius from Acces Point

Cartesian Products Given 2 sets A = { a_i } B = { b_j } create the Ordered pair c = (a_i, b_j ) C = AxB is called the Cartesian product |C| = |AxB| = |A|x|B| . This is called the multplication principle in counting (combinatorics) Select one from column A and 1 from column B This can be extended to ordered J-tuples C= A_1 x … x A_J the J-fold product A vector can be created this way

Power Sets Power Set Pwr(S) is the set of all subsets of any S If a set S is finite, say |S| = N, then |Pwr(S)| = 2^N To see this equate each member of Pwr(S) with the binary string (b_1, b_2, … , b_N ). The b_i in (0,1) label whether element s_i is in the particular subset. There are 2^N binary strings of length N If the cardinality, or size, of A is countable, the power set has uncountable cardinality

Properties of the Power Set Pwr(S) has some closure properties A collection of sets C is closed under set operations: (1) If A in C, A^c in C (closed under complements) (2) If A_1, …A_N in C, their union is in C Often this last property is extended to countable unions. If A_1, A_2,… in C, their union is in C Collections of sets with this property are called algebras and sigma-algebras. When S is uncountable Pwr(S) is too big to assign probabilities to everything – a subtle point

Generating some small power sets function powerset(n); %function powerset(n); n1=n-1; for knt = 0:2^n1-1 C= bitget(knt,n1:-1:1); disp([knt C]) end 0 0 0 0 0 1 0 0 0 1 2 0 0 1 0 3 0 0 1 1 4 0 1 0 0 5 0 1 0 1 6 0 1 1 0 7 0 1 1 1 8 1 0 0 0 9 1 0 0 1 10 1 0 1 0 11 1 0 1 1 12 1 1 0 0 13 1 1 0 1 14 1 1 1 0 15 1 1 1 1

Counting Finite Sets We have already done some counting, using the Multiplications Principle Cartesian product card |C| = |AxB| = |A| |B| All pairs must be possible The size of A and B can be different, but finite There are 52 = 4x13 cards in a deck There are 100 = 2x50 US Senators Lets call a set S an N-set when |S|=N

Ordered Selection w/ Repetition Given a base N-set S In how many ways can we select k objects Assume an ordered selected With replacement. So that repetition is possible We can do this is N^k ways by the multiplication principle Just equate the selection of an object with an ordered k-tuple The elements are independently selected and each drawn from an N-set (repetitions)

Tabular View Repetition With Without W/ Order N^k N!/(N-k)! WO/ Order (N+k-1)!/ (N-1)! k! N!/(N-k)!k! (N+k-1 choose k) (N choose k) It is easier to remember these, when they are considered as a group

Ordered Selection W/Replacement Each Selection is a row Possible Choices along columns 7^12 = 7 x7 …x7 Cartesian Product Illustrates the Multiplication Principle 1 2 3 4 5 6 7 8 9 10 11 12

Examples Cards = 4 suits x 13 face values = 52 cards You can place r balls into n cells in n^r ways n^r = n x n … x n Harder: How many ways can r flags be put on n flagpoles? Ans: n x (n+1) x (n+2) x … x (n+r-1) Why: The second flag can be put above or below the first. So the number of choices increases

Ordered Selection wo/ Repetition Given a base N-set S In how many ways can we select k objects Assume an ordered selected without replacement. So that repetition is not possible We can do this is (N)_k = N(N-1) …(N-k+1) ways, again by the multiplication principle Just equate the selection of an object with an ordered k-tuple The elements are each drawn from sets of decreasing size When k=N, we can select in N! ways

Tabular View Repetition With Without W/ Order N^k N!/(N-k)! WO/ Order (N+k-1)!/ (N-1)! k! N!/(N-k)!k! (N+k-1 choose k) (N choose k) It is easier to remember these, when they are considered as a group

Factorial Interpretation The factorial n! = n(n-1)(n-2) …(3)(2)(1) 0!=1 It gets very big very fast, n! asy sqrt(2pi)n^(n+1/2)exp(-n) Then (N)(N-1)…(N-k+1) = N! / (N-k)! Best ways to compute the factorial is through the Gamma Function (gamma.m) More later. You can permute (reorder) an N-tuple in N! ways

Factorial Growth

Stirling’s Formula

Matlab M-File functioncompareGamma(M); m = [1:M]';half=1/2; fact = gamma(m+1); plot( m, fact, 'b*-');grid title('m! = Gamma(m+1) - Grows very fast'); xlabel('m');ylabel('G(m)'); %try logs to compress the dynamic range figure(2) subplot(121); stir = sqrt(2*pi).*m.^(m+half).*exp(-m); semilogy( m, fact, 'b*-',m,stir,'ro');grid subplot(122); RE = abs(stir-fact)./fact; semilogy(m,RE,'r*');grid title('Relative Error'); figure(1)

Unordered selection wo/ Repetition What if we select without regard to the order. Solve this by reconsidering ordered selection wo/ replacement of k objects from N-set Each such selection resulted a k-tuple that indexed the selection a_i is that object selected on the i^th turn But a k-tuple can be reordered (permuted) in k! ways Therefore we need to reduce our result to N(N-1)…(N-k+1)/k! = N! / (N-k)! K! Combinations - binomial coefficient ( N choose k )

Tabular View Repetition With Without W/ Order N^k N!/(N-k)! WO/ Order (N+k-1)!/ (N-1)! k! N!/(N-k)!k! (N+k-1 choose k) (N choose k) It is easier to remember these, when they are considered as a group

Computing Binomial Coefficient { n choose k } = n! / (n-k)! k! But n! = gamma(n+1) = G(n+1) (gamma.m) { n choose k } = G(n+1)/G(n+1-k) G(k+1) Even better to use loggamma = log(Gamma(m)) { n choose k } = exp( gammln(n+1) -gammaln(n+1-k) -gammaln(k+1) )

More Binomial Coefficient binomialcoef(n) Think (1+x)^n. The coefficients: 1 1 1 2 1 1 3 3 1 1 3 5 3 1 1 4 9 9 4 1 1 6 15 19 15 6 1 Computation – recursion, approximate, logGamma

Matlab Code function binomialcoef(n); %generates an array of binomial coefficients k = [0:n]; lgGn = gammaln(n+1); lgGnk = gammaln(n+1-k); lgGk = gammaln(k+1); bico = exp( lgGn -lgGnk -lgGk ); bico = floor(bico); fprintf('%5.0f', bico);

Unordered w/ Repetition This is the most complicated situation so far Thinks for a long time, then come up with a Clever way to represent a selection bars & balls 4 objects choose 2  o||o| select object 1 and 3 Another choice is |||oo select object 4 twice To select k objects from N-set Use N-1 bars and k balls  string of |||oo But these can be drawn in ( N+k-1 choose k) ways (N+k-1)!/(N-1)!k!

Tabular View Repetition With Without W/ Order N^k N!/(N-k)! WO/ Order (N+k-1)!/ (N-1)! k! N!/(N-k)!k! (N+k-1 choose k) (N choose k) It is easier to remember these, when they are considered as a group