The Amazing Year: Münster (Germany), Covington (KY), Bethlehem (PA), and Huntingdon Gerald Kruse, Ph.D. Associate Professor of Computer Science and Mathematics

Faculty Exchange at Fachhochschule (FH) Münster, during the fall 2006 semester. Note: FH-M is also known as the Münster University of Applied Sciences, or MUAS. It’s Engineering and Technology campus is located in Burgsteinfurt, about 30 miles outside Münster. Taught Algorithmen und Data Strukturen… aka Algorithms and Data Structures. It is between Juniata’s “CS 2” (CS 240) and “Algorithms,” CS 315. Taught Graphical Programming, an one-semester course in Computer Graphics. Sabbatical in Huntingdon, PA, during the spring 2007 semester, exploring modifications to MA 103, Quantitative Methods.

Some mathematics… Let: F = one semester faculty exchange S = one semester sabbatical P = full pay for an academic year F + S = P

We hope to encourage Juniata students to study at FH-Münster by having a faculty exchange first Faculty Gerald Kruse Thomas Weik Gerald Kruse Thomas Weik Juniata StudentsFH-Münster Students Tim Auman, Mike LinkRobin Segglemann, Frank Volkmer Sascha Hlusiak, Morin Ostkamp Moritz Prinz, Thomas Beerman

Departing Huntingdon on the Train

“Our” House in the Village of Leer

Riding Bikes into the Village of Leer

First Day of “Grundschule”

Fuchsjagd (or “Foxhunt” in English)

St. Nicholas

To read more about our travel exploits… If you read our travel blog, you will have the following questions answered: 1. Is it a good idea to have non-prescription medical supplements shipped from the United States to Germany? 2. Can you get a speeding ticket on the autobahn? 3. How bad is “inadvertently” using Juniata’s Pcard to purchase tickets at Legoland ? 4. What percentage of Germans visit Majorca?

Juniata Students For Dinner

A Day in the Life Monday, Tuesday, and Wednesday 06:37 Bus from Leer to Burgsteinfurt Lecture from 08:00 to 09:45 Lab from 10:00 to 13:00 14:51 Bus from Burgsteinfurt to Leer Thursday and Friday Breakfast for the family 08:37 Bus from Leer to Burgsteinfurt Course prep, research, etc. 13:51 Bus from Burgsteinfurt to Leer Afternoon Activities with the Family: Sporthalle, Horseback Riding, Bike Rides, Fußball

The Computer Lab

Observations and Reflections Very little homework to grade… If the student completes all labs, they are, eligible to take the final exam The students were amazing No committee work, a little advising Web-based (foreshadowing to sabbatical) “Psst, don’t share this with my colleagues at Juniata, they might be jealous…”

The Perfect Storm Unscheduled time at the end of the week An Algorithms course filled with eager students, who happen to use a variety of computers and compilers A troublesome/intriguing timing result A problem with a pleasing blend of Mathematics and Computer Science

How Fast is my Sorting Algorithm?  The Sorting Problem, from Cormen et. al. Input: A sequence of n numbers, (a 1, a 2, … a n ) Output: A permutation (reordering) (a 1 ’, a 2 ’, … a n ’) of the input sequence such that a 1 ’ ≤ a 2 ’ ≤ … ≤ a n ’ Note: This definition can be expanded to include sorting primitive data such as characters or strings, alpha-numeric data, and data records with key values.  Sorting algorithms are analyzed using many different metrics: expected run-time, memory usage, communication bandwidth, implementation complexity, … we chose expected run-time  Expected running time is given using “Big-O” notation O( g(n) ) = { f(n): pos. constants c and n 0 s.t. 0 ≤ f(n) ≤ c*g(n) n ≥ n 0 }. While O-notation describes an asymptotic upper bound on a function, it is frequently used to describe asymptotically tight bounds.

Conceptually, a heap can be thought of as a complete binary tree: Heaps

Conceptually, a heap can be thought of as a complete binary tree: Heaps But in practice, heaps are usually implemented as arrays. Notice how 8, 2, and 4 are near each other in the tree, but relatively far apart in the array… By the way, e-Bay uses a “heap-like” data structure to track bids A =

Heapsort Heapsort(A){BuildHeap(A); for (i = length(A) downto 2) { Swap(A[1], A[i]); heap_size(A) -= 1; Heapify(A, 1); }} When the heap property is violated at just one node (which has sub- trees which are valid heaps), Heapify “floats down” the parent node to fix the heap. Remembering the tree structure of the heap, each Heapify call takes O(lg n) time. Since there are n – 1 calls to Heapify, Heapsort’s expected execution time is O(n lg n), just like Quicksort.

Timing Results

Counting Comparisons

A Useful Case-Study in Algorithmic Experimentation: Unexpected Timing Results for Heapsort Dr. Gerald Kruse, Ph.D. Associate Professor of Mathematics and Computer Science / Information Technology Juniata College, Huntingdon, Pennsylvania, USA A Useful Case-Study in Algorithmic Experimentation: Unexpected Timing Results for Heapsort Dr. Gerald Kruse, Ph.D. Associate Professor of Mathematics and Computer Science / Information Technology Juniata College, Huntingdon, Pennsylvania, USA Introduction The study of algorithms, and sorting algorithms in particular, is in the core of the Computer Science curriculum. Detailed asymptotic analysis of the classic sorting algorithms is typically the first in-depth exposure of a student to Big-Oh notation. It is instructive for students to implement these sorting algorithms and experimentally confirm their theoretically derived asymptotic behavior. Heapsort is a well-known sorting algorithm and it has O(n*lg n) expected asymptotic performance. However, its implementation may generate unexpected results. Interestingly, the ratio of the expected running time to the actual running time should converge to a constant as n increases, however occasionally it does not. Timing an implementation of Heapsort for a range of problem sizes and data values can reveal compiler idiosyncrasies and difficulties with large input. Introduction The study of algorithms, and sorting algorithms in particular, is in the core of the Computer Science curriculum. Detailed asymptotic analysis of the classic sorting algorithms is typically the first in-depth exposure of a student to Big-Oh notation. It is instructive for students to implement these sorting algorithms and experimentally confirm their theoretically derived asymptotic behavior. Heapsort is a well-known sorting algorithm and it has O(n*lg n) expected asymptotic performance. However, its implementation may generate unexpected results. Interestingly, the ratio of the expected running time to the actual running time should converge to a constant as n increases, however occasionally it does not. Timing an implementation of Heapsort for a range of problem sizes and data values can reveal compiler idiosyncrasies and difficulties with large input. References “Computing Curricula 2001: Computer Science,” a joint report from IEEE – Computer Society and the Association for Computing Machinery. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, “Introduction to Algorithms, Second Edition,” Cambridge, MA/London, England: The MIT Press/McGraw-Hill, N. Dale, C. Weems, D. T. Joyce, “Object-Oriented Data Structures Using Java,” Boston, MA: Jones and Bartlett, M. T. Goodrich and R. Tamassia, “Algorithm Design: Foundation, Analysis, and Internet Examples,” Wiley: New York: D. E. Knuth, “The Art of Computer Programming, Volume 3: (Second Edition) Sorting and Searching,” Addison-Wesley-Longman: Redwood City, CA, C. C. McGeoch, “Analyzing algorithms by simulation: Variance reduction techniques and simulation speedups,” ACM Computing Surveys, vol. 24, no. 2, pp. 195 – 212, C. C. McGeoch, D. Precup, and P. R. Cohen, “How to find the Big-Oh of your data set (and how not to),” Advances in Intelligent Data Analysis, vol of Lecture Notes in Computer Science, pp. 41 – 52, Springer-Verlag, R. Sedgewick, “Algorithms in C, Parts 1-4: Fundamentals, Data Structures, Sorting, Searching, Third Edition,” Addison-Wesley: Boston, MA, 1997 References “Computing Curricula 2001: Computer Science,” a joint report from IEEE – Computer Society and the Association for Computing Machinery. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, “Introduction to Algorithms, Second Edition,” Cambridge, MA/London, England: The MIT Press/McGraw-Hill, N. Dale, C. Weems, D. T. Joyce, “Object-Oriented Data Structures Using Java,” Boston, MA: Jones and Bartlett, M. T. Goodrich and R. Tamassia, “Algorithm Design: Foundation, Analysis, and Internet Examples,” Wiley: New York: D. E. Knuth, “The Art of Computer Programming, Volume 3: (Second Edition) Sorting and Searching,” Addison-Wesley-Longman: Redwood City, CA, C. C. McGeoch, “Analyzing algorithms by simulation: Variance reduction techniques and simulation speedups,” ACM Computing Surveys, vol. 24, no. 2, pp. 195 – 212, C. C. McGeoch, D. Precup, and P. R. Cohen, “How to find the Big-Oh of your data set (and how not to),” Advances in Intelligent Data Analysis, vol of Lecture Notes in Computer Science, pp. 41 – 52, Springer-Verlag, R. Sedgewick, “Algorithms in C, Parts 1-4: Fundamentals, Data Structures, Sorting, Searching, Third Edition,” Addison-Wesley: Boston, MA, 1997 Sequential Heapsort (in C), from Sedgewick’s text fixDown(Item a[], int k, int N) { int j; while (2*k <= N) { j = 2*k; if (j < N && less(a[j], a[j+1])) j++; if (!less(a[k], a[j])) break; exch(a[k], a[j]); k = j; } #define pq(A) a[l-1+A] void heapsort(Item a[], int l, int r) { int k, N = r-l+1; for (k = N/2; k >= 1; k--) fixDown(&pq(0), k, N); while (N > 1) { exch(pq(1), pq(N)); fixDown(&pq(0), 1, --N); } } Sequential Heapsort (in C), from Sedgewick’s text fixDown(Item a[], int k, int N) { int j; while (2*k <= N) { j = 2*k; if (j < N && less(a[j], a[j+1])) j++; if (!less(a[k], a[j])) break; exch(a[k], a[j]); k = j; } #define pq(A) a[l-1+A] void heapsort(Item a[], int l, int r) { int k, N = r-l+1; for (k = N/2; k >= 1; k--) fixDown(&pq(0), k, N); while (N > 1) { exch(pq(1), pq(N)); fixDown(&pq(0), 1, --N); } } The Experiment Fill the unsorted array with random double values for (i=1; i<=N; i++) { unsortedArray[i] = (double)rand(); } Time the loop which copies and sorts for each of numIter iterations start = clock(); for (j=0; j<numIter; j++) { for (i=1; i<=N; i++) a[i] = unsortedArray[i]; heapsort(); } finish = clock(); sort_with_copy = (double(finish)-double(start))/CLOCKS_PER_SEC; Time the loop which only copies for each of numIter iterations start = clock(); for (j=0; j<numIter; j++) { for (i=1; i<=N; i++) a[i] = unsortedArray[i]; } finish = clock(); copy_only = (double(finish)-double(start))/CLOCKS_PER_SEC; Calculate the time to sort the array once sort_time = (sort_with_copy - copy_only)/numIter; The Experiment Fill the unsorted array with random double values for (i=1; i<=N; i++) { unsortedArray[i] = (double)rand(); } Time the loop which copies and sorts for each of numIter iterations start = clock(); for (j=0; j<numIter; j++) { for (i=1; i<=N; i++) a[i] = unsortedArray[i]; heapsort(); } finish = clock(); sort_with_copy = (double(finish)-double(start))/CLOCKS_PER_SEC; Time the loop which only copies for each of numIter iterations start = clock(); for (j=0; j<numIter; j++) { for (i=1; i<=N; i++) a[i] = unsortedArray[i]; } finish = clock(); copy_only = (double(finish)-double(start))/CLOCKS_PER_SEC; Calculate the time to sort the array once sort_time = (sort_with_copy - copy_only)/numIter; Recursive Heapsort (in C++), from Cormen, et. al. text void maxheapify(int i) { int l, r, largest; double temp; l = left(i); r = right(i); if (l a[i]) { largest = l; } else { largest = i; } if (r a[largest]) largest = r; if (largest !=i) exch(a[i], a[largest]); } void buildheap() { int i; heapsize = N; for (i=heapsize/2; i>=1; i--) maxheapify(i); } void heapsort() { int i; double temp; buildheap(); for (i=N; i>=2; i--) { exch(a[i], a[largest]); heapsize--; maxheapify(1); } Recursive Heapsort (in C++), from Cormen, et. al. text void maxheapify(int i) { int l, r, largest; double temp; l = left(i); r = right(i); if (l a[i]) { largest = l; } else { largest = i; } if (r a[largest]) largest = r; if (largest !=i) exch(a[i], a[largest]); } void buildheap() { int i; heapsize = N; for (i=heapsize/2; i>=1; i--) maxheapify(i); } void heapsort() { int i; double temp; buildheap(); for (i=N; i>=2; i--) { exch(a[i], a[largest]); heapsize--; maxheapify(1); } Suggestions for Use as a Case-Study 1. Have the students implement and time several sorts, with a variety of data (ascending, descending, random). 2. Give the students time in class, preferably in a lab, working with the test driver. This helps promote the significance of calling the sort multiple times, as well as copying the unsorted data into the structure to be sorted before each sort. 3. Encourage the “open-endedness” of this type of study. My assignments typically specify that students must explore some additional factors beyond the standard “best-case” and “worst-case” data. At various points during lecture I typically suggest possible topics for the students to pursue. 4. I still don’t have a definitive answer for the growth in the Heapsort ratio using running time. Have any of you encountered the same results? Suggestions for Use as a Case-Study 1. Have the students implement and time several sorts, with a variety of data (ascending, descending, random). 2. Give the students time in class, preferably in a lab, working with the test driver. This helps promote the significance of calling the sort multiple times, as well as copying the unsorted data into the structure to be sorted before each sort. 3. Encourage the “open-endedness” of this type of study. My assignments typically specify that students must explore some additional factors beyond the standard “best-case” and “worst-case” data. At various points during lecture I typically suggest possible topics for the students to pursue. 4. I still don’t have a definitive answer for the growth in the Heapsort ratio using running time. Have any of you encountered the same results? Analyzing Timing Results Using the Ratio-Test Here we are attempting to verify previously derived expected asymptotic behavior. This is a much simpler problem than attempting to experimentally determine asymptotic behavior. Since we know the expected running time of Heapsort, we simply plot the ratio of the actual running time, T(n) to this expected running time, n*lg n. If this ratio converges to a constant, we can conclude that our implementation exhibits the theoretical expected running time [McGeoch (2)]. The both the recursive and sequential versions of the Heapsort algorithm were implemented in C, C++, and Java, on the Windows and Unix/Solaris operating systems. In addition, various levels of compiler optimizations were tested. Analyzing Timing Results Using the Ratio-Test Here we are attempting to verify previously derived expected asymptotic behavior. This is a much simpler problem than attempting to experimentally determine asymptotic behavior. Since we know the expected running time of Heapsort, we simply plot the ratio of the actual running time, T(n) to this expected running time, n*lg n. If this ratio converges to a constant, we can conclude that our implementation exhibits the theoretical expected running time [McGeoch (2)]. The both the recursive and sequential versions of the Heapsort algorithm were implemented in C, C++, and Java, on the Windows and Unix/Solaris operating systems. In addition, various levels of compiler optimizations were tested. For all these experiments, the ratios are clearly increasing, rather than approaching a constant value. Why? For dramatically large n we could attribute this to the data being “out-of-core” memory. The increase in ratio for these relatively small values of n is probably also explained as being caused by cache memory effects. It is interesting to note that this happens irregardless of platform, language, or version of the Heapsort algorithm. For all these experiments, the ratios are clearly increasing, rather than approaching a constant value. Why? For dramatically large n we could attribute this to the data being “out-of-core” memory. The increase in ratio for these relatively small values of n is probably also explained as being caused by cache memory effects. It is interesting to note that this happens irregardless of platform, language, or version of the Heapsort algorithm. So, is Heapsort really O(n*lg n) ? Yes. We demonstrate this in the following graph, where rather than measuring running time, T(n), the number of key comparisons were counted. The ratio plotted is therefore: # comparisons / (n*lg n) The ratios here do appear to converge to a constant as n increases. So, is Heapsort really O(n*lg n) ? Yes. We demonstrate this in the following graph, where rather than measuring running time, T(n), the number of key comparisons were counted. The ratio plotted is therefore: # comparisons / (n*lg n) The ratios here do appear to converge to a constant as n increases.

Future Work For very large n, we would expect a slowdown for ANY algorithm as the data no longer fits in memory, but it would look like a step function at each layer of memory, not the gradual growth Heapsort exhibits. For very large n, we would expect a slowdown for ANY algorithm as the data no longer fits in memory, but it would look like a step function at each layer of memory, not the gradual growth Heapsort exhibits. Consider the memory access patterns of Heapsort, and attempt to understand, and possibly simplify, the mathematic characterization. No one has really “fixed” the algorithm, either. Explore modifications to the RAM model used in theoretical analysis Explore modifications to the RAM model used in theoretical analysis This is a fun exploration for students, appealing to those with an interest in the mathematics or computer science, look to expand into a student research project

Quantitative Literacy at Juniata Juniata has had “Quant-Math” and “Quant-Stat” skill requirement for graduation since the mid-1990’s. From the Juniata Catalog: Quantitative Skills To demonstrate quantitative literacy, students have three options: (1) complete a "Q" course; (2) complete a mathematical course (QM) and a statistics course (QS); (3) pass a proficiency exam.

Oh, and just what is Quantitative Literacy? “The ability to use numbers and data analysis in everyday life.” Bernard Madison, Univ. of Arkansas “..knowing how to reason and think, and it is all but absent from our curricula today.” Gina Kolata, NY Times “Having comfort with arithmetic, data analysis, computing, modeling, statistics, chance/probability, and reasoning.” Excerpt from Mathematics in Democracy. While a course in quantitative literacy might focus on practical, real-world problems, it still provides the students with a strong mathematical foundation.

MA 103, Quantitative Methods, aka “QM” MA 103, Quantitative Methods, was developed by Sue Esch to serve students who do not have courses with quantitative components in their POE’s. MA 103 is one of the few courses which satisfies both the “QM” and “QS” skills. A large percentage of students at Juniata satisfy their “Q” graduation requirement by taking MA 103, Quantitative Methods (5 sections per year).

Time for a change… From 1996 to 2007, the text used in MA 103 was Quantitative Methods, notes written and maintained by Sue Esch (Bukowski and Kruse added as co-authors later), and produced on campus. Students used two full-feature software packages: Minitab for statistics, and Maple for mathematics. Students used two full-feature software packages: Minitab for statistics, and Maple for mathematics. MA 103 was one of my favorite courses to teach, but I realized that after 10 years it was due for an update.

Search Parameters Published texts preferred Excel-based technology preferred Activity-based Many texts considered, three seriously Frequent consultation with Math department colleagues

And the Winner Is… Chosen Text: Quantitative Reasoning, by Alicia Sevilla and Kay Somers, from Moravian College.

Textbook Highlights Active learning approach Technology informs and enhances the math Modules on Apportionment and Conditional Probability Fall 2007 Student Comment Regarding the Textbook: “Despite being outside my major and one of those required courses people are supposed to hate, I loved this course… The textbook for the course was one of the best I've had at the school -- it was easy to understand, concise, and the assignments taught the material well.”

MAA PREP Workshop at Moravian Shared all my course materials

Course Highlights Pre- and post-assessments of student skills and attitudes Open-ended projects Paper-reduced (assignments posted online, deliverables uploaded) Provided my Math department colleagues with: daily schedule daily notes suggested homework problems solutions to all Activities Web-site evaluation module with Reference Librarian Course web-site:

Fall 2007 Comments “I feel that the methods used in this course were very effective in teaching the course subject matter. Using class time for both lecture and working on assignments was great - we apply what we learned right away and help was readily available when we needed it.” “I thought that the instruction and layout for this course were excellent. I was very nervous about taking a math course, and I had heard bad things about QM from past students, but it must be the fact that the course was revamped that made it so much better.” “I really liked political psychology…”

Fall 2007 Comments, cont. “This was a very good class for me. I am not very confident in my math abilities and this class was a nice way for me to gradually get into college level math. This class did (not) cause any really serious stress…” “I think that this course was really good. Math is not really my favorite subject and I was a little disappointed that I would have to take one for the FISHN requirements, but Professor Kruse did an excellent job. I really feel like I learned a lot of valuable things in his class and he made math really fun. Instead of it being numbers and ideas that I felt like I would never use, I feel like I can take and apply everything he taught us to real life.”

Additional Accomplishments At-large member of International Education Committee FH-Muenster Kolloquium: “Google’s Billion Dollar Eigenvector” SIGCSE Special Session: “A Status Report from the Committee to Evaluate Models of Faculty Scholarship” MAA (Allegheny Mountain Section) Talk: “Are Quicksort and Heapsort Really O(n*lg n)?”

Additional Accomplishments II “Published” in si.com: Jerry of Huntingdon, Pa., Has presented Andrew with such a lengthy resume that the head of our department wanted to hire him. No, Jerry says, just answer one question for me. And what might that be, your Lordship? Will everyone look at what Indy did to the Bears in the Super Bowl, which seemed to mirror what Arizona did to them, which is attacking the Tampa Two defense with underneath stuff, and go at them the same way? Oh sure, they all do that. It's a copycat league. Chicago will look at it, too, and they'll be playing it differently next year. The Cards attacked the Bear nickel and dime, which were vulnerable at that stage of the game. Indy? Superior personnel played a great role in this as well. Joseph Addai was simply terrific, better than the guys trying to tackle him, no matter what defense they were in. Thanks for your compliments, Jerry, and may I leave you with these parting words. Never underrate the matchup of personnel. It's more important than which attack plugs into which defense.

Thank You! Any Questions?

Bibliography T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, “Introduction to Algorithms, Second Edition,” Cambridge, MA/London, England: The MIT Press/McGraw-Hill, N. Dale, C. Weems, D. T. Joyce, “Object-Oriented Data Structures Using Java,” Boston, MA: Jones and Bartlett, M. T. Goodrich and R. Tamassia, “Algorithm Design: Foundation, Analysis, and Internet Examples,” Wiley: New York: D. E. Knuth, “The Art of Computer Programming, Volume 3: (Second Edition) Sorting and Searching,” Addison-Wesley-Longman: Redwood City, CA, C. C. McGeoch, “Analyzing algorithms by simulation: Variance reduction techniques and simulation speedups,” ACM Computing Surveys, vol. 24, no. 2, pp. 195 – 212, C. C. McGeoch, D. Precup, and P. R. Cohen, “How to find the Big-Oh of your data set (and how not to),” Advances in Intelligent Data Analysis, vol of Lecture Notes in Computer Science, pp. 41 – 52, Springer-Verlag, R. Sedgewick, “Algorithms in C, Parts 1-4: Fundamentals, Data Structures, Sorting, Searching, Third Edition,” Addison-Wesley: Boston, MA, 1997