BINF 634 Fall Lec 11 Visualization1 Plan for Tonight A synopsis of our progress through the course Lecture 11 Visualization in PERL Lab 2 Based in Part on Previous Lecture of J. Grefenstette Outline

BINF 634 Fall Lec 11 Visualization2 Where We are and Where We are Going Program 1, Program 2, turned in and graded Midterm turned in and graded Program 3 turned in, Program 4 assigned tonight Program 4 due in two week (11/25/13) Quiz 1, Quiz 2, Quiz 3, Quiz 4, lab1 turned in and graded Lab 2 assigned tonight (11/11/13) Lab 2 Due next week 11/18/13 Take home final provided 11/25/13 Take home final due to me 12/16/13 Miles to go …

BINF 634 Fall Lec 11 Visualization3 Data Visualization Numbers are well and good, but often we get more insight by visualizing data As always, it's a good idea to break the process down into individual steps: data analysis script => output file => plotting script => chart file This can all be wrapped for Web access: CGI script to get user input => data analysis script => output file => plotting script => chart file => CGI script displays chart on web page Viz strategy Based in Part on Previous Lecture of J. Grefenstette

BINF 634 Fall Lec 11 Visualization4 GD::Graph Created by Martien Verbruggen GD::Graph is a Perl module to create charts using the GD module. The following classes for graphs with axes are defined: GD::Graph::lines -- Create a line chart GD::Graph::bars and GD::Graph::hbars -- Create a bar chart with vertical or horizontal bars. GD::Graph::points -- Create an chart, displaying the data as points. GD::Graph::linespoints -- Combination of lines and points. GD::Graph::area -- Create a graph, representing the data as areas under a line. GD::Graph::pie -- Create a pie chart. GD Based in Part on Previous Lecture of J. Grefenstette

#!/usr/bin/perl use strict; use GD::Graph::bars; # File: bargraph.pl (based on # All data arrays should have the same number of entries = ("Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"); = (23, 5, 2, 20, 11, 33, 7, 31, 77, 18, 65, 52); # create a 2-dimensional array, where row 1 is x and row 2 is y = #create a new bar graph object and give it size (in pixels) my $graph = GD::Graph::bars->new(500, 300); # set graph features such as labels on the axes, title $graph->set( x_label=>'Month', y_label=>'Number of Hits', title=>'Number of Hits in Each Month in 2002') or warn $graph->error; # plot the data to create an image object my $image = or die $graph->error; # print the image as a PNG file format open IMG, ">hist.png" or die "Can't open hist.png\n"; print IMG $image->png; exit; 5BINF 634 Fall Lec 11 Visualization GD Based in Part on Previous Lecture of J. Grefenstette

BINF 634 Fall Lec 11 Visualization6 Output % open hist.png GD Based in Part on Previous Lecture of J. Grefenstette

#!/usr/bin/perl use strict; use CGI qw(:standard); use GD::Graph::bars; # File: bargraph.cgi # All data arrays should have the same number of entries = ("Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"); = (23, 5, 2, 20, 11, 33, 7, 31, 77, 18, 65, 52); # create a 2-dimensional array, where row 1 is x and row 2 is y = #create a new bar graph object and give it size (in pixels) my $graph = GD::Graph::bars->new(500, 300); # set graph features such as labels on the axes, title $graph->set( x_label=>'Month', y_label=>'Number of Hits', title=>'Number of Hits in Each Month in 2002') or warn $graph->error; # plot the data to create an image object my $image = or die $graph->error; # print the image as a PNG file format print "Content-type: image/png\n\n"; print $image->png; 7 BINF 634 Fall Lec 11 Visualization Let’s create a web version ! GD Based in Part on Previous Lecture of J. Grefenstette

8 BINF 634 Fall Lec 11 Visualization GD Based in Part on Previous Lecture of J. Grefenstette

#!/usr/bin/perl use strict; use GD::Graph::bars; # File: bargraph2.pl # read a file to get the data: = (); = (); open FH, "hist.dat" or die; while ( ) { my ($xval, $yval) = split; $xval; $yval } close FH; # create a 2-dimensional array, where row 1 is x and row 2 is y = #create a new bar graph object and give it size (in pixels) my $graph = GD::Graph::bars->new(500, 300); # set graph features such as labels on the axes, title $graph->set( x_label=>'Month', y_label=>'Number of Hits', title=>'Number of Hits in Each Month in 2002') or warn $graph->error; # plot the data to create an image object my $image = or die $graph->error; # print the image as a PNG file format open IMG, ">hist.png" or die "Can't open hist.png\n"; print IMG $image->png; 9 BINF 634 Fall Lec 11 Visualization Plotting points from a file. GD Based in Part on Previous Lecture of J. Grefenstette

BINF 634 Fall Lec 11 Visualization10 % cat hist.dat Jan23 Feb5 Mar2 Apr20 May11 Jun33 Jul7 Aug31 Sep77 Oct18 Nov65 Dec52 GD Based in Part on Previous Lecture of J. Grefenstette

#!/usr/bin/perl use strict; use GD::Graph::linespoints; # File: plot.pl = (); = (); # read a file to get the data: open FH, "hist.dat" or die; while ( ) { my ($xval, $yval) = split; $xval; $yval } close FH; # create a 2-dimensional array, where row 1 is x and row 2 is y = #create a new bar graph object and give it size (in pixels) my $graph = GD::Graph::linespoints->new(500, 300); # set graph features such as labels on the axes, title $graph->set( x_label=>'Month', y_label=>'Number of Hits', title=>'Number of Hits in Each Month in 2002') or warn $graph->error; # plot the data to create an image object my $image = or die $graph->error; # print the image as a PNG file format open IMG, ">out.png" or die "Can't open out.png\n"; print IMG $image->png; 11BINF 634 Fall Lec 11 Visualization Creating a plot with lines and points. GD Based in Part on Previous Lecture of J. Grefenstette

BINF 634 Fall Lec 11 Visualization12 GD Based in Part on Previous Lecture of J. Grefenstette

Getting the Chart on the Web This can all be wrapped for Web access: CGI script to get user input (e.g., wrapper) => data analysis script => output file => plotting script (e.g., plot.pl) => chart file (e.g., out.png)  CGI script (wrapper) displays chart on web page To display a file on the web, it has to be in a readable directory in the ~/public_html directory tree 1. Create a directory that can be written to by the web server process: % mkdir ~/public_html/cgi-images % chmod 777 ~/public_html/cgi-images 13BINF 634 Fall Lec 11 Visualization GD Based in Part on Previous Lecture of J. Grefenstette

Getting the Chart on the Web 2. In the wrapper, create the graphics file in the working directory 3. Copy it to the images directory 4. Display it using print img{src=>...} use CGI qw(:standard);... # change to working directory mkdir $dir or die "Can't create directory $dir\n"; chdir $dir or die "Can't change to directory $dir\n"; # create the data file for plotting system "cp /userhomes/faculty/jsolka/binf634/visualization/hist.dat $dir"; # run the plotting program system "/userhomes/faculty/jsolka/binf634/visualization/linegraph.pl"; # copy the output image to the web images directory system "cp out.png /userhomes/faculty/jsolka/public_html/cgi- images/out.png"; # display image file on the web page print img {src=>"/jsolka/cgi-images/out.png"}; 14BINF 634 Fall Lec 11 Visualization GD Based in Part on Previous Lecture of J. Grefenstette

BINF 634 Fall Lec 11 Visualization15 Limitations to GD::Graph GD::Graph is good for simple graphs but has some severe limits for scientific plotting X-axis data assumed evenly spaced 2-dimensional Limited control over labels, arrows, etc. GD Based in Part on Previous Lecture of J. Grefenstette

BINF 634 Fall Lec 11 Visualization16 gnuplot gnuplot is an open source Unix application for creating high quality plots from data Runs interactively and in batch mode Interactive mode: % gnuplot > plot sin(x) gnuplot Based in Part on Previous Lecture of J. Grefenstette

gnuplot Batch mode put commands into a file, for example, histogram.plt run gnuplot with the command file as argument % gnuplot histogram.plt % cat histogram.plt # file histogram.plt set xlabel "Length" set ylabel "Sequences" set title "Sequence Length Distribution" plot "plot.dat" with boxes exit 17BINF 634 Fall Lec 11 Visualization gnuplot Based in Part on Previous Lecture of J. Grefenstette

BINF 634 Fall Lec 11 Visualization18 gnuplot accepts input files with an x,y pair on each line: % cat plot.dat gnuplot Based in Part on Previous Lecture of J. Grefenstette

gnuplot gnuplot can create PNG format files, which work in all common browsers # file hist.plt # usage: gnuplot hist.plt set output "hist.png" # name of output file set terminal png medium mono # type of output file = PNG set size 0.8, 0.8 # adjusts size of text unset key # do not display key set xlabel "Length" # x-axis label set ylabel "Sequences" # y-axis label set title "Sequence Length Distribution" set style fill solid 1.0 # make boxes black set boxwidth 0.9 relative # leave small space between boxes plot "plot.dat" with boxes # plot the data file exit 19BINF 634 Fall Lec 11 Visualization gnuplot Based in Part on Previous Lecture of J. Grefenstette

BINF 634 Fall Lec 11 Visualization 20 Output gnuplot Based in Part on Previous Lecture of J. Grefenstette

# # File: hist3.plt # usage: gnuplot hist3.plt # # assumes data is stored in “myplot.dat" # set terminal png # type of output file = PNG set output "hist3date.png" # name of output file set title "Hits Per Month" # title of graph set xlabel "Months" 0,-1 # x-axis label (down 1 character) set ylabel "Hits" # y-axis label set size 0.75,0.75 # reduce graph size unset key # do not display key set boxwidth 0.9 relative # leave small space between boxes set style fill solid 1.0 # make boxes solid set xrange [0:13] # control size of x-axis # fine tune labels on x axis set xtics rotate ("Jan" 1, "Feb" 2, "Mar" 3, "Apr" 4, "May" 5, "Jun" 6, "Jul" 7, "Aug" 8, "Sep" 9, "Oct" 10, "Nov" 11, "Dec" 12) # plot the data file, creating "hist.png" plot “myplot.dat" with boxes 21BINF 634 Fall Lec 11 Visualization gnuplot Based in Part on Previous Lecture of J. Grefenstette

BINF 634 Fall Lec 11 Visualization22 gnuplot data file % cat myplot.dat gnuplot Based in Part on Previous Lecture of J. Grefenstette

BINF 634 Fall Lec 11 Visualization23 hist.png gnuplot Based in Part on Previous Lecture of J. Grefenstette

Getting the Chart on the Web 2. In the wrapper, create the graphics file in the working directory 3. Copy it to the images directory 4. Display it using print img{src=>...} use CGI qw(:standard); $ENV{PATH} = "/bin:/usr/bin:/usr/local/bin"; # for gnuplot... # change to working directory mkdir $dir or die "Can't create directory $dir\n"; chdir $dir or die "Can't change to directory $dir\n"; # create the data file for plotting system "cp /userhomes/faculty/jsolka/binf634/visualization/plot.dat $dir"; # run the plotting program system "gnuplot /userhomes/faculty/jsolka/binf634/visualization/hist.plt"; # copy the output image to the web images directory system "cp hist.png /userhomes/faculty/jsolka/public_html/cgi- images/hist.png"; # display image file on the web page print img {src=>"/jsolka/cgi-images/hist.png"}; 24BINF 634 Fall Lec 11 Visualization gnuplot Based in Part on Previous Lecture of J. Grefenstette

Perl Data Language (PDL) BINF 634 Fall Lec 11 Visualization25 PDL

A Few Discussions on PDL The next several slides were adapted from the PDL tutorial by David Mertens at this URL BINF 634 Fall Lec 11 Visualization26 PDL

BINF 634 Fall Lec 11 Visualization27 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization28 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization29 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization30 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization31 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization32 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization33 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization34 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization35 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization36 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization37 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization38 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization39 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization40 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization41 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization42 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization43 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization44 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization45 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization46 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization47 Adapted from David Mertens at this URL PDL

BINF 634 Fall Lec 11 Visualization48 Adapted from David Mertens at this URL PDL

More information Online documentation for GD::Graph % perldoc GD::Graph Online doc for gnuplot % gnuplot > help Also see: PLplot PDL 49BINF 634 Fall Lec 11 Visualization Summary

BINF 634 Fall Lec 11 Visualization50 Homework and Reminders Program 4 due next week 11/26/12 Lab 2 due by next week No more quizzes No more programming assignments Only the final Read Chapter 11 Exercise 11.1 and 11.2 Reminders