The Data of Visualization

Data as … The usual sense of “data” is either a number, or a string of letters, dates, or as computer files. Let’s review/consider data … booleans (true / false, 0 / 1) integers (counting numbers, 1, 2, 3, …) float (numbers with a floating point, e.g., 3.14159) [and there are other kinds of floats in computing, called “double”, “long”] aggregates of data (e.g., x * y) alphanumerics (characters, A-Z, 0-9) bytes: characters (a single byte) or characters represented by their binary (0, 1), hexadecimal (base 16), octal (base 8) or similar ideas nominal/categorial data are usually strings (e.g., “good|bad|average”) ordinal, interval, ratio, as we find statistics … Collectively these help us determine what visualization to use and, of course, are stored in computer files. The type of data and the type of file must be considered when ingesting the data for a visualization.

Data types and vis Some data types suggest strongly what kind of representation to use; compare a table versus a chart. Compare these ways of expressing date data (e.g., in YYYY/MM/DD format). If you’re plotting data by a time frame, notice the different ways we can visualize them. Discuss. What kinds of audience, data, and tasks can be addressed by each of them?

Working with your data

1 2 3 4 Typical real-world work scenario most commonly: export the data as comma-delimited or tab-delimited text file source data: cleaned of values outside domain and range; null values,inappropriate 0 values missing text characters; convert case 3 4 script or program to convert the clean data to .json visualize!

Geographic data: example of domain-specific data types (lat, long), combining categorical data, date and time, and numeric values

Models to engage users Besides integrating and balancing data + visual languages, there is the story to be told … and as in any language reflecting on the approach helps to create the visualization as much as it does to help interpret it: Descriptive Persuasive Explanatory Exploratory Process or Predictive Data layout; Spatial layout

Data lifecycle Start with the Needs or Requirements Analysis create a logical model of the system and then The physical model [the programming and implementation] Data Acquisition Processing (data cleaning) Analysis & Data or Statistical Modeling Layout (Composition, Typography, Color, Data) Reference System (part/whole relationships; grids; keys, etc.)

Data lifecycle Projection/Distortion of the image (adjustments) Access to source data by the end-user Review the overall graphic design (ensure a harmony of the data/design for the audience, data, and task) Aggregation & Clustering, redux: allow details on-demand, clustering, expanding data thru interactive techniques, such as zooming, data brushing) Test with the End-Users: do the user-controls create a meaningful, interpretable story Deploy!

Your own skills … <?php echo “\nProcessing File\n”; $lineNo = 1; $xmlWriter = new XMLWriter(); $xmlWriter->openUri(“demo.xml”); $xmlWriter->setIndent(true) $xmlWriter->startDocument(‘1.0’,’UTF-8’); $xmlWriter->startElement(‘root’); $tsvFile = new SplFileObject(‘source.tsv’); $tsvFile->setFlags(SplFileObject::READ_CSV); $tsvFile->setCsvControl(“\t”); foreach ($tsvFile as $line => $row) { if ($line > 0) { echo “\nProcessing $lineNo:\t$row[0]”. “\t$row[1]\t$row[2]\t$row[3]\t$row[4]”; $xmlWriter->startElement(‘data’); $xmlWriter->writeElement(‘refnumber’, $row[0]); $xmlWriter->writeElement(‘location’,$row[1]); $xmlWriter->writeElement(‘name’,$row[2]); $xmlWriter->writeElement(‘age’,$row[3]); $xmlWriter->writeElement(‘other’,$row[4]); $xmlWriter->endElement(); $lineNo++; } $xmlWriter->endDocument(); echo “\nFile done.\n”; ?> On the basis of your experiences with HTML, CSS, JavaScript, and d3.js, encourage in-house testing of integrating local resources, such as local digital library collections, converting tab-separated value file (such as from a spreadsheet program) to .xml. For this example the exported tab- delimited data consists of a reference number, location, name, age, other fields. file name is tsvToXml.php

Even easier … There are a number of online conversion tools and a script to demonstrate the conversion. Python and other languages make it even easier for anyone, including non-programmers, to convert a comma-separated value file into a .json file. In the Unix terminal window, run this line of code at the $ prompt, substitution ‘demofile.csv’ with your own file name: python -c “import csv,json; print json.dumps(list(csv.reader(open(‘demofile.csv’))))” https://www.commandlinefu.com/commands/view/20528/convert-csv-to-json