Jeff Henrikson (jhenriks@umd.edu) Lecture 11 Data Import & Export Jeff Henrikson (jhenriks@umd.edu) http://www.atmos.umd.edu/~jeff/aosc347/ 1

Recap Interactive I/O with input Use apostrophes ('...') or format specifier 's' for text strings; otherwise input is numeric values Output with fprintf Format specifiers %d, %f, %e, %g, %c, %s I/O via *.mat files with save and load Write to ASCII files Open with fopen, close with fclose Write with fprintf File permissions Write/read delimited data in ASCII files csvwrite & csvread or dlmwrite & dlmread Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

I/O vs. Im/Ex image Parentheses ( ) Square brackets [ ] Curly brackets { } Angle brackets or chevrons ⟨ ⟩ Corner and half brackets ｢｣, ⌊ ⌋ Double brackets ⟦ ⟧

Import/Export ASCII File ASCII File with Known Data Format: Import Export csvread dlmread textscan textread fscanf csvwrite dlmwrite Cell Array {..} – access cellplot – visualize celldisp – display Parentheses ( ) Square brackets [ ] Curly brackets { } Angle brackets or chevrons ⟨ ⟩ Corner and half brackets ｢｣, ⌊ ⌋ Double brackets ⟦ ⟧ File with Unknown Data Format: uiimport Import Wizard Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

What is a cell? CELL = container to hold data of any kind: Data can be a mix of different kinds e.g., numbers text strings, arrays or other cells. (p. 240-244) Create a 2x2 cell array >>c {1,1} = rand (2); >>c {1,2} = 'Urbana'; >>c {2,1} = [1:3; 99:101]; >>c {2,2} = sin (1.4); 2x2 array string 2x3 array scalar array = [...] cell {...} = … Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

Visualize cells >> whos [Enter] >>cellplot (c) [Enter] Name Size Bytes Class Attributes c 2x2 548 cell >>cellplot (c) [Enter] 2×2 array of random numbers 1×6 array Each element is a character 2×3 array 1×1 array scalar Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

Display cells >>celldisp (c) [Enter] c{1,1} = 0.9058 0.9134 0.1270 0.6324 c{2,1} = 1 2 3 99 100 101 c{1,2} = Urbana c{2,2} = 0.9854 Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

Access data in cells >>x = c {1,2} [Enter] x = Urbana >>y = c {2,1} [Enter] y = 1 2 3 99 100 101 >>z = c {2,1} (1,3) [Enter] z = 3 Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

Analyze formatted data Assuming the data format is known, textscan can be used to read data from inside the M-file Steps Run M-file M-file will read data M-file will analyze data M-file will plot data Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

textscan C = textscan (file, 'format', N, param, value, ...) C = Variable name N = Number of repetitions param = Parameters that give special instructions Example: C = textscan (file1, '%8.2f', 4, 'CollectOutput', 0) Read from file1 Each number has a format %8.2f There are 4 lines, so repeat format 4 times Read each column of the data into a separate cell array Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

textscan example Copy http://www.atmos.umd.edu/~jeff/aosc347/L10E1.txt Write an M-file L11E1.m to read the data in L10E1.txt Algorithm Clear workspace; clear screen Open L10E1.txt with fopen Read column headers into an array called colhead with textscan Read data into a cell array called data with textscan Save each column into separate cells Close L10E1.txt with fclose a b 0.0 288.15 11.0 216.65 Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

textscan example code Save the following code in L11E1.m clear; clc file1 = fopen ('L10E1.txt', 'rt'); colhead = textscan (file1, '%s %s', 1); data = textscan (file1, '%f %f', 2, 'CollectOutput', 0); fclose (file1); *CollectOutput = 0: save each column in separate cell arrays = 1: save similar data in single cell arrays, e.g., all integers in one array, all decimals in another *We generated ‘L10E1.txt’ in the last class, but if you don’t have, use the file I posted.

textscan example result Run L11E1.m >> whos [Enter] Name Size Bytes Class Attributes ans 1x1 8 double colhead 1x2 452 cell data 1x2 256 cell file1 1x1 8 double Check the values >> colhead{:} >> data{:} Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

Atmospheric Sounding Open a browser & go to http://weather.uwyo.edu Navigate to Upper Air Observations  Soundings From the menu select Region: North America Type of plot: Text List Year: 2012 Month: Jul From: 15/12Z To: 15/12Z Find Sterling, VA (IAD) on the map and click Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

Atmospheric Sounding Save file to: IAD12Z07152012.txt in your browser TIPS: Open text file in MATLAB Delete lines with text Delete lines with missing data Save it into a new file: IAD12Z07152012_d.txt You should have 2 head lines and 120 data lines …… cut/paste until here Or delete after here

textscan Exercise Write an M-file called L11E2.m to read the sounding data and plot PRES (mb) vs TEMP (C) TIPS: Open text file in MATLAB Delete lines with text Delete lines with missing data Save it into a new file: IAD12Z07152012_d.txt You should have 2 head lines and 120 data lines Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

Algorithm - Goal: Write an M-file L11E2 Algorithm - Goal: Write an M-file L11E2.m to read the sounding data and plot PRES (mb) vs TEMP (C) Close all files; Clear workspace; Clear screen Open file IAD12Z07152012_d.txt with fopen Read 11 header fields into 11 columns of a cell array called head with textscan Read 11 variables into 11 columns of a cell array called data with textscan Close file with fclose Draw TEMP vs PRES Reverse vertical axis Add title, labels, legends Write comments Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

textscan sounding code % Close all files; clear workspace and screen close; clear; clc % Open file IAD12Z07152012_d.txt file1 = fopen ('IAD12Z07152012_d.txt', 'rt'); % Read 11 header fields into a cell array called head head=textscan (file1,'%s %s %s %s %s %s %s %s %s %s %s', 4); % Read 11 data variables into a cell array called data data = textscan (file1,'%f %f %f %f %f %f %f %f %f %f %f',120, 'collectoutput', 0); % Close the file fclose(file1); % Plot TEMP=data{3} vs PRES=data{1} plot (data{3}, data{1}); % Reverse the vertical axis set (gca, 'YDir', 'reverse'); help gca PBL Troposphere Stratosphere Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

Import Wizard If we do NOT know the data format? The Import Wizard is a MATLAB feature that determines the type of data file and also the way to extract and display the information It can be used to extract data from ASCII files, Excel spreadsheet files, among others Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

uiimport A more typical way to open the Import Wizard is to type: >> uiimport ('filename.extension') Or USE ‘Import Data’ icon The single quotes around the name of the file are very important: the wizard will not run if they are omitted Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

uiimport uiimport starts the MATLAB Import Wizard Run M-file to analyze/visualize the data Advantage You don’t need to know the format and content of the data file beforehand Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

Import Wizard >>uiimport [Enter] Select File pop-up window Use the Graphical User Interface (GUI) to preview and select data into MATLAB workspace >>uiimport [Enter] Select File Browse and select file IAD12Z07152012.txt to import Or >>uiimport(‘IAD12Z07152012.txt’) [Enter]

Import Wizard Select column separators Click tabs to view data Check Workspace

Import Wizard If you check Generate Function, MATLAB will automatically create an M-File called importfile.m to read the data in *.m files From the next time on, use function: >>importfile(a, b, c)

Import Wizard Excise check Generate Function Save as importfile.m function [PRES,HGHT,TEMP,DWPT,RELH,MIXR,DRCT,SKNT,THTA,THTE,THTV] = importfile(filename, startRow, endRow) >> mfn='IAD12Z07152012.txt' >> [PRES,HGHT,TEMP,DWPT,RELH,MIXR,DRCT,SKNT,THTA,THTE,THTV]=importfile(mfn, 5, 85); Draw TEMP vs PRES: plot(TEMP, PRES); set(gca,'Ydir','reverse');

Summary I/O Introduction to cell arrays Assign cell array Access data in cell arrays Visualize with cellplot Display with celldisp Import text with uiimport Import text with textscan skip columns with * Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang