1. Jeff Henrikson (jhenriks@umd.edu)
Lecture 11
Data Import & Export
Jeff Henrikson 1

2. Recap Interactive I/O with input
Use apostrophes ('...') or format specifier 's' for text strings; otherwise input is numeric values
Output with fprintf
Format specifiers %d, %f, %e, %g, %c, %s
I/O via *.mat files with save and load
Write to ASCII files
Open with fopen, close with fclose
Write with fprintf
File permissions
Write/read delimited data in ASCII files
csvwrite & csvread or dlmwrite & dlmread
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

3. I/O vs. Im/Ex
image
Parentheses ( ) Square brackets [ ] Curly brackets { } Angle brackets or chevrons ⟨ ⟩ Corner and half brackets ｢｣, ⌊ ⌋ Double brackets ⟦ ⟧

4. Import/Export ASCII File
ASCII File with Known Data Format:
Import Export
csvread
dlmread
textscan
textread
fscanf
csvwrite
dlmwrite
Cell Array
{..} – access
cellplot – visualize
celldisp – display
Parentheses ( ) Square brackets [ ] Curly brackets { } Angle brackets or chevrons ⟨ ⟩ Corner and half brackets ｢｣, ⌊ ⌋ Double brackets ⟦ ⟧
File with Unknown Data Format:
uiimport Import Wizard
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

5. What is a cell? CELL = container to hold data of any kind:
Data can be a mix of different kinds e.g., numbers text strings, arrays or other cells. (p )
Create a 2x2 cell array
>>c {1,1} = rand (2);
>>c {1,2} = 'Urbana';
>>c {2,1} = [1:3; 99:101];
>>c {2,2} = sin (1.4);
2x2 array
string
2x3 array
scalar
array = [...]
cell {...} = …
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

6. Visualize cells >> whos [Enter] >>cellplot (c) [Enter]
Name Size Bytes Class Attributes
c x cell
>>cellplot (c) [Enter]
2×2 array of random numbers
1×6 array
Each element is a character
2×3 array
1×1 array scalar
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

7. Display cells >>celldisp (c) [Enter] c{1,1} = 0.9058 0.9134
c{2,1} =
c{1,2} = Urbana
c{2,2} =
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

8. Access data in cells >>x = c {1,2} [Enter] x = Urbana
>>y = c {2,1} [Enter]
y =
>>z = c {2,1} (1,3) [Enter]
z =
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

9. Analyze formatted data
Assuming the data format is known, textscan can be used to read data from inside the M-file
Steps
Run M-file
M-file will read data
M-file will analyze data
M-file will plot data
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

10. textscan C = textscan (file, 'format', N, param, value, ...)
C = Variable name
N = Number of repetitions
param = Parameters that give special instructions
Example:
C = textscan (file1, '%8.2f', 4, 'CollectOutput', 0)
Read from file1
Each number has a format %8.2f
There are 4 lines, so repeat format 4 times
Read each column of the data into a separate cell array
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

11. textscan example Copy http://www.atmos.umd.edu/~jeff/aosc347/L10E1.txt
Write an M-file L11E1.m to read the data in L10E1.txt
Algorithm
Clear workspace; clear screen
Open L10E1.txt with fopen
Read column headers into an array called colhead with textscan
Read data into a cell array called data with textscan
Save each column into separate cells
Close L10E1.txt with fclose a b
0.0
288.15
11.0
216.65
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

12. textscan example code Save the following code in L11E1.m clear; clc
file1 = fopen ('L10E1.txt', 'rt');
colhead = textscan (file1, '%s %s', 1);
data = textscan (file1, '%f %f', 2, 'CollectOutput', 0);
fclose (file1);
*CollectOutput = 0: save each column in separate cell arrays
= 1: save similar data in single cell arrays, e.g., all integers in one array, all decimals in another
*We generated ‘L10E1.txt’ in the last class, but if you don’t have, use the file I posted.

13. textscan example result
Run L11E1.m
>> whos [Enter]
Name Size Bytes Class Attributes
ans x double
colhead 1x cell
data x cell
file x double
Check the values
>> colhead{:}
>> data{:}
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

14. Atmospheric Sounding Open a browser & go to http://weather.uwyo.edu
Navigate to Upper Air Observations  Soundings
From the menu select
Region: North America
Type of plot: Text List
Year:
Month: Jul
From: 15/12Z
To: 15/12Z
Find Sterling, VA (IAD) on the map and click
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

15. Atmospheric Sounding Save file to: IAD12Z07152012.txt in your browser
TIPS:
Open text file in MATLAB
Delete lines with text
Delete lines with missing data
Save it into a new file: IAD12Z _d.txt
You should have 2 head lines and 120 data lines ……
cut/paste until here
Or delete after here

16. textscan Exercise
Write an M-file called L11E2.m to read the sounding data and plot PRES (mb) vs TEMP (C)
TIPS:
Open text file in MATLAB
Delete lines with text
Delete lines with missing data
Save it into a new file: IAD12Z _d.txt
You should have 2 head lines and 120 data lines
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

17. Algorithm - Goal: Write an M-file L11E2
Algorithm - Goal: Write an M-file L11E2.m to read the sounding data and plot PRES (mb) vs TEMP (C)
Close all files; Clear workspace; Clear screen
Open file IAD12Z _d.txt with fopen
Read 11 header fields into 11 columns of a cell array called head with textscan
Read 11 variables into 11 columns of a cell array called data with textscan
Close file with fclose
Draw TEMP vs PRES
Reverse vertical axis
Add title, labels, legends
Write comments
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

18. textscan sounding code
% Close all files; clear workspace and screen
close; clear; clc
% Open file IAD12Z _d.txt
file1 = fopen ('IAD12Z _d.txt', 'rt');
% Read 11 header fields into a cell array called head
head=textscan (file1,'%s %s %s %s %s %s %s %s %s %s %s', 4);
% Read 11 data variables into a cell array called data
data = textscan (file1,'%f %f %f %f %f %f %f %f %f %f %f',120, 'collectoutput', 0);
% Close the file
fclose(file1);
% Plot TEMP=data{3} vs PRES=data{1}
plot (data{3}, data{1});
% Reverse the vertical axis
set (gca, 'YDir', 'reverse');
help gca
PBL
Troposphere
Stratosphere
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

19. Import Wizard If we do NOT know the data format?
The Import Wizard is a MATLAB feature that determines the type of data file and also the way to extract and display the information
It can be used to extract data from ASCII files, Excel spreadsheet files, among others
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

20. uiimport A more typical way to open the Import Wizard is to type:
>> uiimport ('filename.extension')
Or USE ‘Import Data’ icon
The single quotes around the name of the file are very important: the wizard will not run if they are omitted
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

21. uiimport uiimport starts the MATLAB Import Wizard
Run M-file to analyze/visualize the data
Advantage
You don’t need to know the format and content of the data file beforehand
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang

22. Import Wizard >>uiimport [Enter] Select File
pop-up window
Use the Graphical User Interface (GUI) to preview and select data into MATLAB workspace
>>uiimport [Enter]
Select File
Browse and select file IAD12Z txt to import
Or >>uiimport(‘IAD12Z txt’) [Enter]

23. Import Wizard Select column separators Click tabs to view data
Check Workspace

24. Import Wizard
If you check Generate Function, MATLAB will automatically create an M-File called importfile.m to read the data in *.m files
From the next time on, use function:
>>importfile(a, b, c)

25. Import Wizard Excise check Generate Function Save as importfile.m
function [PRES,HGHT,TEMP,DWPT,RELH,MIXR,DRCT,SKNT,THTA,THTE,THTV] = importfile(filename, startRow, endRow)
>> mfn='IAD12Z txt'
>> [PRES,HGHT,TEMP,DWPT,RELH,MIXR,DRCT,SKNT,THTA,THTE,THTV]=importfile(mfn, 5, 85);
Draw TEMP vs PRES:
plot(TEMP, PRES);
set(gca,'Ydir','reverse');

26. Summary I/O Introduction to cell arrays Assign cell array
Access data in cell arrays
Visualize with cellplot
Display with celldisp
Import text with uiimport
Import text with textscan
skip columns with *
Copyright © 2015 University of Maryland. This material may not be reproduced or redistributed in whole or in part without written permission from Xin-Zhong Liang