File I/O 1. Goals of this chapter 2. General Concept, lots of examples 3. High-level I/O 1. Built-in functions that do everything in one step 4. Low-level.

Slides:



Advertisements
Similar presentations
Chapter 3: Editing and Debugging SAS Programs. Some useful tips of using Program Editor Add line number: In the Command Box, type num, enter. Save SAS.
Advertisements

Lecture 7 Debugging Code & Data Import/Export © 2007 Daniel Valentine. All rights reserved. Published by Elsevier.
Managing Grades with Excel Viewing Help To view Help 1.Open Excel on your computer. 2.In the top right hand corner of the Excel Screen type in the.
SUNY Morrisville-Norwich Campus- Week 7 CITA 130 Advanced Computer Applications II Spring 2005 Prof. Tom Smith.
XP New Perspectives on Microsoft Office Excel 2003, Second Edition- Tutorial 11 1 Microsoft Office Excel 2003 Tutorial 11 – Importing Data Into Excel.
CIS101 Introduction to Computing Week 02. Agenda Your questions CIS101 Blackboard Site Excel Project One Next Week.
Division Example 2x - 3y + 4z = 10 x + 6y - 3z = 4 -5x + y + 2z = 3 A*X = B where A = B = >> X = A\B X =
Fall 2006AE6382 Design Computing1 Matlab File & Directory Management Learning Objectives Define file input and output terminology Compare high and low.
SPSS 1: An Introduction to the Statistical Package SPSS Suzie Cro MRC Clinical Trials Unit.
Introduction to Array The fundamental unit of data in any MATLAB program is the array. 1. An array is a collection of data values organized into rows and.
How to Fill Out the CARD Form (Course Assessment Reporting Data Form)
1 Nassau Community CollegeProf. Vincent Costa Session 2 Excel Introduction CMP 117 Business Computing: Concepts &Applications.
Managing Business Data Lecture 8. Summary of Previous Lecture File Systems  Purpose and Limitations Database systems  Definition, advantages over file.
Coding for Excel Analysis Optional Exercise Map Your Hazards! Module, Unit 2 Map Your Hazards! Combining Natural Hazards with Societal Issues.
Software All parts of the computer people can NOT touch, such as programs, files, documents and any other data.
1 -Defined Functions 1. Goals of this Chapter 2. General Concept 3. Advantages 4. How it works Programmer.
Spreadsheets and Microsoft Excel. Introduction n A spreadsheet (called a worksheet in Excel) is a two-dimensional array of cells containing data to be.
"USES OF SPREAD SHEET IN EXCEL"
24 GOLDEN COINS, 1 IS FAKE ( WEIGHS LESS). DATABASE CONCEPTS Ahmad, Mohammad J. CS 101.
REVIEW 2 Exam History of Computers 1. CPU stands for _______________________. a. Counter productive units b. Central processing unit c. Copper.
Miscellaneous Excel Combining Excel and Access. – Importing, exporting and linking Parsing and manipulating data. 1.
Copyright © 2008 Pearson Prentice Hall. All rights reserved. 1 Microsoft Office Excel Copyright © 2008 Pearson Prentice Hall. All rights reserved
Cell Arrays 1.Definition 2.Creating Cell Arrays 3.Referencing Cell Arrays 4.Augmenting Cell Arrays 5.Use of Cell Arrays 1.
File I/O 1. Goals of this chapter 2. Inputs Files 3. Output Files 4. File Formats 5. General Concept, lots of examples 6. High-Level vs. Low-Level 1.
1 Functions 1 Parameter, 1 Return-Value 1. The problem 2. Recall the layout 3. Create the definition 4. "Flow" of data 5. Testing 6. Projects 1 and 2.
Arrays 1 Multiple values per variable. Why arrays? Can you collect one value from the user? How about two? Twenty? Two hundred? How about… I need to collect.
Exploring Engineering Chapter 3, Part 2 Introduction to Spreadsheets.
1 Performing Spreadsheet What-If Analysis Applications of Spreadsheets.
SPREADSHEET BASICS SPREADSHEET BASICS What are the benefits of using a spreadsheet to solve a problem?
Spreadsheets and Non- Spatial Databases Unit 4: Module 15, Lecture 1- Spreadsheet Software.
Introduction to File I/O High-Level Functions 1.Data files 2."High level" File I/O 3.dlmread() 4.xlsread() 1.
CIS101 Introduction to Computing Week 03. Agenda Your questions Online gradebook and quizzes Your Web space Review of charts and today’s homework Excel.
MATLAB FUNDAMENTALS: MATRIX/ARRAY FUNCTIONS THE COLON MATRIX/ARRAY MANIPULATION INPUT/OUTPUT HP 100 – MATLAB Wednesday, 9/3/2014
 Agenda: 4/24/13 o External Data o Discuss data manipulation tools and functions o Discuss data import and linking in Excel o Sorting Data o Date and.
Introduction to MATLAB 7 Engineering 161 Engineering Practices II Joe Mixsell Spring 2010.
Chapter 17 Creating a Database.
Getting Started with MATLAB 1. Fundamentals of MATLAB 2. Different Windows of MATLAB 1.
FILE I/O: Low-level General Ideas. High vs. low-level Opening a file
CMPS 1371 Introduction to Computing for Engineers FILE Input / Output.
File I/O High-Level Functions 1. Definition 2. Is a High-Level function appropriate? 3. xlsread() 4. dlmread() 1.
Logical Ops on Arrays Saving time / avoiding loops 1. Logical Operators 2. Logical Operations 3. Examples: numbers and letters 11.
MATLAB for Engineers 4E, by Holly Moore. © 2014 Pearson Education, Inc., Upper Saddle River, NJ. All rights reserved. This material is protected by Copyright.
Chapter 3: Formatted Input/Output Copyright © 2008 W. W. Norton & Company. All rights reserved. 1 Chapter 3 Formatted Input/Output.
Algorithms  Problem: Write pseudocode for a program that keeps asking the user to input integers until the user enters zero, and then determines and outputs.
Getting Started with MATLAB (part2) 1. Basic Data manipulation 2. Basic Data Understanding 1. The Binary System 2. The ASCII Table 3. Creating Good Variables.
Asking the USER for values to use in a software 1 Input.
ENG College of Engineering Engineering Education Innovation Center 1 More Script Files in MATLAB Script File I/O : Chapter 4 1.Global Variables.
ECE 264 Object-Oriented Software Development Instructor: Dr. Honggang Wang Spring 2013 Lecture 5: Continuing with C++ I/O Basics.
Lecture 5 1.What is a variable 2.What types of information are stored in a variable 3.Getting user input from the keyboard 1.
Using Text Files in Excel File I/O Methods. Working With Text Files A file can be accessed in any of three ways: –Sequential access: By far the most common.
1. FINISHING FUNCTIONS 2. INTRODUCING PLOTTING 1.
Asking the USER for values to use in a software 1 Input.
Chapter 1: Overview of SAS System Basic Concepts of SAS System.
MATLAB for Engineers, by Holly Moore. ISBN © 2007 Pearson Education, Inc., Upper Saddle River, NJ. All rights reserved. This material is.
Lecture 6: Output 1.Presenting results in a professional manner 2.semicolon, disp(), fprintf() 3.Placeholders 4.Special characters 5.Format-modifiers 1.
1 Printing in Python Every program needs to do some output This is usually to the screen (shell window) Later we’ll see graphics windows and external files.
Copyright © 2010 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Starting Out with Programming Logic & Design Second Edition by Tony Gaddis.
1 Lecture 4 Post-Graduate Students Advanced Programming (Introduction to MATLAB) Code: ENG 505 Dr. Basheer M. Nasef Computers & Systems Dept.
Computing with SAS Software A SAS program consists of SAS statements. 1. The DATA step consists of SAS statements that define your data and create a SAS.
Early File I/O To help you get started with your final project 1. Definition of “high level” 2. Is using a High Level function appropriate? 3. xlsread()
Intro to Loops 1.General Knowledge 2.Two Types of Loops 3.The WHILE loop 1.
Files in Python The Basics. Why use Files? Very small amounts of data – just hardcode them into the program A few pieces of data – ask the user to input.
Chapter 3: Formatted Input/Output 1 Chapter 3 Formatted Input/Output.
FILE I/O: Low-level 1. The Big Picture 2 Low-Level, cont. Some files are mixed format that are not readable by high- level functions such as xlsread()
Fundamentals of Python: First Programs
Advanced Data Import & Export Jeff Henrikson
First Python Program Professor Hugh C. Lauer CS-1004 — Introduction to Programming for Non-Majors (Slides include materials from Python Programming: An.
Fundamentals of Data Structures
funCTIONs and Data Import/Export
Presentation transcript:

File I/O 1. Goals of this chapter 2. General Concept, lots of examples 3. High-level I/O 1. Built-in functions that do everything in one step 4. Low-level I/O 1. Built-in functions that are specific to one task 1

1. Goals of this chapter Understand why external files are greatly used in programming, such as.txt,.xls,.dat… Recognize when to use a file vs. hardcoding data Understand how Matlab sees any file Define and recognize delimiting-characters Depending on which type of file, choose an appropriate built-in function Remember the overall idea behind each built-in function Remember the logical order of reading/writing a file 2

2. General Concept What is an “input-file”? It is file that contains the givens needed to solve the entire problem, or maybe just a part of the problem. LOTS of data (as little as 5 lines to billions of lines…) What is an “output-file”? It is a file which contains the results (once the problem has been solved). Most common example: I _ V O _ C _ 3 Some examples of input files…

Property Sales in Flagler (North of Daytona Beach) 4 INPUT FILE

Property Listing in Ormond Beach (North of Daytona Beach) 5 Online data dated March 21 st 2010.

Hurricane Data 6 Online data dated March 21 st 2010, nhc.noaa.gov INPUT FILE

Hurricane Data... What? 7 Same as previous file… A text file doesn’t always look pretty. Try different softwares to open it until one shows clearly the columns and rows! -Notepad -Wordpad -Internet explorer -Firefox -word INPUT FILE

Coefficient of Friction 8.xls, Excel format.txt, text format(INPUT FILES)

File Formats Files are stored in essentially two different formats: ASCII text, and binary. ASCII text consists entirely of characters which have meaning within the ASCII table. The only special codes are already present in the ASCII table, such as newlines and tabs. Notepad, for example, works only with ASCII text files.

File Formats, cont. Binary files may contain some ASCII text, but will also contain data which has meaning to the program which was intended to use the file. MS Word, for example, embeds codes for bold, underline, HTML, etc in the file. These codes only have meaning to Word – not to programs like Notepad. Word can work with Notepad files, but Notepad cannot work with Word files.

General Concept, cont. When reading data from a file, it continually requests values from the file data stream until the end of the file is reached. As the data is received, the program must identify the delimiting characters (tabs, spaces, enter key, special characters..) and reformat the data to reconstruct the organization of the data as represented in a file. When writing data to a file: to preserve the organization of the data, the appropriate delimiting characters must be inserted into the stream of information. 11 FileProgram An I nput Stream of information ProgramFile An O uput Stream of information

General Concept, cont. Using the data contained inside input-files Analyze, min, max, average Estimate trends (stock market) Use it as a database for one or two variables (coefficient of friction) Use it to give user possibilities to pick from Plot Using output-files, once populated with data Print them them Publish in a report Reuse as an-input file for another software 12 Examples…

Analyzing Flagler Sales Price of properties function of square footage Linear Fitting of all data points (red) Standard deviation (green) 13

Attendance Sheet 14 OUTPUT FILE

Analyze grades 15 INPUT FILE OUTPUT FILE

Another Hurricane Data This one shows all coordinates (latitude/longitude) of the storm at different times during the day, as well as the wind speed. 16 INPUT FILE

Plot Hurricane Data This is a final project in JAVA (CS225) in Fall 2007 that analyzed the hurricane input file. 17

General Concept, cont. Although there are only a few ADVANTAGES to using files, the ones that do exist are sufficient in themselves!!! LOTS of data, usually contained in very little variables (possibly only 1 cell-array!) The software is developed knowing that the file cannot change format, but definitely knowing that the file can grow in size! Although the hurricane data obviously gets updated yearly, the software to analyze the data does not change a bit! User does not have to enter the data: less prone to errors. Long term saving/backup of data. 18

General Concept, cont. Only certain formats can be analyzed.txt.xls.csv.dat.jpg.png.gif.trk The following is not possible under MATLAB.doc 19

NOW THAT YOU HAVE AN IDEA.. How is all this done? 20

General Concept, cont. There are two types of built-in functions High-level functions: one (and 1 only) command opens the file, read or writes data, and closes the file. All is stored immediately in variable(s). Low-level functions: 1. one command needs to open the file, 2. another command (maybe more than one) needs to read or write the data, 3. and finally another command closes the file. Learn both. High-level functions can only be used with some very specific files. 21

3. HIGH LEVEL FUNCTIONS In one command-line, the function grabs all the data 22

High Level, cont. Load data from external sources (INPUT) Save data back to a file (OUTPUT) When reading data, it is a good habit to explore the data in the file first. What type of file is it? (excel, text, jpg..) Find the organization overall (data in rows, or data in columns) Recognize the delimiters (space, tabs, new lines, dash, colon, or any specific symbol. What makes it obvious it is a new column? What makes it obvious it is a new row?) Recognize the data types (all numerical? all strings? Or combination of both) 23

High Level, cont. NUMERICAL AND CHARACTERS Excel worksheet xlsread() (sometimes does not work depending on which system you have) NUMERICAL DATA ONLY (of course, not in an excel file…) General-delimited dlmread() Tab-separated dlmread() (same function as tab) Comma-separated csvread() Most can be interchanged. These are powerful functions. Remember xlsread() and dlmread() mostly! 24

Whatever you do, the files (text,.m,.mat) should all be in the same directory for function calls to work! 25

3.1 Using xlsread() The function recognizes the data types of each cell, and groups similar data- type together >>[v1 v2 v3] = xlsread('grades.xlsx') First variable (v1) will have all the numerical data Second variable (v2) will have all the character data Third variable (v3) groups ALL the data together in one cell-array. Not column by column >> [names age grade] = xlsread('grades.xlsx') 26

xlsread(), cont nd return value: characters 3 rd return value: The data all together 1 st return value: numerical

xlsread(), cont. txt = nbs = 'Name' 'Age' 'Grade' 'Fred' '' '' 'joe' '' '' 'SaLLy' '' '' 'CharliE' '' '' 'mary' '' '' 'Ann' '' '' raw = 'Name' 'Age' 'Grade' 'Fred' [ 19] [ 78] 'joe' [ 22] [ 83] 'SaLLy' [ 98] [ 99] 'CharliE' [ 21] [ 56] 'mary' [ 23] [ 89] 'Ann' [ 19] [ 51] >> [txt nbs raw] = xlsread('grades.xlsx‘)   Variable names are up to the programmer. If placed in wrong order by mistake… BAD. The order of the return values is important.

xlsread(), cont. Simply omit the 2 nd and 3 rd return value to collect only numerical values. values = textread(‘grades.xlsx’); If a project needs all the data together, collect the 2 nd and 3 rd return values into a dummy variable. [trash trash data] = textread(‘grades.xlsx’); If there happens to be ‘holes’ in the spreadsheet, Matlab fills it with a NaN value (not a number). The function isnan() can help determine where those ‘holes’ are. (This function will not work under Citrix, since Excel is not installed on Citrix. Come to the lab to do labs and homeworks.) 29

Writing to an excel sheet? After manipulating, analyzing, or creating an array, excel spreadsheets can also be written-to using: xlswrite(,,, ) The and arguments are optional. 30 clc clear %create phony data table = rand(5,3)*100; %print to excel xlswrite('testingTesting.xls',table) Then use excel to plot and analyze the data!

3.2 Using delimited files – Numbers only Columns are delimited by tabs, and/or spaces. Rows are delimited by the new line character (enter key, invisible to the human eye). 31 Matlab automatically filled in with zeros where data was missing.

Same function used for Comma-delimited numbers Added feasibility to ‘skip’ columns 32

High-Level, conclusion The variables are automatically created by MATLAB. Sometimes a regular array [ ] Other times, a cell array{ } Use knowledge from previous chapter to analyze the data in the arrays. FOR loops, A(position), A(row,col), A{row,col}, max(arrayName), mean(arrayName), sort(..)