Data Organization Data files Databases Data streams Data files Databases Data streams

The organization of data  Data/information must be organized  Computer/network accessible  Stored or streaming  Data/information must be organized  Computer/network accessible  Stored or streaming

Stored information  Spreadsheet  Worksheet oriented  Cells, columns, rows  Can be set up to look like a table  Database  Table oriented  Column headings describe a row  Rows are records  Tables can be related (relational database)  Spreadsheet  Worksheet oriented  Cells, columns, rows  Can be set up to look like a table  Database  Table oriented  Column headings describe a row  Rows are records  Tables can be related (relational database)

Spreadsheets in Excel  Limited in size to 64K rows, 256 columns  Slow to compute changes  Accessed via GUI or Visual Basic  Can be exported to relational database table  Limited in size to 64K rows, 256 columns  Slow to compute changes  Accessed via GUI or Visual Basic  Can be exported to relational database table

Excel Worksheet

Relational Databases  Licensed Databases  MS Access, MS Sequel Server, Oracle  Free databases  MySQL, PostgresSQL, MaxDB, Firebird, Ingres (  Interfaces: GUI, SQL, Programming languages (Java, Perl, PHP, C++, VB)  Licensed Databases  MS Access, MS Sequel Server, Oracle  Free databases  MySQL, PostgresSQL, MaxDB, Firebird, Ingres (  Interfaces: GUI, SQL, Programming languages (Java, Perl, PHP, C++, VB)

Relational Table

Relational Table Elements

Relating Tables

One to One Relationship

One to Many Relationship

Many to Many Relationship

Data Streams  Not stored in a file  Serial data  Real-time  Underlying model to discover changes over time  Drift in the data mining/analysis tools needed  Incremental adaptation of the learning algorithm  2006 inaugural conference   Not stored in a file  Serial data  Real-time  Underlying model to discover changes over time  Drift in the data mining/analysis tools needed  Incremental adaptation of the learning algorithm  2006 inaugural conference 

Data Stream Examples  Network Packets  AP news wire  Live audio / video  Control system data  Customer click streams  Telephone records  Large sets of web pages  Retail transactions  Network Packets  AP news wire  Live audio / video  Control system data  Customer click streams  Telephone records  Large sets of web pages  Retail transactions

Summary  Data is stored or streaming  Data is organized  Data is accessible programmatically  Need to know  How data is organized  How to access the data  How to apply DI tools to the data  Data is stored or streaming  Data is organized  Data is accessible programmatically  Need to know  How data is organized  How to access the data  How to apply DI tools to the data