Computing and the Web Databases: Controlling the Information Deluge

Overview n Information Overload n Overview of Data Collection n Retrieving Data n Visualization of Information n Transforming Data into Information n Database Structure & Design n Software Applications

Information Overload n Sheer volume of data that is available –Processed multiple ways to provide information –Much of the data / information is interrelated n It is projected that 1 terabyte of data is put online on the internet each day –Assuming that a 500 page book takes 4,750M –1 terabyte would hold 220 books –That’s the equivalent of 6,600 books per month

Information Overload n Relationships based upon the data are more complicated due to computers and automation n Example: Grocery store checkout –Item is scanned by bar code reader –Bar code is looked up and resolved to item name –Current price of item is looked up –Inventory is automatically updated to reflect sale –If inventory drops below established levels, an automatic order can be generated to replenish the inventory levels of the item

Overview of Data Collection n Previous methods of data collection –Survey / inventory : paper & pencil –Keypunch cards –Optical mark scan forms n Current methods of data collection –Bar codes & bar code scanners –Proximity devices (Easy pass) –Data probes –Satellite sensing (imagery, infrared, etc.) –Intelligent devices generating “alerts”

Retrieving Data n How data will be retrieved will figure prominently in determining how to store it –Who will use it (single or multiple users) –How often will it be needed –How will it be utilized (select information or all) n Example: Library system –All books have a unique identifier –All patrons have separate identification numbers –Indexes are built based upon book or patron number

Retrieving Data n Complexity of data is growing with time n Reducing everything to a number is not “natural” in relation to our brains n Example: FBI fingerprint files –Database contains over 30 million sets of prints –Fingerprints can be divided into 1,024 groups –Before computers: 1,400 techs / 24K req per day –After computers: 700 techs / 30K req per day –Nature of fingerprints makes it difficult to reduce them to a number

Visualization of Information n Too much data can be a liability instead of an asset n Need exists to reduce or simplify the data n Visualization allows large amounts of data to be represented in a graphical way –Superimpose values on charts, maps, or graphs –Example: population growth chart in book

Transforming Data into Information n Data transformation is another means of simplifying large amounts of data n Statistics is the most commonly employed means of data transformation –Percent –Probability –Sampling: limited selection (beware skewing) –Normal distribution –Correlation n Connection of two related pieces of information n False correlation looks normal but is erroneous