1. Selene Bainum RiteTech LLC

2.  Doing ColdFusion & SQL development for more than 1/3 of my lifetime  Chief Architect @ RiteTech  RiteTech is my company – completely independent for the first time!  Likes:  SQL and databases!  Love making money on the internet ▪ www.smartsupplies.net  Dislikes:  DC Traffic  Rude People

3.  Intro  Importing Data  Tools  Examples *  Exporting Data  Tools  Examples *  Resources * Any custom tags/functions shown in examples are free!

4. SIMILARITIES  Data stored in tabular format  Both contains rows and columns  Headers/column names denote data DIFFERENCES  Spreadsheets often contain repeated rows of data  Database tables should be normalized – repeated data among rows kept to a minimum  Spreadsheets used column delimiters – not as reliable as database columns  Spreadsheets often contain no unique key

5. IMPORT  Data only available as:  Web pages  Text files  Excel spreadsheets  Etc…  Need data in your database! EXPORT  Export raw data so that others can view/manipulate it in Excel  Create reports

6.  Bad/non-standard data  Spreadsheets may take hours and hours to clean up the data manually  Ill-placed commas and quotes can wreak havoc  Data types can get misread/lost  Matching imported data can be daunting  It helps to have good knowledge of SQL

7.  Import CSV or Excel files via SQL Server  Loop over file line by line *  CFHTTP *  Custom Tags  cfx_text2Query  cfx_Excel2Query  cfx_Excel * Problematic as CF doesn’t recognize empty string elements

9.  Find a site that contains regularly formatted data  Using list of US Representatives as an example  Copy the data and paste into notepad  Perform any formatting necessary

10.  May not work on all databases  Using SQL Server 2005 in example  Not all installs of databases may support this type of import  May not have access to database

19.  Insert all rows first  Start with query result set  Create/modify table as needed in database  Loop over query results  Insert results  Perform other database inserts/updates as needed

22. -- Declare the variables DECLARE @RepID INT, @StateID INT, @DistrictID INT, @LastNameTx VARCHAR(50), @FirstNameTx VARCHAR(50), @PhoneTx VARCHAR(25), @RoomNb INT, @StateCd VARCHAR(2), @DistrictTx VARCHAR(10) -- Create a cursor DECLARE cRep CURSOR FOR SELECT FirstName, LastName, State, District, Phone, Room FROM RepListImport WITH (NOLOCK) -- Open the cursor OPEN cRep -- Loop over the cursor FETCH NEXT FROM cRep INTO @LastNameTx, @FirstNameTx, @StateCd, @DistrictTx, @PhoneTx, @RoomNb WHILE @@FETCH_STATUS = 0 BEGIN SET @StateID = NULL SET @DistrictID = NULL -- State exist? SELECT @StateID = StateID FROM State WITH (NOLOCK) WHERE StateCd = @StateCd IF @StateID IS NULL BEGIN SET NOCOUNT OFF INSERT INTO State (StateCd) VALUES (@StateCd) SELECT @StateID = Scope_Identity() SET NOCOUNT ON END -- District exist? SELECT @DistrictID = DistrictID FROM District WITH (NOLOCK) WHERE DistrictTx = @DistrictTx IF @DistrictID IS NULL BEGIN SET NOCOUNT OFF INSERT INTO District (DistrictTx) VALUES (@DistrictTx) SELECT @DistrictID = Scope_Identity() SET NOCOUNT ON END -- Insert the rep INSERT INTO Rep (FirstNameTx, LastNameTx, PhoneTx, RoomNb, StateID, DistrictID) VALUES (@FirstNameTx, @LastNameTx, @PhoneTx, @RoomNb, @StateID, @DistrictID) -- Get the next record FETCH NEXT FROM cRep INTO @LastNameTx, @FirstNameTx, @StateCd, @DistrictTx, @PhoneTx, @RoomNb END -- Close the cursor CLOSE cRep DEALLOCATE cRep

23.  Create table structures and import scripts  Schedule retrieval of files  Email attachments  External files  Process import when new files available  Run additional database scripts  ColdFusion  Database scheduler

24.  Much easier than imports  Data is already cleaned and normalized  Tricky part is formatting

25.  Manually create a CSV file via ColdFusion  Custom Tags/Functions  generateExcel  cf_Excel_XML  cfx_Excel  cfx_Query2Excel

26.  Manually created CSV file  Created by looping over the query results  Can add custom column headings  Can format data in cells – date, number, etc…  generateExcel  Easier than manual creation  Cannot add custom column headings ▪ Column heading is the same as the column name  Automatically formats numbers and dates

27. SELECT R.RepID, R.FirstNameTx, R.LastNameTx, R.PhoneTx, R.RoomNb, S.StateCd, D.DistrictTx FROM Preso..Rep R WITH (NOLOCK) INNER JOIN Preso..State S WITH (NOLOCK) ON R.StateID = S.StateID INNER JOIN Preso..District D WITH (NOLOCK) ON R.DistrictID = D.DistrictID ORDER BY S.StateCd, D.DistrictTx "State","District","LastName","FirstName","Phone","Room" "#StateCd#","#DistrictTx#","#LastNameTx#","# FirstNameTx#","#PhoneTx#","#RoomNb#"

28. SELECT R.RepID, R.FirstNameTx, R.LastNameTx, R.PhoneTx, R.RoomNb, S.StateCd, D.DistrictTx FROM Preso..Rep R WITH (NOLOCK) INNER JOIN Preso..State S WITH (NOLOCK) ON R.StateID = S.StateID INNER JOIN Preso..District D WITH (NOLOCK) ON R.DistrictID = D.DistrictID ORDER BY S.StateCd, D.DistrictTx generateExcel(qryGetRepList, "StateCd,DistrictTx,LastNameTx,FirstNameTx,PhoneTx,RoomNb");

29.  Creates real Excel file  Uses Excel’s XML format  Custom Settings  Header/column names  Multiple data formatting options  Multiple sheets per file  Advanced Formatting  Header color/text  Alternating row colors  Column width

30. SELECT R.RepID, R.FirstNameTx, R.LastNameTx, R.PhoneTx, R.RoomNb, S.StateCd, D.DistrictTx FROM Preso..Rep R WITH (NOLOCK) INNER JOIN Preso..State S WITH (NOLOCK) ON R.StateID = S.StateID INNER JOIN Preso..District D WITH (NOLOCK) ON R.DistrictID = D.DistrictID ORDER BY S.StateCd, D.DistrictTx <cf_excel_xml filename="export" sheets="1" collist1="StateCd,DistrictTx,LastNameTx,FirstNameTx,PhoneTx,RoomNb" headerlist1="State,District,Last Name,First Name,Phone,Room" colwidths1="50,75,100,100,75,50" query1="#qryGetRepList#">

31. SELECT FirstNameTx, LastNameTx, PhoneTx, RoomNb, DistrictTx, StateCd FROM qryGetRepList WHERE StateCd = '#StateCd#' ORDER BY DistrictTx ac["collist#counter#"] = "StateCd,DistrictTx,LastNameTx,FirstNameTx,PhoneTx,RoomNb"; ac["headerlist#counter#"] = "State,DistrictTx,LastNameTx,FirstNameTx,PhoneTx,RoomNb"; ac["colwidths#counter#"] = "50,75,100,100,75,50"; ac["query#counter#"] = "#qryGetReps#"; ac["sheetname#counter#"] = "#StateCd#";

32.  Importing and Exporting can be:  Easy  Difficult  Both  Great tools exist – use them!  Search regularly for new tags & code  If you know ASP.NET, even better

33.  Adobe ColdFusion Exchange  cf_Excel_XML ▪ http://www.adobe.com/cfusion/exchange http://www.adobe.com/cfusion/exchange  Ryan Emerle  cfx_text2query  cfx_excel2query  cfx_query2excel ▪ http://www.emerle.net/old/programming/ http://www.emerle.net/old/programming/  cfTopper  Generate Excel Function ▪ http://www.cftopper.com/index.cfm/blogId/1/tag/ColdFusion http://www.cftopper.com/index.cfm/blogId/1/tag/ColdFusion

34. selene@ritetech.net