© OCS Biometric Support 1 Updating an MS SQL database from SAS Jim Groeneveld, OCS Biometric Support, ‘s Hertogenbosch, Netherlands. PhUSE 2010 – CC04.

Slides:



Advertisements
Similar presentations
PHP II Interacting with Database Data. The whole idea of a database-driven website is to enable the content of the site to reside in a database, and to.
Advertisements

Copyright © 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 5 More SQL: Complex Queries, Triggers, Views, and Schema Modification.
Query Methods (SQL). What is SQL A programming language for databases. SQL (structured Query Language) It allows you add, edit, delete and run queries.
Axio Research E-Compare A Tool for Data Review Bill Coar.
Characteristic Functions. Want: YearCodeQ1AmtQ2AmtQ3AmtQ4Amt 2001e (from fin_data table in Sybase Sample Database) Have: Yearquartercodeamount.
What is MySQL? MySQL is a relational database management system (A relational database stores data in separate tables rather than putting all the data.
Let’s try Oracle. Accessing Oracle The Oracle system, like the SQL Server system, is client / server. For SQL Server, –the client is the Query Analyser.
By Hrishikesh Gadre Session II Department of Mechanical Engineering Louisiana State University Engineering Equation Solver Tutorials.
ASP.NET Database Connectivity I. 2 © UW Business School, University of Washington 2004 Outline Database Concepts SQL ASP.NET Database Connectivity.
DT211 Stage 2 Databases Lab 1. Get to know SQL Server SQL server has 2 parts: –A client, running on your machine, in the lab. You access the database.
ASP.NET Programming with C# and SQL Server First Edition Chapter 8 Manipulating SQL Server Databases with ASP.NET.
DAT702.  Standard Query Language  Ability to access and manipulate databases ◦ Retrieve data ◦ Insert, delete, update records ◦ Create and set permissions.
Made in USA Software Development Services Ayoka, L.L.C. 202 E. Border Street, Ste 334 Arlington, TX By Steve Chang.
Chapter 18: Modifying SAS Data Sets and Tracking Changes 1 STAT 541 ©Spring 2012 Imelda Go, John Grego, Jennifer Lasecki and the University of South Carolina.
Understanding SAS Data Step Processing Alan C. Elliott stattutorials.com.
Databases with PHP A quick introduction. Y’all know SQL and Databases  You put data in  You get data out  You can do processing on it very easily 
Chapter 5 Introduction to SQL. Structured Query Language = the “programming language” for relational databases SQL is a nonprocedural language = the user.
Managing Passwords in the SAS System Allen Malone Senior Analyst/Programmer Kaiser Permanente.
Making ODBC easier By Dave Doulton University of Southampton.
© OCS Consulting The flexible extension to your IT team 1 Jim Groeneveld, OCS Consulting, ´s Hertogenbosch, Netherlands. PhUSE 2011 Comparing dataset metadata.
PHP Programming with MySQL Slide 8-1 CHAPTER 8 Working with Databases and MySQL.
1 MySQL and phpMyAdmin. 2 Navigate to and log on (username: pmadmin)
ADO.NET A2 Teacher Up skilling LECTURE 3. What’s to come today? ADO.NET What is ADO.NET? ADO.NET Objects SqlConnection SqlCommand SqlDataReader DataSet.
Chapter 7 Working with Databases and MySQL PHP Programming with MySQL 2 nd Edition.
SAS Efficiency Techniques and Methods By Kelley Weston Sr. Statistical Programmer Quintiles.
Introduction to MySQL Lab no. 10 Advance Database Management System.
1 Structured Query Language (SQL). 2 Contents SQL – I SQL – II SQL – III SQL – IV.
7 1 Chapter 7 Introduction to Structured Query Language (SQL) Database Systems: Design, Implementation, and Management, Seventh Edition, Rob and Coronel.
M1G Introduction to Database Development 2. Creating a Database.
Stored Procedure. Objective At the end of the session you will be able to know :  What are Stored Procedures?  Create a Stored Procedure  Execute a.
Stored Procedures Week 9. Test Details Stored Procedures SQL can call code written in iSeries High Level Languages –Called stored procedures SQL has.
6 1 Lecture 8: Introduction to Structured Query Language (SQL) J. S. Chou, P.E., Ph.D.
Define your Own SAS® Command Line Commands Duong Tran – Independent Contractor, London, UK Define your Own SAS® Command Line Commands Duong Tran – Independent.
Programmatic SQL Shaista Khan CS 157B. Topic Embedded SQL statements in high-level programming languages.
BIS Database Systems School of Management, Business Information Systems, Assumption University A.Thanop Somprasong Chapter # 8 Advanced SQL.
Jessica Bennett, Advance America Barbara Ross, Flexshopper LLC PharmaSUG 2015 Paper #QT06.
Guide to Oracle 10g ITBIS373 Database Development Lecture 4a - Chapter 4: Using SQL Queries to Insert, Update, Delete, and View Data.
© OCS Biometric Support 1 SAS macro %_COUNT_ Jim Groeneveld, OCS Biometric Support, Leiden, the Netherlands. CC01 – PhUSE 2008.
SQL Basic. What is SQL? SQL (pronounced "ess-que-el") stands for Structured Query Language. SQL is used to communicate with a database.
DT228/3 Web Development Databases. Querying a database: Partial info Search engines, on-line catalogues often need to allow user to search a database.
© OCS Biometric Support 1 APPEND, EXECUTE and MACRO Jim Groeneveld, OCS Biometric Support, ‘s Hertogenbosch, Netherlands. PhUSE 2010 – CC05 PhUSE 2010.
Get Rid of Cron Scripts Using Events Sheeri Cabral Senior DB Admin/Architect,
Chapter 5 : Integrity And Security  Domain Constraints  Referential Integrity  Security  Triggers  Authorization  Authorization in SQL  Views 
Week 8-9 SQL-1. SQL Components: DDL, DCL, & DML SQL is a very large and powerful language, but every type of SQL statement falls within one of three main.
>> Introduction to MySQL. Introduction Structured Query Language (SQL) – Standard Database Language – Manage Data in a DBMS (Database Management System)
A Guide to SQL, Eighth Edition Chapter Six Updating Data.
Starting with Oracle SQL Plus. Today in the lab… Connect to SQL Plus – your schema. Set up two tables. Find the tables in the catalog. Insert four rows.
IMS 4212: Application Architecture and Intro to Stored Procedures 1 Dr. Lawrence West, Management Dept., University of Central Florida
MYSQL AND MYSQL WORKBENCH MIS2502 Data Analytics.
ADO .NET from. ADO .NET from “ADO .Net” Evolution/History of ADO.NET MICROSOFT .NET “ADO .Net” Evolution/History of ADO.NET History: Most applications.
Copyright 2009 The Little Engine That Could: Using EXCEL LIBNAME Engine Options to Enhance Data Transfers between SAS® and Microsoft® Excel Files William.
More SQL: Complex Queries, Triggers, Views, and Schema Modification
Creating Database Objects
Chapter 10: Accessing Relational Databases (Self-Study)
Chapter 5 Introduction to SQL.
SQL and SQL*Plus Interaction
Database application MySQL Database and PhpMyAdmin
Views, Stored Procedures, Functions, and Triggers
Chapter 8 Working with Databases and MySQL
Microsoft Office Access 2003
Unit I-2.
SQL-1 Week 8-9.
Session - 6 Sequence - 1 SQL: The Structured Query Language:
Tutorial 6 PHP & MySQL Li Xu
Stata Basic Course Lab 2.
Copyright © 2013 – 2018 by Curt Hill
Information Management
IST 318 Database Administration
Updating Databases With Open SQL
Updating Databases With Open SQL
Presentation transcript:

© OCS Biometric Support 1 Updating an MS SQL database from SAS Jim Groeneveld, OCS Biometric Support, ‘s Hertogenbosch, Netherlands. PhUSE 2010 – CC04 PhUSE 2010

© OCS Biometric Support 2 Updating an MS SQL database AGENDA / CONTENTS 1.Creating an MS SQL database table 2.The write access dillemma 3.Limited write access solution 4.Overview of basic steps and obstacles a.Emptying target table (structure and content) b.Refilling target table, structure and content independently i.Structure: variable types, conversions if incompatible ii.Content: values, conversions if incompatible types 5.Conclusions

© OCS Biometric Support 3 Updating an MS SQL database Creating an MS SQL database table LIBNAME SQLLIB +un+pw………………; LIBNAME SASLIB ………………; 1.a data step, copying a SAS dataset to an SQL table: DATA SQLlib.table; SET SASlib.dataset; RUN; 2.PROC DATASETS, copying a dataset to an SQL table: PROC DATASETS NOLIST; * (or outdated PROC COPY) ; COPY IN=SASlib OUT=SQLlib; SELECT dataset; * becoming a database table; QUIT; 3.SAS PROC SQL, copying a SAS dataset to an SQL table: PROC SQL; CREATE SQLlib.table AS SELECT * FROM SASlib.dataset; QUIT;

© OCS Biometric Support 4 Creating an MS SQL database table The write access dillemma 1.needs general write access in MS SQL database; 2.selective write access to not yet existing, new tables is not possible; 3.full access is a security flaw; 4.SAS should only have limited write access.

© OCS Biometric Support 5 Write access dillemma Solution: limited write access 1.write access to only one or some specific, existing tables; 2.overwriting those tables without actually recreating them; 3.which is just modifying tables; 4.by emptying them completely (data and structure); 5.and refilling them with a new structure and data.

© OCS Biometric Support 6 Solution: limited write access Basic steps (5) in more detail 1.connecting to the database (with username and password); 2.emptying the table, removing all existing data, still leaving the structure; 3.removing the existing structure, the columns and their attributes; 4.defining the new structure (variables and attributes from the SAS dataset); 5.filling the table with data from the SAS dataset.

© OCS Biometric Support 7 Basic steps in more detail Obstacles to overcome The 5 basic steps seem simple but there are many obstacles to overcome. The steps will be discussed one by one. A SAS macro %SAStSQLt will be presented that performs all steps automatically.

© OCS Biometric Support 8 Basic steps in more detail Connecting to the database 1.with username and password; 2.using ODBC engine; 3.two methods, both applied: a.* MS SQL lib ODBC engine; LIBNAME MSSQLsrv ODBC DSN=”SQL server” USER=username PWD=password; b.* Pass-Through Facility; PROC SQL; CONNECT TO ODBC (DSN="SQL server" USER=username PWD=password);

© OCS Biometric Support 9 Basic steps in more detail Remove the table’s contents 1.remove database table’s data via the SQL Pass-Through EXEC statement: EXEC (DELETE FROM &SQLtable) BY ODBC; 2.this could alternatively as well be done using the SAS SQL statement: DELETE FROM &SQL_table; (In here and the following SAS macro variables are being used to denote datasets, tables, variables and lists, like &MSSQLsrv, &SQLtable, etc.)

© OCS Biometric Support Basic steps in more detail Determine existing columns 1.build list of existing column names: SELECT Name INTO :SQLvars SEPARATED BY ' ‘ FROM DICTIONARY.COLUMNS WHERE UPCASE(LIBNAME)='MSSQLSRV' AND UPCASE(MEMNAME)="&SQLtable"; 2.alternatively using PROC CONTENTS: %LET SQLvars = ; %* create empty variable; %Var_List (DATA=&SQL_table, StoreVar = SQLvars, Delim = %STR(,)) /* aux. macro */ %PUT SQLvars=&SQLvars; %* feedback of existing columns in MS SQL table; 10

© OCS Biometric Support Basic steps in more detail Remove the table’s structure 1.unlike a SAS dataset an SQL table should minimally have one column. Thus firstly add a Dummy column: EXEC (ALTER TABLE &SQLtable ADD &Dummy FLOAT) BY ODBC; * &Dummy is unique name; 2.remove all other columns (variables): EXEC (ALTER TABLE &SQLtable DROP COLUMN &SQLvars) BY ODBC; In here the macro variable &SQLvars contains the list of existing columns. 11

© OCS Biometric Support 12 Basic steps in more detail Define the table’s new structure-1 Add column declarations (&SASvars) to the SQL table (&SQLtable): EXEC (ALTER TABLE &SQLtable ADD &SASvars) BY ODBC; The content of &SASvars must meet MS SQL syntax, including VARCHAR(…), FLOAT, DATETIME, comma delimited. SAS has to generate these declarations from the variable types and formats. MS SQL has a specific DATETIME type, not separate DATE and TIME types.

© OCS Biometric Support 13 Basic steps in more detail Define the table’s new structure-2 When directly CREATing an MS SQL table from SAS, SAS automatically and implicitly takes care of the type interpretations and conversions to MS SQL of the column declarations and its values from the SAS formats. In this alternative case the new table structure has to be explicitly defined, including the later value conversions.

© OCS Biometric Support 14 Basic steps in more detail Define the table’s new structure-3 So, every numerical SAS variable has to be checked for its associated format. If the format is one of the many explicitly checked DATETIME, DATE or TIME formats then the corresponding MS SQL column will be of the DATETIME type: %LET DtTm_Fmt = 'DATETIME' 'DATEAMPM'; %LET Date_Fmt = 'DATE' 'DAY' 'DDMMYY' 'DOWNAM'...many more...; %LET Time_Fmt = 'TIME' 'TOD' 'HHMM' 'HOUR' 'MMSS' /*'TIMEAMPM'*/; IF (Format IN: (&DtTm_Fmt)) THEN Attr=TRIM(Name)||' DATETIME'; ELSE IF (Format IN: (&Date_Fmt)) THEN Attr=TRIM(Name)||' DATETIME'; ELSE IF (Format IN: (&Time_Fmt)) THEN Attr=TRIM(Name)||' DATETIME'; The formats have been determined form PROC CONTENTS.

© OCS Biometric Support 15 Basic steps in more detail Define the table’s new structure-4 After all that the mentioned column declaration command can be executed: EXEC (ALTER TABLE &SQLtable ADD &SASvars) BY ODBC; Subsequently the temporary Dummy variable may be removed: EXEC (ALTER TABLE &SQLtable DROP COLUMN &Dummy) BY ODBC; Finally the SQL Pass-Through facility is no longer needed: DISCONNECT FROM ODBC; Now the table must be filled with values.

© OCS Biometric Support 16 Basic steps in more detail Fill the table with data values-1 The data to be fed to SQL, using the SAS SQL INSERT command, are actually unrelated to their already declared column names while inserted. So care should be taken for transferring the data in the correct order with the correct type and with the correct DATETIME value conversion if necessary. The command is: INSERT INTO &SQL_table SELECT * FROM &SAS_dataset; But firstly value conversions needed......

© OCS Biometric Support 17 Basic steps in more detail Fill the table with data values-2 Before doing that certain DATE values need conversion to the DATETIME format. DATE values must be multiplied by (seconds/day). ARRAY ChronoVar _NUMERIC_; * all numeric variables; DO OVER ChronoVar; * (old style DO-loop over vars); IF (VFORMAT(ChronoVar) IN: (&DtTm_Fmt)) THEN ; ELSE IF (VFORMAT(ChronoVar) IN: (&Date_Fmt)) THEN ChronoVar = * ChronoVar; * Convert Date; END; * Only then perform the INSERT command here; INSERT INTO &SQL_table SELECT * FROM &SAS_dataset; PROC SQL may be terminated: QUIT; * Finish PROC SQL;

© OCS Biometric Support 18 Updating an MS SQL database Conclusions 1.Transferring data from SAS to MS SQL seems easy, but is cumbersome while just having limited write access; 2.Yet it is possible by modifying an existing table (structure and data) for which one has write access; 3.Everything discussed here (& more) is part of the macro %SAStSQLt that can be used as a basic macro to transfer data from SAS to MS SQL.

© OCS Biometric Support 19 Updating an MS SQL database QUESTIONS & ANSWERS