Managing Large Data Sets For Finance officers Sean p. Canning mpa, qpa scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org Managing Large Data Sets For Finance officers Sean p. Canning mpa, qpa
Goals of this seminar Add information for ease of automating requested or required analysis using: Google Forms (NEW in 2016!!) Cloud based files Cloud based work order systems Existent web based open source or other data Examples Overtime compilation by departments using Google Forms template for ease of publish for governing body Integrating cloud based work order systems for FEMA reimbursement or other reporting How to make use of open source web scraping through google, other format data for ease of integration scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Assumptions As CMFO’s or financial professionals there is a basic level of usage of excel This may act as a refresher for some advanced techniques or pique interest into pursuing further Tools Will be using Google Forms Table creation Power query tools Pivot tables and charts Allow for updating automatically saving you time scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Example 1 Overtime Automated Reporting This can be used for any updated reporting for governing body or others Budget updates, monthly budget reporting to department heads, FEMA calculations, overtime weekly impacts Cloud based data: Google Forms, One drive, drop box or other prepared template file that can be imported Should be prepared by CMFO and mandated for use by department heads Cloud based input file DPW or multiple department payroll sheet = ADP reports etc Query through excel into database Automated Pivot tables and charts refreshed upon data input scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Link to existing file in cloud Departments prepare overtime report Can be merely downloaded from program such as ADP or form creation Important to have in your source file as a table Can be a single file or multiple files Designated by YOU Updated weekly or after major events Google sheets, updating will be real time Name, rate, overtime rate, hours, pay period scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Google forms method Overtime entry form scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Link to existing file in cloud Using Data New QueryFrom File From Workbook Click on your file sheet having data Place into cell a1 CTRL+T put your data into table format If its existing file merely click “Refresh all” no need to link new query scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Link to existing file in cloud Insert pivot table Pay period on Rows Department on Columns Capable of drilling down Specific employee Percent by department Actual cost per employee FEMA uses will provide for Accurate timely cut and paste scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Insert Pivot Table scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Insert Pivot Table scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Insert Pivot Table scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Example 2 Cloud Based Data Reporting Organizations that have field data entered DPW, water utilities, zoning, assessments in the field etc Ipad or mobile device based data entry into a cloud based software (Iworqs, Pubworks others, google forms) Data can be downloaded directly or often exported from software in a csv file If using google platform is real time Cloud based input file DPW or multiple department payroll sheet = ADP reports etc Query through excel into database Automated Pivot tables and charts refreshed upon data input scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Example 2 Cloud Based Data Reporting Raw Data Download Field based supervisors with smart phones or tablets Use cloud based work order and entry systems Example is raw data from Mount Olive Township Morris County They use Iworq system for public works and public buildings and grounds organizations scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Example 2 Cloud Based Data Reporting Raw Data Download From cloud based system into csv file 2440 line items for 6 months Same process as before Cloud data Raw Data Excel Filter through Pivot scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Example 2 Cloud Based Data Reporting scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Example 3 Types of Open Source Data More Open Source Data SQL used for scripts, is widely used on the web Azure marketplace data Connect and download from data sources such as Lexis Nexis oData Data Source Microsoft creation for restful API’s ODBC – Open data base connectivity XML – Web based data From Access Microsoft tables From Website From Text (text files) scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
NYC Department of Finance Parking Violations Data NYC Open Data NYC Open Data 1500 Data sets updated for open source Several ways to download For this it was a csv for export scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
NYC Department of Finance Parking Violations Data Added one column finding the day of week from the date =text(a2,”ddd””) scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
NYC Department of Finance Parking Violations Data Quick Pivot Table Tells quickly most summonses and fines are weekends Wednesdays and Thursdays runner up Staffing model then would indicate a drill down Are there more parking agents on weekends? Are there more free time activities on weekends giving rise to parking offenses? Do we hire additional or transfer parking enforcement to weekends? What is the ROI of such a move? scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
NYC Department of Finance Parking Violations Data NYC Violations by County and Precinct scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Web scrapers www.Import.Io scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Add ons Wolfram Alpha Takes your searches, returns data about subject and inserts into Google Sheets Example is all about Gloucester County scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Ability to Quickly Manage Large Data Sets Cloud based Into Pivot Tables or other data sets Setup templates Quickly produce reports that can be amended at a click for your reports SAVE TIME!!! scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org
Take Action!! Automate your reporting Identify areas you can implement forms and receive instant data Create a system whereby your stakeholders can receive these reports automatically Sean P. Canning 862-228-3563 www.TheCanningGroup.org Scanning@TheCanningGroup.org scanning@thecanninggroup.org, 973-706-7813 www.TheCanningGroup.org