1. National Science Foundation Office of Inspector General Update
National Conference on College Cost Accounting 2015 Annual Conference
October 13, 2015
Brett M. Baker, Ph.D, CPA, CISA
Assistant Inspector General for Audit
National Science Foundation

2. Overview NSF OIG overview Audit update DATA Act and IPERA
Data analytics in the federal government
Applications in grant oversight
Dr. Brett Baker, AIGA, NSF OIG

3. Inspectors General 72 Federal Inspectors General (IGs)
Promote economy and efficiency
Prevent and detect fraud, waste, and abuse
Audit and evaluate agency programs and activities
Investigate allegations of wrongdoing
Operate a fraud hotline
Review agency compliance with legislation and regulations
Advise agency head and Congress
Dr. Brett Baker, AIGA, NSF OIG

4. Inspectors General Do Not….
Operate programs or decide funding
Perform management functions
Make agency employment decisions
Enforce implementation of OIG recommendations
Suspend or debar recipients
Dr. Brett Baker, AIGA, NSF OIG

5. Who We Are Expertise in areas of research,
Inspector General
Assistant IG for
Audit
Counsel to the IG
Assistant IG for Investigations
Financial Statement
and IT Audit
Administrative Investigations
OIG Management Support
Performance Audit
Civil/Criminal Investigations
External Audit
Investigative
Legal / Outreach
Audit Services and Data Analytics
Expertise in areas of research,
grant, and contract administration
Investigations Specialists and Support Staff
Dr. Brett Baker, AIGA, NSF OIG

6. OUTREACH Two Sides of the House Audit Investigations
External Audit Program
Audits of awardee financial, administrative and internal controls.
Reports findings to NSF for audit resolution (e.g., return of funds).
Internal Audit Program
Audits of NSF programs and operations to determine program compliance and efficiency of operations.
Report findings to Congress, National Science Board, and NSF.
Investigations
Civil and Criminal
Cases of Fraud, Theft, Conspiracy, Embezzlement, False Statements, etc.
Consequences: Restitution, Fines, Prison, Compliance Plans.
Administrative
Cases of Research Misconduct; Breach of Confidentiality; Human Subjects, Animal Welfare.
Consequences: Debarment, Certifications and Assurances, Remedial Training, etc.
OUTREACH
Semiannual
Reports
to Congress

7. Audit Planning Work Required by Law: Discretionary Work:
Agency Financial Statement Audit (CFO Act)
Federal Information Security Management Act (FISMA)
Improper Payment Elimination and Recovery Act (IPERA)
Conference spending and government purchase card oversight
DATA Act
Discretionary Work:
OIG Risk-based Assessments
Internal NSF operations
External award oversight
Referrals from Investigations
NSF Management Challenges
National Science Board and NSF Suggestions
Congressional Requests
Dr. Brett Baker, AIGA, NSF OIG

8. Distribution of Audit Work
Dr. Brett Baker, AIGA, NSF OIG

9. Greater Attention to Analytics in Government
DATA Act
Promotes data sharing across government agencies
Treasury data analytics center for OIGs – automated oversight
Government-wide structured data standards for financial reporting
USASpending data should be standardized and machine-readable
OIGs will audit data quality
Improper Payments Elimination and Recovery Act (IPERA)
Amends the Improper Payments Information Act of 2002
IPERIA strengthens estimations
Strengthens detection, prevention, and recovery efforts
Pre-award and pre-payment checks with Do Not Pay
Annual risk assessments of covered programs
Published improper payment estimates with reduction targets
Goal to reduce improper payments by $50B and recover $2B in 2 yrs
Dr. Brett Baker, AIGA, NSF OIG

10. Programs with Improper Payment Estimates Over $1 Billion in FY 2014
Source: GAO
Dr. Brett Baker, AIGA, NSF OIG

11. Risk Identification for Oversight
Why look for risk?
Helps ensure programs are working and funds are being used properly.
Government responsible for all activity, risk-based approaches help focus effort on things we need to pay more attention to.
General risks
What an institution is….
Some contract and grant awards are riskier than others
-- Smaller institutions tend to have weaker internal controls
Activity-based risks
What an institution does….
Award actions that stand out from normal activity
-- Large drawdown on a single date – end of a fiscal year
-- Spending out remaining award fund just before and after the award
Challenges
General risks can be more obvious
Activity-based risks are more revealing, but can be harder to see
Dr. Brett Baker, AIGA, NSF OIG

12. Focus on Risk Many to the Few
600, Grant award drawdowns annually
totaling $7 billion
Each assigned a risk score
Spikes, near award expiration charges
40, Active awards
Summarize drawdown risks by award
2, Institutions
Summarize award risk and add CCR, EPLS, FAC risks
20 Audits of higher risk institutions
Each audit tests all expenditures
for all awards
with automated risk indicators
15,000,000 Transaction-level expenditures tested
Dr. Brett Baker, AIGA, NSF OIG

13. Forensic Audit Framework
Set Objectives and Define Universe
Structured brainstorming with subject matter experts
Map out End-to-End Process
Identify systems, key processes and controls
Obtain Transaction-Level Data
Build Targeted Business Rules and Run Against Data
100% review with automated business rules based on risk
Award/institution anomalies and changes in patterns over time
Examine Anomalies
Dr. Brett Baker, AIGA, NSF OIG

14. Risk Identification through Automated Oversight
Improved risk identification
100% transaction review – limited statistical sampling
Automated business rules based on risks
Focus review on higher risks
Key data analytics software techniques
Join databases (need linking field)
Summarize data (many to the few)
Apply risk indicators using computed fields
Develop risk profiles by institution, award-type, transaction-type
Summarize risk into one number
Agencies and recipients can use similar data analytics techniques
Monitor grant spending
Identify anomalies early
Dr. Brett Baker, AIGA, NSF OIG

15. Comparing Data Files (Three Bucket Theory)
Vendors
Paid and
In Vendor
Table
Vendors
Paid but not
In Vendor
Table
Vendors
Not Paid
Yet
Vendor Table
(Valid vendors)
Disbursing
Transactions
Linking field allows
for comparison
Dr. Brett Baker, AIGA, NSF OIG

16. Normal Distribution Anomalous Activity Anomalous Activity
Normal Activity
Dr. Brett Baker, AIGA, NSF OIG

17. Building a Forensic Capability
Develop organizational capability
All staff should have basic skill with data analysis tools
Forensic units perform more sophisticated analyses
Targeted reviews are more efficient in time and cost
Phased development
Hardware and software
Access to internal and external data
Staff: system savvy, analytical, business process knowledge
Training, then immediate application to work
Very important component is tone at the top
Dr. Brett Baker, AIGA, NSF OIG

18. Transaction-level Data
Framework for Data Analytics Using Government and Publicly Available Data
Award-level Data
Grants, Contracts
Transaction-level Data
Payee, Contract No, CLINs, Payment Amount, Date
Award
Systems
Payment
Systems
Disbursing
Systems
Federal Reserve
System
Commercial
Bank
Data Download
Data Download
Data Download
Data Download
Contract Invoices
Grant Pmt Req’s
Data Analytics
Join databases
Apply risk indicators
Risk rank transactions
Identify anomalies for testing
Oversight
Review by
Auditors
Investigators
Agencies
Data Download
Data Download
Data Download
Data Download
Data Download
GuideStar
(non-profits)
SAM
(CCR, EPLS)
Federal Audit
Clearinghouse
Master Death
File (SSA)
CPARS, FPDS
Examples of systems that can be used to help understand payment transactions
Dr. Brett Baker, AIGA, NSF OIG

19. U.S. Financial Assistance Overview
$600 billion in awards
88,000 awardees and 26 Federal grant making agencies
Project and research, block, and formula
Outcomes are designed to promote public good
Challenges
Limited visibility of how Federal funds are spent by awardees
Support for funding requests much less than for contracts
American Recovery and Reinvestment Act (2009)
$840 billion of assistance to stimulate the economy
Greater accountability and transparency over spending than ever
Opportunities to enhance oversight with less
Automated oversight
Dr. Brett Baker, AIGA, NSF OIG

20. Framework for Grant Oversight
Data analytics-driven, risk-based methodology to improve oversight
Identify institutions that may not use Federal funds properly
Techniques to surface questionable expenditures
Life cycle approach to oversight
Mapping of end-to-end process to identify controls
100% review of key financial and program information
Focus attention to award and expenditure anomalies
Complements traditional oversight approaches
Techniques to review process and transactions are similar
Transactions of questionable activities are targeted
Recipients and Agency Officials can use data analytics
Identify high risk activities through continuous monitoring
Dr. Brett Baker, AIGA, NSF OIG

21. Grants Differ From Contracts
Promote services for the
Public Good
Merit review (competitive)
Multiple awardees
Award budget
No government ownership
Grant payments
Summary drawdowns
No invoices for claims
Expenditures not easily visible
Salary percentages
CONTRACTS
Specified deliverables
(Goods and Services)
Competitive process
One awardee
Contract Price
Government ownership
Contract payments
Itemized payment requests
Invoices to support claims
Detailed costs
Salary hourly rates
Dr. Brett Baker, AIGA, NSF OIG

22. End to End Process for Grant Oversight
CASH
DISBURSEMENTS
PAY/
ENTITLEMENT
PRE-AWARD
REVIEW
CASH
REQUEST
AWARD
CLOSE-OUT
SOLICITATIONS
PROPOSALS
POST AWARD
AWARD
PRE-AWARD RISKS
ACTIVE AWARD RISKS
AWARD END RISKS
No /Late Final
Reports
Cost Transfers
Spend-out
Financial
Adjustments
Unmet Cost
Share
Funding Over Time
Conflict of Interest
False Statements
False Certifications
Duplicate Funding
Inflated Budgets
Candidate
Suspended/Debarred
Unallowable, Unallocable, Unreasonable Costs
Inadequate Documentation
General Ledger Differs from Draw Amount
Burn Rate
No /Late/Inadequate Reports
Sub-awards, Consultants, Contracts
Duplicate Payments
Excess Cash on Hand/Cost transfers
Unreported Program Income
DATA ANALYSIS
Dr. Brett Baker, AIGA, NSF OIG

23. Look at Red Flag Areas
Use Data Analytics to identify anomalies that are potential fraud indicators, such as:
breaks in trends, outliers… 2 3
The more red flags,
the higher the risk.
Proposals
Solicitations
Recipient Project System
Pay System
Acctg System
HR System
Reports
Internal Grants Portal
Awards System
Proposal System
External Grants
Portal
Award Close-Out
Post Award Monitoring
Award Notification
Pre-Award Review
Research
Idea 5
RISK 6 1
The less red flags,
the lower the risk. 7
Funding
RISK 4 8 9
Dr. Brett Baker, AIGA, NSF OIG

24. Data Sources Internal External Recipient financial system records
Proposals: budgets, panel scores
Agency award systems, recipient reporting
External
Excluded Parties List System (EPLS)
Central Contractor Registration (CCR/SAM)
Public tax filings
Federal Audit Clearinghouse (A-133 audits)
Recipient financial system records
General ledger and subsidiary ledgers
Effort reporting
Property
Travel and purchase card
Subaward monitoring
Dr. Brett Baker, AIGA, NSF OIG

25. Identification of Higher Risk Institutions and Transactions
Dr. Brett Baker, AIGA, NSF OIG

26. Anomalous Drawdown Patterns$$
Spending
Remaining
Grant funds
(before expiration)
Spending
Remaining
Grant funds
(after expiration)
Start up
costs
Drawdown
Spike
Normal drawdown pattern
Grant
Award
Grant
Expiration 26
Dr. Brett Baker, AIGA, NSF OIG

27. Early Drawdown
Dr. Brett Baker, AIGA, NSF OIG

28. Draw Spike
Dr. Brett Baker, AIGA, NSF OIG

29. Spend out Pattern
Dr. Brett Baker, AIGA, NSF OIG

30. Does this drawdown pattern look okay?
Dr. Brett Baker, AIGA, NSF OIG

31. Common Audit Findings Pre-Data Analytics Audits Data Analytics Audits
(stat sampling projections)
Unsupported costs
Effort reporting
Effort reporting (subaward)
Pre-award charges
Data Analytics Audits
(actual transactions)
Unallowable, unallocable, unreasonable costs
Excess salary
Indirect Costs
Equipment
Dr. Brett Baker, AIGA, NSF OIG

32. When should you contact OIG?
Report significant administrative or financial problems
Report allegations of wrongdoing
Research misconduct
Fraud / theft involving NSF funds
Violation of regulation, directive, or policy
(anonymous hotline)
(business hours), or
(non-business hours)
Online hotline through our website webform.
NSF Hotline
Dr. Brett Baker, AIGA, NSF OIG

33. OIG Outreach Presentations at conferences and seminars
For students, PIs, and administrators
OIG outreach visit can be requested
Fact sheets and brochures, briefings, conference presentations
OIG Semiannual Report
Requests:
Dr. Brett Baker, AIGA, NSF OIG

34. Questions?
Dr. Brett M. Baker Assistant Inspector General for Audit National Science Foundation Office of Inspector General Phone: