1. Eric C. Glass & Dana Neacsu
Data Mining (scraping) & Interdisciplinary Research in Law:   Gov Docs: Domestic and Foreign
Eric C. Glass 
&
Dana Neacsu

2. Web Scraping
Extracting and parsing formatted data from a web page *(HTML, XHMTL, JSON etc.).
Automated or manual
Python
Beautiful Soup -
 Toolkit for dissecting a document and extracting what you need. It doesn't take much code to write an application
Manages encodings
Sits on top of popular Python parsers like lxml and html5lib
Gathering election results example:

3. Web Scraping Automated tools (no programing) Import.io Webscraper.io
Cloud based web application
No longer free apparently, but free trial is available
Webscraper.io
Chrome browser plug in available free
 Sitemap building, data extraction and export are all done within browser
Have not used, but there is Youtube:

4. Web Scraping Premade tools (many applications on GitHub)
Example – NYPD Crash Data Band Aid
On Github -
NYPD released data based through “idiotically obfuscated PDFs”
Tool is built in python and on top of xpdf and wget

5. Training and Help Lynda.com (through libraries license) Code Academy.
Python: Programming Efficiently
Code Academy.
Python intro in addition to a variety of web based APIs
Digital centers in the libraries
Python open lab
R open lab
Collaboratory at Columbia University
An appointment-based free consulting service for students and researchers at Columbia University that offers assistance with planning and executing data driven research projects, including help with data visualization, analysis and prediction, both in conceptual terms and with concrete software implementations.

6. Read your question for research clues:
Q. 1. What is the connection between gun ownership and public health?

7. Open Data Map of open data policies: NYC Open data NYS Open Data
NYC Open data
On March 7, 2012, former Mayor Bloomberg signed Local Law 11 of 2012, more commonly known as the “Open Data Law,” which amended the New York City administrative code to mandate that all public data be made available on a single web portal by the end of 2018.
NYS Open Data
FOIA & FOIL requests

8. Collections
Inter University Consortium for Political and Social Research (ICPSR)
A rich data archive of over 7,500 titles presented with full documentation and most with data formatted for use in standard statistical packages.
ProQuest Statistical Insight
Provides statistical data from U.S. government publications from 1973, state and private sources from 1980, and international organizations from 1983.
Historical Statistics of the United States
ProQuest statistical abstract of the U.S. 
DSSC data catalogs
Social Sciences resources -

9. Preliminary meta-research 1. Free of charge databases. 2
Preliminary meta-research 1.Free of charge databases? 2. Fee-based databases? How do you choose? Who publishes them? Where can you access them?
Open data:
USA.gov, Data.gov, Data.un.org
Government Agency websites
Sunlight Foundation
Closed data:
ICPSR
Govistics
Columbia Spatial Data Catalog

10. Read your question for research clues:
Q. 2.  Creating a false appearance of active trading in the market by investors is a domestic and international problem. Can it be regulated?

11. Clarify your question (break it into smaller concepts) Do you need contextual information? (literature search) Can you go for the primary source, the rule? Where do you start?

12. More preliminary meta-research domestic law research 1
More preliminary meta-research domestic law research 1.Free of charge databases? 2. Fee-based databases? How do you choose? Who publishes them? Where can you access them?
Remember a mere Google search may give you the right starting point

13. Answer – US regulations
Doing the research!
Depending on your research needs you may or may not skip the literature search (regular law-based databases)
Find gov docs free-of charge databases
Use a library guide
Gov reports (CRS reports)
Agency reports and other activities
Use fee-based databases
Bloomberglaw.com
Practical Law (from Westlaw)

14. More preliminary meta-research foreign and international law research 1.Free of charge databases? 2. Fee-based databases? How do you choose? Who publishes them? Where can you access them?
Remember a mere Google search may give you the right starting point

15. Answer – EU regulations
Doing the research!
Depending on your research needs you may or may not skip the literature search (regular law-based databases)
The main EU database is free of charge: Europa
Use a library guide
Use the database itself
Find legislation on your topic
The main UN databases are free of charge
UNCITRAL
Google Searches

16. Questions?
Eric C. Glass
Dana Neacsu