REGNET Gloria Lau, Kincho Law, Gio Wiederhold June 8th, 2004 Legal Information Retrieval and Application to E-Rulemaking

1 Motivation  Multiple sources of regulations  Multiple jurisdictions: federal, state, local, etc.  Different formats, terminologies, contexts UK DDA in HTMLADAAG in HTML   Amending rules, conflicting ideas IBC in PDF

2 Motivation  Multiple sources of regulations  Multiple jurisdictions: federal, state, local, etc.  Different formats, terminologies, contexts  Amending rules, conflicting ideas  Need for a repository  Locate relevant information  E.g., small business: penalty fees for violations  Need for analysis tool  Complexity of regulations  Multiple jurisdictions  Understanding of regulations & their relationships

3 Example 1: Related Provisions ADAAG Appendix … Such a curb ramp opening must be located within the access aisle boundaries, not within the parking space boundaries. CBC 1129B.4.3 … Ramps shall not encroach into any parking space. Exception: 1. Ramps located at the front of accessible parking spaces may encroach into the length of such spaces …  CBC allows curb ramps encroaching into accessible parking stall access aisles, while ADA disallows encroachment into any portion of the stall.

4 Example 2: Related but Conflicting Provisions ADAAG Slope. …Transitions from ramps to walks, gutters, or streets shall be flush and free of abrupt changes… CBC 1127B.5.5 Beveled lip. The lower end of each curb ramp shall have a ½ inch (13mm) lip beveled at 45 degrees as a detectable way- finding edge for persons with visual impairments.  ADAAG focuses on wheelchair traversal; CBC focuses on the visually impaired when using a cane.

5 Relatedness analysis Repository development Scope  Repository development  Relatedness analysis  Performance evaluation, results and applications

6 Repository development

7 Sources of data  Accessibility standards  Americans with Disabilities Act Accessibility Guide (ADAAG)  Drafted chapter for rights-of-way access  Associated public comments  Uniform Federal Accessibility Standards (UFAS)  British Standard BS 8300  Scottish Technical Standards, Part S  International Building Code (IBC), Chapter 11  Drinking water standards  Code of Federal Regulations, Title 40 (40 CFR)  California Code of Regulations, Title 22 (22 CCR)  Fire code  International Building Code (IBC), Chapter 9

8 Computational properties of regulations  Hierarchical tree structure  Referential structure  Discipline-centered, e.g., ADAAG for accessibility  Shallow parser to capture computational properties … … Assembly areas with fixed seating shall comply …...

9 Relatedness analysis Repository development Scope  Relatedness analysis  Performance evaluation, results and applications

10 Related elements: door and entrance Relatedness analysis ADAAG 4.1.6(3)(d) Doors (i) Where it is technically infeasible to comply with clear opening width requirements of , a projection... UFAS Minimum Number Entrances required to be accessible by 4.1 shall be part of an accessible route and shall comply with...

11 Relatedness analysis  To utilize the computational properties of regulations for a complete comparison  Measure  Degree of relatedness: similarity score f (A, U)  (0, 1)  Nodes A and U are provisions from two different regulation trees f  (0, 1)

12 Base score f 0 computation  Linear combination of feature matching  F ( A, U, i ) = similarity score between Sections ( A, U ) based on feature i  N = total number of features   = weighting coefficient   Feature matching   Based on the Vector model using cosine similarity as the distance between feature vectors   Non-Boolean features   A measurement of “2 inches max” can be a 70% match to “2 inches”   Synonyms exist, e.g., ontology defined for chemicals   Perform vector-space transformation prior to cosine computation

13 Score refinements based on regulation structure  Neighbor inclusion  Diffusion of similarity between clusters of nodes in the tree

14 Score refinements based on regulation structure  Reference distribution  Diffusion of similarity between referencing nodes and referenced nodes in the tree  E.g., f (A5.3, U6.4(a)) updates f (A2.1, U3.3)

15 Relatedness analysis Repository development Scope Relatedness analysis  Performance evaluation, results and applications

16 Performance evaluation  Conduct a user survey of rankings of similarity  10 randomly chosen sections from the ADAAG and UFAS  Ranks 1 to 100 in the order of relevance  Root mean square error ( RMSE )  = user-generated ranking vector  = machine-predicted ranking vector

17 Survey results - Tabulated RMSE’s  Compared our analysis to Latent Semantic Indexing (LSI)   = structural weighting coefficient   = feature weighting coefficient  Average RMSE smaller than LSI  Measurement feature performs best  No improvement in result observed for structural comparison

18 Results of comparisons: ADAAG vs. UFAS  Related accessible elements: door and entrance  Neighbor inclusion reveals higher similarity

19 Results of comparisons : UFAS vs. BS8300   Terminological differences - revealed through neighbor inclusion

20 Results of comparisons : UFAS vs. Scottish Technical Standards  Terminological differences - revealed through reference distribution  Stairs and ramps

21  Application domain: e-rulemaking  Comparison between draft of rules and the associated public comments  ADAAG Chapter 11, rights-of-way draft  Less than 15 pages  Over 1400 public comments received within 4 months  Comments ~ 10MB in size; most are several pages long  New regulation draft can easily generate a huge amount of data that needs to be reviewed and analyzed  Parsing of the draft and comments  From HTML to XML  Recreate structure of the draft using our shallow parser  Extract features from the draft and comments  Treat individual comments as provisions Application: e-rulemaking

22 E-rulemaking Drafted regulations compared with public comments

23  Related section in draft and public comment Results from e-rulemaking application

24 Results from e-rulemaking application  No related provisions identified  Concern not addressed in the draft

25 Conclusions   A framework for regulatory repository   Structure of regulations recreated in XML   Feature extractions   Prototype for similarity comparisons   Contextual comparisons   Domain knowledge   Structural comparisons   Performance Evaluation, Results and Applications   User survey and comparisons with LSI   Observations of comparisons between Federal, State, non-profit organization mandated codes and European standards   Application on e-rulemaking

26 Future research directions  In the legal domain  Regulatory competition  Cross border data transfer laws  Especially in the polyglot countries in EU  Regulatory updates  Track changes in updates  Track cross references between regulations  Extension of application to other domains of semi- structured documents  Software specifications  User manuals  Similarity/relatedness is settled - how about differences and conflicts?  A lot of almost identical provisions

27 Thank You!