1 Peer Review Casualty Actuaries of New England Spring Meeting April 2, 2008 Jeff Kucera

2 Table of Contents  Introduction  Background information and Peer review overview  Why peer review?  Peer review process  Small firm issues

3 Quality measures and audits  Assures that methodology is appropriate and reasonable for task at hand  Helps to deliver more consistent products and services  Quickly addresses any problems that occur and will often reduce potential damage

4 Background information  C onference of Consulting Actuaries- Committee on Professionalism- On CCA website (  Peer review guidelines  AAA Peer Review discussion paper- --- On Academy website (  2005 edition replaced 1997 publication  Basic need to follow Code, ASOPs, Qualification Standards

5 Peer review  Not a required measure  Large firms typically have peer review policies in place  Some smaller firms have peer review policies while others are looking into it

6 What is peer review? Evaluation of a work product or advice by an independent qualified professional Evaluation of a work product or advice by an independent qualified professional Exchanging ideas, results, and answers to improve work product Exchanging ideas, results, and answers to improve work product Generating ideas to ensure information communicated is reasonable and appropriate with respect to the assignment Generating ideas to ensure information communicated is reasonable and appropriate with respect to the assignment

7 Peer review vs. checking Checking is more detailed than peer review Checking is more detailed than peer review Peer review is done to ensure results are reasonable and the process used by the author is appropriate Peer review is done to ensure results are reasonable and the process used by the author is appropriate Peer review may include confirming that the scope of the assignment has been addressed, and that the communications are clear Peer review may include confirming that the scope of the assignment has been addressed, and that the communications are clear

8 Why peer review? Could improve compliance with Code, ASOPs, law/regulation and applicable actuarial practice & standards Could improve compliance with Code, ASOPs, law/regulation and applicable actuarial practice & standards Could improve compliance with business and professional standards, thereby enhancing the quality of work Could improve compliance with business and professional standards, thereby enhancing the quality of work Can get an additional perspective into the problem or assignment Can get an additional perspective into the problem or assignment Will help to satisfy Professional responsibilities Will help to satisfy Professional responsibilities Can develop expertise and improve skills Can develop expertise and improve skills

9 Why peer review? (cont.) Could enhance the overall quality of the final work product Could enhance the overall quality of the final work product May result in better advice in many cases May result in better advice in many cases May catch embedded errors before they become big May catch embedded errors before they become big May help to minimize errors and omissions exposure May help to minimize errors and omissions exposure May help to write document clearly that can be understood by broader audience May help to write document clearly that can be understood by broader audience

10 Why peer review? (cont.) Client/supervisor will have more confidence when the work products and advice they receive are peer reviewed Client/supervisor will have more confidence when the work products and advice they receive are peer reviewed Can demonstrate that extra care was taken Can demonstrate that extra care was taken Can improve methodology and assumptions Can improve methodology and assumptions Can help to deliver creative and effective solutions to our clients’ business problems Can help to deliver creative and effective solutions to our clients’ business problems

11 Steps involved in Peer review Technical Technical Process Process

12 Technical peer review Have applicable laws, regulations and guidance been appropriately reflected? Have applicable laws, regulations and guidance been appropriately reflected? Are applicable Actuarial Standards of Practice appropriately reflected? Are applicable Actuarial Standards of Practice appropriately reflected? Have uncertainties been appropriately addressed? Have uncertainties been appropriately addressed?

13 Technical peer review (cont.) Are actuarial methods and assumptions appropriately described and supportable? Are actuarial methods and assumptions appropriately described and supportable? Does data seem reasonable? Does data seem reasonable? Does the advice seem appropriate? Does the advice seem appropriate? Do the results fall within reasonable expectations? Do the results fall within reasonable expectations?

14 Technical peer review (cont.) Have appropriate caveats and limitations been communicated? Have appropriate caveats and limitations been communicated? Does the work deviate from standards? If so, disclose and ensure that that the work can be supported Does the work deviate from standards? If so, disclose and ensure that that the work can be supported

15 Process peer review Is the assignment well defined? Is the assignment well defined? What question did the client ask? What question did the client ask? Does the question solve client’s problem? Does the question solve client’s problem? Does the work performed appear to be consistent with the assignment? Does the work performed appear to be consistent with the assignment? Does the actuarial analysis appear to be logical? Does the actuarial analysis appear to be logical? Do proposed solutions appear to be appropriate? Do proposed solutions appear to be appropriate?

16 Are proposed solutions of value to the end user? Are proposed solutions of value to the end user? Do recommendations appear to be consistent with professional and ethical standards? Do recommendations appear to be consistent with professional and ethical standards? Have alternative solutions and approaches been explored? Have alternative solutions and approaches been explored? Do written communications appear to be clear? Do written communications appear to be clear? Process peer review (cont.)

17 When to peer review? Peer review should start in early stages of the project, and should end with the review of the final product Peer review should start in early stages of the project, and should end with the review of the final product If peer review is not complete before the release, the document should be marked : If peer review is not complete before the release, the document should be marked : DRAFT -- SUBJECT TO CHANGE UPON PEER REVIEW DRAFT -- SUBJECT TO CHANGE UPON PEER REVIEW Follow-up with final document after the review is completed Follow-up with final document after the review is completed

18 Qualification for peer reviewer Proper education Proper education Technical knowledge Technical knowledge Skills and experience Skills and experience Independent status Independent status As knowledgeable as the original actuary As knowledgeable as the original actuary

19 Scheduling and process Agree on what type of peer review needed Agree on what type of peer review needed Agree well in advance on a schedule Agree well in advance on a schedule Provide supporting documentation to peer reviewers Provide supporting documentation to peer reviewers Document peer review Document peer review

20 When peer review could be most helpful? Assignments with significant financial implications, such as mergers & acquisitions Assignments with significant financial implications, such as mergers & acquisitions Non-routine projects Non-routine projects Advice based on assumptions and methods Advice based on assumptions and methods High risk assignments- labor negotiations High risk assignments- labor negotiations Innovative or uncommon solutions Innovative or uncommon solutions

21 Documenting a Peer Review Peer review documentation is always a good idea Peer review documentation is always a good idea Peer review documentation may include: Peer review documentation may include: Identification of type of peer review; Identification of type of peer review; Initials of reviewer and date review Initials of reviewer and date review Reviewer’s comments and the resolution of any professional disagreements Reviewer’s comments and the resolution of any professional disagreements

22 Peer reviewer’s role Gain knowledge of the project Gain knowledge of the project Ensure what is expected from reviewer Ensure what is expected from reviewer Provide feedback in a timely manner Provide feedback in a timely manner

23 Reconciling Differences of Opinion Peer reviewer advice should not be ignored Don’t leave issues unresolved Don’t leave issues unresolved Identify alternatives to resolve disagreement Identify alternatives to resolve disagreement Engage third party Engage third party ABCD for counseling ABCD for counseling Ultimate responsibility for work product Ultimate responsibility for work product

24 Implementing a Peer Review program Small Firm (Company) vs. Large Firm (Company) considerations Small Firm (Company) vs. Large Firm (Company) considerations Costs Costs Availability of Peer Reviewer Availability of Peer Reviewer

25 Small Firm Issues---Overview  Small firm needs review as much as larger firm  Organized peer review process --- very valuable, not always feasible  Integrate peer review into broader quality control  Screening of assignments  Organized processes  Selection of team

26 Small Firm Issues---Overview  Resources are a particular challenge  Situations vary depending on the assignment  Consider in context of broader quality control issues: peer review is optional and part of a portfolio of quality control possibilities

27 Thinking about quality control  Incorrect work can harm user in some cases and in others not, degree of harm can vary greatly  Example of potential major harm --- valuing company too highly in potential acquisition  Example of very limited harm --- statement that is not correct in magazine article  Some people specialize in specific areas  Examples: small pension plan advisors, expert witness, divorce calculations, personal actuaries, etc.

28 Finding resources for quality control  Some reviewers are big picture thinkers, some are very focused on the details and calculations, others on communication  Firm with multiple professionals --- can train everyone and exchange work  Issues similar to larger firms

29 F inding resources for quality control  One person firm/department  Peer review is not mandatory – decide when needed  Work with outside persons  Maintain a list of possible reviewers/their qualifications  Make an agreement with someone who you exchange with  In some cases, client may provide reviewer

30 Different kinds of assignments  Well defined and repeating assignments --- e.g. reserve calculations, rate indications, etc.  Assignments needing framing and definition --- e.g. predictive modeling, Enterprise Risk Management

31 Working as a sub-contractor Client may provide peer review Client may provide peer review Quite common to work as independent contractor for former employer Quite common to work as independent contractor for former employer In some cases, there is no separate work product --- contractor simply becomes member of team In some cases, there is no separate work product --- contractor simply becomes member of team Important to define who is responsible for work product Important to define who is responsible for work product

32 Questions?