Becoming a Data-Driven Organization LIFELONG LEARNING 2007 GREG MARSELLO

Too many decisions in lifelong learning organizations are made today using the wrong approach. It not only leads to the wrong decisions being made, but it also actually takes more time to make the wrong decisions.

The Wrong Approach The wrong approach usually involves a number of people making the decision, with the decision based more on the opinions of those people rather than the data.

Use a data-driven decision-making approach if you are making decisions on: Activities to offer Teachers or presenters Budgeting Pricing Canceling courses Promotion Marketing Length of time Course/event format Location Brochure copy or any other programmatic or marketing kinds of decisions.

The Wrong Way The wrong way is to make program and marketing decisions based on: Opinion. Many decisions are based on one or more person’s opinion, rather than the actual facts and data. Consensus. Group decisions often try to achieve consensus, more to preserve the group cohesiveness than to make the right decision.

The Wrong Way Personality. The personality of a leader may move a decision in a certain direction. Other personalities, from those who talk a lot to those with purchasing power to those with seniority, often have undue influence. Group morale. Some decisions are actually made more to maintain group morale than to make the right decision. One person might gain deference on a given decision because s/he has not “won” another decision, or certain persons feel left out, or a given decision makes everyone feel better.

Meetings Bad * Committee generally bad * Staff meetings bad It takes time to… Plan Convene Make sure everyone has arrived Do introductions and pleasantries Distribute information Consider exceptions Handle diversions Ignore interruptions Make decisions.

The time spent in meetings, furthermore, takes time away from actually doing the programming and the marketing. Group decision making takes longer, and often deadlines are missed, causing a decrease in registrations.

The Right Way & Person The right way is to make data-driven decisions based on: Customer behavior Customer evaluations Customer demographics Previous history Other data.

The Right Way & Person The right person to make these decisions is the person who has the: Most information, and/or Best information. In almost all programming and marketing circumstances, there is one person who has access to or experience with previous history and more data than anyone else. That person should make the decision.

When Groups Help The rare circumstances when groups make better decisions than individuals. These almost always involve: High dollars, those decisions involving $100,000 or more High risk, those decisions with little or no previous history, often involving precedent-setting or new activities. Little or no data, where there is little or no data available from which to make a data-driven decision.

When Groups Help In these situations, groups often make better decisions than individuals. But these circumstances do not apply to most day-to-day, session-to-session, or event decision making for most programmatic and marketing decisions.

Redefine Decision-Making Authority 1.Assign responsibility. For every ongoing and regular decision involving programming and marketing, assign a person to make the decision. This often can be written into a person’s job description. It can also be conveyed verbally or with a simple .

Redefine Decision-Making Authority 2.Develop levels. Mentally, devise a dollar scale of decision making. For example, front line staff can make decisions up to $100 for such things as purchasing, refunds, discounts, and exceptions to rules. Professionals make decisions up to $10,000 for such things as courses/ events, instructor pay, promotion costs, and so on. Executives are involved in decisions over $10,000.

Redefine Decision-Making Authority 3.No precedent setting. Staff understands they cannot make decisions which are precedent setting. 4.Decision guidelines. By your decision making, by your reporting and staff briefings, by sharing of information, and by demanding information, you make it clear that decisions are to be data- driven, based on customer behavior, participant evaluations, customer demographics, and other data.

Redefine Decision-Making Authority 5.Trust. Initially, top executives will need to trust the decisions made by staff. After a while, most staff will develop data-driven judgment and decision-making skills, and you can take corrective action with those people not making good decisions.

Redefine Decision-Making Authority 6.Advocate and object. If you are asked to serve on a decision-making committee, advocate for data-driven decision making by one or two individuals, document the inefficiencies of the committee, and mentally keep track of the bad decisions made for the next committee assignment. Advocate for data-driven decisions.

Use a data-driven decision-making approach if you are making decisions on: Activities to offer Teachers or presenters Budgeting Pricing Canceling courses Promotion Marketing Length of time Course/event format Location Brochure copy or any other programmatic or marketing kinds of decisions.

Data to Collect Demographics: Registration Promotion Method: Registration/During Activity Evaluations: During Activity/Spot Check Costs: Invoices Preferences: Surveys Not best to collect all data at same touch-point.

Demographics Address Birth Year Gender Occupation Job Title Best that you determine occupation and job title.

Promotion Method Distribution Method Mailing List Snail or Electronic Mailing 1, 2, 3… Word of Mouth Web Site

Evaluations Satisfaction Teacher/Presenter New Activities

Model Programmer Course Evaluation Form Course__________________Teacher__________________ Programmer Evaluation. At the conclusion of the above course, the following information should be generated. A copy should be passed on to your supervisor, a copy should be put in the teacher’s file and a copy should be put in the course file. A. Student Evaluations 1. Overall, were you satisfied with the course? ____ YES ____NO 2. Fill in your response to the following questions: 1–very poor, 2–needs improvement, 3–okay, 4–good, 5–excellent Understood the subject matter._____ Was well prepared for each session._____ Made the goals and objectives clear at course. _____ Stimulated discussion and group involvement._____ Provided individual help when needed._____

3. Comments. On a separate sheet of paper, list key comments on what participants liked about the course and improvements they suggested. 4. Testimonials. On a separate sheet of paper, list key testimonials along with the person’s name. Make sure to hold their evaluation form on file because it has their signature approving the use of their testimonial. B. Teacher Evaluation 1. Overall, were you satisfied with the course? ____ YES ____NO 2. Fill in your response to the following questions: 1–very poor, 2–needs improvement, 3–okay, 4–good, 5–excellent Was the organization’s staff helpful? _____ Was your room prepared for your course?_____ Did the learning environment meet your needs?_____ Did the description clearly outline your course?_____ Did your programmer give you feedback?_____

3.Comments. On a separate sheet of paper, list key comments on what teachers liked about the course and what improvements they suggested. 4. Testimonials. On separate paper, list key testimonials along with the teacher’s name. Make sure to hold their evaluation form on file because it has their signature approving the use of their testimonial. C. Performance Analysis 1. Price of the Course _____ 2. Total Registrations _____ 3. Total Withdrawals_____ 4. Total Income (after withdrawals) _____ 5. Total Cost of Promotion _____ *If there was additional promotion beyond the catalog listing, attach a copy to this evaluation. *To figure the promotional cost of a course in a catalog, divide the total cost of the catalog (desktop, printing, mail, etc.) by the number of courses.

6. Total Cost of Production (instructor fee, materials, space) _____ 7. Total Direct Costs (5 + 6) _____ 8. Operating Margin (4 – 5 + 6) _____ D. Half-Life: The number of weeks/days prior to the course when 50% of registrations were generated _____ E. Participant Analysis Report. On separate paper, breakdown the students’ demographics: Age, Sex, Geography, Education Level. F. Programmer Evaluation 1. Overall, were you satisfied with the course? _____ 2. Rate the teacher on the Likert Scale (1–5)_____ 3. What action steps would you take to improve this course?

Costs Promotion Production Administration $____100% $____10-15% $____45-50% $____60% $____40% $____35% $____5%

Preferences Whatever You Need to Know More Surveys, Not Less

The Language You Should Be Speaking Repeat Rate Lifetime Value Operating Margin Cancellation Rate Brochure:Participant Ratio or Response Rate Staff Productivity Average Participants Average Fee Satisfaction Level Development Costs Go/No Go Point Promotion Cost:Income Ratio

Making It Happen LERN Program Planner Tool LERN Contract Training Tool LERN Segmenting Tool Web-Based Software

80/20 7 Primary Market Segments Push Down Data Analysis

Dig Deeper Analysis by market segment involves taking each of your market segments or primary customer audiences and analyzing sales.

Total sales by market segment What is my top market segment by sales? What are my four top market segments by sales? My four top market segments are what percent of total sales? My next three top market segments are what? What market segments are growing faster than others? Are any market segments stable or declining in sales? Do I keep them?

Average sales per customer What is my top market segment by average sales per person? What are my poorest market segments by average sales per person? Do I keep targeting my poorest market segments?

Percentage growth What market segments are growing as a percentage of total sales? What market segments are declining as a percentage of total sales? What is my strategy for my segments that are growing? What is my strategy for those segments not growing?

Market share/market potential As market share, how are my market segments doing? As market potential, how are my market segments doing? What do I keep as my primary market segments? What do I drop as my primary market segments? What do I add as my primary market segments?

By product How are sales of each of my product lines by market segment? Are there new products/programs or product lines I need to develop for a particular market segment?

Top customers Who, by name, are my top 1,000 customers for this year, by dollar sales? Who, by name, are my top 100 customers? Who, by name, are my top 10 customers? Is there anything I want to write, say or do for my top customers?

Thank You! Be Data-Driven!