The What and How of Measuring Electronically Mediated Work:

The What and How of Measuring Electronically Mediated Work:
A Review of Different Countries’ and Private Organizations’ Experiences Anne E. Polivka July 2019

BLS The views in this presentation do not represent official BLS policy or necessarily the views of other BLS staff.

Outline of Talk Discuss what is trying to be measured
Why may want a particular measure How measurements have been made Compare various estimates Concluding thoughts

What Included in Review
Electronically Mediated Employment/Platform Work/Digital Platform Work Will be using terms interchangeably Excluding sharing economy, collaborative platform work Where there is no remuneration for work or lending of an asset i.e. Couchsurfing, Zipcar, 1000Tools, Freecycle Discussing from the worker’s perspective not business’s

Included in Review Early “modeled” estimates
Government administrative data Private company data Household survey data covering Private sector, academic, government sponsored, and national statistical offices’ surveys 24 Surveys

What To Measure

What is Platform Work? Concepts of platform work differ
No agreed upon definition What is included varies along many dimensions

General Conceptual “Definitions”
EU describes the main features of platform work as: Paid work organized through platforms Three parties involved: platform, client, worker Aim is to conduct specific task or solve specific problem Form of outsourcing/contracting out Break-down of ‘jobs’ into tasks On-demand services

BLS describes electronically mediated work as arrangements where workers Use a company’s website or mobile app to connect to clients or customers and obtain short jobs, projects or tasks Are paid by or through the company that owns the website or mobile app May do short jobs, project or tasks in person or online Chooses when and whether to work

U.S. Commerce Department describes digital matching companies as: Using information technology (IT systems), typically via web-based platforms such as “apps” or internet-enabled devices to facilitate peer-to-peer transactions Relying on user-based rating systems for quality control Allowing individuals providing services via platforms flexibility in deciding their typical hours of work Relying on workers using own assets and tools U.S. Commerce Department describes digital matching companies as exhibiting the following characteristics: Use information technology (IT systems), typically available via web-based platforms such as “apps” or internet-enabled devices to facilitate peer-to-peer transactions Rely on user-based rating systems for quality control, ensuring trust between consumers and service providers who have never meet Individuals who provide services via digital matching platforms have flexibility in deciding their typical working hours To the extent that tools and assets are necessary to provide a service, digital matching firms rely on the workers using their own. Unlike employees of traditional firms that use assets owned by the firm to perform services, these workers either own or have personal access to the assets that are used to provider services. Often, the assets must meet a set of criteria that the firm dictates

Scandinavian countries describe platform work as: A triangular relationship between the platform company, those who provide labour and those who purchase labour services.

Non-official (personal) working definition of digital platform work is at a minimum: Paid work where an app or website (digital platform) plays an integral role in helping to connect workers to customers ….. Want to exclude Job search for a longer term job e.g. CareerBuilder or Monster.com Employment placement agencies Social media e.g. LinkedIn Would need to define what is meant by integral. May want to add “to complete short, discrete tasks or projects” May want to add “or for the rental of an asset owned by the worker” May want to add “or the sale of a good owned by the individual selling it” But will discuss these “add ons” along with and after the discussion of what to measure

In Determining What to Measure
There are additional questions have to address when deciding what to measure Identified 8 questions need to be aware of Although researchers/designers may not always be intentional in these decisions Could be implicit in what data decide to use or formulation of question wording

Does location of where work is done matter?
Platform work can be done at a customer’s physical location (in-person), or virtually (completely on-line) For some purposes may not want to make a distinction For some purposes may want to measure both, but separately For other may want to restrict to in-person work First question, does the location of where the work is done matter? Measurements tend to have focused on the in person work or in-person and online, but others have looked at just online and some have not made a distinction.

Does the type of customer matter?
Want to restrict the measure to peer-to-peer transactions? Want to include peer-to-business transactions, but make a distinction between peer-to-peer and peer-to-business? Don’t care about the type of customer? Second question, does the type of customer matter? Most measures have either not made a distinction or have focused just on peer-to-peer transactions.

How much labor services does the transaction need to involve?
Transactions done through digital platforms occur along a labor service-capital spectrum Only labor services Labor using assets Selling goods made Renting asset, some maintenance service Selling own used goods Providing financial capital Third question, how much labor services need to be involved in the platform transactions? This really is a question of what types of activities to include in the measurement. Measurements have been across the spectrum. Most do not include providing financial capital, but a few such as one of the Canadian measures do.

How much labor services does the transaction need to involve?
Transactions done through digital platforms occur along a labor service-capital spectrum Only services provided – e.g. baby sitting, dog walking Services provided using personal asset – e.g. ride-hailing, repair tasks Selling new goods produced or obtained for sale - e.g. making handcrafted items or buying antiques Renting asset, maintenance service may be involved - e.g. renting spare bedroom or vacation home Selling own used or surplus goods Providing financial capital –e.g. providing start up capital

How much control does the platform company need to exhibit?
Transactions done through digital platforms can have various levels of platform company involvement in the customer, client interaction Minimal Workers posting on bulleting boards Intermediate Worker posts profile including wage Customer selects worker Company can facilitate payment and background checks Intermediate Company maintains lists of tasks on site Worker chooses task Payment trough site Customers rating of work on site Extensive Worker posts profile including wage Company algorithmically matches First to accept gets work Payment via app Customers rate via app Very extensive Company algorithmically matches Company determines wage and price Customers rate via site Worker can only turn down a few assignments and must maintain minimum rating Fourth question, how much control over workers does the platform need to exhibit? Control can range from minimal, where workers maintain almost complete control. For example workers, posting ads on electronic bulletin boards stating they are looking for work and what they charge. Workers can then determine what customers to accept and how and when to perform the work. The worker/customer matching basically is done “manually” and customers pay workers directly. (For example posting on Craig’s or Angie’s list for plumbing work or tutoring jobs). On the other end of the spectrum are companies that algorithmically match workers to customers via an app. The company determines workers’ wages and the price customers are charged. Companies facilitate customers’ payments. Customers can rate workers, and workers need to maintain a high rating in order to continue on the platform. Workers have the flexibility to choose when to work, only working when they have the company’s app on. However, when workers have the app on, they only have a limited number of customers that they can turn down prior to being barred from the platform. Workers use their own equipment, but the digital platform company also can set standards for the equipment that workers use. For example, Uber and Lyft. Most measures intentionally or not have focused on company control from intermediate level to very extensive. However, some have included from minimal to very extensive in a single measure.

How much control does the platform company need to exhibit ?
Minimal Workers post ads on electronic bulletin board Customers contact workers themselves (by phone or electronically) Workers set own price (wage) Workers can turn down customers Workers determine how and when to complete work Customers pay workers directly -E.g. Craig’s list, Angie’s list

Intermediate Company recruits workers for provision of specific type of service Workers post a profile of themselves including rate of pay will accept Customers search profiles and select a worker Company can facilitate payment through website Company can maintain workers’ background checks -E.g. Care.com Sittercity.com

Intermediate Company maintains list of customers’ tasks workers can review and select to complete on website Payment is through the app or website Customers’ rating of workers can be posted on the platform -E.g. Amazon Mechanical Turk

Extensive Workers set up a profile including their schedule and rate of pay will accept Companies algorithmically match customers’ requests and workers based on workers profile First worker to accept obtains assignment Payment is through the app Workers can be rated via the app -E.g. Taskrabbit

Very Extensive Company algorithmically matches worker and customer when worker has app on Company determines worker’s rate of pay and customer’s price Customers pay through the company app Customers can rate worker via the app Workers can only turn down a limited number of assignments and must maintain high rating -E.g. Uber

Over what time period does the activity need to have occurred?
The reference period in which the activity could have occurred can be wide or narrow Ever (or unspecified) In the last year Last 6 months Last month Last week Fifth question: Over what time period should the digital platform activity have occurred in. To match with labor force surveys (LFS) measures of employment better to have the periodicities for measuring other employment and platform employment match. But can have more than one periodicity in a survey (will discuss more) Periodicities chosen for measurement have varied widely, and each of the ones specified on this slide have been used.

How intensively did a person have to undertake the activity?
This question related to reference period, but it is a distinct question How often did a person do the activity within the reference period Only once at some point Only once in the last year Once a month in the last year Several times a month in the last year Weekly How many hours within a time period Sixth question, how intensively does a person have to have worked on the digital platform. Should the measure be confined just to those who work on digital platforms extensively. Measures could be restricted to those who work on digital platforms at least once a month. Some surveys specifically construct a series of measures based on intensity. Others include people who only worked via a platform once at some point (this is an “ever worked” on a digital platform measure)

Does the proportion of income earned from the activities matter?
Should the measure only include those who earn more than a threshold amount of their income from the activities? If so What is the threshold - 25%, 50%? Note that having a threshold will skew the measure toward lower income people Should the threshold be for all activities combined or separate categories, platforms Seventh question, should measures be restricted to those who earn above a set proportion of their income from a digital platform? Some measures are restricted to those who earn 25% or 50% of their income from digital platform work, but the majority do not have this restriction

How much commitment to working and availability of work is there?
Is it sufficient for a person to be registered with a platform or does the person have to have worked on the platform? Does it matter how long people have been registered with a platform and when they last worked, if ever, through the platform? Do workers desire more work and/or have experienced difficulties obtaining all the platform work they desire? Eighth question, there can be a difference between people being registered on a platform and actually obtaining and working on a platform; should there be a distinction in the measure between being registered and offering to work versus actually obtaining work? Does the commitment and availability of work inherent in actually obtaining work matter? Some measures only require people to have offered their services. As is seen in the next to slides, being registered or offering one’s services does not directly translate into actually having worked on a digital platform.

In Norway only 1.3% of registered Upwork freelancers did an assignment every month Australian Survey 14.3% of respondents who participated in digital platform work more than 12 months ago stated despite attempting, ‘they didn’t get any work’ ILO Survey 58% of respondents reported availability of tasks was insufficient and an additional 17% said that the did not find enough well-paying tasks

Why Want A Measure

What question/issue is being addressed
Digitalization of the economy and/or “complete” income measures Use broad measure (labor services, rental, selling, capital investment), may not need distinction by type of activity, but do need to have obtained income through platform (just registering insufficient) Number of workers involved May want to restrict to labor services, or at least make distinctions by type of activity Need to consider what question or issue you want to address or can address with the data you have. If it is …..

Application of national labor laws, health and safety regulations, or local market effects Restrict to in-person labor services, considerable platform control, may want to condition on number of hours worked International labor markets or means of economic development for developing countries Restrict to online tasks, may want to include financing platforms If considering applicability of national labor laws may not want to include “incidental” workers who do platform work for very few hours, sporadically. May want to have people who are “actively” doing platform work, excluding those who have only done platform work once, or have only offered their services, but have not obtained any assignments. When considering applicability of national labor laws probably do not want to include the selling of items, renting property, or situations where platform companies have minimal control over workers.

Evaluate importance as a form of employment Restrict to labor services, use reference period similar to labor force surveys (e.g. 1 week) Workers’ payment rights May want to restrict to online and/or companies with intermediated amounts of involvement or greater Effects on specific occupations or industries Ask about specific activities (i.e. ride-hailing, property rental)

How Measurements Have Been Made

Included in Review Early “modeled” estimates Private company data
Government administrative data Household survey data covering Private sector, academic, government sponsored, and national statistical offices surveys 24 Surveys All approaches have some drawbacks

Early “Modeled” Estimates
Combine data from various sources to obtain indirect measure Estimate for a small subgroup Web-scrape data for “all” groups (subgroup and others) Scale estimate for subgroup using Web-scraped data

Early “Modeled” Estimates (cont.)
For example Harris and Krueger (US), De Groen and Maselli (EU) Estimate of the number of Uber drivers Google trend estimates of Google searchers for 26 platforms and Uber Scale the number of Uber drivers by the ratio of search estimates for 26 platforms to Uber searches #𝑃𝑊= 𝐺𝑜𝑜𝑔𝑙𝑒 𝑠𝑒𝑎𝑟𝑐ℎ𝑒𝑠 𝑓𝑜𝑟 26 𝑝𝑙𝑎𝑡𝑓𝑜𝑟𝑚 𝐺𝑜𝑜𝑔𝑙𝑒 𝑠𝑒𝑎𝑟𝑐ℎ𝑒𝑠 𝑓𝑜𝑟 𝑈𝑏𝑒𝑟 x #Uber workers

Early “Modeled” Estimates (cont.)
Involve strong assumptions Largely superseded by direct estimates

Private Company Data Bank account data
For example JP Morgan Chase study Sample of 39 million U.S. Chase checking accounts Track payments through 128 platforms (number in 2018, fewer in earlier years) Estimate how many families participated in online platform economy Have estimates for current month, st quarter, and in prior year Platforms meet the following criteria: - Connect independent suppliers to customers - Mediate the flow of payment from customer to supplier -Empower participants to enter and leave the market whenever they want

Private Company Data (continued)
Cannot identify payments through 3rd parties such as PayPal Can be influenced by changes in payment process (i.e. switches to debit cards) Not universal coverage, particularly not geographically Chase does not have a branch in 23 states including VA and DC. Tax data show DC 4th highest proportion employed in platform work (2.5% 2016) and VA 7th highest by state (1.14%) Based on specifically identified platforms Unit of measurement families Data not available to general researchers Sources: Data on proportion of people in Washington, DC urban area and state estimates from (Collins, Garin, Jackson, Koustas, and Payne) Count of states from JP Morgan Chase 2018 report. Data not available to general researchers to do additional analysis or verify estimates. JP Morgan Chase researchers do not reveal which platform companies they included.

Private Company Data Web-scraping postings for projects/tasks
For example Online Labour Index (constructed by Oxford Internet Institute) Track all projects/tasks posted on 5 largest English language labour platforms Represents at least 70% of the market traffic Measures utilization of labor platforms across countries and occupations Not an estimate of filled “orders” or measure of workers, but can follow trends Since 2017 observe work activities on 4 major platforms –still not a measure of proportion of employed

Government Administrative Data
Tax filing or tax information forms For example analysis of 1099-K forms in the U.S. 1099-K forms are “information” forms sent to people by companies if a person received payment via a credit card Informational only, not a tax bill Researchers screen forms based on names of platform companies and “unduplicate” so only count individuals once even if receive multiple forms For example, screen forms to look for name of a ride-hailing company. If a person receives forms from two different ride-hailing companies only count the person as an online platform worker once.

1099-K Analysis/data Country specific, not generalizable internationally Data not available to researchers in general Does not have rich auxillary data that surveys often have Restricted to platforms identified by researchers Collins, Garin, Jackson, Koustas, Payne National analysis 50 important labor platforms Bernhardt, Prohofsky, Rothstein California, 20 platforms

Tax forms Sent for tax collection purposes, not research Effected by changes in laws and rules Platform companies prior to 2016 were being cautious; since, due to a tax ruling, only sending 1099-K’s to those with $20,000 or 200+ transactions California may make W-2 workers Interpretation of rules can vary by state VT, MA, $1 earned must send state 1099 Belgium required taxes to be withheld for all ride-hailing workers as if they were wage workers.

Tax filings Often subject to minimum income thresholds Concerns in EU that Value Added threshold not meet by many platform workers Sensitive to cyclical variation During economic downturns less likely to exceed Compliance can be less than complete Changes over time and sensitive to economic conditions Effected by changes in laws and rules Study found starting to issue 1099-K, increased reporting by effect group more than 20% Value Added concern and how passing threshold varies by demographics of workers noted by Balaram, Warden and Wallace-Stephens for the U.K. (2017) and Pesole et al COLLEEM report (2018) 1099-K Study Slemrod, Collins, Hoopes, Reck and Sebastani examined information reports received by sole proprietors from tax years 2004 to The empirical analysis found that approximately 10 percent of Schedule C firms report their gross receipts within 5 percent of the gross amount on the 1099-K’s they receive. We estimate that Form 1099-K caused more than 20 percent of taxpayers in this particular group to start filing Schedule C. For firms in this group that had already been filing Schedule C, 1099-K caused increases in reported receipts of up to 24 percent, although these firms also increased their reported expenses by as much as 13 percent

Registration information More of a European data source Can vary across time, countries, and industries For example, Denmark trying to install seats in Uber cars to measure ridership

Household Survey Data There will always be a role for household survey data Challenge is conveying to respondents what is meant by a digital platform Type of sample will influence estimates Sample probably more important than mode of data collection embedded in sample, but mode of collection can also influence Believe there will always be a role for household survey data both in obtaining an estimate and because of the other information that household surveys usually obtain. For example rich demographic information can indicate whether the disabled, particular ethnic or racial groups, or recent immigrants are more likely to be digital platform workers. In household surveys can obtain information about workers’ preferences for the work, whether they have used platform work as a stepping stone to other work, or whether they are obtaining as much platform work as they desire.

Household Surveys–Common Understanding of Platform Work
Surveys have taken 4 broad approaches to conveying what is meant by a digital platform and electronically mediated work Use term “digital platform” or other technical terms Ask about specific platforms by name Ask about specific activities done on platforms Describe and ask about attributes of platform work

Understanding term Platform Work Using Term Directly or Technical Terms
ONS Pilot, Norwegian 1st Pilot, Sweden, Eurobarometer, Rand American Life Panel (Katz and Krueger for missed work), Finnish LFS Pilot E.g. Sweden Have you ever tried during the past year to get an assignment via digital platforms(e.g. Uber, Urb-it, Taskrunner, Offerta etc.)?

Understanding term Platform Work Using Term Directly or Technical Terms
E.g. Eurobarometer A collaborative platform is an internet based tool that enables transactions between people providing and using a service. They can be used for a wide range of services, from renting accommodation and car sharing to small household jobs.* Have you ever provided services on these platforms? * Asked about knowledge and use of first

Understanding Term Platform Work Using Term Directly or Technical Terms
E.g. ONS Pilot In the last 12 months have you used a digital platform to find work on a short term, payment by task basis? E.g. Rand ALP (Katz and Kruger) for missed work Did you work on any gigs, HITs or other small paid jobs last week that you did not include in any of your answers so far? - HITs is the platform work

E.g. Finnish LFS Pilot Products and services are traded through digital platforms on the Internet that include, for example, Airbnb, UpWork and Uber. Do you use these types of digital platforms to get work assignments or income? Denmark asks a somewhat similar question TJENTARB T2S1. Have you made money in the past 12 months by performing work tasks found through websites or apps - for example via Uber? instructions: Uber provides online driving services where car owners can make money by transporting passengers from A to B. In addition, it could for example be work via Upwork, Meploy or Ahandyhand. This also applies to freelance tasks found through websites and apps. There is no question of selling used property - e.g. the car base or the DBA As did Borinin and Rinne for Germany Even if you are not doing it now, have you ever done work in exchange for money, for orders that you received over the Internet or an app?

ONS, Finnish and Norwegian Pilot found the term digital platform was poorly understood Picked up people advertising online or obtaining customers, those whose work involved a series of tasks (e.g. plumbers), those searching for work on the Internet/Facebook, online scheduling and Facebook Buy and Sell Groups Change or plan to change approach Found through interviewer feedback and backing up in questionnaire and other qualitative analysis At proportions of earnings question from found error Rand ALP (Katz and Kruger) know of no release of estimates or discussion of question

Understanding Term Platform Work Asking About Specific Platforms
Finnish Labor Force Survey Have you during the past 12 months worked or otherwise earned income through the following platforms: 1. Airbnb, 2.Uber, 3. Tori.fi/Huuto.net, 4. Solved, 5. Some other, 6. None of the above. Follow up question asking about proportion of income earned through platform work Only included selling on Tori.fi/Huuto.net if more than 25% of income Wanted to exclude “selling your child’s used skis” - Only included selling on Tori.fi/Huuto.net if more than 25% of income Tori.fi and Huuto.net are Finland's 2 largest and best known market places. Found in testing if did not include these sites respondents were either unclear about whether selling items on these sites should be included or would include selling on these sites

Basing whether someone is counted as a platform worker on the proportion of income the value of what is sold can skew the results Value is not the same as volume Selling constituted approximately half of activity The selling of cars and houses would be high value and house and car selling sites were frequently mentioned in the “some other category” in the Finnish LFS Still some confusion over what was meant Investment sites, betting sites and Facebook mentioned With the restriction that people had to earn at least 25% of their income through various digital platforms, it was estimated that in % of Finns 15 to 75 years old had done this activity in the last 12 months. When the restriction was removed the proportion of Finns who had earned income via a digital platform increased to 7%. Most of this increase came through people identified as selling items online.

Norway 2nd pilot Recently, there has been a lot of attention around companies that use apps and websites to convey work and services. This is usually called the sharing economy. Below are a list of such companies. Have you done any assignments or paid employment through one or more of the following companies in the last 12 months? 1. Uber 2. Foodora 3. weClean 4. Upwork 5. Konsus 6. Haxi 7. FINN småjobber 8. Other ______ 9. No Captured people using service not providing

Specific platforms are not present in all countries. Complicates cross country comparisons if use names Existing platforms can go out of business and new ones can arise Uber was declared illegal in Scandinavia for part of the time Norway was testing Shortly after completing found if were redoing Finland would have to add 2 new platforms Treamer.fi and palkkaaminut.fi.

Understanding Term Platform Work Asking About Specific Activities
Canada (both LFS and Internet Use surveys), CIPD (for UK), Sweden (SOU), Norwegians (3rd Pilot), Royal Society for the Encouragement of Arts, Manufacturing and Commerce (U.K. Balaram et al.) The number and type of activities asked about varies across surveys Canadian LFS just asks about 2 In the past 12 months, did you offer ride services such as Uber, Lyft, etc.? In the past 12 months, did you offer private accommodation services such as Airbnb, Flipkey, etc.? -

CIPD (UK) asked about 7 activities Thinking about the LAST 12 MONTHS, which, if any, of the following have you done via an online platform (i.e. website) or app (i.e. mobile device application) to earn money? (Please tick all that apply) - Provided transport using my vehicle (e.g. Uber, BlaBlaCar etc.) - Rented out my vehicle (e.g. EasyCar, Zipcar etc.) - Rented/shared my accommodation(e.g. AirBnB, Tripping, HomeAway etc.) -

CIPD (UK) activities (continued) - Delivered food or goods (e.g. Deliveroo, City Sprint) -Performed short-term jobs via online platforms that connect people looking for services (e.g. TaskRabbit, Upwork, PeoplePerHour etc.) - Sold things I have created via online platforms (e.g. Etsy) - Other work arranged through an online platform (open-ended)

SOU (Sweden) also asked about 7 activities In which, if any, of the following ways have you ever personally carried out paid work using a website or mobile phone application? 1. Providing a driving or taxi service, for a fee, by finding passengers through a website or app such as Uber or BlaBlaCar 2. Providing professional work, such as consultancy, legal advice, accounting services, through a website or app such as UpWork, PeoplePerHour or Freelancer They are not, however, exactly the same 7 activities as in the CIPD

SOU (Sweden) also asked about 7 activities 3. Providing creative or IT work, such as writing, graphic design, or web development, through a website or app such as UpWork, Freelancer, PeoplePerHour, Fiverr or Toptal 4. Providing administrative work, such as data entry or ‘click work’, through a website or app such as Clickworker, PeoplePerHour or Freelancer 5. Providing skilled manual work, such as plumbing, building, electrical maintenance and carpentry, through a website or app such as Rated People, MyBuilder or TaskRabbit

SOU (Sweden) also asked about 7 activities 6. Providing personal services, such as cleaning, moving, or DIY tasks, through a website or app such as TaskRabbit, Hassle or Handy 7. Providing delivery or courier services, through a website or app such as Deliveroo, UberEATS or Just Eat

Confines measurement to those activities being asked about has risks Excludes new activities that could arise Taskrabbit started 2008, Uber 2009, Fivver 2010 Not all activities of interest to all countries E.g. Canada asks “The following questions are about various digital methods of making money, including raising funds through crowdfunding platforms, selling new and used goods online and providing peer-to-peer or freelance services. 04: Selling live animals via online bulletin boards (e.g., Craigslist, Kijiji, Facebook)” The number and type of activities asked about varies across surveys Canadian LFS just asks about 2 In the past 12 months, did you offer ride services such as Uber, Lyft, etc.? In the past 12 months, did you offer private accommodation services such as Airbnb, Flipkey, etc.? - All the activities Canadian Internet Survey ask about are: 01: Raising money through crowdfunding platforms (e.g., GoFundMe, Kickstarter) 02: Selling unused or second-hand products via online bulletin boards (e.g., Craigslist, StubHub, Kijiji, eBay, Facebook) 03: Selling new products that you personally produced via online bulletin boards (e.g., Etsy, Kijiji, eBay, Facebook) 04: Selling live animals via online bulletin boards (e.g., Craigslist, Kijiji, Facebook) 05: Generating money by posting or selling creative content online (e.g., YouTube videos, blogging) 06: Providing peer-to-peer accommodation, ride sharing or delivery services (e.g., Airbnb, Uber, Skip the Dishes, Uber Eats) 07: Providing freelance specialized online services (e.g., Upwork, Freelancer) 08: Selling other services using an online platform (e.g., Craigslist, Kijiji, Facebook) 09: None of the above

Different activities entail different levels of platform company involvement Combining all activities into a single measure may not be that useful Need to take care when aggregating activities because people can be involved in more than one activity Does provide somewhat the types of skills work involves But may still want more specificity, if workers skills is of interest -

May capture people who use the service rather than provide the service Norway changed wording between pilots possibly to address this issue, Switzerland added an introductory paragraph to address the issue Asking how much was earned on platform and allowing back ups will help, but not a “best” practice

Norway 3rd pilot Recently, there has been a lot of attention around companies that use apps and websites to convey work and services. This is usually called the sharing economy. During the last 12 months, have you done some of the following ... 1. Did you work as a bicycle courier for Foodora? 2. Worked as a cleaner for WeClean? 3. Worked for Upwork or Konsus? 4. Worked as a driver for Haxi? 5. Did a job you found on FINN småjobber?6. Did you do a job on Mitt anbud.no?7. Rented a home on Airbnb?8. Done assignments you have found on other apps or websites _______ 9. None of the aforementioned

Understanding Term Platform Work Describe and Ask About Platform Attributes
Ask whether worker/client interactions involve specific facilitating functions of digital platforms Connecting workers to clients/Involved in obtaining clients Katz and Krueger, French LFS ad hoc module, Eurostat, Gallop (1st half- Abraham, Hershbein and Houseman) Connecting workers/clients plus other functions PEW survey, COLLEEM Survey (for European Commission 14 EU countries), CPS Contingent Worker Supplement, Gallop (2nd half)

Involved in obtaining clients (RAND, ALP Katz and Krueger) Do you do direct selling to customers on your main job or a secondary job, or both? Does your direct selling involve goods or services? Do you work with an intermediary, such as Avon or Uber, in your direct selling activity? Do you work with an online intermediary to find customers, such as Uber or TaskRabbit?

Involved in obtaining clients (French Labor Force Survey, ad hoc module) asked of those identified as self-employed How do you mainly get in touch with your clients? Many answers possible (if the respondent can't choose) Don't read item 5. 1. Clients come into the shop or contact you directly (phone, mail, Internet etc.) 2. Clients go through a platform or through a third party business that redirect them to you. 3. You're directly looking for clients / contact yourself the clients. 4. Other 5. Not meaningful This question was actually designed to measure “dependent contractors” not platform workers specifically Bolded response category is what is used to determine platform workers

Involved in obtaining clients/work (Eurostat) Have you obtained paid work by using an intermediary website or apps (e.g. Upwork, TaskRabbit, Freelancer, Amazon Mechanical Turk) in the last 12 months? Note that this is a very broad measure. This question was included in the 2017 and 2018 COMMUNITY SURVEY ON ICT USAGE IN HOUSEHOLDS AND BY INDIVIDUALS Had hoped the word “intermediary” would help, but Eurostat decided not to release estimates from this question and did not included it in their 2019 survey.

Connecting workers to clients (Gallop Education Consumer Pulse Surveys - Abraham, Hershbein and Houseman 1st half) For any of the work you did in the past seven days, did you connect directly with new customers or clients through a mobile app or online platform? (Added on for half of sample) For example, you might have given rides to people using a ridesharing app; used an app to find people looking for cleaning, delivery or handyman services; or used an online platform where people can bid on data entry or other tasks

Connecting Workers Plus Other Functions (COLLEEM) Have you ever gained income by providing services via online platforms, where you and the client are matched digitally, payment is conducted via the platform and the work is location-independent, web-based? Have you ever gained income by providing services via online platforms, where you and the client are matched digitally, payment is conducted via the platform, but the work is performed on-location? Note: The COLLEEM asks additional questions about frequency of work to construct alternative measures of the number of platform workers.

Connecting Workers Plus Other Functions (PEW) Some people find paid jobs or tasks by connecting directly with people who want to hire them using a particular type of website or mobile app. These sites require workers to create a user profile in order to find and accept assignments, and they also coordinate payment once the work is complete. In the last year, have you earned money by taking on jobs through this type of website or mobile app (for example, by driving someone from one place to another, cleaning someone’s home, or doing online tasks)? This is the only question I have seen that includes the attribute of creating a user profile. Pew developed this question based on focus groups with platform workers. Actually a “mixture” question because it also asks about specific activities. I will discuss these “mixture” type questions in a little bit. PEW question is in the “describe and ask about attributes” type questions because PEW reports an estimate of the number of platform workers based on this question.

Connecting Workers Plus Other Functions (CWS) Some people find short, IN-PERSON tasks or jobs through companies that connect them directly with customers using a website or mobile app. These companies also coordinate payment for the service through the app or website. For example, using your own car to drive people from one place to another, delivering something, or doing someone’s household tasks or errands. Does this describe ANY work you did LAST WEEK? Y/N Was that for your main job, your second job, or other additional work for pay?

Connecting Workers Plus Other Functions (CWS) Some people select short, ONLINE tasks or projects through companies that maintain lists that are accessed through an app or a website. These tasks are done entirely online and the companies coordinate payment for the work. For example, data entry, translating text, web or software development, or graphic design. Does this describe ANY work you did LAST WEEK? Y/N Was that for your main job, your second job, or other additional work for pay? Note: The CWS questions ask separately about in-person, physical tasks and those done completely online (the introduction to the question notes that both types of Electronically Mediated work will be asked about). Describes attributes (connecting/selecting and payment through the company) that we hoped would be robust to types of platforms and platform activities Hoped to eliminate capital intensive services by referring to “tasks” and jobs and because it was imbedded in a labor force survey. Also hoped combination of connect (select) wording and payment coordinated by company would eliminate those using a website just to look for work The question tries to clarify what is meant by giving examples, but without using the name of platforms

Connecting Workers Plus Other Functions (CWS)
Understanding Term Platform Work Describe and Ask About Platform Attributes Connecting Workers Plus Other Functions (CWS) Unfortunately the questions did not work as intended Captured workers who could not have been in an EME arrangement e.g. a local police officer, a surgeon at a large hospital, a railroad engineer Complicated complex question Respondents and interviewers focused on the examples Respondents whose worked involved some digital assistance answered affirmatively e.g. a gravel delivery person who used an app for directions, a real estate agent who obtained customers through the web, fast food worker who prepared orders placed through an app Workers who used a computer at work answered “yes” Team manually recoded false positive answers based on other collected information in the CPS Note: Examined a sample of “no’s” to the questions in industry may have expected EME work and found the vast majority of “no’s” were correct

Questions can get long when describing attributes Knew CWS questions were complex Would have been better to split the attributes up into separate questions But restricted to 4 questions, and tests via cognitive interviewing indicated questions worked “okay” May need to add additional attributes that more uniquely defined EME I.e., Can choose when to work by turning on app, need to have app or website on in order to work, need a profile in order to use app or website, can be rated by customers through app or website Our experience indicates -The importance of testing questions on those for whom the situation being asked about does not broadly apply (majority of cognitive testing was done on those for whom the situation applied or those in closely related activities such as being a taxi driver for a taxi company or a food deliver for a pizza store). How having interviewers who are going to administer the questions in the field review the questions and provide feedback would be helpful. How having larger scale field tests (or a large test panel), so there can be iterative rounds (such as Norway was able to do), would be beneficial.

Understanding Term Platform Work “Mixed” Ask Platform Attributes and Activities
Explain attributes and then ask about specific activities (Swiss Labor Force Survey) Internet platforms and apps make new income opportunities possible today. You are put in contact with the client and generally paid directly via the platform Have you rented a room, apartment, or a house to somebody via an internet platform or app such as Airbnb or Flipkey in the past 12 months? Have you provided taxi services via an internet platform or app such as for example Uber or Lyft in the past 12 months?

Explain attributes and then ask about specific activities (Swiss Labor Force Survey) - continued Internet platforms and apps make new income opportunities possible today. You are put in contact with the client and generally paid directly via the platform c. Have you sold goods via an internet platform or app such as Ricardo or Ebay in the past 12 months? Please only answer “yes” if you previously collected or bought the goods with the specific aim of reselling them d. Have you provided other services via an internet platform or app such as cleaning, handiwork, deliver services or online programming in the past 12 months? Note: The paragraph explaining the attributes was added after testing showed people were saying “yes” just when they had used a service. Don’t repeat the paragraph between activities, just on this slide for illustrative purposes Had particular trouble with the “renting” accommodations and transportation questions Are processing data now, and still are finding they need to “clean” the data for too many false “yeses” Editing based on interviewers notes and subsequent questions asking about earnings from platform work and number of hours people did platform work.

Ask about specific activities and then ask if work was electronically intermediated (SHED) 1. In the past month, have you been paid for each of the following activities? a. Child or elder care services, b. Dog walking, feeding pets, or house sitting, c. House cleaning, yard work, or other property maintenance work, d. Driving or ride-sharing, such as with Uber or Lyft, e. Paid tasks online, such as posting YouTube videos (do not include GfK surveys), f. Other personal tasks, such as deliveries, running errands, or helping people move

Ask about specific activities and then ask if work was electronically intermediated (SHED) continued 2. In addition, in the past month, have you been paid for each of the following activities? a. Sold goods yourself at flea markets or garage sales, b. Sold goods at consignment shops or thrift stores, c. Sold goods on-line, such as on eBay or Craigslist, d. Rented out property, such as your car or your house, e. Any other paid activities that you have not already mentioned in the survey 3. Some people doing these activities use companies that connect them directly with customers using a website or mobile app. For example, Uber and Lyft help drivers find riders. Last month did you find any paid work or customers using a company like this Note: SHED report on the proportion of employed who were platform workers is based on the third question. There are some odd patterns in the data need to investigate more, because the proportion of “yes” responses to question 3 are almost equal to the proportion who just said “yes” to the ride-sharing question. Respondents to question 3, may have just focused on the Uber and Lyft examples, but more investigation is needed to confirm this.

Household Survey Data Online samples
A few explicitly exclude completing online surveys. Most do not and at least one explicitly said to include completing the survey Sometimes weight to demographic characteristics of the entire population or proportion of population that have internet access. Some do not Do not believe weighting will completely counter biases, selectivity of online samples (those who do not have internet access should be included as “no’s”) Quota samples particularly prone to biases because more frequent users more likely to answer and probably more likely to do platform work PEW survey specifically provides completing surveys as an example of what to include. This will increase the number and should make the estimate of platform workers almost 100% of respondents. Norway after their pilots (which were online) decided the optimal way to conduct their survey was by phone interviews because of the biases in online surveys.

EU Community Survey on Internet Usage (sampled of internet users)
Household Survey Data Online samples –Respondents frequency of Internet Use vs. General Public (Select countries) EU Community Survey on Internet Usage COLLEEM Survey (sampled of internet users) COLLEEM/ICT Daily Use Daily Weekly Less U.K. 88% 5% 7% 93% 2% 106% Spain 67% 10% 23% 87% 3% 130% Germany 78% 9% 13% 112% Italy 66% 1% 33% 96% 145% Romania 42% 14% 44% 92% 219% Source: Platform Workers in Europe Evidence from the COLLEEM Survey (Pesole, Brancati, Fernandes-Macias, Biagi, Gonzalez Vazquez Note: The COLLEEM isn’t even strictly a quota sample.

Comparison of Estimates

Results of Measurements
Measurements have intentionally or implicitly addressed differently the decision about what to measure There have been a wide range of measures, even for the same country and same time periods Undoubtedly related to methodological differences not real differences -

Range of Estimates for the U.S.
24.0% PEW (any income from a platform), 7,8,12/2016 22.4% Upwork & Freelancers Union, 2018 8.0% PEW(only labor), July, Aug, Dec 2016 6.0% CPS-Internet Use Supplement, Nov. 2017 3.0% Gallop May 2017-May 2019? 2.8% SHED, 2018 1.4% JP Morgan Chase, 1st Q 2018 1.1% IRS (Collins et. al), 2016 1.0% CPS-CWS, May 2017 0.5% Rand ALP, Oct, Nov. 2015

Results of Measurements Broader Concepts Yield Higher Estimates
CPS Internet Use Supplement just specifies a service has to be “offered” no work needs to have been done, combines labor and platform work, and can entail minimal platform company involvement Have you offered own services for sale via the Internet? (Examples include offering rentals on Airbnb and driving for Uber or Lyft. Do not include any goods or possessions sold online, such as clothing, shoes, or crafts.) CPS CWS asks about work done, restricts to labor platforms, intermediate company involvement Internet Use estimates 6% of employed, CWS 1% (after editing, 3.3% before) -

Results of Measurements Online Surveys Yield Higher Estimates
COLLEEM Online Survey/Eurobarometer phone interviews Source: Measuring Platform Mediated Workers (OECD Digital Economy Papers No. 282 April 2019) For example, Great Britain in Eurobarometer was 2.0% of employed, whereas in COLLEEM it was 10%; Germany was about 2.5% in Eurobarometer, and a little more than 8% in COLLEEM These differences arose despite the Eurobarometer having a broader concept than COLLEEM

Results of Measurements Including Selling Items Boast Estimates
PEW 24% of Americans earned money in the platform economy in 2016 18% by selling something online 8% from taking on a job or task 1% from renting out accommodations Canadians 28% made money through an online platform Most common means was selling through an online bulletin board - As an aside the PEW estimates illustrates that an individual can participate in several activities (so it is a mistake to add up each of the separate activities to obtain a total)

Results of Measurements Including Selling Items Boast Estimates
Finland 0.3% of Finns earned income through a digital platform when impose the ¼ of income rule Half of this is from people selling items online If do not impose ¼ of income rule the estimate increases to 7.0% of Finns

Results of Measurements Frequency and Intensity of Activity Affects
Source: “Platform Workers in Europe Evidence from the COLLEEM Survey” Pesole, Urzi Brancati, Fernandez-Macias, Biagi, and Gonzalez Vazquez

Australia (March-April 2019) 7.1% of respondents worked (or offered to work) at the time of the survey or had done so within the past 12 months 13.1% had at some point in the past 80.7% report digital platform work makes up less than half of the their total annual income 28.3% engage with the platform less than once a month 47.2% spend less than 5 hours per week working or offering services

JP Morgan Chase, 2018 58.3% working on a transportation platform had earnings in just 3 or fewer months 68.1% of those working on a non-transportation platform had earnings in 3 or fewer months Platform earnings represent a major source of income for families during the months they participate But overall platform income represents 20% of income on average of those who have participated at any point in the year

Concluding Thoughts

Concluding Thoughts Important to think about what want to or are measuring Clearly label what have measured and think about what questions can answer with the measurement Think about whether want to include selling items Is in-person, online distinction important Household survey data and administrative data are complements Each has strengths and weaknesses Just because have had “growing pains” with survey questions does not mean good questions cannot be written Hope I have shown you that the “What and How” of what is measured under the rubric of electronically mediated employment/platform work matters.

Concluding Thoughts The most promising avenues for identifying platform workers in surveys seem to be describing attributes or “mixed” attributes and activities, maybe just asking about specific activities Can have two reference periods in same survey for identification of workers In Labor Force Surveys it is important to use the LFS reference period to match to other forms of employment In the past year can also be asked Care needs to be taken on order and reference period for other questions if use 2 periods A series of questions asking about whether specific attributes describe a persons job followed by questions asking about the type of work or platform used would allow both identification of platform workers and the skills involved in the platform work. However, again, asking about activities can be restrictive and make the questions less durable across time and countries.

Concluding Thoughts Useful to ask additional questions in surveys about frequency and intensity of platform work Particularly for policy analysis May be useful to ask about name of platforms used Can help with editing and identify types of workers’ skills that are being used Frequency and intensity probably more useful than proportion of income work makes up because proportion of income will skew measurement towards lower income individuals, and value and volume of work are not necessarily equivalent (particularly if the sale of items is included in the measure)

Concluding Thoughts Focus on work having been completed
Just whether “offered” services, because workers can be unsuccessful, is not that useful Can ask whether workers think of themselves as employees or independent contractors, satisfaction with platform, or reasons stopped working on platforms For an estimate representative of the entire population use address or phone samples Online surveys have inherent biases If interested in number of workers just offering services is an insufficient measure. Australia asked whether workers thought of themselves as employees or self-employed % thought platforms treated them as employees. Can ask people why stopped working on platforms. Was it because they found other more desirable work, because they couldn’t get assignments/find enough work or because they were barred from the platform Can ask about working conditions (insurance coverage etc.)

Employment Research and Program Development Staff, Research Chief
Anne E Polivka Employment Research and Program Development Staff, Research Chief Office of Employment & Unemployment Statistics

The What and How of Measuring Electronically Mediated Work:

Similar presentations

Presentation on theme: "The What and How of Measuring Electronically Mediated Work:"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

The What and How of Measuring Electronically Mediated Work:

Similar presentations

Presentation on theme: "The What and How of Measuring Electronically Mediated Work:"— Presentation transcript:

Similar presentations

About project

Feedback