 So, then we come to the most important part of today's conference and it is my great pleasure to and I would like to say Mr. Crowdworking and Mr. Crowdsourcing to have him here at this conference and I wouldn't have expected that I'm so happy to have him here because he's also having studied information systems and this apparently was of very much use today because without him we wouldn't have fixed this problem here and so I didn't expect that it would be so helpful in many ways to have you here and to give you a short introduction. So Jan-Marco Lehmmeister did a diploma in business administration at the University of Fohenheim where he also did his doctoral degree and thereafter he moved to Munich to the technical university where he did his habilitation and then in 2008 he got his first professor in information systems at the University of Kassel and there he become also the director of the information systems and research center for information systems design and in 2014 he then became a full professor and also director of the Institute of Information Management at the University of St. Gallen. He has published in very many journals, he's also the editor of many journals among others, he's the associate editor of the European Journal of Information Systems, he's a senior editor of the Journal of Information Technology and he's an editorial board member of the Journal of Management Information Systems and he was also a long time abroad so he went to many US universities like Harvard University, the University of California at Berkeley, Columbia University at the East Coast and most recently he was the number four or received the place number four among 2824 business researchers in Germany which is ranked by the Wirtschaftswoche and today he will speak about understanding a new type of digital labor, how the nature of work affects satisfaction and identification among crowd workers and we are very much looking forward to your presentation and very much appreciate that you are here so thank you very much. So thank you very much for this very kind introduction and after listening to what you said I can only fail now right because expectation management and reality will not match so let's be clear about that right from the beginning and I'll do my best to make it as graceful as possible when I fail but I'm sure I will fail. So I'm very grateful to have the opportunity to be here it's an interesting discussion that I could already observe and watch and a lot of the things that the talks addressed I think are of vital importance for our European community because what we're seeing here around crowd work is a little little little glimpse on what the future of digital work could look like right this is not everything that we will see in digital work it's one type there will be many others but to me it's like digital work on steroids so a lot of the things that we see in crowd work structurally will also happen in other types of work so that's why when Paul said that the phenomenon doesn't have that magnitude that might be true for crowd work but the underlying problems have a huge magnitude and I myself am a good example of what you would call atypical employment right having multiple appointments at the same time working for different companies for different universities for different setups for a certain period of time in my environment that's not unusual and it happens more and more when I look at the younger generation that becomes more and more the dominant pattern and all the challenges that we have around our social security system to deal with that type of things not so much about crowd work but everything else so maybe that's just to contextualize right and I will be standing here because I don't see my slides right so every time I walk there I watch there it's not because I want to be rude or something and show you my back it's just because I don't know what's on the slides right so forgive me for that so let me contextualize a little bit about the background of the research that I do so this is my team and most of the work that we do is joint work right so I will not take credit for everything by no means very often it's their hard work and me just communicating it that's also true to some degree for the study about today the study of today is joint work with Evo Blom and David Dover and it's coming out of one of the currently 23 projects we do at our institute it's a project funded by the Hans-Böckler Stiftung for those of you who are not familiar with that that's the foundation of the German unions which means the research is funded by unions worrying about what their take on the future of digital work will look like I find that very interesting right so they worry about what does that mean for social systems social security and so forth and it is part of a context of five projects where we deal with crowds or crowd type of work settings and since I didn't know what to expect today I prepared a very classical paper-based talk so the talk that I will give is basically about one paper and one study we've done but I'm more than happy to share everything that is happening right now in the other projects because to me it makes only sense if we see the broader picture and how these things relate if I was to present what we're currently doing I would be speaking only about internal crowd work so crowd work applied within organizations and across organizations this paper here is about external crowd work so this is the type of out in the wild open call internet based no employment contract whatsoever around right so I would like to follow this agenda which means I'll try to lay out my definitions and understanding of the crowd phenomena how it also relates maybe to the definition of work and employment and gainful employment because that's very very important to see how we can address regulatory issues or things alike would then give a little bit of context about this particular theory background that we apply here we will work with self-determination theory which is a very popularly widely used theory in work psychology and to use that to better understand is there a different type of perception of digital work among crowd workers and if so what drives it and is it any structurally different to other types of digital worker employment and I will in a in a couple of seconds give you the punch line and you'll see that there might be some special things around that exactly when it comes to the method that is up to your interest we're doing a rather sophisticated mediated moderation analysis I've learned a lot working on that paper especially with evil he's a method guy coming from that field I now feel comfortable that I can present that without major issues but rest assured the first time we ran into that that was quite an effort for me because it's quite different to what we've been doing before and at the end I will also give you some implications on different layers right so just me thinking about what could that mean for either theory development in certain areas but also when it comes to implication for practitioners for crowd workers and organizations so what I usually do is I structure my talk in three parts first of all I will tell you what I will tell you then I tell you and then I tell you what I told you so this is what I will tell you I will show you that satisfaction with work with crowd work mediates the effects of several task characteristics and it'll mediate the effect on to us the final focal variable which is identification with crowd work you might wonder why is identification with crowd work relevant that is the key variable to make a sustainable work life a work perception measured like the colleagues in work psychology do right so satisfaction is rather temporarily shorter so you can be satisfied with an episode or something but if you're fully convinced of your work setting then you will identify with it so that's the long term variable that we have in mind we will also show that the effects that we see are stronger for crowd workers that are able to realize a greater financial compensation what does that mean our data makes us believe that we see a plateau effect of salary there are other theories that work with that as well that you need a certain level of employment before other characteristics kick in I mean this is very much a muscle type of logic that you see him or health two factor theory would also say that we will show that we have that but on top of that is like a catalyst for making after that plateau level the effects of task characteristics even stronger what that would mean is that we can we can we can discuss later and we will also see that from a psychological point of view what you very often see in crowd work is a tailoristic approach you make tasks smaller smaller smaller but there's a limit to that decomposition where the advantages of making tasks smaller and less knowledgeable in execution has a negative trade-offs if you push things too far when it comes to the satisfaction and the overall identification with that type of activity that you see within the crowd so that is basically the punchline of the whole talk and let's see how we get there um to me the overarching term for crowd work is crowd sourcing crowd sourcing is the principle right it's not new you've probably heard about it multiple times the the most prominent source where it was uh named for the first time goes back to Jeff Howe in the in the Wired Magazine in the mid-2000s and back then and that's how the article started he was describing something he observed in the Silicon Valley that's it right where you would see startups having e-commerce business models without any technologist in the team and against common wisdom you would assume if you're a technology-based business you would probably need someone knowledgeable about technology in your team and he was seeing the contrary and that was when he was starting to look into that and he found out that because of the scarcity of some tech talent in the in the Bay Area they were working project-based in parallel in multiple settings and that's basically where he described for the first time this crowd sourcing thing and the key components he mentioned was the open call and it's the voluntary participation so that's the principle right that has got a lot of attention in the work environment especially if you go to the IT field um where we have the concept of cloud computing as a way of delivering IT services when you need it you use it when you don't need it you switch it off and that requires of course the same concept for work right so basically it's the cloud computing principle applied to human work and for the first time large scale we came across it in in the IT field right and where it wasn't so much about the startup scene but ordinary jobs in data centers or an application development were the same concept was was pushed and if you talk to IT executives they would still love that concept right because it would allow them to scale and adapt the workforce and workload exactly like they feel the market cycles why is crowd sourcing potentially interesting when it comes to solving problems I mean that very much relates to wasla what last honof was presenting it gives us a couple of advantages compared to the way how companies usually work on problems this best one is that you usually as a company search within your area of expertise that would be one of the peaks in those profiles once you extend that search environment and you access people who might be in other contexts they might be coming from those environments or those environments they can add type of solution information to your process or to your tasks that you otherwise wouldn't be able to get hold of and that is true for innovative problem-solving as well as for any type of challenge that you would pose to people in work and this goes back to research that has been done in the open innovation field in the the programming environment would go on and on and on every time you need creative solutions that's the potential you can bring to the table compared to any other organization so this here would be our maximum if we are in this organization and this search environment and we could jump up to that level by having someone with that skill set mindset coming from the other organization to work with us and this is very much similar to a lot of things we see in digital work where collaboration patterns do not stick to boundaries of organizations but rather emerge across it so software development is very interesting if you think like of platforms like github it's very easy to set up projects no one cares which organization you work for actually it's the setup where you are thinking like plug and play to get a project going and this is an environment where that could help right the other thing when we talk about the niche-ness of the phenomenon a couple of tasks of organization nowadays are mainly mainly mainly done using crowdsourcing principles right and if you look at the big brands for instance in the last 10 years 85 percent of them have applied it at least once more than 50 percent applied regularly does it replace what they have been doing for instance in product development or in marketing campaigns no it doesn't it's broadening their repertoire so this means if you think like consumer brands like l'oreal for instance they adopt those mechanisms for certain campaigns for certain elements and now they have a broader repertoire and more choices to to choose how to play to play their game right the other thing is what is it that the crowd can do and if you take a depiction a porter style of an organization you would see that in almost every function of an organization you would find at least one successful crowd-based service provider operating not doing exactly the same or the full fledge of every activity but some of it right and let's use some example so that you can illustrate that right we know that when it comes to r&d ideation and and open innovation has been around for quite a while that's not nothing surprising i think that's also related to what last honof was presenting that's that's a well-known suspect but let's look into other areas right so for instance in the area of services you would find companies like crowd guru or mila that would offer crowd-based after-sales services mila for instance is a swiss com subsidy that offers neighborhood services customers help customers and it's a way to complement the existing service office that swiss com has and it is basically a matchmaking platform where the crowd can help each other right and it's a economically very interesting model when you go to marketing and sales for instance you would use service providers like a cross that could help you with any type of multi-language material that you want to use so basically complimenting or replacing translation services or also very interesting you would have i don't have it on the slides but it's currently being bought by dhl a company that does uber for your trunk which means it's it's a crowd-based mechanism for intraday logistics right so people think of cosmopolitan of metropolitan areas with a lot of traffic happening is there a way how we can make use of all those empty trunks being driven around is that a case we can use of and of course we can and that's that's also similar mechanism and one of the most interesting areas where to me crowd work and crowd testing is changing the game is software testing especially user-based software testing if you think of testing mobile apps or something like that to me there is no other way how to do that reasonably than without crowd testing because the variety of devices that people have combined with operating systems combined with software versioning it's it's not controllable you would have millions of combinations so standard testing mechanisms can control for maybe 30 30 40 percent of what you would need to test with a crowd you can almost cover everything and so the most dominant players toss test birds or also prass brains from switzerland they are basically taking that market right so that's one of the cases where crowd-based soft business models definitely winning the market against their traditional competitors and you could go on and your vote was an interesting case when it comes to innovation support during product development and so on now but the talk is not about crowdsourcing it's about crowd work right and now i'm not an expert in work law or in any type of law i'm just a lay person working every now and then with some people from law and that is basically my internal framework how i try to communicate them right so first of all work is an effort for accomplishing a task that's a general definition that's the least common denominator that my colleagues came up with so then we go to paid work right which is goal directed effort for creating income now we're getting closer to gainful employment right then a subset of that would be digital work which means it's an effort to create digital goods that makes substantial use of digital tools for creating income right so that's the digital work part and this one here this one is growing this one is growing quickly and in multiple ways and crowd work being one in there right now which means there is work that is not paid right customers sharing ideas on platforms that's not paid that's not part of our digit work definition right so that's why you would see that we would have crowdsourcing paid and unpaid and once it's paid we would call it crowd work so that's just for me to be very clear and simple on what we look at and this study will look at crowd work so it will not look on customers sharing creative ideas on user communities or on Wikipedia or anything because it's not paid and the structural behind that would be different is that relevant there's been a 2018 study from the world from the OECD trying to measure the size of the phenomenon they came up with 1.2 million in Europe in in Germany we're having 42 million roughly speaking as as employees or as a workforce so we would speak of a rather small percentage interestingly now coming from Switzerland Switzerland has almost the same amount of crowd workers but just one tenth of the workforce so there the magnitude of the effect seems to be much stronger my explanation is very simple multiple employments are much more common in Switzerland people doing multiple jobs at a time than it is in in Germany that's probably one of the easiest explanations for that the other thing that we know from other studies is that crowd work usually the full-time crowd worker is a myth in the studies that we've done so far that was below five percent of the crowd work is active but it's a huge amount of people doing second third or fourth jobs and especially for covering for seasonal differences or for life stages baby pauses or whatever when those things go up and become more relevant in the proportion of income that people get yeah and you see that there are differences between the UK and between Italy and one of the things that we're currently looking at looking at the OECD data is we have the assumption that there is a correlation if we take Switzerland out of the equation that the unemployment rate and the amount of crowd work is directly correlated so the more unemployment there is the more people work in the crowd because it's an access to work and that is very straightforward which would mean that for our German environment given our good economic development the last 10 years we're all lucky we're all lucky right so once that changes there will be probably much more demand for crowd work than we would have we've seen that indication in the data from the US as well when 2008 was very very hard for people there platforms like Amazon Mechanical Turk had three times four times the amount of people looking for jobs now what happens if the amount of work doesn't go up and just more people look for work prices go down so those mechanisms are the dynamic that we haven't seen here because of our economic development and a lot of the studies that we've done in the last couple of years for instance never came up with data from Germany saying that we have abuse of market power or something like that or work below a minimum wage type of things no that wasn't something we could report on but the big elephant in the room is during economic times where there is no pressure on the system right so those data have to be seen very careful and interpreted very carefully because we don't know the strength of economic downturns on those systems right so the research gap that we try to look at we look at crowd working platform as labor markets we look at it in the study like an organizational focus right and we have a selection bias in what whatever we do because the only access to data that we get is voluntary participation in interviews and surveys right so this is a big big constraint nothing we can do about it other than trying the best we can to compare and triangulate with other data sources and see whether the patterns we see are stable across studies and across and across settings the second thing that we look at is the motivation of crowd workers where we see differences and there's a lot of research on that recently coming out in different areas what intrinsic and extrinsic motives or motivations of crowd workers are in different settings one of the big constraints of the studies that we've seen and I think I have the literature screened I have it screened quite well is that there are so many different types of crowd work that whatever study you see is true for that type of crowd work not for all different types of crowd work so if you do a study on Amazon Mechanical Turk what you would find there for motives would be true for micro-tasking and if you would not be able to transfer it to software testing for instance and vice versa right so there's this big constraint that the type of work is still so heterogeneous that we're still even within crowd work comparing apples to oranges and that's a big big problem that we've had in the past and very often the perception of motivation on psychological work outcomes and research is neglected all right so we look at performance but we don't look very often at the workers perception especially not over time so there's a huge gap for those of you looking for interesting study topics the workers perspective in the crowd setting is under research compared to all the other settings and it's contextualized in a very raw and immature way so far which is good right which means opportunities for good research and we will look explicitly on task design so how is the task designed the crowd work is work on because we know from the work psychology that the task design determines mainly the overall work perception and this is nothing special to digital work this is true since we know the assembly line and the negative effects of too much Taylorism that that can have a negative effect on satisfaction and all the other effects and the other thing that makes task design very interesting is and that is something that is quite new in the in the crowd work environment we now see more and more intermediaries apply different ways of automation and combine it with crowd work so the first wave of crowd work platforms basically use standard technologies for decomposing and aggregating tasks now we see more advanced and more sophisticated machine learn type of setting text mining based approaches so what is the the division of labor and the and the management task behind that is also getting more and more automated with different effects on the task design of the crowd worker itself so this is a very interesting area as well so for this study what we try to address is how the perception of task characteristics and financial compensation jointly influence the professional well-being of crowd workers because you might recall this is funded by the union the foundation of the union what they care about are good working conditions for people right so the question is what is the role of financial compensation for good working conditions and the nature of the tasks for good working conditions and for those of you not coming from Germany you know that we you might know that we have a minimum wage which is has been introduced not so long ago which is still an issue in in public debate whether that was a smart move to do or not so it's a regulatory issue right and the argument back then was from the union side that we need to control for a minimum level of income so that good work principles on top of that make in the combination a decent life possible right out of your own work and that's basically the same thing that that we try the same logic we try to follow here now when we speak about crowd work we have at least three structural characteristics that have to be existent before looking at anything else first of all financial compensation right so it has to be financially re-renumerated and it has to be a full-time part-time or whatever type of portion of employment the other thing is we believe that crowd work always requires autonomy of the person being in the crowd so he or she chooses when to work where to work on what to work so this is the core element to me of crowd right so the self-determination the self-selection of the people working there and what is also necessary condition for us is IT facilitation which means that a large chunk of the value creation a large chunk of the value creation has to be done on the platform because then we can apply all those mechanisms we know from platform work right like the scalability and all those sort of things so if you asked me is uber crowd work no it's not because the large part of the value creation is not done on the platform right it's just the matchmaking it's just this small part of supply and demand matching whereas all the examples that we will look at they have a large part of the work itself being done on the platform so now coming to the conceptual background the task characteristics we know that the perception of task characteristics are key antecedents of satisfaction with work and identification with work so the difference between satisfaction identification is basically the duration they refer to satisfaction usually can be on a task-based level identification usually is with a work setting right so i'm proud to be a researcher that's my identification i'm satisfied with my data analysis the last time i did it this is much smaller chunk of task that we will refer to so what we're after is basically this long-term perspective on identification with work we know that there are important contextual factors that influence these relations and we're drawing a lot on the literature on digital on computer mediated work that's nothing new that's been around since we have the late 90s telework was the first time this was coined and used in social sciences and in computer science and we know that most of those constraints and limitations there are also true for for the more modern types of digital work the other thing we know is that task decomposition also affects the nature of work we know that from the industrial revolution and the assembly line and that sort of things and we know the same also from the digital work environment and we know that the tasks characteristics also have a strong impact on intrinsic motivation that is grounded in performing a certain task so these are things we know from from literature and we use the self-determination theory a rather mature theory coming from the work psychology environment and the colleagues decky and ryan have been using it i think for 20 years 25 years replicating it and they have a huge present for us which is wonderfully tested scales right so for most of the things we look at they have a huge repertoire of great scales so no need to reinvent the wheel by developing new scales and it's also nice because that way we can reference back these types of results to other settings where the instruments that go back to that theory have been used and the most dominant one is the work design survey instrument which is also used by the european union which is also used by the oecd so that's a that's a good starting point right so in the self-determination theory it's about professional well-being and this is affected by the individuals intrinsic and extrinsic motivation and in the working context that's the specific we distinguish between controlled motivation and autonomous motivation so the controlled one is that comes from the setting of the organization and the other one is the one that is inherited in us and we would see that we have conflicting results regarding financial compensation in crowd work why because on the one hand side it's an extrinsic motivation and on the other side it can also be a signal for appreciation making you feel valuable and respected that you get that type of compensation the other thing we know and this concept we call internalization so that it'll become like an intrinsic motivation that way the other thing that we look at are the task characteristics and how they match to the different motivation types and we like to believe that these are considered to be additive so they add to each other so the overall motivation is the sum of the the the different pieces and so why do we look at identification as the focal variable and i'll just show the model with you in a second so that you can see where this all leads to well as i stated before it's a higher level psychological work outcome it's the long-term focus it's the pivotal antecedent to professional well-being which is at the core of self-determination theory and it has this long-term orientation and it's part of one's self-awareness right so we can report on it it is how we feel about it so this is the model and we are assuming what we would call a moderated mediation effect so on the left hand side we see the different task characteristics like the perceived autonomy the perceived task variety the perceived task identity and the perceived feedback as task characteristics by working on the task and we know that from the theory that is an antecedent of satisfaction and that is an antecedent or indeterminate of the identification with the crowd work and we assume that financial compensation will be the moderator for this mediation right you would find many studies that try to connect this directly with this right and we will show that this model that way explains more variance and you would also see that you would have financial compensation directly influence the perceived identification and satisfaction we control for all of that and I will share that with you in a couple of seconds so our assumptions are we will see positive effect on satisfaction and identification that's the core anchor here we assume positive effects of the task characteristics on identification and we assume that financial compensation moderates the indirect effect of task characteristics on identification work through satisfaction so this is a rather complex model but you will see that it's worth the effort because variance we explain will go up significantly by by playing it this way so the data collection is based on the biggest crowd work data collection that I'm aware of only German speaking and filtered only for German platforms right because the web is very hard to measure by nation the best thing we can do is language so the German speaking internet is the first the first setting the second one is the platform where we collect the data if they're all located in Germany so that there might be a little bias there might be some Swiss people working on on a German platform or some Austrians working on a German platform but that's the best we can do to control for that we see four different types of crowd working platforms very very typical one so it's basically testing innovation click work and I forgot the fourth but it's covering all major types of crowd work platforms so it's a cross platform setting that's the first thing right it's the first study to the best of my knowledge that does that across all platform types and the demographics that we see is a typical digital nomad digital worker type of setup right so average 35 mainly single not so male dominant right so a little bit skewed but not that much 66 percent employed or self-employed and higher education level than the average of the of the population so we have more people with A levels or Abitur that we would have in the population right and we see that we asked of course the crowd workers for their wages by average 50 percent of them of their total income is achieved through crowd work so they are heavy workers right and the average salaries here are above minimum wage and the average income they generate is 534 right so depending on where you are and what your social situation is in Germany this can be substantial right if you think of someone being on social on social security 500 euros is a lot of money to them right so it's not just for fun how do we operationalize the constructs we use the worked sign questionnaire as introduced before how do we measure financial compensation we use the monthly average income from crowd workers over the last three months so that we control a little bit for the season allergies and we related to the hours worked on platform now here comes the big the big the big the big the big methodological problem like with any study on not taxed income right we have to trust what people say that's the best we have right so if you say you've worked that many hours I have no way of controlling that right so that's the big the big methodological problem the controls we use the total income the share of crowd work to total income the employment status for your other job right because this is usually the second or third job how active you are on the platform gender age educational level and we have dummies for all platform types that's the most important one so that we can see if there are significant differences between click work testing and so on when it comes to the results on how they perceive the work how do we do the construct validation I mean that is all probably known to you who work with with survey research we do exploratory confirmatory factor analysis we check for the convergent validity the discriminant validity and the reliability of factors if you're interested in those details feel free to approach me any given moment yesterday we got finally the acceptance letter of the paper so I don't have to tweak anything no more now it's published I'm more than happy to share all the details and all the data with you so no problem what do we apply as an empirical strategy we use ordinary least square with a non-parametric bootstrapping approach right and we use the variation of causal step regression procedures while also ruling out alternative models that was the one that they forced us over the process of the review right that's where I had to learn it the hard way that that is a different thing and the bootstrapping approach tested directly for the moderation mediation effect that way we can rule that out now which means now we're on the safe side to show that these things are true and we have low type one errors and high power in assessing the moderate mediation effects and we have a higher predictive validity than causal steps or the product of coefficients approach so the so-called test so these are the regressions if you feel like that is interesting to you we test the direct effect of the independent variable on perceived identification the direct effect on satisfaction and on satisfaction as a mediator it's all tested in all in all variations now let's look at the regression results so we see we calculate three different types of equation assuming the different types of how you could explain what is happening and we see here the values for the tested effect of perceived autonomy as independent variable task variety task identity perceived feedback and the second one you would see on perceived satisfaction with crowd work this is perceived identification you see the values go up here right and in the third one you see that the dependent variable on perceived identification with crowd work and it has almost no significance on the first ones right but we have perceived satisfaction with crowd work on a mediator rather strong and significant the moderator financial compensation that's quite interesting right and the perceived feedback and financial compensation and that leads basically to us then doing a bootstrap and the bootstrapping result for the moderate mediation effect where you basically distinguish the person tails of the independent variable and the color coding gives you an indication about the overall pattern of the effects and the mediation effects of perceived autonomy perceived task variety and perceived feedback they're all moderated by financial compensation and that was our initial assumption right and the task variety that's the only one that is not moderated which is quite interesting for theorizing so let's go to the results basically most of the hypotheses hold except for the task variety and identification that is not moderated so the first one that was satisfaction and identification is rock solid the satisfaction mediates the effect of perceived autonomy task variety task identity and feedback on identification and the mediated effect of is moderated by financial compensation across all those right so basically the model holds true and we explain much more variance that way that's the interesting one we almost increase the variance by 40 percent that is quite nice then robustness uh they gave us a very hard time because this is survey research when it comes to common method variance and all those types of nasty things for those of you who are not familiar with common method variance that's a methodological problem that when you control or that when you collect data from the same individual um for the independent and the dependent variable there might be a flaw in it right so we've done all the tests that we could get hold of um potzakov and the harm and single factor tests um no indication whatsoever that we have uh that we might have a common method variance issue uh we also used unmeasured latent market construct technique to do that and also that gave no indication for common method variance so it seems to be not an issue here uh endogene endogeneity i love that word um there are two potential sources of that right so a predictors value is not given but rather deliberately chosen based on the purposes of the crowd working platforms or the crowd workers with high levels of identification might have self-selected themselves to participate in our survey those could be the two sources for flaws right uh we do that uh we we check for that with a two-step regression approach we estimate the effects of instrumental variables on each independent variable and the second step is that we test the effect of the independent on the dependent variable while controlling for the residuals of the first steps regressions so that's the way how we try to rule that out and the data there is also um highlighted in the tables here so the complexity specialization interaction with platform the cognitive demands and the equipment use are instrumental variables that we use i i find it very hard to report that in a talk because it's very dense um but those details basically just give you the the message that we tested for all possible uh flaws that might be in the data right and um bottom line is um all the data uh results or the analysis results that we are getting gives us no indication whatsoever that there might be a flaw and that is the good news right um the robustness of results when it comes to end the second source of endogeneity the satisfaction with the crowd uh so that again is task significance problem solving context factors and meaningfulness as instrumental variables and again um that are values that make us believe that there is no issue whatsoever there next one would be propensity score matching so the moderating effect of financial compensation could be only appropriately judged if crowd workers perform perform the same task which by definition is not true because we sampled across different platform types so by definition that might be an issue right so this is hardly fixable for us this is something we have to acknowledge right and the second best approach to deal with this is um and the propensity score matching where we divided the respondents into a high and a low financial compensation group and we're using task characteristics um to match each respondent in both groups against his or her closest counterfactual so that's a way of how to try to control for the difference in that in the tasks and to see whether their behavior changes in a certain way but that also turned out to be quite quite sound right so the match sample consists of 286 responses and the results remain consistent so that way we believe we can use the full data set if you have questions any given moment right now what does that mean in results well the first one is and that's quite interesting for us from a theoretical perspective we introduce a multi-platform view on crowd work right remember that the challenge always was apple and oranges this is a setup where we can control for those apple and oranges and see that the phenomenon of interest is stable um we think that this is a nice work um to show that specific type of crowd of crowd work and one specific type of platform studies cannot address but what we can and we also think we can overcome selection biases because we're at least a little bit more generalizable right with all the constraints that I mentioned before and what I find also very interesting we had that in the break I think uh with um collection IDA um very often we get feedback if we have national data in international journals well that's a German data set I think in this case it's an advantage because most of the studies that you would get from international contexts a large percentage of the crowd workers are coming from third world countries right so Amazon Mechanical Dirkus is a good example where you have a huge proportion of Indian Pakistani and so on this is now one coming from the first world only right with a very specific setup where it's very clear that this is something that is happening in a highly developed country in good economic times so that is definitely something that you can use to inform um decision making and and discussions about how to regulate and how to deal those things right and the other thing the moderate mediation effect is a nice one because it extends the prior research on motivational factors which looked at motivational factors isolated but that's basically not getting it it is the combination of things and their interaction that explains much more uh variance and it is the task designing crowd work um that we can now address when we look at the psychological outcome of it we see the interaction of intrinsic and extrinsic motivation and they jointly shape the satisfaction and identification and that's the interesting part to me and the satisfaction reflects the generative mechanism through which the task characteristics and financial moderation facilitate identification so it's a different logic that we see what is that helping us um for understanding the financial compensation well first of all the financial satisfaction financial compensation does not affect satisfaction contrary to prior research right and the satisfaction is a short-term evaluation of work outcome um and and the financial compensation does directly and indirectly affect the identification with the crowd work so it's the repeated positive short-term evaluations that the identification measures and um we see that this is very much in line with um what the unions had advocated for once you get to a certain level let's call it minimum wage that's the necessary precondition to have decent work conditions on top of that if you don't have the necessary uh income level achieved you can have the greatest work design it'll not be effective when it comes to satisfaction identification with the task because the prerequisites are not there um an alternative interpretation of the effects of increasing compensation so once we get over the plateau effect is that salaries or uh income can also be perceived as informational cues and it's that way an internalized extrinsic motivation so the fact that you make more money on that task compared to others is also a social message right which gives you self-esteem in that type of um activities and the indication of appreciation and competence that crowd workers have built up uh on crowd working platforms is something I don't know if you came across that in most crowd work platforms your most intangible most relevant intangible asset you have as a crowd work is your profile because it documents your skills your expertise your routine and for instance in software testing that determines 90 percent of the type of jobs you get exposed to so it has value to you so having that um that profile um is your precondition for getting the better paid jobs and if you were to change the platform for instance you would start in a new system with a with an empty profile and you would have to build your profile from scratch again right and you can see that for those who stay in platforms um that type of profile is something that they actually use in signal right to showing the expertise also to peers so that is an interesting element um that most platforms actually don't use it wisely enough right so they don't always have these different levels of expertise um showing it to the other crowd workers so that they can use it as a social comparison or a social um currency um the other thing that when it comes to theorizing is about task decomposition we use it as a contextual element that affects the design in the perception of crowd work and we see that higher levels of task identity lead to higher satisfaction so the more identity you're given to the tasks um the higher the satisfaction and it turns over time into identification which means lower degrees of task and composition lead to higher satisfaction so this is basically where you see especially in the click work setting when things are being pushed too far so that there's no way of how to um have um um your levels of task identity that you would need to be satisfied with it um from an economic point of view or from a management point of view we would advise those companies to give more degrees of freedom into the task to improve work outcomes and also the satisfaction with the type of work setting so these overly decomposed tasks may hinder um positive outcomes and actually that's something that we also see from studies on amazon mechanical tech um so that's also might maybe true across cultures right and um we also see um the mechanism uh being contingent on level of financial compensation so task identity only has an effect on identification once we have a certain level so again this minimum wage argument kicks in this becomes relevant the moment people say okay that's enough so that I can live on it or whatever it is that I perceive as the threshold right there are a lot of practical implications you can derive from that study uh governance mechanisms that encourage crowd sources to create tasks that allow for autonomy variety and identity and also to give um feedback they would directly influence um the satisfaction identification so giving crowd workers the chance to give feedback and to share with other crowd workers we can measure has positive effects we see that uh over Taylorism approaches are negative right um also for um the click work areas and we also see that especially if you run the click work on cent levels I mean that is again a catalyst for even worse um satisfaction identification levels I find that very intuitive and on the other hand we see that those who do it well um those crowd workers develop a strong psychological bond we found it in our case studies that we've done on software testing very strong bond to what they call their community right so this is also where sometimes the terms literature get a little bit blurry um so um for instance with um with test birds one of the big German crowd testing um providers um they call their crowd workers their birdies and so um they have the bird manager which is the community manager and they would never use the word crowd right never never for multiple reasons right because it's attributed negatively uh because it's not how they feel about it because the definition of community is the sense of belonging together that's what they want to create they want the crowd workers to feel that they belong there so this is where maybe also some of the theories that you read that are being used around crowd work could better be uh replaced by theories we know from online communities in that field um the other thing is we see churn rates of crowd workers and churn rates of crowd workers are very much in line with our results a direct function of how good do you implement good working conditions right so the click work has a huge churn whereas the software testing has almost no churn this gives you a strong indication if you want to do that in a sustainable way well you need to design reasonable working and payment structures so that people stay there for longer and um but the the other thing is how do you get the new labor upskilled and into the system and that's where those mechanistic tasks still have legitimacy so that people have this fast return on having done something successful right so it's probably also something where you have to control and account for the life cycle of the crowd workers all right further practical implications crowd worker exchanges um yeah I think those those effects are quite quite straightforward and the um the deteriorating strength of marginal effects of additional payment that's of course the the relevant one so the one you want to identify where is the the minimum wage level that you need for your community and our data shows it's different right so for software testing it's higher and it would be for click work once you have that sweet spot there is not much need to go beyond that right you rather need to do other things to keep the crowd engaged and uh yeah so what we would love to do and what we're currently trying to do is a follow-up studies um we wanted to convert this one time study into a panel study um so far we have three more measurements um but our sample size is shrinking right so now we're at 180 that have four measurements which is still nice but borderline for some of the stats we've done so that might be the next study to see if there is a dynamic overtime in it and um there are a lot of things um to to examine here uh interrelations between work performance psychological work outcomes and so on so for those of you interested in that type of empirical uh work on digital on digital work um a plethora of open things right so and since there is a dfg group here I think uh there are hundreds of opportunities to do reasonable studies and and interesting stuff which if interpreted well can help us make better decisions on how we want to build a good future for digital work because that's at the end of the day what we should do right because if we don't do it um it'll not happen it'll not happen that's something we can see um already just looking at the US data so my takeaways satisfaction mediates the effects uh of task characteristics on identification the effects are stronger um once we have um a greater financial compensation and there are psychological limits to task decomposition in crowd work that's it thank you very much and that at the end of the day brave you thank you very much for the very informative talk as far as I understood you have created the data set so and then you publish this test to the crowd how did you identify which features do you have to cover in the data set and how did you so the the data should have the features in order to reflect the information that you want to have as outcome right how did you identify the features how many features did you have and how did you um distribute these features among the data that you had that would be very interesting to me so what we've done is we've collected data on using the um the work design questionnaire um that full fledge requires some 40 minutes to fill out extended it so that it's in one hour survey to fill out and posted that survey across 86 Germans mainly German speaking crowd work platforms that has which means we have a vast variety of data in there and um so from that we then isolate the things that we believe are relevant so the features that you refer to are mainly coming from the case studies we've done with different different types of platforms so we've run now I think 10 to 15 platforms case studies on each type of platform and from that basically the phd students then came up with their conceptualization so the features came from the previous studies that how to what should be the questions in the questionnaire in there yeah okay yeah yeah and I can only I can only um share with you my my uh usage experience with the work design questionnaire it covers a lot it covers a lot really covers a lot so that will be my follow-up question um what is your experience with the work design questionnaire and what other dimensions of the work design design questionnaire actually work well with crowd design work because what we found in our research is that not all the dimension fit on crowd work now that you have this small task level true true so there are a couple of extensions um on the work design questionnaire that go for more granular type of work I don't know if you're familiar with that literature that's because that's a phenomenon of the digital times right so the granularity gets lower um we we used the latest version and the extensions um that helped us uh and we also had um yeah uh batteries and scales that were just not applicable that's that's because of the nature of work right so everything that is about the physical elements doesn't make sense in a digital environment right assuming that people don't do physical work so why would you would you control for that yeah yeah yeah I have another question if I'm still on turn um rather than the beginning of your talk you you spoke that the task that you find on code web platforms actually evolving now that you have data mining or um algorithm or learning mechanisms um do you think that the tasks themselves and content or that the role the crowd worker plays on the and the crowd web tasks are changing too and does this also um change task characteristics and the way your model works yes yes three times yes three times yes we see that um the task of um of the um crowd manager is changing substantially and now there are big differences in the type of crowd work right so let's use the the easiest case that's the software testing where you can see how the job of the test manager uh has so much potential by automation to be more efficient to produce his test cases faster to cut the chunks that he hands out to the crowd in different ways and now we see also algorithms that try to do um combinations of test cases so that you reduce the error I mean the version one would be you give twice the same task to two different people if it matches you're good if it's different um you give another task out to check for it that's a very simplistic way the more modern um algorithms they would cut the cases differently so that way you would have more cross-checkings and more more options to test whether what's coming back qualifies as a bug or not and um they see quality increases that are substantial because of that so it's not so much what the crowd worker in testing is doing it's the way how you design the task that that allows higher quality and of course higher productivity levels so for instance the test birds have been reporting on substantial increase there and a swiss company that is specialized in that field uh was talking about productivity increases of 50 percent over one year just by changing the way how you design and allocate the tasks and to me this is like a moment in time where everyone is trying what works right you see a competition of concepts there and so far there is no dominant pattern what is what is what is winning what definitely is promising is everything was text mining because they this is the the 2080 rule right with 20 percent of the effort you can cover for 80 percent of the work just by pre-automating it and in many tasks that that feedback comes in text and that is very powerful so it's not so much um something that the crowd worker the individual crowd worker experiences but something that's optimizing the process and background that was the examples that i gave you now in the um in the in the text mining um you would see that you ask different things people to do right so also the variance of tasks goes up because the algorithm is doing more advanced and different things so that is an example where um the autonomy and and the task identity could be influenced in a positive way so this is also nice because it shows us that sometimes technology can make work better by making it less routine and less standardized thank you uh yeah thank you for your interesting talk um i just wondered because i'm um currently doing research on this uh topic two on autonomy for example and using the self-determination theory if you also include the counterpart monitoring in another study for example because i wondered how um crowd workers are monitored and how this could probably affect their satisfaction or your identification yeah um we have not controlled for that in that study but this is something that i'm 100 convinced of being highly relevant for internal crowd work if you apply the same logic within an organization that's what we're currently studying and working on um this negative thing of being controlled and monitored is much more relevant to what determines am i happy with that type of work so we're currently working with a big tier one supplier in the car industry um that is launching that in pilots across his r&d departments and the logic goes like this um for 20 of your time so the fifth day of the week um you are free to work on projects that you wish and here is the infrastructure the platform where you can choose them if you wish and uh there the question of control and accountability uh is is definitely very relevant um to measure to see whether um identification and that type of things are influenced in a positive or negative way because there's also a lot of dependence on the company culture and let's say the the the previous history of how management and employees have interacted whether that is a trustful environment or whether there's more control elements in it so there we can see it already in the case studies that it's relevant um i would also have a question i have a microphone so i don't take the other one uh yes uh so um my my question would be what you're saying is that we actually should also pay crowd workers more right i think in a normative sense that might also make sense in a certain way but i'm thinking is that not also wishful thinking in a certain way right i mean being an economist i would say well there's supply and demand right we determine the price and if the price is very low bad luck right now if we artificially increase the price i mean that would be kind of let's imagine you have three postdocs right one you can finance the second one you can't and the third one you can't either but you could pay for a research assistant right and then the guy doesn't you know like he's working uh you know for the research assistant wage and uh then what you're saying is if you increase it to a minimum wage well this guy would have not been in academia in the first place right and now we ask actually whoever it is the taxpayer the platform whoever to pay the full wage for him or for her and uh but doesn't that in some way skew the market right i mean now everyone gets in everyone who wants to do it i mean they're getting bad pay uh but we're giving them good pay because we think it's actually the thing we should do right so your assumption is that my my reasoning is pay them more that's not correct my assumption is if you are running a platform you need to know do i want my crowd workers to be around with me for a long time that's my precondition in testing that's very important because the type of jobs that i get requires experienced crowd workers much more than unexperienced you usually have excess supply of unexperienced but what you need are the experts uh and if you want to keep experts you need to keep them happy in order to do so that's basically what's coming out of the data pay them in a reasonable way um so that the other characteristics of the task for instance that it's more fun to work with you that it's more interesting to do the things that they can choose more things they become more relevant once you go beyond that level that's the implication right if you are in the in the amazon mechanical turk model there's no need right but does it mean that the testing platforms for instance don't get it like there's a market failure that they're paying too little but they pay the decent wage so there's no need for regulation for that i'm not arguing for regulation either um not not with the data that we have right now there is no evidence whatsoever in all our studies of abuse of power or of market failure that let's be very clear here right that's nothing we see in our data but we see superior results and better performance for those who treat their crowd workers better that's basically a very straightforward economic argument uh you have happy employees they do better jobs so right so that's basically the logic and it's a calibration thing and you'd be surprised to see how many platforms actually have not thought well about where to set let's say the the entry level for the compensation that way they give away potential right but if you if you check for that what is the best way how to get into that entry level or how you build careers within the crowd so that they get quickly to that level you would attract more of the of the scarce crowd workers that you want to have and that's and you can see that those who are successful they're better at that than the others so that's the evidence we get from the case studies that what we see here in the large-scale data set is also happening in the in-depth case studies but we should discuss market failure once we see economic downturns I'm very sure that the ability of the market to absorb twice the amount of crowd workers is not there and then we will see market failure but that's my personal guessing which is of course also logic which is reflecting the ancient Fordism if you want to keep good workers among you as an employee you have to invest into them and which of course works when they have some skills and the less skills they have the more weak and the less from an employer side or a platform side you want to invest into them and then comes in pops in the question do you have to correct at one question or well it's a question or remark it's it was on work the definition of work which was an element of accomplishing a task income so but traditionally in most of the social laws there isn't third element which is regularity it should have a regular pattern so one activity with income is not work but when that income is with some regular pattern because then the idea is that you try to earn income to live your life upon it to earn your living and here they most of the systems they they they struggle how to measure that regularity is that I was just about to say that yeah so what do most what do some systems do in Europe they say we we do it differently we look at a minimum amount of income you earn from that activity and if that minimum is reached let's say 500 years a month or 600 years a month or 6000 years a year or dollars or whatever then we consider that to be regular so the amount you make yeah would control for regularity yeah and what do you do with those that are highly professionalized in in testing for instance that do two hours three hours a month that's one of my points but when I see that systems at least social protection systems more and more start to lay upon that kind of approach in more and more countries they say we give it up to find what is regularity we simply look at the income you generate and if you generate enough income we consider that activity to be work now most of the time it's around 6000 euros a year or 600 months a month so if I saw the averages already in Germany they drop out I mean most of them there is also a second line of reflection they say and that's also confirming the German well at least the sample if you already have an activity what's the sense of bringing that second activity on the platform into the systems of pensions and so on and let us exempt that I mean because they are already protected oh which is of course creating false competition but that's the direction in which mode well there is a kind of tendency where you see that countries go into its minimum thresholds and if it's a side activity let's exempt as well which I think is it should be the other way around you should broaden your income source as much as possible and so on so forth and when I listen carefully to your story also for the element of satisfaction it could be an additional element in well my kind of activity is also making sure that I can have supplementary pension whatever or supplementary and so on well I might be biased by the by the swiss system and in the swiss system we don't distinguish between the sources of income we pay social security and everything which I think is a way to go for okay so we have one more a question and then we have to head towards the the conference dinner and I will try to where it is after the the question is answered try to keep it short okay thank you very much I have a brief theoretical question because we've been doing some research on digital transformation of organizations and we find one key challenge is identity professional identity of employees so um for example how do I make sense of what I do here and what I am relative to iot or something like that and because you've been talking about identification as well and you briefly touched on identity issues like I'm a crowd work I'm asking based on the case studies you've done what type of identity dynamics have you seen or are are they relevant at all sort of in the sense that does the nature of crowd work affect what people consider themselves to be and what what they do and stuff like that so what we have not looked into is the identity perception of the individual overall because we see people mainly being in multiple employment what we've checked for is basically their identity perception once being part of the crowd so the example of the test birds is a good one where they have very strong bonds and they consider themselves to be a birdie a test birdie right so the very active ones they call themselves test birdies and that's something would this individual say if you ask him in the morning what are you I'm a test birdie I don't think so but it's it's it's a different it's an element of and the the previous studies we've done that we're looking for is there something like an identity of a crowd work no no no no no no why because the type of work is so different and usually your object you refer to is the community or slash crowd you work for not the type of work right so it would be like I'm a I'm a Daimler person rather than I'm a I'm a mechanic right so my professional understanding would be I'm with Daimler rather than I'm a mechanic the same we would see