 Imagine if you have a set of thousands of video files, audio files, images or documents you need to perform some cognitive tasks on them like transcription or translation. So the solution will is crowd sourcing, people here will be knowing what crowd sourcing is. So software requirements were to import tasks into the files on which we want to perform the cognitive task, distribute the task among qualified users, we will be coming on to that later, allow users to provide appropriate tags or transcriptions i.e. to perform those operations and evaluate the user provided type. Technologies we used a PyBOSA, a Python based crowd sourcing framework, PLASC, a Python based web framework, PostgreSQL as a database, SQL Alchemy as an object relational mapper and PgAdmin3 as a database administration tool. We have used PyBOSA, it's a framework which actually supports crowd sourcing, all the built-in functionalities, basic functionalities required for crowd sourcing are already there. Problem is that it doesn't support local file importing, it only supports via some Google Drive or Dropbox or anything like that and it also requires a Python script to be run, then convert the things into CSV using the Python script and then it gets converted to JSON. It doesn't support importing from the local machine. So this is our homepage, so it has a recommended project which the site admin can choose. So suppose Sakej wants to contribute to a project, so he just selects a project, he knows a lot about Mohen Chhuzar's civilization, he wants to contribute. So he basically contributes? So first we have to answer a quiz so that the project owner knows that he is eligible to answer. He is eligible. Thanks. So he selects a quiz? Yes, he is eligible. So he answers a quiz? He is eligible. Sir, he needs to answer it. Log in as some eligible user and so. I can't do a full testing here. First we are having some eligible user in your system or you are not tested at all. Yes, sir. So this is the question. So the quiz has been given now and this is the Contribution. So what is this image? So suppose it's an animal. It has been tagged. He moves on to the next task. Now what is this? So this is a text document. So you can write anything. Actually they are simply tagged. Tell me, how many iterations you are going to do with the one image? No, there is a task redundancy parameter which is present in pipe also. So you can set it. Suppose you, so you set it to 30 then the same image will be given to 30 users and 30 tags for the user will be. So show me an image given to 30 users. Just tell me, how many iterations you are going to do with the one image? There is a task redundancy parameter which is present in pipe also. So you can set it. Suppose you, so you set it to 30 then the same image will be given to 30 users and 30 tags for the user will be. So you are not able to show me any tested end to end result. You have done a system, right? Yes sir. To show me. Sir, these things are already present in pipe also. So that's not what we have actually done. So what have you done? So we have done the local importer and we have added the quiz inside the system. So because the current system was not considering the user because he was giving the tasks to the users in any fashion. He was not considering how the user is good, whether it is good or not. So that is why. That I don't agree. That it is to be done by a quiz. Why it should not be done by a quiz? No sir, quiz is just for a starting evaluation. Why? Just to evaluate him. Why? So that we can. Sorry, I am very upset. Who are you? Me, I have got people saying Drupal is horrible and now I have got people saying your pipe also is horrible. It is a working system for a long time. It is a collaborative community. It is supposed to be working. Thousands of people are using it. Four upstarts here come and say I will add a quiz without attempting the quiz. I will not let you contribute. I don't know whether that is an acceptable thing in a collaborative community. Sir, there is an option. If the project admin wants a quiz to be taken before the... Then only... Then only he can choose if he wants it. Correct. So all the people who are using PyBosa, all the project admins who are using PyBosa are stupid people. They don't want to do that quiz. You are the people who want a quiz to be added. How can you change a process which is accepted by a collaborative community? Sir, we are actually... You added a quiz. Correct. So that is only... That is only the entry point. Okay. Yes, sir. What else have you added? We have added the local... No, no. I have not seen what you added in the quiz. No, sir. Quiz actually can be... Quiz is what you added now. Yes, sir. Explain the quiz to me. It is basically a set of MCQ questions to which the project admin already knows the answer to. Correct. So if the user... So set up a quiz for me. Yes, sir. Yes, show me the interface. Don't try to... Okay, okay. Quiz name I'll accept. Then what do I have? So question text. Question text. Option one, option two. How many I can give? Four options. Options? No, how many questions I can give? Multiple. Multiple. As much as you want. As much as I want. Yes, sir. Okay. What is the criteria? What is this image about? Where is the image? Sir, I am uploading the image. Oh, I have to upload an image. Okay, alright. So I have to upload an image. That's fine. I upload an image. I ask some question about the image. Yes, sir. I give four options and I choose. Yes, sir. So that quiz setting you have implemented. Yes, sir, we have implemented. Okay. How do I specify a rule? For acceptance and rejection? No, sir. The correct answer can be said. Correct answer I know. I have given 100 questions. That is what I am. I am stupid. I mean, no. I want to select. So if I want to give a selection, I will give 100 images. 100 answers. Pura tum. Das. Ganta kaam koro. Phi baat mein mai sochunka. Tum ko allow karne kai ki nahi. That is your model. But, okay. I accept that model. Okay, sir. Where do I specify passing marks? Sir, there are no passing marks. No, sir. Then what is the point in giving 10 questions? No, sir. All 10 are correct. I will take and one wrong. I will not take. What are the rules? No, sir. The one who is better at tagging a particular image or a document or a video. We are actually categorizing it accordingly. And then we will just give the tasks. Show me. Show me how you are categorizing. Because I don't see any category here. Whether it is image or a video. I don't see that. I am taken from the extension of the file. Extension of the file. Okay. I have written this quiz. Show me any quiz you have created. Okay. And the results of the people giving that quiz. Show me. No, sir. The results are not actually shown to the user. Their results are just views. Admin locker ke dikhaw na. I want to look at test data, sir. No, sir. If you are not tested, then don't even talk to me about this. No, sir. That software is useless. If you have written quiz, I need to see the quiz result from an admin. Please show it to me. Sir, the quiz results are not displayed to the user because. Admin. We are not even the admin actually. Just because we just want to. You said who is going to make the call. He is good at video. No, sir. He is good at text. He is good at something else. No, sir. It's not going to be done manually. It will be sorted basically. If he is good at images and audios, the best. He gets the highest score there. Then he will be given images and audios. How do you do the highest score? By giving the quiz. I want to see. You have put in a different process. Okay. I am just evaluating whether your process is correct or not. And after this, you walk out. Okay. Tell me whether the process is correct or not. That is a major contribution. It seems you are done. No, one more thing. There are many other things. There are many other things we have done. First, this itself is wrong. Then I will not go for further thing. I have no time. I am running three hours late. No, sir. Its image, audio, video and document is basically there. Then according to his answers, he will get a score for separately. And that 10,000 people who have given scores. Abhi kya karna hai uska score ka? Where he stored it? Show me the database where you have stored it. No, sir. According to his scores. Then he will be getting… Suppose he gets the maximum score for an image and an audio. So, he will be… So, I get 0, 0, 0 and 1 for every somewhere. 100 questions I have asked. So, 1, I am qualified. What are the rules, sir? You are setting up a quiz. And you are not telling me what are the… How… The answers you are using… Sir, the tasks are distributed dynamically among users by algorithms known as schedulers. So, they… Have you written the scheduler? Yes, sir. Yes, sir. There is scheduler. There is scheduler. It is based on the maximum score which you get. According to that… Do one thing that show us a workflow diagram. That is the best way. Yes, sir. In spite of talking randomly. Just show the workflow diagram. Yes, sir. Yes, sir. And now explain the thing. There is a project admin who creates a task. Uses the task importer for creating it. Project is created, then tasks are added. Task is any document has been added. Then it goes to the server. This is the crowd. These results go to the task scheduler. And seeing the results, the task scheduler takes the tasks and gives them to the user. So, you have modified the task scheduler. Yes, sir. So, there are different varieties of task scheduler. So, we have added one. So, you have modified… Yes, sir. Yes, sir. Yes, sir. Yes, sir. Which takes as input with data. Yes, sir. Correct, sir. Yes, sir. Okay. So, then the image audio… Tell me one user. I will take a test case of one user and 100,000 users. And you tell me what your task scheduler does. Because I have got the viewer written, no? Okay. So, it gives it… One user is slow mark. What I want to know. That also actually is going to get some tasks. But that will be done on a random basis. And then when he contributes the task, according to that, we can train it. Two users. One user is zero mark. One user is one mark. What is your task scheduler going to do? So, for the one mark one, wherever he gets one mark, that is image… I am a very simple man. Okay. So, he gets… Zero marks everywhere. Two user right. What does your task scheduler do? Okay. So, then it will distribute it according to the default one. No, sir. Every… If everything is same… I think you have done it properly. It is not properly tested. Okay. 100% not properly tested. You are not just not a problem. You are just writing software. Same problem with a lot of other people also. It is not tested. I know it is not tested. I know you are sure me. Do you have a sample case available where you can demonstrate… This has happened. This has happened. This is the output result. Do you have any sample case? We have not done that. No. The testing we have not done with many users. He is appointed. Okay. If you want to talk about… What else you have done? Okay. Sir, quickly… Quickly just present. It is not tested at all. Important. Quickly present the things. What else you have done? Just continue. Existing Pibosa has two importers like Google spreadsheet and CSV. We have developed one imported. That is a local importer. That will be accepting zip format or tarzZ. And it will take the file and that will be categorized into four types like images, documents and videos and audios. And user can choose the files which they want to upload. And he can add the questions for particular category here. See, these things they have explained. You explain the other features. Continue. There is a editing question module also. We have added task presenter. Existing Pibosa has a default task presenter but that is… Either Pibosa existing software has only task presenters relevant to an image or relevant to a video. It can't handle all of these based on what the input data is coming. So we wrote a task presenter which sees the input data as a video image document. How you chose this? How you chose this by extension of the file? Yes, sir, by extension. Is it a correct way? Sir, actually… If I got a PDF file, rename it as a JPG and then what it will do? Sir, but if that happens in all the operating systems also. If we rename the file as .jpg even the operating system becomes confused as to it is a JPG or a PDF. It shows that… It is a lot of space for the hackers like me who can confuse your software very well. And the one advantage is that using RESTful APIs any other project who wants to crowd source their data to us can integrate with us. We expect and give results in JSON. Okay, thanks. Next group. Thank you.