MTurk- FAQs

Follow
Mechanical Turk

 

MTurk CrowdSurf FAQ *Under Construction*

 


1. Where can I find my MTurk ID?

ANSWER: It’s on your dashboard on Mechanical Turk. Look for it under the Your Account Tab at https://www.mturk.com/mturk/dashboard.

 

Back to the top

2. Where can I find the Job ID?

ANSWER: The Job ID is available at the bottom of the Help Tab on every HIT. Please included the Job ID and your worker ID in messages to CrowdSurf Support.

 

Back to the top

 

3. When can I expect to be paid?

ANSWER: There both regular base payments as well as volume base payments.  All payments are made through Amazon to your Amazon Payments Account.

All regular base payments for work performed in Mechanical Turk are paid after the HIT has been quality controlled. Each HITs base rate is visible on the right side of the HIT queue. Once the HIT has passed the quality control step a payment is instantly credited to your individual MTurk account.

Volume bonuses for Transcription Tasks are paid on Mondays (excluding holidays) depending on the end date of the bonus (note that bonus extensions may result in payments delayed until the close of the bonus period):

    • Monday payment reflects earnings from the previous Monday 12:00am through 11:59pm Sunday PST
    • All Volume Bonuses are published on the CrowdSurf Blog.

 

Back to the top

 

4. What are your standard base payments?

ANSWER: Base rates for Mechanical Turk HITs are subject to change.  Base rates are determined on both the length and priority of the HIT.  When you log in to your MTurk account search for "CrowdSurf Transcription" to find transcription HITs. Longer HITs will earn you more.

Back to the top

 

4. What are your bonus rates?

ANSWER: *Bonuses change often.  Updated Transcription bonus information can be found on our website.

Back to the top

 

6. I think I was paid too much/too little, what should I do?

ANSWER: Oops! Mistakes happen and if you feel like you have been paid incorrectly please contact us at [email protected]  Please always include your MTurk ID in the email.

Back to the top

 

7. Can I retake my CrowdSurf qualification test?

ANSWER: No, but you can email us at [email protected] with your MTurk ID and we can set you up with new test material or assign qualifications to you.  Please always include your MTurk ID in the email.

Back to the top

 

8. I have issues with a transcription or a review HIT,  what can I do? ANSWER: Email us at [email protected] Provide screenshots or Job IDs when possible (You can find the job ID on the bottom right corner of the media tool when you click on the "Help" tab on the top right corner.)

Back to the top

 

9. What does my MTurk qualification score mean and why does it go up and down?

ANSWER: Generally the qualification score reflects the volume of work completed and approved but not necessarily the quality of your work. This score reflects when a HIT is approved (score increases by one point) and when a HIT is rejected (score decreases by one point) Scores will remain the same for HITs that were scored "poor".

In order to continue working CrowdSurf HITs, the score must be above 900. At CrowdSurf we use or own measurement to maintain the quality of our transcripts. It's called the QC score (Quality Control Score).   

After you submit a transcript we have other workers that review those transcripts give it a score and make edits if necessary. These workers use a 1- 4 scale.

  • A score of a 1 means fraud, transcript contained several errors like misspellings, punctuation, wrong words, etc that seriously affect the meaning of the transcript.
  • A score of a 2 means poor, transcript contained several errors like misspellings, punctuation, wrong words but it was rather sloppy work than an intentional fraud.
  • A score of a 3 means good, transcript contained some errors like misspellings, punctuation, wrong words that could have been transcribed correctly but it was mostly decent work and the majority of the transcript is correct.
  • A score of 4 means perfect, the transcript perfectly matches the content of the video, the reviewer had to make minimal changes.

On this scale, a HIT is being rejected if it receives a 1 or a 2 (your qualification score decreases on Mechanical Turk), or is accepted if it receives a 3 or a 4 (your qualification score increases on Mechanical Turk when you score a 4).  We work to regulate both sides of this process, so we consistently contact workers with low scores, and we eliminate consistently harsh reviewers from our workforce to protect transcribers.

Although you have unrestricted access to view your qualification score whenever you want, we encourage you to email [email protected] with your MT ID and request information about your average QC score.(Please always include your MT ID in the email).  We ask that you limit these requests to once a week, and please be patient with us, it may take up to a week for us to respond with a quality report.

Back to the top

 

10. How do I score a perfect 4 on every transcription HIT?

ANSWER:  Our goal is verbatim transcripts with correct style and formatting. Use our style guidelines, and scoring criteria, and email us with questions. We are always happy to help!

Back to the top

 

11. Im working on CrowdSurf review HITs, what do I do?

ANSWER:  Your job as a CrowdSurf reviewer on MT is very important!  You help decide the average score of workers when you award their transcript a Fraud, Poor, Good, or Perfect rating.  Use the training guide to help you review transcripts.

Our review process is completely automated, so just like any qualified worker can complete the transcription HITs on MT, any qualified worker can complete the review HITs on MT.  Use our scoring criteria to make sure you are accurately and fairly reviewing each transcript.  

CrowdSurf will revoke review qualifications for any workers who are fraudulently or unfairly reviewing transcripts.

Back to the top

 

12. How do I continue to work on CrowdSurf HITs?

ANSWER: To keep your CrowdSurf qualifications on MT you must maintain an average score of 3.5 or higher and an approval rating 95% or higher.   We will revoke your qualifications if we feel that your quality/accuracy is too low or you are incorrectly reviewing transcripts. In most cases, we send you a warning first if your score falls below 3.5 or if our data shows that your reviews have been too harsh or too lenient.

However, if we see that there is no improvement or if we see a very low score and a high number of completion we will revoke your qualifications without a warning. If you feel like this action was unjust you can contact [email protected].  Please always include your MT ID in the email. and we can look into your case further to determine if we will give you your qualification back or not.

Back to the top

 

13. How come there are so many CrowdSurf HITs at times and so few HITs at other times?

ANSWER:  CrowdSurf provides an on-demand media captioning service.  When our clients have a lot of media, we have lots of HITs available.  Likewise, when our clients are creating new media, we’re slow to upload HITs.  

Please understand that CrowdSurf is not leaving MTurk when volume is low.   We simply have inconsistent volume, sometimes more, sometimes less.    We do our best to supply you with a steady stream of HITs, so keep checking MTurk.   To stay up to date with volumes feel free to check our website, follow us on Twitter or like us on Facebook.

Back to the top

 

14. How do I find CrowdSurf HITs about specific subjects or that are high priority?

ANSWER:  All of our media content is mixed among all of the different queues in MTurk,  so there's really no way to predict where lectures, sports, news, or cartoons will end up.  We also cannot predict when we will have urgent HITs. It depends on our customers and it also depends on total volume in the system.  But you can sort HITs by reward amount in MTurk to find the highest paying CrowdSurf HITs.

Back to the top

 

15.  How do I find the best paying CrowdSurf HITs?

ANSWER:  Search CrowdSurf Urgent in the search bar within MT.  Make even more money by avoiding rejections: skip or return HITs that you feel are too difficult.  In general we pay higher for higher priority, our levels starting from lowest paid to highest paid are :

    • Low Priority
    • Medium Priority
    • High Priority
    • Urgent Priority

Back to the top

 

16. I am reviewing a clip with challenging audio and the worker used [INAUDIBLE] OR [UNKNOWN], but I can not understand the speaker?

ANSWER: Please make a judgement call. If you feel like it was easily audible or the transcriber could have easily looked up the word in a search engine, then you should grade a 2=poor. If it was difficult to understand and it took you several times yourself to understand what was being said please grade a 3= good.  When in doubt please score a 3=good.

Please also consider that if the content is very specific, e.g. a higher education chemistry class, etc., and you are a chemist, that the word may be easy for you to understand and very difficult for others that have only basic or no knowledge of the field.

Back to the top

 

17. What does CrowdSurf do with my transcript?

ANSWER: After you complete one of our HITs, your transcript gets edited by other workers just like you. Our transcripts go through a first pass in MT with transcribers and reviewers, and then snippets are reassembled to form a complete transcript before going through a second pass for final editing. Depending on the client’s needs we will perform special handling, speaker identification, or quality control on your transcript. Once complete, your micro-transcript forms part of educational lectures, cartoons, sports broadcasts, news segments and much more!

Back to the top

 

18. I want to learn more about "Freelance Transcribing", who do I talk to?

ANSWER: In addition to the CrowdSurf style guidelines and scoring criteria, there are lots of helpful resources available for you on the web.

  •  Check out the CrowdSurf Subreddit  to connect with experienced CrowdSurfers and get help from the CrowdSurf community.
  • Check out the TurkerNation forum. See what other turkers have to say about CrowdSurf, making money, and being a successful freelance transcriber in Mturk.

Back to the top

 

19. How do I learn about new CrowdSurf offers and bonuses n MTurk?

ANSWER:  We will post daily messages for turkers directly within the HIT itself on the right side of the media player for your convenience!  Check for guideline updates and other important info straight from within the HIT you are working on.  You can also navigate the tabs on the right of the media player for helpful hints and job-specific notes.

Back to the top

20.I keep receiving empty HITs/I keep receiving HITs with foreign language, what do I do? 

ANSWER: Email us immediately at [email protected].  Please always include your MTurk ID in the email.  Provide screenshots or Job IDs when possible (You can find the job ID on the bottom right corner of the media tool when you click on the "Help" tab on the top right corner.)

Back to the top

21. My HIT froze, what do I do?

ANSWER: When the player freezes, copy what you have transcribed to that point and then click on "HITs Assigned to Me" to retrieve the hit. Paste the text back into the text box and continue on from there. Just make sure to copy anything that has already been transcribed because if not, it will be lost when you reopen the HIT through the Assigned HITs area.

 

22. My QC score for Review HITs is low, what's wrong?

 I am new to review HITs and I am disappointed that I have a series of ratings of mostly 3s with one 2 and one 4, for an average of 3.

On the review HITs, I can't click on the number to see the feedback like I can on transcription HITs.

I can't improve if I don't know what I'm doing wrong. What can I do about to improve these QC scores. 

ANSWER:  The HIT QC Scores that you see on your Work History Dashboard are the scores that you award those HITs based on the edits made and errors marked. We will contact you if we received a dispute about your HIT scoring. If we receive more than 3-5 valid disputes for your reviews in 1 week, you will be blocked from Reviews until you can complete further training.

 

 

Still don't see your question? Suggest a new FAQ in an email to [email protected].

 

 

Was this article helpful?
1 out of 1 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk