Transcription Update- HIT Dispute Tool Guide

HIT Dispute Tool

---

We are so excited to introduce a new automated way to dispute your HIT scores!

We understand that there will often be minor disagreements between workers about how to apply CrowdSurf style and General Guidelines. For the first time ever we have an efficient, effective way to mediate these HIT score disagreements!

CrowdSurfers, meet the HIT Dispute Tool!

The HIT Dispute Tool makes your life easier by wrapping the entire dispute process into a neat little button on your Work History Dashboard.

 

This dispute button will send us all of the required information for a HIT Dispute.

This includes:

1. Your worker ID
2. The job ID
3. Your transcript
4. The score awarded by the reviewer
5. And a brief description of why the review is incorrect

Then we take all of this information and evaluate the review. You should hear back from us within two business days regarding the status of your review.

The best part is, is when you use the automated Dispute Tool we are able to adjust your score on successfully disputed HITs. That’s right, for the first time ever we are able to adjust your HIT score on disputed HITs!

Here’s how you dispute an unfair HIT score:

1. Go to your Transcription Work History Dashboard.

Your Work History Dashboard is available from the Stats tab of every accepted CrowdSurf HITs in Mturk.

Alternatively, if you would like to bookmark the link for the Dashboard please just insert your Mturk ID where it says YOURMTURKID:

https://work.crowdsurfwork.com/reports/work_data/worker/?crowd_worker_id=YOURMTURKID&crowd_name=Mechanical%20Turk

2. Click on a blue score to view your Review Feedback

HITs are randomly sampled to be reviewed. Click on the blue score to review your HIT feedback and use the dispute tool if necessary. The HITs with dashes were not reviewed and cannot be disputed.

 

3. Click Dispute on an unfair HIT

If you disagree with a HIT Review and you have available disputes, then you can click Dispute to have a member of the CrowdSurf team review the dispute.

 

You will have a limited number of disputes, so we encourage you to use your disputes wisely. You will have a maximum of 3 disputes for every 100 HITs that are reviewed. It may make sense for you to spend your disputes on your lowest scored HITs.

4. Write a brief description why this score deserves to be disputed

Once you click on Dispute, a box will pop up prompting you to “Provide a specific reason why the score is not correct and our team will evaluate the review and your reason.”

 

Please explain exactly why you feel the score is not correct, and what guidelines, if any, the reviewer has violated. When you’re done click Submit Dispute.

5. Once your dispute has been submitted

You’ll receive a notification that states “Your dispute has been submitted and will be reviewed by our resolution team within the next 48 hours.”

 

No further action is required at this point, the dispute request has been received by our system and will be processed by the CrowdSurf Team.

6. Return to your Dashboard to check status

Disputed HITs will be highlighted red on your Work History Dashboard. Once your dispute has been reviewed by the CrowdSurf team, it will say “Approved” or “Rejected” and your score will be adjusted accordingly.

 

Things to know about the HIT Dispute Tool:

1. You will have a maximum of 3 disputes at any given time.
2. Disputes do not refresh based on calendar dates, but rather new disputes are accrued based on the number of reviewed HITs in your work history. This means that available disputes is tied to the volume of HITs you complete.
3. You can earn another dispute for every 100 HITs reviewed.
   1. If you have 300+ HITs reviewed in your entire work history with CrowdSurf you will start with 3 disputes available.
   2. If you are new to CrowdSurf, you will start with 0 reviews and will earn your first one after 100 HITs have been reviewed.
4. You cannot accrue new disputes if you already have the maximum number of disputes available.
5. If you already have the maximum number of disputes, you must use a dispute (and work more HITs) to accrue a new dispute.
6. The tooltip that appears when you hover over the Dispute button will tell you how many disputes you have left.
7. The Dispute Tool will change your score on an approved dispute.
8. If you have a concern about a reviewer that you would like to bring to our attention beyond submitting a dispute, please follow the manual How-to Dispute your HIT Score Guide: http://crowdsurfwork.com/2015/07/how-to-dispute-your-hit-scores/.