In this experiment, you are working with an AI that is answering questions about images. You cannot see the image, you can only see the AI's answer. Your task is to decide if you trust the AI's answer or not.
If your decision is correct, you gain a reward of $0.10. You will see your total reward at the end of the experiment. Your payment is $2.0 plus whatever you accumulate during this experiment.
To help you make your decision, you will see the AI's answer, along with an explanation, and some information about how reliable the explanation is. After getting each piece of additional information, you will be asked whether you think the AI is correct or not.
(-/-)
Thank you for participation in our study.
Your total reward is $0.00 (0 questions answered).
Please navigate to Prolific or with the code C18F0V8M.
Thank you for testing out our study.
Your total reward is $0.00 (0 questions answered).
Please complete the exit questionnaire for providing feedback on the user study.
Welcome to our user study. In this experiment, you are working with an AI that is answering questions about images. You cannot see the image, you can only see the AI's answer. Your task is to decide if you trust the AI's answer or not.
You will first be shown the question that the AI is trying to answer, as well as the AI's answer.
In the first step, you will be asked whether you think the AI's answer is correct or not. You can also indicate that you are unsure.
You will next be shown an explanation from the AI. You will be asked again whether you think the AI's answer is correct or not.
Finally, you will be provided with some information about how reliable the explanation is. You will be asked again whether you think the AI's answer is correct or not.
The experiment will have you verify 10 questions in total.
You are not permitted to leave this tab for the duration of the experiment. This will be monitored and violation of this can result in your submission being rejected.
The average time of the experiment is 10 minutes.
You will not be paid if you already participated in this study before.
Click Start study if you understood all the instructions and want to start with the first
question.
Feel free to re-read the instructions again (Previous).
Question:
The AI thinks the answer is:
AI's Explanation:
The AI system thinks that the answer is correct because...
AI Confidence that all the visual details in the explanation are correct: xxx AI Confidence that the other options are incorrect: YYY
