I've been thinking recently about the role of confidence in chess. My theory has always been that there are some things that it helps to be unconfident in (like driving) and some things it clearly helps to be overconfident in (public speaking, fighting), and that chess clearly belonged to the latter group. Last weekend, while coaching at the Nationals Girls Championship, it occurred to me that I was hearing the phrase "I don't know if this is right, but..." a crazy amount (though interestingly, not from the very best players or the very worst). Which inspired me to try an idea for a research experiment that Jean Hoffman and I had come up with a couple months ago.
I'm trying to test whether girls are less self-confident in their chess abilities than boys, if that affects their ability to find the right move in different situations, if it impacts their time usage, and whether one gender is more accurate in their self-assessment (i.e. maybe girls and boys both answer a question incorrectly, but only girls are aware of it). I'm in the middle of producing the survey and test positions, so I'm eager for your feedback or ideas. I'm excited about doing this well, because I feel like I'm in a uniquely good situation to gather data: many coaches have already offered to help.
Students are given 12 positions and asked what move they would play if they had this position in a game. While the problems are not labeled as such, there are four types of answers: tactics, attacking combinations/moves, positional moves, and positions where you must respond to your opponent's threat. After deciding on a move, students are asked
a) how sure they are that their move is a good move
b) how sure they are that their move is the best move.
They can choose from very sure, sure, medium sure, not sure, and it's a guess.
There are two worksheets, one for tournament players 800-1100, another for those rated 1100-1500. Students may take as long as they wish to answer the questions, but they are asked to report the total time.
After* completing the problems, students are asked a few questions: age, gender, ethnicity, rating, time spent on chess, time spent on this survey, and how good they see themselves as being, relative to others in their chess club. (What else should I ask?)
Any statisticians want to offer any advice on what I need for this to be remotely valid? Or where to go to learn how to analyse the data? (I know what a standard deviation is and I can use excel, but that's about it.)
ALSO: If you are a chess teacher and would be willing to give this to your students, please let me know. It should take about 30 minutes to complete, and I would be happy to send you the answers, so you could use it as a lesson.
*There's been research that shows that when African Americans (and maybe other minorities who are negatively stereotyped) are asked about their ethnicity before taking a standardized test, they perform measurably worse than when they are asked after completing the test.