Hi, I am obviously stirring the pot as I want to level the way we perform our work. I have been a JA for 3 years now. I have judged for an additional 3 or 4 years. These kids are impressive, but the way we as JA’s are going about things is vastly varied. This hurts the kids centered philosophy and makes for bad experiences for the kids, judges, and coaches, long and short everyone. Last year I pushed my judges to work the interviews for details. I must say few of the top note booking teams fared well under this interview style. Teams that had really amazing codes largely failed in more than a basic level of understanding while teams that struggled with code really understood where they were on the code and just ran out of time to finish the fixes. It made for some interesting debates. This year with the changes to how we proceed to press for details as we need more details for scoring and the debate, what things are you considering and why? How are you planning on moving beyond the intro question or 2. I am thinking of pressing for better more defined understanding of how the code works and why (short answers), press for 2 design features one team decides and one judge just picks. Really need to move them away for the rehearsed and practiced talking points to see if they really understand their bots. I am hoping for a minimum of 10 minute interviews with follow-ups for the top teams. What are you all thinking?
Thanks for your question. Please keep in mind that I am coming at this from the curriculum side rather than the judging side, so take this as one more voice in the room.
The pattern you saw last year is the whole argument. When the interview asks for detail, it separates teams that own the work from teams that can present it. We see the same thing in classrooms, which is why we do so much around conversation-based grading.
The judge-picked design feature is a good idea. The team-picked feature tells you what they rehearsed. The judge-picked one tells you whether they understand the robot. The same applies to code: point to a specific line or value and ask what would happen if it changed. That is a great way to elicit evidence of understanding.
I would do the same with the notebook. Instead of asking about it, ask from it. Open to a page and say, “What happened here, and what did you do next?” A team that lived the process can talk about any page. A team that wrote it up afterward can talk about three.
“Pressing” can slide into an interrogation, and a nervous student can shut down for reasons that have nothing to do with understanding. The best question I know is, “Tell me about something that didn’t work.” It is disarming, almost impossible to rehearse, and every team that actually built the robot has an answer. Here is a great article that discusses this.
I hope this helps.
I must say I agree with avoiding more of the rehearsed speeches. My teams of judges and myself have noticed some teams struggle when asked something that is not rehearsed or worse yet only 2 to 3 members of a larger team can answer. So asking but not pressing for details is a better choice. In tournaments I worked as a judge or a JA, asking for things that were not rehearsed impacted the deliberations significantly. I think tha after the event if asked by teams, coaches or parents, pointing out that all aspects of the robot and code are fair game and students must be ready for that is a fair reminder. You can’t tell them how to answer, but reminding them the whole of the data presented, robot, notebook, code and now summaries are fair game. The summeries are not graded and will take you to likely the best prepared areas. So as an adult and parent, I understand this and will look to at least one other area to get a score. Thanks for responding. The more opinions shared the better and more lelvel the playing field will be for all.
Agreed, David.
What we have always done here as judges is to first collaborate in the judges room with the advice of the JA to determine some key questions that are good openers for the individual team interviews. Generally those open-ended questions require the team to tell how they overcame a challenge, or what a particular line of code does, or what does this thing do (as we point to something out of the ordinary on their robot).
This then leads to further questions as they respond to the openers. Or, if a team is more shy, they need more friendly leads into each question as they do not want to speak, or they have a language barrier (we do not always have a translator in our judging partners, but the teams understand the interviews are in English).
Cheers! 
This was a great article Jason. Thank you!
Through this we can see how to elicit a response from a student in a positive way.
"Ask the student to explain what is wrong.
- How is the robot moving in their project?
- How should the robot be moving?"
This example can be fitted to our judges simply for the physical robot, the code, or programming questions.
The basics of teaching are pretty universal. 
Thanks, Paul. You named it. The basics are universal. A good judging interview and a good classroom conversation make the same move: ask something a student can only answer if they did the work, then listen.
Your point about shy students and language barriers matters too. We’re looking for evidence of understanding, and a student who freezes gives us no evidence either way. Friendly openers aren’t going easy on a team. They’re how you get to the real answer.
As the JA at my event maybe events, I will be telling the judges one maybe 2 easy warm up questions than go for aspects of code, notebook and design not mentioned. Interviews will be 10 minutes. So the teams will not be allowed much of the rehearsed areas. Last year, I used this approach and it really did weed out teams, even top performing teams. The judges also really enjoyed the experience more. I will have a few returning judges, which will be helpful.
I do LOVE seeing more people posting.
I did read the article. Since I am not an educator, but a trainer and long in the tooth chemist. I can say I use this concept with those I train. Knowledge is power. So knowing how to convey, confirm message is recieved as intended and having the trainee later “train you” I have found is working the best. So the reflection, editing and all play into more than just the competition, but helps to build the foundation for their futures. Keep the educator ttips coming. Thanks!
