AI: Testing assignments
Joseph Reagle
2023-09-27
Our question(s)
- How good is AI at my assignments?
- What to do?
- Improve prompts and questions.
- “Lock down” with technology or policy.
- Replace or abandon the assignment.
- Embrace/integrate within assignments (next session).
How good is AI?
- 90 SAT
- 99 GRE verbal
- 80 GRE quantitative
- 54 GRE writing
- 88 LSAT (OpenAI 2023)
What to do?
Improve prompts/questions
- Be specific with respect to course readings and discussions.
- “According to our discussion of the Boyd reading …”
- Engage higher level
cognition
- Hypothetical: “Suppose … had been the case, would the outcome have been the same?”
- Summary: “What themes or lessons have emerged from our readings on …?”
- Pre-test and exploit AI mistakes and hallucinations.
- “In visidata, how do you sort a selected rows only?” – is currently wrong.
“Lock down” the assignment with technology or policy
- Technology
- AI detectors
- warning: many false positives
- Respondus LockDown Browser
- warning: clumsy, buggy, and it’s easy to have an alternative device on hand
- AI detectors
- Policy
Replace or abandon the assignment
- Move toward oral, interactive, or performative modalities.
- Develop outcomes over multiple stages/iterations.
- Require authentic reflection on the work performed.
- Integrate AI within assignments (next workshop).
- e.g., “Write a 300 word prompt that creates a tutor for a course concept: the user should define and differentiate the term, receive critique by the AI, amend their work, and respond to a multiple-choice question for self-quizzing. Write a 300 word reflection on your process and if you think the result worked well.”
Reading
- Kalley Huang Huang, 2023, Alarmed by A.I. Chatbots, universities start revamping how they teach
- Ethan Mollick, 2023, The future of education in a world of AI
- NC State University, 2023, Designing assignments and activities with ChatGPT and generative AI in mind
Questions?
If not, let’s start testing at perplexity.ai!