Teaching AI Proficiency

Discussion prompts

1. 1Have students interact with a chatbot for five minutes, then describe what they think is happening 'inside' it. Compare descriptions across the class.
2. 2Ask: what is the difference between a tool that behaves as if it understands, and a tool that actually understands? Does that difference matter for how we use it?
3. 3Discuss: why might designers want AI to seem warm and relatable? What are the benefits and risks of this design choice?

Activity ideas

1. 1The ELIZA test: show the original ELIZA transcripts from Weizenbaum's 1966 paper. Ask students to identify exactly what they thought ELIZA was understanding and why.
2. 2Classification exercise: give students 10 AI behaviors and ask them to label each as 'simulation of cognition' vs. 'actual cognition'. Discuss where they disagree.
3. 3Have students try to find the 'seams' — moments where an AI's behavior breaks the illusion of understanding. What triggers these?

Discussion prompts

1. 1Present the Mata v. Avianca case. Ask: who should have been held responsible, and who actually was? Do students agree with how the court assigned liability?
2. 2When a physician follows an AI recommendation that harms a patient, does the presence of AI reduce the physician's moral responsibility? Why or why not?
3. 3Ask: is 'the algorithm recommended it' ever a legitimate defense? When, and when not?

Activity ideas

1. 1Responsibility mapping exercise: use the Responsibility Mapper tool as a class. Project each scenario and have students vote on allocations before comparing to real-case outcomes.
2. 2Scenario writing: have students write a brief scenario where AI involvement makes responsibility ambiguous, then swap with another group to analyze.
3. 3Debate format: 'Resolved: AI developers should be legally liable when their tools cause harm through professional misuse.' Evidence required on both sides.

Discussion prompts

1. 1Ask students to reflect: in the past month, how many times have they changed an answer after seeing AI output? Was the change always justified?
2. 2Introduce the Bainbridge 1983 'ironies of automation' concept: as tasks become more automated, humans get worse at performing them manually. How does this apply to AI use today?
3. 3Discuss: what would 'calibrated reliance' look like in your field? What would you still do independently, and when would you defer?

Activity ideas

1. 1Run the Calibration Test tool as a class. After results, group students by their reliance profile and have each group discuss what patterns they recognize.
2. 2Pre/post AI exercise: have students write answers to 10 factual questions first, then compare to AI answers, then decide. Track how often they switch and whether the switch was warranted.
3. 3Skill audit: have students identify three tasks in their field where AI assistance is most useful, and three tasks where manual competence is still essential. Discuss the criteria they used.

Discussion prompts

1. 1Ask students to state a position they hold confidently, then ask an AI to argue against it. How hard did AI push back? Did it capitulate when they pushed back?
2. 2Why would AI be trained to agree with users? Discuss the incentive structure of human feedback-based training and what it optimizes for.
3. 3Ask: if an AI always makes you feel confident and validated, what might you be losing? What's the value of disagreement in an intellectual process?

Activity ideas

1. 1Run the Sycophancy Detector as a class before discussing sycophancy. Reveal the categories after students have made their selections.
2. 2Pushback experiment: have students take a clearly wrong position on a factual question and see how long it takes AI to agree. Document the conversation.
3. 3Prompt engineering challenge: design a system prompt that would make AI less sycophantic for a specific professional context. What constraints are hardest to enforce?

Discussion prompts

1. 1Show two versions of the same false claim — one vague, one with specific statistics and a citation. Ask students which seems more credible. Why does format affect perception of truth?
2. 2What does it mean for a source to be credible? How is AI output different from human expert writing in terms of what credentials it carries?
3. 3Ask: when should we trust fluent, specific AI output more than vague AI output? When should we trust it less?

Activity ideas

1. 1Run the Credibility Rater as a class. Have students predict which passage type they'll rate highest before starting. Discuss surprises.
2. 2Citation verification lab: have students generate AI responses to a research question, then verify every specific claim and citation. Tally accuracy rates.
3. 3Fluency manipulation exercise: take an accurate statement and a false statement and have students rewrite them in styles that would make each seem more or less credible. Discuss the craft of persuasive-sounding language.

Discussion prompts

1. 1Ask students to describe their typical AI use in a single workday — how many tools, how many simultaneous threads, how much context switching.
2. 2Introduce cognitive load theory (Sweller 2011). Ask: what is the 'intrinsic load' of the actual work you're doing, and what is the 'extraneous load' added by managing AI tools?
3. 3Discuss: what would it mean to use AI in a way that reduces total cognitive load, rather than just shifting it? Is this possible?

Activity ideas

1. 1Have students run the Workflow Audit individually, then share results anonymously. Map the distribution of profiles in the class.
2. 2Attention residue experiment: have students intentionally close all AI threads before switching tasks for one week. What do they notice about their focus and output quality?
3. 3Tool audit: list every AI tool you use in a week. For each, ask: what problem is it solving? Is the orchestration overhead worth the output? Would anything be faster without AI?