Stefanie Demetriades (Assistant Professor, College of Communication) presented “I Think, Therefore AI: An experimental assignment writing about AI, with AI” at the AI in Teaching Symposium, October 18, 2024.

Demetriades discusses an experimental assignment she designed for her “Communication Technology and Society” course. The assignment challenges students to critically engage with AI by using ChatGPT to write an op-ed about the role of AI in higher education.

Goals and Structure

The assignment aims to help students develop critical AI literacy by going beyond theoretical discussions and actively engaging with AI tools. It also encourages students to take ownership of their work, emphasizing that they are responsible for the final product even when using AI. The assignment consists of two parts:

1. A process and reflection document: This guides students through preparing prompts for ChatGPT, reflecting on the outputs, and iteratively revising the text with AI assistance.
2. A clean copy of the final op-ed: This represents the finished product of the assignment.

Both parts are equally weighted in the final grade, highlighting the importance of both the process and the outcome. To support students, Demetriades provided materials such as assignment instructions, op-ed guides, a step-by-step process guide, and a grading rubric.

The Process: Engaging with AI

The step-by-step process encouraged students to:

• Start with a clear position statement on AI in higher education.
• Prepare a prompt based on this statement.
• Evaluate the initial output from ChatGPT using the provided rubric.
• Engage in iterative revisions, choosing how and when to use AI.
• Critically reflect on how AI shaped the writing process and the final product.

Demetriades emphasized that students could write the entire op-ed themselves if they found the AI output unsatisfactory, but they had to justify their decision. She stressed that students were ultimately accountable for the final product.

Evaluation and Student Outcomes

The final op-eds were graded based on criteria such as originality, argument strength, language and style, persuasiveness, and evidence integration. Despite initial concerns about AI potentially levelling out the quality of assignments, the grades demonstrated a typical distribution, with most falling in the B to A- range. Interestingly, while AI excelled in areas like structure and flow, it struggled with originality, insight, and personal style.
Student reflections revealed valuable insights:

• AI’s strengths: Students recognized AI’s usefulness for brainstorming, structure, and organization, providing a solid foundation for further development.
• AI’s limitations: Students observed a lack of depth, personalization, and creativity in the AI outputs, often finding them repetitive and formulaic. They also noted AI’s difficulties with integrating evidence and citations accurately, even fabricating sources and quotes in some instances.
• The importance of ownership and balance: Students realized the need for careful prompting and thoughtful engagement with AI, recognizing that its effectiveness depended on their own knowledge and input. They also highlighted the importance of critical thinking and not relying solely on AI as a “crutch”.
• Evolving perspectives on AI: Initial student views on AI were often polarized, but the assignment prompted more nuanced and critical perspectives, leading to a better understanding of AI’s potential benefits and drawbacks in educational contexts.

Pedagogical Takeaways and Future Directions

Demetriades outlines several key insights for both students and educators:

• Student agency: Students possess unique writing capabilities, depth, and humanity that AI cannot replicate. AI should be used as a tool to enhance, not replace, their skills.
• Responsible AI integration: Educators should encourage ethical and responsible AI use, guiding students to become independent thinkers and writers.
• The value of process-based learning: Engaging students in the process of creating, reflecting, and revising allows for deeper learning and skill development.
• Student accountability: Students are responsible for their work and the educational value they derive from it, regardless of the tools used.
• Embracing experimentation and collaboration: Engaging with new technologies like AI requires an experimental and collaborative approach, acknowledging that both educators and students are learning together.

The presentation concludes with a student quote that highlights the importance of engaging with AI now to prepare for its future impact on society. Demetriades encourages continued experimentation and discussion on effectively integrating AI into educational practices.

“This is the perfect time to test and see what the best way to implement AI could be. That way when it becomes a much larger discussion or is implemented into everyday life, people won’t be blindsided.

People would be more prepared if we work with it first.

People in college and university are the people who will lead the next generation, so it is important they are all capable of dealing with these issues.”