As a teacher and researcher of feedback, I’ve long been fascinated by the popular discourse about feedback to children. The alleged praise-happy softness of American parents has been a target of scorn by George Will, the bow-tied conservative columnist, and held up as a competitive weakness relative to the super-achieving progeny of Chinese “Tiger Moms” [ Will2010ses ; Chua2011wcm ]. Interestingly, research indicates that there are both “perils and promises of praise” [ Dweck2007ppp ]. That is, the type of praise one gives is critically important: does it reinforce a belief that intelligence is a fixed trait and not susceptible to improvement, or that improvements can be had with further effort.

This insight is based on work from psychologist Carol Dweck and colleagues, and more recently popularized in the 2009 best-seller NurtureShock [BronsonMerryman2009nsn]. In one experiment by Mueller and Dweck, of those fifth-graders who were praised for their intelligence, 69% reported they preferred tasks that “weren’t too hard” and “pretty easy” such they continue to show “they were smart”, compared to 12% of children praised for their effort. Conversely, 80% of the children praised for their hard work preferred learning goals: “problems that I’ll learn a lot from, even if I won’t look so smart”. Through a series of subsequent experiments that varied the difficulty of tasks and opportunities afforded to students, they found that those students praised for intelligence cared more about performing well than learning: they chose easier tasks, displayed less persistence, resilience, and enjoyment, and actually did (significantly) worse on subsequent tasks than the children praised for effort. Children that were praised for their intelligence also preferred learning about the performance of other children rather than learning new strategies, and were more likely to lie to (distant and anonymous) children about their own performance [MuellerDweck1998pic].

While my own time as a fifth grader preceded this research, I do recall report cards with both performance and effort grades. I remember being disdainful of the system when I got an “A” for performance but a “C” for effort – “stupid teacher” I thought. However, (hypocritically) I did cling to the “A” in effort when I had to explain the “C” in performance to my parents. Yet, by the time students come to my classroom, this bifurcated grading system is a distant memory. And, while I do recognize and encourage effort in feedback (formative assessment), I feel the final grade ought to be based on performance (summative assessment). I stress that a grade is not a reflection of students’ personality, efforts, or life circumstances. Yet, as one colleague described it, students are increasingly “brittle.” Is this because my students have always succeeded previously, and hence are operating under an innate-intelligence model? Or, could it be that they have been assessed on effort, and hence a less-than-exemplary assessment is viewed as a critique of their efforts? Hence, at what age – between primary school and professional work – do we draw the line between, or at least balance, the praising of effort, so as to further progress, and assessing performance?

this entry posted to career/teaching ;
comments (0)

My year away from teaching while at the Berkman Center meant I approached this year with a lot of enthusiasm. In the fall, I continued with what had worked well for me in the past. I taught sections of an upper-level elective, New Media Culture , for which I chose many excellent texts. I asked the students to write reading responses and I facilitated each class as a discussion (inspired by Brookfield and Preskill’s 2005 Discussion As a Way of Teaching ), sometimes around particular themes, and often simply as a “go around” in which people spoke about their response. There were no quizzes or exams, just two significant essay assignments. Based on student evaluations, they were quite pleased with the course.

This spring’s section of Media Culture & Society was more challenging. This is an introductory level class required of students in the media concentration. I find the required text to be lacking for many reasons, including that the students find it to be stale and unengaging – and I agree. Much of the grade is based on quizzes and exams, which I don’t have a lot of experience with. However, I approached this class with the intention of having lots of great class exercises available (inspired by Barkley’s 2009 Student Engagement Techniques ) and a desire to take greater advantage of technology.

• I greatly expanded my materials on learning and referenced them often, but I’m not sure if the students bothered to consult them.
• I tried to link the quizzes to specific reading and learning practices, but students found the quizzes to be too hard. If I keep the quizzes, which I may not, I will have to rethink them again.
• I tried to increase student choice and collaborative opportunities. However, no student ever proposed quiz questions or continued our collaborative outlining (using Google Docs) beyond what I prompted immediately in class.
• I do think our collaborative exam review, in-class exam design, worked well and I will repeat this. Still, I’m puzzled by students who got questions wrong on the exam that were proposed by student groups, discussed, and answered in class and the resulting document.
• I did lament our use of the textbook but tried to cast it as a lemon from which we are challenged to make lemonade. I think this backfired as students then associate the textbook with me and the course. In the future I will be more specific in the reading prompts, and more selective in the pages assigned with more supplementary material provided. I’m also considering handing out my notes on the book.
• I made greater use of class slides; DZSlides is HTML5 which means I can easily embed images, webpages, and YouTube videos. I include lots of prompts and questions and exercises as part of the slides. However, there is always that push by some students to simply be told what they need to know for an exam, and that slides and notes be provided before class. I’m concerned this mitigates higher-level learning and discussion and furthers lazy or rote learning, but I had classes as a student where the instructor provided notes, which I appreciated.

I’m looking forward to having another go at this class in the fall. I do plan to simplify where I gave a lot of choice, and be more clear about my expectations and recommendations; otherwise, there is still much to continue thinking about and experimenting with.

this entry posted to career/teaching ;
comments (0)

This weekend I placed last semester’s New Media Culture course up at Github under a CC license; it includes the source files for the syllabus, my notes and slides. Like everything I’ve put up at Github, I’m not confident it will be of use to a lot of people, but I like to share, and I’m often pleasantly surprised. Coincidently, a recent link on /r/Python/ led me to a great talk by Greg Wilson on What We Actually Know About Software Development, and Why We Believe It’s True . While this talk is 2 years old, I also had a look at his blog. It just so happens that while I was sharing my class on github, Wilson was asking “ would it be possible to create a ‘GitHub for education’ ”? Why does need one even ask this question? The answer is because:

because today’s learning content formats make merging hard. PowerPoint remains the tool (and format) most commonly used for individual lessons, but there aren’t good open tools to merge PowerPoint files. (Wilson 2011)

Indeed, PowerPoint is a binary file and no fun at all. However, most every text that I author is markdown or HTML/XML, which is then converted to some other format as needed. I’ve long maintained such files in version controlled repositories. For example, Good Faith Collaboration was written in markdown (and existed in CVS, subversion, and git as it went from dissertation to book). My class’s slides are authored in markdown and then converted to DZSlides via pandoc .

Hence, I think I’ve addressed the format concern — in a rather geeky way. Someone can easily fork my class, make changes to my slides, diffs of which I can easily view and merge back in. However , there are still some issues that would provide some friction to sharing:

1. Each instructor’s take on a course like Media, Culture, and Society will be novel. I imagine my slides on a widely used text, like Jenkin’s Convergence Culture , might be of use to someone else, but I would not expect someone to adopt my course wholesale. (Perhaps this is less true for standard courses like Physics 101.)
2. Furthermore, courses on the same topic and even using the same readings can still differ significantly depending on how many courses there are in the tri-/se-mester, how long the classes last, and how many students are in each class.
3. My use of freemind for my syllabus is novel as far as I know. David Weinberger has been a big advocate of Syllabus XML wiki , and perhaps something will pan out there, but XML is not and easy format to fork and merge content in.
4. The instructor’s ability to share course materials can be a confusing issue . I believe I have the right to share these materials, and have/will make some efforts to confirm this belief further, but I’d like to see more universities develop explicit open access and sharing policies.

this entry posted to career/teaching ;
comments (1)

I'm at that point in the semester where I'm asking students to think about what they want to do for their more open-ended assignment. Rather than simply answering questions I've asked or applying course material to a particular case, I request that they propose a topic they would like to research. This is a source of difficulty and anxiety for some. Granted, there is an element of risk in the openness but I never had much difficulty with choosing a topic myself as a student, so it's difficult to understand how I can best help as a teacher. For example, for a film class I wrote a brief essay on Blade Runner that I really enjoyed working on and am quite fond of. I didn't get the grade I thought I deserved -- and I suspect the instructor didn't "get it", so I appreciate the risk -- but I had no problem conceiving the topic and executing the argument. (Fortunately, the essay would be widely read on the Web, for which I would get a lot of responses and it was even translated into Italian -- not too bad for an undergraduate essay!)

So while I always liked these type of assignments, some bright students can have difficulties. To address this I do the following:

1. Ask the students to send me a proposal with a sense of the topic, argument, concepts and readings that will be used. (I started this in my second semester of teaching and it yielded better results.)
2. Provide example topics and/or even an example proposal.
3. Encourage students to review their reading responses or bring relevant news items to the attention of the class throughout the semester, so as to build a repository of ideas.
4. Provide a list of themes/concepts at the beginning of the course and highlight them throughout.
5. Encourage them to brainstorm a number of (provocative) arguments they could make as they research and outline their topic.

But, still, some students experience difficulty with choosing a topic. Are there any resources you would recommend in guiding students through the writing of open-ended assignments?

this entry posted to career/teaching ;
comments (0)

I've not made much use of in-class exams before, but I gave the first one of this semester and it prompted the question in my mind of why do we give exams at all? Or, if we do give exams, why not simply give all the answers beforehand?

By reflex, we might respond that exams test learning. But do they? And if so, is the testing the sort of learning important for the students' future success -- rather than the ability to do well on exams?

A cynical response is that exams simply test the students' intellectual endowment, be it socio-economic (having gone to a good high school) or cognitive (of a certain type of intelligence). And we can't give everyone an A, and so we need some sort of filtering mechanism. I once heard this argument about the business school of an elite institution: it could be little more than a sieve selecting from quality students who then use the opportunity to network. What is actually taught in the two or three years there is inconsequential.

A reason I sense in my own motivation is to satisfy myself that the students are paying attention. That is, my choice of questions for the exam is a sample of all possible things the students ought to have learned. I had this experience as an undergraduate myself, especially in humanities-type courses. I went to class, paid attention, and I would know what was likely to be on the exam. Yet, to give exams so that students pay attention only so they do well on the exam is circular. When I was a student, I had a particularly poor calculus teacher to whom it made no sense to pay attention to. Instead, I took class time to study, do extra exercises, and visit the TA. I also had a wonderful statistics teacher who provided us with handouts to help us make sense of the material, upon which we were tested. I even had a computer science teacher who gave us a practice exam, and it turned out that it was also the in-class exam. (And I still know people have failed it!)

A more interesting reason is that the exam prompts students to study and prepare. So the exam itself isn't as important as the learning that one does in preparation. An even more interesting extension of this is that we use exams to teach students how to be good learners. That is, exams prompt students how to identify what's important, how to think critically, and to adopt tools for note-taking, mnemonics, etc. But how many teachers actually teach this?

There are many options, across varied disciplines, for giving exams: pop, planned, preceded by study questions, open notes, open book, or the take-home format. Does anyone know of a nice articulation of the types of exams and what types of learning they are intended to further?

this entry posted to career/teaching ;
comments (2)

It's been a relatively quiet summer but there's plenty to blog about as the new semester begins.

An exciting bit of news is that I'm joining the faculty of Communications Studies at Northeastern. I'm also happy to remain involved with the Berkman Center as a faculty associate. (Now, I'll be riding my bike across the Charles.)

In the upcoming semester I'll be teaching two sections of a course called New Media Culture . I'm really looking forward to the class; we'll be using Jenkin's Convergence Culture and Gershon's Breakup 2.0 and also touching on Lulz, Anonymous, celebrity, and collaborative cultures online.

I will be posting more news about Wikimania 2011, Wikipedia and gender, and the book's Web edition in the upcoming month.

this entry posted to career/teaching ;
comments (0)

I've been reading a slew of interesting -- and often depressing -- links from academia from the end of 2010. Flavia's entry on students sharing more about their "lives than I have any desire to know" made me think of my own struggles with this issue. I generally want to respect the privacy and autonomy of students. For example, students have a specific number of absences that are "freebies" so they need not explain or justify any absence to me. However, if they do need to speak to me, I tell them to speak to me sooner, rather than later, about a proposed solution, rather than gory details of the problem.

While I am quite sympathetic to well-timed and reasonable proposals, otherwise, and especially when it comes to grading, I typically respond along these lines:

I would never presume that "school should come first" relative to the other events in people's lives, but nonetheless my obligation is to assess what happens in the classroom. Honestly, it is impossible to play Solomon and judge the often difficult circumstances of people's lives, or to try to figure out how things might have been absent those challenges. So, I pretty much stick to the grading criteria in the syllabus.

this entry posted to career/teaching ;
comments (0)

With the end of the semester comes the opportunity to review what students think of my teaching. As a (relatively) new teacher I take the reviews seriously. However, with four years of practice and data I do struggle with how to interpret the reviews and use them as constructive feedback that yields measurable improvements in subsequent evaluations.

This is not to say that I have not attempted to improve my teaching. For example, I believe I'm much more consistent in reviewing the concepts encountered in a session at its end -- and the students seem to appreciate this. However, when I plot the trend for the overall ratings from those four years I hoped there'd be a strong uptick with time, but there's no consistent trend.

Also, confoundingly, there is the disparity of opinion. For some things people naturally have different preferences: more or less outside readings, lecturing, student discussion, etc. (Students do seem to universally love watching video clips.) However on other things, like the completeness of the syllabus or the clarity of the grading system, I am confused.

As a follow-up to my experiment to quiz the students on the content of the syllabus at the beginning of the semester, students did surprisingly poorly. So I know some of them are not paying attention to the syllabus while rating the syllabus as less than complete. I feel similarly about the grading system as it is clearly described in the syllabus and I do four detailed grade reports throughout the semester, each time saying I would be happy to discuss their performance so far, but some students still occasionally fill-in the bubble suggesting that their grade is an opaque mystery. Also, while one evaluation rated me as a poor instructor overall, the vast majority rate me as very good or better. In fact two wrote in the comments that this was the best class they've taken at Steinhardt and NYU respectively. How to reconcile these disparate evaluations?

I conclude that there will be a student or two that, for whatever reason, doesn't like me or the class. (Most characterize me as nice, friendly, and sometimes even funny, but one noted I was intimidating...?) Yet, since my school finally provided departmental wide statistics last semester, I know I am right in the middle of the distribution. I rate higher than half of the faculty, but also lower than the other half. So I know there is room for improvement. However, given it only takes one disenchanted student to skew the averages and that I've not yet been able to implement changes that clearly manifest in the evaluations, I'm not sure how, or if I should be overly concerned? But I am very curious as to how the instructors in the top quartile manage it.

this entry posted to career/teaching ;
comments (2)

One of my favorite blogs is MIT's Tomorrow's Professor , and I particularly appreciate the essay on Explaining the Reasons for Criticisms of Students’ Academic Performance . Barbara Walvoord spends some times discussing both student and (end of year) teacher assessment. It seems to me that the question of how to make feedback connect and count with the student is central to the exercise of teaching. In the past year, I have experimented with breaking my final assessment feedback on an assignment down into four categories: engagement, understanding, writing, and scholarly support.

I think this works well, though it can be difficult to ascertain in the course evaluations. While the majority of students assess the completeness of the syllabus and my feedback as relatively high (one student likened the point-based system and frequently e-mailed assessment reports as being "like a science") I still do have the infrequent evaluation where this is not the case. It is a puzzle to me how student could say my syllabus was not complete, or why after I give feedback on one assignment they repeat the same mistakes in the next assignment. So this semester I'm going to try two more experiments: a quiz and self-evaluation. I plan to give a quiz on the actual syllabus (e.g., how many freebie absences are students allowed, do medical notes count against those freebies?) and as part of the first assignment ask students to evaluate what they are submitting in term of the four categories. I'm very interested to see the results.

this entry posted to career/teaching ;
comments (0)

My sense in teaching over the past four years is that I have been assessing higher grades. (I shy from the term "giving grades" as it sounds like a gift based on character or my fondness for the student.) Beyond an anecdotal report on what the department median grade is (for which I appear to be one half letter grade above), I have no other information for the grading distributions in other classes in my department or at NYU, including other sections of the classes I teach. So, my philosophy is to tell students that if everyone performed excellently, that would be accordingly reflected. I then remind students frequently of how I evaluate their work , based on the departmental criteria, and at the beginning of the course provide exemplars of what I consider to be excellent work.

If there is an improvement over the initial semester, this doesn't surprise me in that I feel like my classes are now more honed, with exemplars students have a better sense of my expectations, and I've debugged assignment specifications. I also feel that while the material and assignments in the Media, Technology, and Society (MTS) class are more difficult than Conflict Management (CM), the students are more consistent. So, I performed a five number summary and generated the following box plots (with outliers below 70 truncated):

My conclusion is that while I assessed lower grades in my first semester of teaching each course, there is otherwise no consistent trend. Also, my sense of the MTS students being more consistent in performance is confirmed.

this entry posted to career/teaching ;
comments (0)