42 undergraduates in an introductory psychology course at Iowa State were randomly assigned to two groups. Each was told their memory would be tested later. They watched a 65-second video of a lecturer explaining why calico cats are usually female. . . . One group watched the fluent video, in which the speaker stood in front of a desk, spoke without notes, maintained eye contact with the camera, and gestured with her hands for emphasis. The other students watched the disfluent one. The same lecturer stood behind a desk and delivered identical content, but read haltingly from notes as she hunched over a podium, pausing occasionally to glance at the camera.

After watching the videos, the students in both groups were asked how well they thought they had learned the material, how much they predicted they would remember 10 minutes later, and how organized, effective, and knowledgeable the speaker was.  The students who had watched the fluent lecture were about twice as likely as those who had watched the disfluent one to predict that they would remember what they had heard and to say they had learned the material. The fluent speaker was also rated as significantly more organized, knowledgeable, and effective than the disfluent speaker.

All pretty much what we’d expect, right? The more clearly delivered lecture should result in better learning.  But it didn’t happen.  Students in both groups did equally well (actually, equally badly):  they got about 25% of the material correct.  So while the fluent lecture gave a stronger perception of learning, and that lecture received higher ratings, it did not lead to greater knowledge. Would you get the same result from a longer lecture, or a semester-long course? This experiment can’t answer that question. But it does point to a quickly-formed perception/learning gap.

The second article also examines the tie between student evaluations and student learning.  A study of more than 10,000 Air Force Academy cadets randomly assigned students to different sections of a course and used identical syllabi and identical final exams (graded by someone other than the professor). The students also all took mandatory follow-up classes, which again had identical final exams. Students of the less experienced professors did better on the exam in the initial course, and gave their professors higher evaluations.  However, the students of the more experienced professors did better on the exam in the follow-up courses.

The authors theorize that the more experienced professors broadened their coverage, while the less experienced ones “taught to the test.” The broader coverage may have led to deeper understanding of the material, which in turn helped the students do better in more advanced classes (while feeling less happy in the intro course).

Does this mean that student evaluations are worthless? Certainly not (and the Air Force experiment may demonstrate a flawed alignment between the coverage of the introductory and advanced courses).  But it is further evidence that when we use student evaluations we need to be quite clear about what questions to ask the students, and we need to understand that there is a gap between what people think they have learned and what they actually learn.  It’s also another reason not to base decisions about the quality of teaching for tenure and promotion on student evaluations alone.

Sharing student work on a course blog is an example of what Randall Bass and Heidi Elmendorf, of Georgetown University, call “social pedagogies.” They define these as “design approaches for teaching and learning that engage students with what we might call an ‘authentic audience’ (other than the teacher), where the representation of knowledge for an audience is absolutely central to the construction of knowledge in a course.”

Blogs are not the only medium through which students can produce meaningful work for a public audience — think of the possibilities created by Wikipedia or other online resources.  This spring, a number of groups organized “Wikipedia edit-a-thon” events, designed to increase participation in Wikipedia editing by diverse communities. (The average Wikipedia editor is a college-educated, 30-year-old male resident of the US or Western Europe).

Could an edit-a-thon work as a class project? To add meaningful content to an entry (and provide suitable documentation), students would have to use their research, information literacy, critical thinking, and writing skills.  Whether the goal of the edit-a-thon is to make corrections across Wikipedia entries (as when feminist groups reacted to the revelation that American women novelists were being moved from the category “American Novelists” to the sub-category “American Women Novelists“) or the goal is to create or enhance a single entry related to the course being taught, students would also have to practice valuable communication and teamwork skills.

Interested?  This Prof Hacker blog from the Chronicle has a host of sensible suggestions about how to make an edit-a-thon work.

]]> http://blog.smu.edu/cte/2013/05/31/wikipedia/feed/ 0 http://blog.smu.edu/cte/2013/05/31/present-like-a-pro/ http://blog.smu.edu/cte/2013/05/31/present-like-a-pro/#comments Fri, 31 May 2013 22:20:30 +0000 Elizabeth Thornburg http://blog.smu.edu/cte/?p=407 Continue reading →]]> Hitting the academic conference trail this summer?  Presenting at a conference is a way to shape a professional reputation — and requires some of the same skills as great teaching.  Yet how many times have we sat through deadly readings (tempted, like our students, to check email or play Free Cell), or longed for the chance to participate more actively?

If you’re going to be the moderator, a recent article in the blog of the Harvard Business Review has a list of helpful suggestions (and, by extension, these ideas also have implications for individual speakers).  All of them won’t work for every context, but most are good advice for avoiding “death by panel discussion.”  The full article explains more fully, but here’s a partial list:

1.  Don’t prepare with multiple conference calls agreeing on the content of every speaker’s talk. The resulting canned presentations can be ponderously dull.

2.  Think really hard before using PowerPoint slides — they can lengthen and straightjacket the presentation, especially if it’s not a visual topic.

3.  State your objective briefly and clearly at the outset.  What is this panel, or this presentation, really about?

4.  Panelist introductions should also be brief, and to the point (3 lines, max), especially if bios are in the program materials. (Organizers:  consider a mobile app like Guidebook that links schedule and presenter information — free for groups up to 200).

5.  Involve the audience within the first five minutes (but don’t lose control).

6.  Don’t go down the line of all panelists for every question.  How often does the fifth response to the same question add anything new or interesting?

7.  Invite panelists to ask each other questions.  If they have overlapping expertise, these questions may be more insightful or provocative than audience or moderator questions.

8.  Divide the time among big picture issues, specifics, and genuine opportunities for audience participation.

9. Don’t ask the panelists for “one final thought.”

10.  Stick to the allotted time.

Can we re-envision the panel as the catalyst of a collaborative conversation? Perhaps not in every discipline, but I’d love to see it tried.

One of his experimental findings has some implications for faculty members wishing to motivate their students.  Imagine a student assignment, and different degrees of feedback to the student.  In the experiment, subjects were given a piece of paper with letters on it and asked to circle all of the places where the same letter occurred twice in a row.  In one version, the tester receives the student answers, looks at them, makes a noncommittal sound, and places the student paper on a pile (acknowledged).  In the second version, the tester receives the paper, neither looks at it nor makes a sound, and puts it in the pile (ignored).  In the third version, the tester takes the student paper and places it directly in the shredder (shredded).

The results had both good and bad news for those who want to motivate others.  The bad news:  ignoring the performance of students is almost as de-motivating as destroying it in front of their eyes.  Turns out failure to acknowledge work makes the work far less meaningful, and decreases future motivation.  The good news: modest, informal acknowledgment dramatically increases students’ motivation.  The left-hand yellow bar in the graphic at the beginning of this blog post shows how many more times the subjects would repeat a task (each time for less money) when their work was acknowledged as opposed to when it was ignored (middle column) or shredded (right column).

At this time of year, we are focusing on final exams:  labor-intensive, formal assessment of our students’ work.  But this is a good reminder for the future that we can encourage greater student motivation to do the assigned work by building in occasions for more informal, low stakes feedback throughout our courses.

Much of the criticism of tenure seems to be that it provides job security to lazy professors—those who don’t bother to conduct research, those who don’t care about and don’t invest time in good teaching, and those who don’t do either. For example, a recent Harvard Business Review article explains that, “despite a technological revolution,” “[a]cademic teaching techniques remain calcified.” “Professors’ reluctance to use technology to revamp the way they teach is understandable,” the article continues, because “tenured professors, of course, don’t have to.” Another critic argues: “The best way to improve the quality of education . . . is to get professors to focus more on teaching. And to do that we need to ditch the tenure system and start evaluating professors on the basis of their teaching ability, without any guarantee that they will keep their jobs if they don’t continue to measure up over the years.” These seem to be the common choruses of tenure critics.

One of the most common responses to these critics is that tenure is necessary for academic freedom in scholarship. For example, one scholar has explained that “tenure allows a professor the ability to speak up about issues on which he or she is an expert” and also “allows a professor to purse [sic] research regardless of its political or controversial nature.” Another scholar has stated that, “[w]ithout the freedom granted by tenure, the conditions necessary for creativity will be threatened,” and “data and experience suggest that the overwhelming majority of faculty members at our best universities and colleges are highly self-motivated individuals who strive to produce new, important discoveries, and to write books that will redefine their fields well after they receive tenure.”

Academic freedom, though, is also salient to teaching. As the American Association of University Professors has stated, “[a]cademic freedom is essential to” the common good, which “depends upon the free search for truth and its free exposition.” “Academic freedom . . . applies to both teaching and research. Freedom in research is fundamental to the advancement of truth. Academic freedom in its teaching aspect is fundamental for the protection of the rights of the teacher in teaching and of the student to freedom in learning.” Although defenders of tenure often focus on tenure’s importance to scholarship, tenure remains important to teaching as well. There are nontraditional approaches to teaching that could potentially benefit students but that might be unpopular with students (and thus lead to lower student evaluation scores) or that might be questioned by colleagues more comfortable with traditional teaching techniques. Take “flipping the classroom,” for example. Many teaching experts laud the advantages of this teaching method, but some anecdotal evidence suggests that students sometimes don’t appreciate this approach with which they are unfamiliar. I’ve been advised not to experiment too much with various teaching techniques until I have job security, i.e., tenure. My guess is that if tenure protections are abolished, some professors may feel less free to experiment with teaching techniques that could very well benefit students. And without the freedom to experiment and innovate, student learning and student experiences could suffer. Thus, in defending the importance of tenure, supporters of tenure ought to consider how tenure advances teaching and student learning in addition to scholarship. Policy decisionmakers, too, should consider the role that tenure plays in faculty approaches to teaching.

Amherst’s rejection of edX, decided by a faculty vote, could mark a new chapter for MOOCs—one in which colleges revert to their default modes of deliberations and caution. “I think we’re at the early stages of that honeymoon period coming to an end,” says Richard Garrett, vice president and principal analyst of the consulting company Eduventures.

Then on April 30th, the Chronicle of Higher Education reported that Duke University has recently announced that “there will be no credit-bearing online courses at Duke in the near future”:

The university’s Arts & Sciences Council, the governing arm of the undergraduate faculty, voted down a proposal to join a consortium of top colleges offering for-credit online courses through 2U, a company that specializes in real-time, small-format online education.

Perhaps in tension with this decision, Duke still offers MOOCs through Coursera.

Could it be that MOOCs and other online courses are just a passing fad? Or will Amherst College and Duke turn out to be either outliers or just slow adopters? MOOCs can offer the potential benefit of providing education even to those who cannot pay high tuition rates, but can participating universities afford this investment, and can universities that do not participate survive? Should credit be offered for students participating in MOOCs? If not, how should potential employers view MOOC enrollment? These are just some of the questions still swirling, and it’s probably still too early to determine the full panoply of advantages and disadvantages offered by MOOCs and other online courses.

A democracy relies on an electorate of critical thinkers. Yet formal education, which is driven by test taking, is increasingly failing to require students to ask the kind of questions that lead to informed decisions. . . .

Informal learning environments tolerate failure better than schools. Perhaps many teachers have too little time to allow students to form and pursue their own questions and too much ground to cover in the curriculum and for standardized tests. But people must acquire this skill somewhere. Our society depends on them being able to make critical decisions, about their own medical treatment, say, or what we must do about global energy needs and demands. For that, we have a robust informal learning system that eschews grades, takes all comers, and is available even on holidays 

Recent years have seen many calls for enhancing, overhauling, or abolishing the traditional premed requirements. Critics argue that the pace of scientific discovery and its clinical application have outstripped the requirements; that information technology has made memorizing vast amounts of content unnecessary; that the requirements lack clinical, scientific, and social relevance; that they’re used to cull the herd of talented aspiring physicians; that they disadvantage minority and female students; that they crowd out studies of bioethics, social justice, and health policy; and that rigidly structured premedical and medical school curricula hinder students from becoming self-directed lifelong learners. Furthermore, the current model has perpetuated “premed syndrome,” a culture of aggressive competition for grades that conflicts with the precepts of medical professionalism: academic and intellectual rigor, creative thinking, collaboration, and social conscience.

Muller doesn’t say that entering medical students shouldn’t take science courses. For the past 25 years his medical school (Mount Sinai’s Icahn School of Medicine) has run a Humanities and Medicine “early assurance” admissions program for humanities majors. The students must take some “clinically relevant” science courses the summer before their senior year and some basic science the summer before entering medical school, but that’s about it. Oh, and these students don’t have to take the MCAT, either. By most metrics, the humanities do as well as the science majors, and the program is being expanded next year so that half of the entering class can have any major under the sun. There are various other requirements, and the school will provide enhanced guidance and advising functions for these nontraditional students. The payoff, they hope, will be to

dramatically expand the educational, cultural, and socioeconomic diversity of entering classes and our health care workforce. By eliminating MCAT use, outdated requirements, and “premed syndrome,” we aim to select students on the basis of a more holistic review of their accomplishments, seeking those who risk taking academically challenging courses; are more self-directed than traditional medical students; pursue more scientifically, clinically, and socially relevant courses; and pursue independent scholarship.

Who knows how quickly these kinds of changes will be adopted by other medical schools and most especially by the Association of American Medical Colleges? It will be interesting to see if we are at the beginning of a revolution in medical education that will rival that of Abraham Flexner’s revolution a century ago. If we are, SMU will need to radically rethink and revise the way it prepares its pre-meds for medical school and ultimately medical practice.

Imagine taking a college exam, and, instead of handing in a blue book and getting a grade from a professor a few weeks later, clicking the “send” button when you are done and receiving a grade back instantly, your essay scored by a software program.

And then, instead of being done with that exam, imagine that the system would immediately let you rewrite the test to try to improve your grade.

EdX, the nonprofit enterprise founded by Harvard and the Massachusetts Institute of Technology to offer courses on the Internet, has just introduced such a system and will make its automated software available free on the Web to any institution that wants to use it. The software uses artificial intelligence to grade student essays and short written answers, freeing professors for other tasks.

There is, of course, a backlash:

a group of educators who last month began circulating a petition opposing automated assessment software. The group, which calls itself Professionals Against Machine Scoring of Student Essays in High-Stakes Assessment [actually, it appears they call themselves Human Readers], has collected nearly 2,000 signatures, including some from luminaries like Noam Chomsky.

“Let’s face the realities of automatic essay scoring,” the group’s statement reads in part. “Computers cannot ‘read.’ They cannot measure the essentials of effective written communication: accuracy, reasoning, adequacy of evidence, good sense, ethical stance, convincing argument, meaningful organization, clarity, and veracity, among others.”

And, lest you think the opposition group consists of a bunch of cranky old Luddites, it includes — in addition to Prof. Chomsky — Les Perlman, “a retired director of writing and a current researcher at M.I.T.”

As much as I would like to think there is a software-based alternative to slogging through hundreds of essay answers at the end of each semester, doesn’t it seem just a little incredible?

One of PAMSSESHA’s objections is the lack of statistical evidence based upon a comparison of computer grading versus that of human graders. Until there are studies that validate this technology (preferably from independent third-parties), this strikes me as a leap too far. Indeed, although Wikipedia (I know, I know) states there are plenty of studies that show automated grading programs outperform human graders, there’s an impressive pile of research attached to the PAMSSESHA petition that negates that claim.

More, I am sure, later . . . .

Maybe not. I first learned about the dysfunctional-family dynamic fostered by the tenure system in some academic departments when I was a grad student selecting a committee. I couldn’t ask Dr. X to serve with Dr. Y because they didn’t speak to one another. Today, I see the strain of maintaining “perfection” on the faces of faculty members, especially the pre-tenured. They must be “outstanding in either teaching or research” and “of high quality” in the other area.  And when we’re discussing workshop topics for CTE events, I frequently suggest that presenters share something that didn’t work and how they overcame a classroom disaster—I don’t recall anyone volunteering. We may encourage our students to take risks, but our internal culture is commonly risk-adverse.

CTE launched two Faculty Learning Communities last fall, in part to provide a safe haven for professional growth and vulnerability. I’ve had no doubt about the valuable contributions that each of these FLCs is making to pedagogy at SMU. But I wondered about the personal-enrichment experiences of the FLC participants. Here are some responses from the FLC on writing:

I thoroughly enjoy finding out about common concerns—and creative ways to address those issues—with faculty across campus. . . . I look forward to going each time we meet! –Paige Ware

It has been one of the best experiences I have had at SMU! . . . It might be luck that our group was perfect for me – there was a lot of knowledge about a subject I knew little about! –José Lage

I think my biggest response so far is how can our disciplines be so different yet we struggle with so many of the same issues with respect to students and writing assignments. –Sheri Kunovich

I’m happy to share that one of my favorite takeaways from the FLC has been to learn how my colleagues are using technology to facilitate writing/editing assignments at the individual and group level. –Sandy Duhé

From the technology FLC:

I found out who is knowledgeable on campus about how these technologies work, if I should become brave enough to use some of them. This is helpful because I will know who to ask. –Lynne Stokes

. . . being on the committee was a spur for me to follow up on doing some tech-based things I had considered but not carried out. – Mark Kerins

Of course I get some good ideas for my courses from the FLC, but I think the most valuable thing I gain from the FLC experience is meeting with other faculty on campus and learning from them how they handle different classroom situations, address new challenges, and their experiences with different learning/teaching techniques.  It really is enjoyable to discuss teaching (using technology or otherwise) with a group of faculty who are passionate and engaged. –Paul Krueger

We’ve had a meeting or two where we basically threw out the agenda because we got started on a different technology-related topic.  So it’s been spontaneous at times.  Also, it’s just been good to touch base with faculty and staff across the various schools to see how they’re using technology in their particular fields.  This type of interaction has really given me some new ideas I want to try out. I think our group members actually like each other.  To me, it’s really been a positive experience. –David Son

I have enjoyed the opportunity to work with and learn from faculty I might not otherwise have the opportunity to meet. It is exciting to talk about how we can do our work better! –Barbara Morganfield