Steven Volk, March 7, 2016

What has been the most frequently assigned text at Princeton in the last 15 years? What about Harvard? Yale? For Princeton (along with Columbia), Samuel Huntington’s The Clash of Civilizations was the chart-topper. At Harvard, pivoting in the opposite (ideological) direction, the most frequently assigned text was Martin Luther King, Jr.’s “Letter from the Birmingham Jail.” Yale, for its part, returned to the classics with Plato’s Republic. Much to think about there!

More? OK, any guesses on Oberlin’s most assigned text? Would you be surprised if I reported that it was the “Communist Manifesto” by Marx (a text which ranks 5^th at Brown and 3^rd at Wesleyan, behind…wait for it… Huntington’s Clash of Civilizations and Hobbes’ Leviathan). Clash of the Titans!

 

 

         vs.

 

 

Frankenstein was the text most frequently assigned by English faculty at U.S. (and some international) colleges and universities over the past 15 years (followed by Canterbury Tales, Paradise Lost and Hamlet). Horace Miner’s wonderful spoof study (“Body Ritual among the Nacirema” which appeared in the American Anthropologist about a billion years ago and whose secret is given away in the title) was the third most frequently assigned text among Anthropology faculty, while Adrienne Rich’s “Compulsory Heterosexuality and Lesbian Existence” ranked at the top the Women’s Studies’ lists.

Big data analytics has met the college syllabus in the “Open Syllabus Project” (OSP) a joint project of scholars at Columbia and Stanford which went public with a beta version this past January. The project, housed at The American Assembly at Columbia, is supported by a grant from the Sloan Foundation with assistance from Columbia University’s Library and Department of English. It brought together several research groups interested in exploring what one can learn about undergraduate education by aggregating the data gleaned from hundreds of thousands of course syllabi. This effort, which brought in researchers from Harvard, the University of North Carolina-Chapel Hill, and Swarthmore College, built off the 2002-2009 “Million Syllabi” database created by Dan Cohen, the Executive Director of the Digital Public Library of America.

The OSP hoovers up faculty syllabi by one of three methods: searching publicly accessible university websites, gaining access to syllabus archives through agreements with individual universities, and collecting syllabi sent in from individual faculty members. It aggregates the data from these syllabi in order to find out what articles or books are most widely assigned within specific disciplines or departments, at individual colleges and universities, or in specific states. It also includes more spotty data from universities in the English-speaking world, largely from the UK.

Once the syllabi are collected, project personnel apply meta tags to the data, noting subject, texts used, school and date. The OSP currently holds the metadata on more than 1 million college and university syllabi, mostly from the United States. Using this data, it can disclose rankings by frequency (a teaching ranking) that can tell you which are the texts that are most often used in different areas. It can also provide some interesting data on what texts are most likely to be taught together. All of this is done without disclosing any of the underlying data which could reveal individual class syllabi, thereby protecting privacy and copyright.

Writing in The New York Times, Joe Karaganis and David McClure, two directors at the Open Syllabus Project, described the Syllabus Explorer as “mostly a tool for counting how often texts [have been] assigned over the past decade.” Using frequency as a proxy for influence, the Project assigns an overall ‘Teaching Score’ to each text, providing another metric for gauging the impact of certain books.

It’s not my purpose here, nor do I have the statistical chops, to suggest how accurate these rankings are. But others have noted that the rankings are statistically slanted toward the humanities as science and engineering classes tend to assign fewer titles.

The project designers themselves admit to some wariness as to the value of such metrics, noting that “Many academics are uncomfortable with this sort of numerical reduction of intellectual work.” But they suggest that current metrics (citation indexes, or the “journal impact factor,” which scores journal articles based on journal rankings that are determined by the journals’ own frequency of citation in other journals, often take a long time to appear and are limited in what they can tell us. Ultimately, they argue that “the academy is better off when it has multiple methods for valuing the wide range of work academics do.”

As Joseph Esposito of “The Scholarly Kitchen” blog notes, “When a scholar cites another, that tells us something; but it also says something about a work when it becomes adopted in course after course around the country; and it says more and different things when it is used in classrooms in different courses and even different fields.” He concludes, “It’s probably fair to say that the data is highly suggestive but not definitive. Metrics mavens will want more.”

But even in its current beta version, I have found the OSP to be both useful (and a lot of fun). The fun part is to see what reading trends are at different universities and in different disciplines. The utility comes in many guises. Let’s say that you are planning to teach a course in urban sociology and wanted to see whether colleagues in Sociology were assigning Alice Goffman’s hugely discussed study, On the Run: Fugitive Life in an American City (Chicago, 2014). Head over to the “Open Syllabus Explorer,” limit your search to “Sociology,” enter the title and click. You would soon discover that Goffman’s article “On the Run: Wanted Men in a Philadelphia Ghetto,” the book’s precursor which appeared in the American Sociological Review, had the highest ranking of any article and that her book ranked second. It would also give you a ‘Teaching Score’ (TS), which is a numerical indicator of the frequency with which a particular work is taught.  The score, I am told, is derived from the ranking order of the text, not the raw number of citations, so that a book or article that is used in four or five classes gets a score of 1, while The Republic, which is assigned 3,500 times, gets a score of 100. (I must admit that I remain baffled about exactly how “teaching scores” and “assignment counts” are calculated, nor did I get much help from the Project’s FAQs. See above on my lack of statistical chops.)

If you then wanted to see who was using the Goffman at different universities, you can filter for institution where you would discover, among other things, that Alice Goffman is more frequently assigned (at least in its article form) at Oberlin than at New York University. That should make your day!

As interesting, and perhaps more useful, is OSP’s ability to generate a list of other titles that are frequently taught together with a selected text. In the case of On the Run, just click on the book title and a list is generated which notes that the most title that is most frequently taught along with Goffman is a 1989 article in Human Rights by Jack Katz and Vicki Quade, “The Seductions of Crime” and Jane Jacobs’s 1961 classic, The Death and Life of Great American Cities. The texts most likely to be used by faculty who assign Goffman’s article are a 2002 article in the American Sociological Review by Annette Lareau, “Invisible Inequality: Social Class and Childrearing in Black Families and White Families,” and Devah Pager’s “The Mark of a Criminal Record” (American Journal of Sociology, 2003).

None of this, obviously, should be used as a way to avoid the hard work of putting together one’s own syllabus – essentially allowing the OSP to crowd-source your work for you. But it can reveal new connections that you have not made before and at least give a sense of where the field has been going.

Should Syllabi Be Shared?

There are understandable reasons why some faculty, particularly pre-tenure faculty, are reluctant to share their syllabi to a local or much broader community. Besides significant questions of ownership of intellectual property, many pre-tenure (and probably tenured, as well) faculty have legitimate academic freedom concerns. It is not too great a stretch to say that faculty worries about a weakening of their ability to teach a subject based on their own expertise and training were ratcheted up when Texas passed a law in 2009 (HB2504, it took effect in the fall 2010 semester) requiring that all syllabi be publicly available on the internet, searchable, accessible without a user name or password and no more than three links away from the school’s home page. It doesn’t take much time before families (or trustees) are forwarding their complaints to the faculty or deans when they find their daughter is reading Fanon or Baldwin. These are some of the reasons are why the Open Syllabus Project only provides metadata in their results, and they are not to be dismissed easily.

On the other hand, at Oberlin, and I’m sure elsewhere, we have talked for some time about making syllabi more broadly available via ObieMaps, OCTET, Blackboard or elsewhere in the context of making our scholarly work, and our teaching, more readily available to a wide community. Sharing syllabi within a single institution can help us see how our courses align with others in diverse disciplines, who else is teaching Foucault or Locke, whether Frankenstein being taught in History courses as well as English; whether faculty are using John Berger or other visual literacy texts in different departments. Developing our internal syllabus database can provide this information. Such a project can also provide some useful information as to what kinds of texts students are reading so that we can better understand what they are likely to have been exposed to and what not.

For all the caveats, there are powerful reasons why the Open Syllabus Project and similar attempts to create an open forum for discussion of what is being taught, also holds great potential at Oberlin and other small liberal arts colleges…as long as intellectual property rights can be protected as well as the teacher’s academic freedom upheld. Let me trace out a few:

• Faculty at liberal arts colleges have a pressing interest in helping students integrate their learning. Sharing syllabi can allow for increased intra-campus and inter-campus collaborations. Faculty in multiple departments can search the syllabi of related departments and programs to see where similar themes are being explored with the intention of bringing classes together periodically either physically, if they meet at the same times and can arrange it, or opening opportunities for students in different departments and disciplines to collaborate on work, thereby incorporating a number of disciplinary approaches to a problem.

• Faculty preparing new courses in their field can search syllabi at other institutions to get a sense of what materials tend to be taught together, not to simply copy what is done elsewhere, but perhaps to open new ways of thinking about how to situate a particular text.

• Faculty who have taught the same course a number of times can get a sense of what texts are becoming more highly regarded in the field. Again, this doesn’t suggest that we should teach a text because it’s being taught elsewhere, but it is always instructive to see how a field that you have taught for a while has been moving in terms of its textual requirements.

• In terms of student access, while access to current syllabi should never been seen as a guarantee that the course will be taught the same way in the future – past performance is not guarantee of future success! – being able to read past syllabi can give students a better sense of what will be taught than they are likely to get in a short catalog description. Further, to the extent that syllabi can be linked by tags, it can also allow them to link courses in a more intentional fashion.

So take a look at the Open Syllabus Project and see whether you like what it offers. And if it does, then let’s talk about creating our internal syllabus database project.