Showing posts with label history. Show all posts
Showing posts with label history. Show all posts

Wednesday, April 3, 2013

Programming Historian 2 Lessons I'd Like to See

I've been actively part of the Programming Historian 2 team for the past two years and I've been really pleased to see so many people using and learning from the site, including a number of university courses. I learned to write Python code from the original Programming Historian, and I still regularly reference skills and techniques found in the lessons in my day-to-day research.

My role as an editor of the project means I help guide lessons contributed by others through peer review and editing. I'm also always looking around the blogosphere for people working on cool new techniques or writing guides of their own that I think would be useful for practicing historians. For the most part this is a passive process. I sit, I wait, and I watch. But every once in a while I come across something I'd really like to see. So rather than wait, I thought I'd post my personal wish list of Programming Historian 2 lessons I'd like you to write for all of us.

In no particular order:

  • How do you turn a spreadsheet into a database and write custom queries? The jump from an Excel spreadsheet which you can see to a MySQL or sqlite3 database that you can't see is not an easy one. A lesson on making this leap would be well received and widely used I would imagine.
  • What the heck do you do with topic models? The entire digital humanities world seems fixated on topic models these days. Our most popular lesson by far is a tutorial on Getting Started with Topic Modeling and MALLET. But what are the cool things we can do once we HAVE generated topic models? What can we know? How do we use it responsibly? How do I interpret all these numbers and topics?
  • What can we do with our sources once they've been downloaded? I see so many people using programming to curate sources, but far fewer people asking historical questions of their sources using programming. What are some of the ways we can actually answer questions about the past with programming?
I'd be very happy to hear from anyone who'd like to take on these challenges and create a Programming Historian 2 lesson, or from anyone with an idea of their own they think others could benefit from. Check out our submission guidelines and be in touch.

Tuesday, January 25, 2011

Distant Reading by Using a Written Index

Today, gauging the general pulse of what people are saying or talking about or reading is fairly easy. Twitter's "Trending Topics" are one of many methods for seeing what people are interested in right now. Others include a scan of the top stories in today's newspapers, or a comparison of today's blog posts by keyword.

But how do we gauge that same pulse historically? What were people interested in 200 years ago? Can we really know?

One method is to look at what people read. For example, in 18th century London, the Gentleman's Magazine was the most successful magazine that targetted a wealthy and powerful audience. The Gentleman's Magazine published for over 200 years, starting in 1731 and during the late eighteenth century was England's most respectable magazine. In the first issue, the editor declared the magazine was originally conceived to "give monthly a view of all the pieces of wit, humour, or intellligence, daily ofered to the Public in the Newspapers, (which of late are so multiplied, as to render it impossible, unless a man makes it a business, to consult them all)".

Since we know that the magazine sold well, it's fair to assume that the editors were providing content that people wanted to read and is therefore a relevant measure of what people found interesting, broadly construed. That's not to say everyone who read it thought every article spoke to their soul. It's just to suggest that people vote with their wallets and one of the strongest indicators of a publication's influence is the number of copies it sells and for how long its able to continue publishing.

This magazine has since been digitized by Google Books, but to my knowledge, there is no accessible machine-readable text version freely available. However, there is a paper subject index of all the essays written for the magazine. The index was compiled in 1821 by some poor chap whose father insisted upon the task. The index won't give you the same full-text search option we've come to love in search engines. It is organized thematically into categories selected by the indexer. This means there is a fair degree of bias in terms of what was categorized, how it was categorized, and what was overlooked. However, with 675 pages of topics, along with the various page references to find the article, the index itself provides a useful gauge of what was appearing in the magazine during the 30 year period covered by the index (1787-1818).

reading John Bull's mindEach page is split into two columns. Using the columns as a unit of measure, I have compiled a list of the 34 most prevalent subjects from the Index and graphed them as a word cloud. France had the most 11.5 columns of entries and Africa had the least with half a column. Given the status of the magazine with Britain's upper classes, and given the fact that the magazine was successful, and therefore publishing content its readers wanted to consume, I believe it is fair to say that this word cloud is a fairly good - yet rough - illustration of what was important to wealthy Englishmen from 1787 to 1818.

Given the international affairs between Britain and various countries of the world during this era, the countries that appear most prominently are perhaps not a surprise. However, there are a few observations that I think are worth noting. Overall, I would suggest this wordcloud shows us what the wealthy people of late eighteenth, early nineteenth century England were worried about. Without reading the articles, or even knowing what the titles of the articles are, it would seem that these "Gentlemen" were writing essays about countries with which England was at war or at odds: France, Ireland, America. Less frequently appearing are Scotland and Wales, both passive at this point in terms of relations with England, along with Switzerland who tended to stay out of things.

The third most common term is "Naval Action" and "Buonaparte" is high on the list. Fires (7 columns), Storms and Murder all appear on the list. The first two threaten commerce, and murder is an obvious concern. Cowpox (which had been discovered could be used to vaccinate against Smallpox) is as prominent as Scotland or England. However, notice theft, rents, or poverty don't appear at all.

By applying my historical knowledge of Britain during this era, my distant reading of the Gentleman's Magazine suggests to me the following conclusions:

Wealthy Englishmen in the late eighteenth, early nineteenth century were interested in whichever country was currently causing the most trouble. They wanted to be kept informed of things that could kill them, or things that could disrupt their trade. They were interested in discussing the structure of the Anglican church, but less interested in discussing other religions, or directly engaging with the Bible. And finally, London was more important than America.

Is this perfectly accurate? No, of course not. But it is a macro-analysis of the interests of wealthy male Englishmen from 1787 to 1818. And it only took 20 minutes.

I would be interested to hear from others who are engaged in distant reading, or comments and concerns you may have about my methodology and conclusions, from digital humanists and historians alike.

Wednesday, June 11, 2008

The Canadianization of Zotero

Zotero is a pretty amazing computer program for historical researchers using the internet. What makes it even better are the translators. Translators allow you to click on a little icon in the address bar and automatically save all the bibliographic information found on the website. This is a great time-saver as those of you who use Zotero for this purpose already know.

If you're not familiar with this feature and already have Zotero installed, go to www.amazon.com, search for your favourite book, go to the entry and click on the little blue book icon on the right hand side of your address bar. Then take a look at what got saved in Zotero. Pretty nifty for one click of a mouse.

However, they're not as easy to make as they are to use. The translator for Amazon.com only works for Amazon.com. Each website that is supported - and there are a lot - have a custom-coded translator, specific to that site.

Right now, most of the sites that are supported are American. I say it's time for a change.

So, in the interest of promoting Canadian history research, I'm offering you a chance to get the translator of your dreams, free of charge.

I am taking requests for translators for sites that are used by CANADIANS for research.
These sites can be in English or French (or both), and priority will go to historical databases, and requests made by UWO history professors who gave me good grades.

If you know of a site that fits these criteria that you would like a translator for (or if you operate such a site), please post your request HERE, and include the word "Canada" somewhere in your message.

I will do my best to fulfill all suggestions, provided they are posted prior to July 15, 2008.

To be eligible, the site must contain:
  • a large database of records (1000+ entries).
  • each entry must have its own page with a stable URL (if you can cut and paste the URL into a blank browser's address bar and it takes you to the entry, then it's stable enough).
  • Each entry must have a title.
  • The entries must be searchable via a search box.
  • I must be able to access the records. (That means if it's password protected, it must either be accessible to me via the library at the University of Western Ontario's subscription, or you must provide me with access.)
  • The site cannot be under construction, or planning changes to its structure/design in the near future.
Examples include:
  • Canadiana.org
  • Glenbow Library and Archives
  • the Globe and Mail
  • CAIN
  • BCain
  • UWO Library
Please forward this request to any of your colleagues who may find this helpful.

Remember, only until July 15, 2008. After which time I'll be on to other things.