#apiworkshop Reflection: free the data

I recently attended Bill Turkel's Workshop on APIs for the Digital Humanities held in Toronto and had the pleasure of coming face to face with many of the people who have created the historical data repositories I have used so enthusiastically.

What I came away with was an even stronger conviction that data in humanities repositories should be completely liberated. By that, I mean given away in their entirety.

The mass digitizations that have occurred recently have provided a great first step for researchers. I no longer need to fly to London and sit in a dark room sifting through boxes to look at much of the material I seek. But, I'm still - generally - unable to do with it what I like.

Many repositories contain scanned documents which have been OCR'd so that they are full text searchable, but that OCR is not shared with the end user, rendering the entire thing useless for those wanting to do a text analysis or mash up the data with another repository's content.

Most databases require the user to trust the site's search mechanisms to query the entire database an return all relevant results. If I'm doing a study, I'd prefer to do so with all the material available. Without access to the entire database on my hard drive, I have no way of verifying that the search has returned what I sought.

Many of those at the workshop who administered repositories were willing and eager to email their data at the drop of a hat, but that is not yet the norm. Most of my past requests for data have been completely ignored. When it comes to scholarly data, possessiveness leads to obscurity.

As humanists become increasingly confident programmers, many will define research projects based on the accessibility of the sources. Those who are giving their data away will end up cited in books and journal articles. Those desperate to maintain control will get passed by. If someone asks you for your data, think of it as a compliment of your work, then say yes.

Publishing for a Wide Audience: Grad Student Workshop

Monday October 19, 2009
London, ON, Canada

Fifty thousand screaming readers rush the newsstand to get a copy of your latest research. Okay, maybe they're not screaming, but the numbers probably aren't that far off. While peer reviewed journals may make the academic world go round, it's through magazines and newspapers that your work can make its way into homes across the country – and you might be surprised to find out how interested Canadians are in what you do. Did we mention that you also get paid, and the amount of work is probably less than you spent on your first undergrad paper?

The Network in Canadian History & Environment (NiCHE) is sponsoring a graduate student workshop on Monday, October 19, 2009 in London, Ontario which will teach participants how to sell an article about their work or experiences to a popular publication. The workshop will be hands-on and by the end of the day all participants should have a proposal finished and ready to submit to an editor.

Accommodation grants are available for out of town participants. Priority will be given to students studying topics on Canada and the environment and those who are registered as NiCHE members. Membership is free and you can sign up at http://niche-canada.org/user/register.

Participants are also invited to attend an optional public lecture that evening by Harriet Ritvo, president of the American Society of Environmental Historians. Ritvo will be discussing her new book, The Dawn of Green: Manchester, Thirlmere, and Modern Environmentalism (Chicago UP, 2009).

Space is limited, so if you are interested please contact Adam Crymble at acrymbl@uwo.ca as soon as possible. Formal registration to follow. No fee.

See the event poster (PDF).

Photo credit: "Ode to Jack Keruac", Oliver Hammond.

Learning Unix

I was recently given the task of learning Unix on my Mac. Recently, in this case means last week. I make no claims of being a Unix expert, but I am making some good headway. So, I thought I'd compile a post that tells others in a similar situation what resources I found helpful. If you have other suggestions, please add them in the comments.

Read:

Hayne, Cameron. "Unix FAQ (for OS X)", Hayne of Tintagel.

It's fairly long, but it provides a great overview of what you're trying to learn. There is plenty of opportunity to practice while reading through the various sections.

As you read, keep the following reference chart handy:

"An A-Z Index of the Apple OS X command line". This basically tells you what all the commands do and shows you the syntax required to use them.

Next, try the following questions, compiled by Bill Turkel to keep me occupied while he did more important work:

• Open a terminal.


• What directory are you in?


• What files are in the directory?


• Can you list the files in the directory in different ways?


• How do you go to a different directory?


• What files are in that directory?


• How do you see only a few of the files at a time? Say ones that begin with the letter 'a'?


• How do you see only the .html files?


• How do you copy a file?


• How do you move a file to a new filename? Is that the same as renaming it?


• How do you make a new directory?


• How do you get rid of a directory?


• How do you list the contents of a .txt or .html file to the screen?


• How do you break a text file into screen-sized chunks if it is too big to display on the screen?


• How do you show the first part of a text file to the screen?


• How do you show the last part of a text file to the screen?


• How do you find a file in a different directory if you know (part of) its name?

You should be able to use the reference list posted above to solve all these problems. If you're totally stuck, leave a note in the comments.

Then you can move on to some more Turkel questions. Characters in quotes are commands to type into your Unix terminal. Don't include the quotation marks:

• What do the "uname", "hostname" and "set" commands do? (minus quotes)


• Try "ls -l" and "ls -1". Read "man ls"


• Practice using "touch". Read "man touch"


• Practice using "grep".


• Make sure you read about . .. ~ and / .


• Read "man less"


• Read about pipes and redirection (|, <, >, >>), sed, split, chmod, curl and lynx

Once you're done that, you can try out some of the free quizzes on the bdv-unix-skills website.

Hope that helps, and since I'm still learning too, more advice is most welcome!

Google's Geographic Bias

If you're sitting in Atlanta, it makes sense that when you Google "Hair Dressers" you are probably most interested in services near to you. It doesn't do you any good to know "Jimmy's Haircut Shop" in Vancouver BC doesn't require an appointment; you need to know who in your area can cut your hair.

Google's searches have several mechanisms in place to help with this. For example, an American will automatically be directed to Google.com when they navigate to Google. Here in Canada, you get sent to Google.ca, and in Jolly Old England, you go to Google.co.uk. Each is designed to help users find search results most likely relevant to them. Canadians are more likely to see Canadian sites ranked higher.

However, this "helping" can effectively mean people in some countries are blind about what occurs elsewhere in the world. Many people do not look beyond the first page of results when they have performed a Google search. Hence the intensely competitive "Search Engine Optimization" industry that works hard to ensure sites are ranked highest. What this all means for a researcher new to the internet or unaware of the perils is that depending on where you live, Google might not point you to the same resources. This is especially the case if you are trying to research another region.

For example: the Google search for "england history" on the US, UK and Canadian instances of Google all provide 83 000 000 matches.

However, while the page "The History of England" ranks 6th on the UK version of Google, it is ranked 66th on the Canadian and American versions of the search engine, deep enough down that most people wouldn't likely bother to check it out.

If I were interested in the Network in Canadian History & Environment (NiCHE), hosted at niche.uwo.ca, and did a search for "niche", what I found would depend on where I lived:

Google.ca rank: 3rd
Google.com rank: 15th
Google.co.uk rank: 420th (that's the 42nd page!)

The .ca attached to the address means that Google has decided that this page is likely irrelevant to Brits. Now, granted, I should probably refine my search to get beyond the fact that a search for a word like "niche" is a bit ambiguous. But, the important thing to take away here is that Google is vetting what knowledge we find, in many cases without us even being aware of it.

What can we do about it? Probably not very much that will get Google to change their ways. But as responsible researchers and internet users, we should be aware of this. When you're doing research, make sure you don't just accept Google's top 10 suggestions at face value. Try another search engine (there's lots!). And if you're doing research on another country, make sure you try out their version of Google as well as your own.

Technology is great, but we've got to know it's limits.

Do You Have a Wiki Yet?

If you're doing research without a personal Wiki, you're livin the hard life. Whether you love or hate Wikipedia, you can't deny it's got appeal and a large part of that is the wiki environment. It's extremely easy to link pages together, edit and keep back ups of previous work all at the same time.

Working on multiple projects? A central wiki can branch off into each project and keep things organized for you.

Don't have a big research budget to spend on software? How's free sound? If you already have webspace you can install the open source MediaWiki, the same package that Wikipedia runs on. If you don't have server space of your own, try Wikispot who will provide free wiki access to anyone working on a collaborative project.

If you're concerned about privacy or having your unfinished research show up on Google Searches, fear not; there are ways to prevent search engines from Indexing your wiki, usually found in the preferences, depending on which wiki you use. If you're running your own wiki, you'll have the option to password protect it.

So, do you have a wiki yet?

History: Enough with the Quizzes

I came across the Dominion Institute's newest quiz this morning. The quiz involves identifying famous Canadians based on their photographs. It included everyone from Wayne Gretzky to Sir. Fredrick Banting. The Dominion Institute issues these quizzes knowing full well that most Canadians will do poorly on them. The resultant statistics then make for timely Globe and Mail or National Post articles around Canada Day in which we are shamed for our lack of Canadian history knowledge.

I took the quiz, and got 10 out of 10, but not without a great deal of difficulty. In one case (Nelly McClung), I had to figure out the answer by process of elimination, ruling out Celine Dion and Michelle Jean from the list of choices.

The average Canadian did much poorer than I did. But, who cares? Why is it only history that we have to quiz? How come the Physics institute of Canada doesn't go around polling everyday Canadians with 3rd year physics problems, only to complain we have been neglecting our physics, as a nation?

So, I've created my own 1 question chemistry quiz, of comparable difficulty level:

Identify the following molecule:

Choices:
Apple Juice, Butter, Mustard, Saturated Fat, Vitamin C, Garlic, Aspartame, Hydro-Chloric Acid, Yeast, Caffeine, Tomato Juice, Folic Acid, Vitamin B12, Chlorinated Salt, Vitric Acid.

---

If you didn't recognize that as Caffeine, you're either evidence that chemistry isn't given enough funding / emphasis / time in schools, or that's just not something that's ever been important enough to your life to memorize or learn how to interpret.

Enough with the quizzes. People don't like to be shown how stupid they are. Not everyone yearns to play trivial pursuit. And, the inability to recognize a remote historic figure is not a sign that Canadian people are stupid.

If you'd like to take the quiz it is available here (for now): http://www.dominion.ca/

Digital Humanities: Where to Begin?

I was speaking with someone today with a MA in History, who was interested in getting involved with the digital humanities. She's keen to learn, but didn't know where to start.

My suggestion: take the best essay you think you've ever written and turn it into a webpage.

This will ground you in principles that will be forever useful as you progress to bigger and more complex projects, and it will give you a tangible 'deliverable' that you can be proud of. It is also something that you can teach yourself to do, with little instruction and a lot of experimentation.

Creating a simple website has many advantages. A lot of the advanced work you will do as a digital humanist will require you to work with webpages. This may involve dissecting pages from their source code, to extract content. You'd expect your surgeon to know what was connected to your knee bone; and I'd expect my digital humanist to know what was in my <:head> tag.

Also, markup, which is the language-type simple websites are written in, is much easier to learn than programming languages that use commands, such as JavaScript or Python.

To get started, see my previous post "Putting a History Essay Online".

What about the other digital humanists out there? What's the best first step for an aspiring student?