I'm pleased to announce my latest endeavour to teach historians how to program computers in aid of their research has been published and is freely available on the Programming Historian 2 website.
Full Lesson: Downloading Multiple Records Using Query Strings
The hands-on lesson builds upon some introductory Python tutorials also available on the site and is intended to teach practicing historians how to use Python to discriminately download a copy of sources found in scholarly databases to their own computer for further processing. Thanks to Fred Gibbs, Miriam Posner, and Sara Palmer for their assistance working the lesson into shape. The abstract is available below:
Downloading Multiple Records Using Query Strings
Downloading a single record from a website is easy, but downloading
many records at a time – an increasingly frequent need for a historian –
is much more efficient using a programming language such as Python. In
this lesson, we will write a program that will download a series of
records from the Old Bailey Online
using custom search criteria, and save them to a directory on our
computer. This process involves interpreting and manipulating URL Query Strings.
In this case, the tutorial will seek to download sources that contain
references to people of African descent that were published in the Old Bailey Proceedings between 1700 and 1750.
For Whom is this Useful?
Automating the process of downloading records from an online database
will be useful for anyone who works with historical sources that are
stored online in an orderly and accessible fashion and who wishes to
save copies of those sources on their own computer. It is particularly
useful for someone who wants to download many specific records, rather
than just a handful. If you want to download all or most of the records in a particular database, you may find Ian Milligan’s tutorial on Automated Downloading with WGET more suitable.
The present tutorial will allow you to download discriminately,
isolating specific records that meet your needs. Downloading multiple
sources automatically saves considerable time. What you do with the
downloaded sources depends on your research goals. You may wish to
create visualizations or perform various data analysis methods, or
simply reformat them to make browsing easier. Or, you may just want to
keep a backup copy so you can access them without Internet access