snatch 0.6 (The Man Who Couldn't Cry) – the page image grabber - Core Historical Literature of Agriculture

Core Historical Literature of Agriculture (chla)

Description

The Core Historical Literature of Agriculture is exactly what it sounds like: a repository of literature about agriculture.

The copyright page contains the following text:

As a publicly supported institution, Mann Library generally does not own rights to material in its collections. Therefore it does not charge permission fees for use of such material and cannot give or deny permission to publish or otherwise distribute material in its collections. It is the obligation of the user to determine and satisfy copyright or other use restrictions when publishing or otherwise distributing materials found in the Mann Library collections.

Formats

The chla module supports the following download formats:

-f value	Description
pdf	Pages are returned as one PDF file per page.
image	Pages are returned as one GIF image per page.

ID

The collection contains both books and journals. The IDs for each type of work can be found by navigating to the item and finding the portion of the URL that says:

id=XXXXXXX

The "XXXXXXX" is the ID. Journal IDs are in the format "XXXXXXX_XX_XX".

History

0.4.4: Fixed renumbering bug
0.4: Added module

Module Printout

# chla : Core Historical Literature of Agriculture
#
# 
# This module will allow you to download page images from
# the Core Historical Literature of Agriculture site. See
# the documentation for additional details

$module_c = "chla";


$module_default_format = "pdf";

%module_formats = (
  'image'  => 'gif',
  'pdf'    => 'pdf',
);


$module_format = module_check_format($module_default_format,keys(%module_formats));
$config{'ext'} = $module_formats{$module_format};


if ( $module_format eq "pdf" ) {
  print_v("Renumbering pages!");
  $config{'renumber'} = 1;
}


$module_url{'sitebase'}  = "http://chla.library.cornell.edu/";
$module_url{'plistbase'} = $module_url{'sitebase'}."cgi/t/text/pageviewer-idx?c=$module_c;idno=$idno;page=pagenav";
$module_url{'imagebase'} = $module_url{'sitebase'}."cgi/t/text/pageviewer-idx?c=$module_c;idno=$idno;size=l;view=$module_format;page=main;seq=";
$module_url{'touchbase'} = $module_url{'sitebase'}."t/text/gifcvtdir/$idno/";


print_v("Getting page listing... (".$module_url{'plistbase'}.")");


$res = $ua->get($module_url{'plistbase'});

if ( $res->is_error ) {
  print STDERR "Could not get page listing\n".$res->status_line;
  exit();
}

$res->content =~ m!<option value="(\d+)">.+?</option>\n</select>!;
$module_available_high = $1;
print_v("This volume has $module_available_high pages...");

$module_actual_high = module_set_limit($module_available_high);


foreach $module_i ( $config{'start'} .. $module_actual_high ) {
  $module_image_url = $module_url{'imagebase'}.$module_i;

  if ( $module_format eq "pdf" ) {
    push(@urls,$module_image_url);
    print_v("PDF URL: $module_image_url");
  }
  else {
    $module_j = sprintf("%08d",$module_i);

    $module_touch_url = $module_image_url;
    $module_image_url = $module_url{'touchbase'}."$module_j.tifl.gif";

    print_v("Touch URL: $module_touch_url");
    print_v("Touch URL: $module_image_url");

    push(@touchurls,$module_touch_url);
    push(@urls,$module_image_url);
  }

}