snatch 0.6 (The Man Who Couldn't Cry) – the page image grabber

Kentuckiana Digital Library (kdl)

Description

The Kentuckiana Digital Library is part of the Kentucky Virtual University and contains many items about Kentucky history.

The copyright page contains the following text:

Many items offered by the Kentuckiana Digital Library may be protected by the U.S. Copyright Law (Title 17, U.S.C.). Some items may have restrictions imposed by the copyright holder or the repository owning the physical items. The holding repositories have made best efforts to identify the copyright status for online items. This information is offered as a service to the general public in determining the proper use of an item and is found in collection finding aids and/or upon inquiry to the holding repository. However, it is always the user's responsibility to determine copyright restrictions and obtain the permission of the copyright holder.

Formats

The kdl module supports the following download formats:

-f value	Description
pdf	Pages are returned as one PDF file per page.
image	Pages are returned as one GIF image per page.

ID

The ID for each work can be found by navigating to the item and finding the portion of the URL that says:

id=AXX-XXX-XXXXXXXX

Where "A" is a letter and each "X" is a number.

History

0.6: Fixed bugs due to site navigation changes
0.5: Fixed renumbering bug
0.4.2: Fixed bug preventing GIF downloads
0.4: Added module

Module Printout

# kdl : Kentuckiana Digital Library# 
# This module will allow you to download page images from
# the Kentucky Digital Library site. See the documentation
# for additional details


$module_c = "kyetexts";

$lcidno = lc($idno);

$module_default_format = "pdf";

%module_formats = (
  'image'  => 'gif',
  'pdf'    => 'pdf',
);


$module_format = module_check_format($module_default_format,keys(%module_formats));
$config{'ext'} = $module_formats{$module_format};


if ( $module_format eq "pdf" ) {
  print_v("Renumbering pages!");
  $config{'renumber'} = 1;
}

$lc1 = substr($idno,0,1);
$lc2 = substr($idno,1,1);
$lc3 = substr($idno,2,1);


$module_url{'sitebase'}  = "http://kdl.kyvl.org/";
$module_url{'plistbase'} = $module_url{'sitebase'}."cgi/t/text/text-idx?c=$module_c;idno=$idno";
$module_url{'touchbase'} = $module_url{'sitebase'}."cgi/t/text/pageviewer-idx?c=$module_c;idno=$idno;size=l;view=$module_format;seq=";
$module_url{'imagebase'} = $module_url{'sitebase'}."cache/$lc1/$lc2/$lc3/$lcidno/";

print_v("Getting page listing... (".$module_url{'plistbase'}.")");


$res = $ua->get($module_url{'plistbase'});

if ( $res->is_error ) {
  print STDERR "Could not get page listing\n".$res->status_line;
  exit();
}

@content = split(/<div class="indentlevel1"><span class="scopingpage">/,$res->content);
$module_available_high = $#content;
print_v("This volume has $module_available_high pages...");

$module_actual_high = module_set_limit($module_available_high);


foreach $module_i ( $config{'start'} .. $module_actual_high ) {
  $module_touch_url = $module_url{'touchbase'}.$module_i;

  $module_j = sprintf("%04d",$module_i);

  if ( $config{'ext'} eq "pdf" ) {
    $module_image_url = $module_url{'imagebase'}."$module_j.tif.1.pdf";
  } else {
    $module_image_url = $module_url{'imagebase'}."$module_j.tifl.gif";
  }

  print_v("Touch URL: $module_touch_url");
  print_v("Image URL: $module_image_url");

  push(@touchurls,$module_touch_url);
  push(@urls,$module_image_url);
}