snatch 0.6 (The Man Who Couldn't Cry) – the page image grabber - Our Roots: Canada's Local Histories Online

Our Roots: Canada's Local Histories Online (roots)

Description

Our Roots features books that highlight Canada's local history.

Formats

The roots module supports the following download formats:

-f value	Description
image	Pages are returned as one JPEG image per page.

ID

The ID is an integer (4 characters). To get the ID, navigate to the table of contents page for the book you wish to retrieve. The URL will have a portion that looks like:

id=XXXX

The XXXX is the ID. Note that if you are viewing a specific page of a book you will see a 6 digit number:

ID=XXXXXX

This is the ID for the page, not the book; return to the Table of Contents to find the correct number.

History

0.6: Additional updates to fix broken URLs from site changes
0.5.2: Updated based on site changes (thanks iclysdale!)
0.4: Added module

Module Printout

# roots : Our Roots: Canada's Local Histories Online#
# This module will let you get materials from the
# Our Roots site. See the documentation
# for details.

$module_default_format = "image";


%module_formats = (
  'image'  => 'jpg',
  'jpg'    => 'jpg',
  'jpeg'   => 'jpg',
);


$module_format = module_check_format($module_default_format,keys(%module_formats));
$config{'ext'} = $module_formats{$module_format};


$module_url{'sitebase'} = "http://www.ourroots.ca/e/";
$module_url{'plistbase'} = $module_url{'sitebase'}."toc.aspx?id=".$idno;


print_v("Getting page listing... (".$module_url{'plistbase'}.")");
$res = $ua->get($module_url{'plistbase'});

if ( $res->is_error ) {
  print STDERR "Could not get page listing\n".$res->status_line."\n";
  exit();
}

if ( $res->content =~ m!src="roots/([A-z0-9]{3})/! ) {
  $module_url{'imagebase'} = $module_url{'sitebase'}."roots/".$1."/".$1."c";
} else {
  print STDERR "ERROR: Could not determine source\n";
  exit();
}

@imgids = split(/<option value="/,$res->content);
$module_available_high = $#imgids;
print_v("This volume has $module_available_high pages...");


$module_actual_high = module_set_limit($module_available_high) - 1;


foreach $module_i ( $config{'start'}-1 .. $module_actual_high ) {
  $module_i = sprintf("%04d",$module_i);
  $module_image_url = $module_url{'imagebase'}.$module_i.".".$config{'ext'};
  push(@urls,$module_image_url);
}