SobekCM Builder / Bulk Loader: Release History

VERSION 3.04 January 28, 2012

• Builder wait value moved into the system-wide settings accessible through the web app (in seconds)
• Number of days builder logs are retained is now a system-wide setting accessible through the web app
• Update the statuses used with the builder to add a PAUSE option which does not cause execution to completely stop
• Corrected issue when loading a METS files which does not have a VID indicated
• Corrected issue when an office document fails to convert to PDF for some reason which was causing all subsequent packages to be skipped
• Corrected issue creating JPEG2000 files
• Added better error-checking and error-reporting functionalities, especially on initial launch
• Moved all configuration information to an external config file
• Added ability to query the SobekCM Builder from the command-line for version information ( --version )
• Added ability to edit the configuration from the command-line ( --config )
• Updated the command-line help ( --help )
• Created a new deployment project which automatically sets necessary folder permissions and launches configuration tool as necessary
• Updated all libraries to be in sync with version 3.04 of the web application

VERSION 3.0.2 September 15, 2011

• Corrected issue that some portions of the SobekCM Bibliographic Package library had not been updated
• Corrected issue with METS-less resource files causing the metadata to revert to an earlier version
• Changed process so the RSS feed and XML file creation occurs no more frequently than every 10 minutes
• Moved the privacy email addresses into the database, from the application config file
• Page counts are now included for resources without page images but with a PDF
• Any errors encountered during auto-conversion of Word DOCs and PPT to PDF are written to the error log
• Changed how OAI-PMH records are stored; now in the database
• If a file is loaded for a PUBLIC resource which is BORN DIGITAL, then it is marked complete as well

VERSION 3.0.1 August 23, 2011

• Corrected the application configuration to hold only the bare minimum of necessary data
• Updated the SobekCM library to the newest version
• Added ability to call out to an external OCR engine while post-processing or handling new incoming data

VERSION 3.0.0 August 8, 2011

This represents a major update to the logic and further code cleaning. In addition, a new post-processing step was added and support was added for converting Word and Powerpoint files to PDF, pulling text from the PDF, and creation of thumbnails from a PDF file, as well as a number of other changes.

• Post-processing step included
• Added ability to convert PPT, PPTX, DOC, and DOCX files to PDF
• Added ability to pull full text from a PDF file
• Added ability to automatically create the main thumbnail from a PDF
• Added regular expression check for social security number in the loading text
• Added final call to the database to build the cached links between aggregations and metadata for quick metadata browsing
• Added final call to optimize the Solr/Lucene indexes before shutting down
• Solr/Lucene index updated while loading new items
• Added ability to control which types of requests can be processed from different input locations

VERSION 2.9.11 March 24, 2011

• Updated with new SobekCM Library and SobekCM Bib Package class libraries (dlls)
• Performed rather major code refactoring to remove unnecessary section and complete code commenting
• Removed tables in the database for incoming file destinations, image servers, application servers, and search servers
• Replaced these tables with five settings in the builder settings table
• Added new table in the database to support multiple incoming folder locations with different settings
• If a package has a TIFF, image derivatives will be checked and generated if necessary
• Added back DELETE processing if they are allowed in that particular incoming folder
• Setup file generated for 64 bit machines now

VERSION 2.9.10 January 9, 2011

• Updated with new SobekCM Library and SobekCM Bib Package class libraries (dlls)
• METS Types are now COMPLETE, PARTIAL, and METADATA_UPDATE
• Search engine sitemaps now include last modified date and limited to 30,000 per file
• Pointed all status updates to the new UFDC database, and removed all references to the old tracking DB

VERSION 2.9.9 December 20, 2010

• Updated with new SobekCM Library and SobekCM Bib Package class libraries (dlls)
• Modified save routines to only save the behaviors for a brand new item
• Builder no longer supports DELETE type METS files
• Added support for new DarwinCore zoological taxonomies in the METS files

VERSION 2.9.8 November 23, 2010

• MarcXML report and static aggregation pages built first thing the builder launches everyday
• Updated MarcXML report creating to validate against LC's schema
• Modified loader to create new static pages for each loaded item for search engine support on web server
• Modified loader to create new static browse all page for each item aggregation
• Updated METS --> MARC mapping when creating the static MARC files
• Updated with new SobekCM Library and SobekCM Bib Package class libraries (dlls)

VERSION 2.9.7 October 26, 2010

• Updated bib package for new secondary structure map
• Updated logging system to use a new log each day and remove old logs ten days old
• Move from registray abort system to database abort system

VERSION 2.9.6 September 27, 2010

• Updated builder with all the new code from the SobekCM web application
• Updated static page builder for new URL format

VERSION 2.9.4 July 20, 2010

• Corrected issue with the MARC thumbnail link
• New incoming packages that have OCLC and ALEPH numbers are automatically flagged to go to Endeca

VERSION 2.9.3 July 12, 2010

• Updated item saving routines
  • Recursively saves institutions ( if tagged to institutional division, also links to parent institution )
  • Saves size of package in KB to database
  • Saves additional metadata fields ( MIME Type, etc.. )
  • External record numbers ( i.e., OCLC, ALEPH, etc.. ) saved in seperate table
  • Saves ticklers and user entered comments
• Updated all SobekCM libraries to match web application
• If an item comes over tagged as METADATA UPDATE, but does not exist and has entire package, this is treated as a NEW item
• Static pages are now saved within the digital resource folder
• RSS feeds/Site maps updated for above change

VERSION 2.8.9 March 24, 2010

• Updated footprint save routing and bounding box calculation

VERSION 2.8.8 March 18, 2010

• Reincorporated all the new code from the SobekCM_Library and SobekCM_Bib_Package into the builder

VERSION 2.8.4 February 1, 2010

• Updated UFDC Builder to work correctly with new item aggregations
• Now accepts metadata updates outside of folders in the inbound folder directly
• Now builds OAI and RSS feeds for every subaggregation of an aggregation when building that aggregation

VERSION 2.7.9 October 7, 2009

• Corrected issue that was not clearing out the server information before loading new server information. Resulted in the same servers appearing in the server collection multiple times, causing the build to go slower.