Batch Uploading High-Resolution Images

We have two means of uploading images to the SCAN/LepNet database, a direct route using the iDgiBio appliance below and several alternative DropBox solutions for batch image uploading.

DropBox

These two DropBox options will allow for individual collections to batch upload images to their museum folder on DropBox. Then one designated person (Cat Chapman) will process all of the images for all collections through the iDigBio appliance from the Dropbox folders. The images need to be linked to records in the database.  We will plan to make a copy of all the images at the end of each month and store them elsewhere and then delete all images that have been uploaded for that month.

  • By default we have shared the Dropbox folder to everyone we thought would upload images. To share, you probably need to have a paid Dropbox account, a free DropBox account will likely force you to go over your quota. If storage space is a concern or you do not  wish to use a Dropbox account, we can set up a File Request for you through DropBox. In either case, you simply provide the images you wish to batch upload to us in one of two ways:
    • Shared folder Option (Need a DropBox Account):
      • You have continuous access to your files. If you have to rename anything or delete anything you can do so at your leisure. It also is a bit more seamless if you are using the Dropbox client on your computer as it will seamlessly integrate with your filesystem; no real need to deal with web pages.
      • You will be given access to the root image folder, so you will be able to see everybody else’s folders and files. Only put images in your own folder! Your folder will be named after your institution.
      • This does come at the cost of using Dropbox storage space. If you have over 2GB (the free Dropbox account storage limit) of photos this option will require a Dropbox account upgrade, which is not free. It is very easy to go over 2GB of photos.
    • File Request Option (Does not use a DropBox Account)
      • Uploading via a file request does not require a Dropbox account, and it will not count against your Dropbox storage space. You will simply provide the files to us via a web interface. There is no known limit and it’s no more difficult than uploading an image to social media.
      • You will not have access to these files after they are loaded. If you need to change anything, you must have one of us make the change for you. If you accidentally uploaded the wrong files, we will have to delete them for you. This method certainly requires double and triple-checking of the data before it is loaded.

If you would like to DropBox and you do not have DropBox access to the folder, contact Neil.Cobb@nau.edu and let us know which institution you are contacting us on behalf of, and we will set you up. As with any batch image loading to Symbiota, it is REQUIRED that the catalog number is in the image filename in the SAME format as it is on the digital record in Symbiota. E.g. a specimen with the catalog number EXAMPLE0001234 can have an image linked to it ONLY if the image’s filename contains the EXACT catalog number “EXAMPLE0001234” in its filename, for example “EXAMPLE0001234.jpg” or “EXAMPLE0001234_dorsal”. As long as the verbatim catalog number is present in the filename, the image can be linked to the specimen.

 

 

iDigBio Image Ingestion Utility + Symbiota Quick Start Guide

For Use with up-to-date Symbiota installations

  1. Ensure that you have the latest version of the iDigBio Image Ingestion tool – it may be found here: https://github.com/iDigBio/idigbio-media-appliance/releases
    1. Installation guide for the utility: https://github.com/iDigBio/idigbio-media-appliance/wiki/Installing
    2. It is highly recommended that you also have the latest version of Symbiota installed. Contact your systems administrator if you are unsure of the version of Symbiota your data portal is running.
  2. In order to run the utility, you will need a UUID and API key from iDigBio. This is like a username and password. If you are a Symbiota data collection manager and do not have either of these things, you may request them from the iDigBio Technical Team (idigbio@acis.ufl.edu).
  3. Launch the image ingestion utility. Log in with your UUID and API key.

PC users should use a browser other than Internet Explorer.  In other words, use Chrome or Firefox.

For Windows users to launch the iDigBio Media Appliance once the install is finished, go to the start menu and type: idigbio_media_appliance.

You can also create a shortcut to the program by creating a new shortcut and entering that command in the “command to run” box. This should open a command prompt, and then shortly afterwards your default browser should open a web page pointed at the iDigBio Media Appliance.

To close the appliance once you are done using it (and all uploads have finished). Close the browser tab with the appliance in it, then click the X box in the upper right hand corner of the appliance command prompt window.

 

  1. From the main page, you will see options for generating a CSV. This CSV is used to index the images you wish to load into iDigBio (and later your Symbiota portal).
  2. You will see three fields: Upload Path, GUID Syntax, and GUID Prefix.
    1. Upload Path: The directory where your images are stored. Clicking within this field should bring up a window allowing you to point-and-click to where the directory is located on your computer. (Note: the window may pop up in the background, minimize your browser to check) If your directory has nested subdirectories, be sure to tick the “Also Search Files in Subdirectories” box.
    2. GUID Syntax: For linking images to records within Symbiota data portals, you’ll need to choose the [GUID Prefix][File Name] option. Your file name MUST start with the catalog number of the corresponding record in Symbiota. After the catalog number follow with an underscore and use whatever information you want to describe the image. For example, a record’s catalog number is WFBM0000008, and the image filename is “WFBM0000008_dorsal.jpg”
    3. GUID Prefix: If your institution code should already a part of your image filenames, you may leave this field blank. Otherwise, enter your institution code and/or collection code as instructed by the data portal manager.

  1. Click “Generate CSV and Upload Files” to begin the upload to iDigBio. Depending on the amount of images you are uploading, this may take a long time.
  2. When the upload procedure is complete, from the Upload History tab, click the green “Download Media as CSV” button. You will use this file to link your uploaded images to Symbiota.

  1. Log in to your Symbiota portal and navigate to the collection whose images you had loaded into iDigBio.
  2. From the collections administration menu, select “Processing Toolbox”, and then the “Image Loading” tab.

  1. If you don’t see an option for “iDigBio CSV Upload” like in the above screenshot, it will need to be set up by the portal manager. This only needs to be done once:
    1. Select “iDigBio CSV Upload” from the Upload Type drop-down menu.
    2. Enter a regular expression that will correctly capture the catalog number from the image file name. Remember to enclose the portion of the expression representing the catalog number in parentheses. Regular expressions can be difficult to set up http://www.regexr.com is a useful resource in setting one up. If problems persist, don’t hesitate to ask Cat Chapman (cac469@nau.edu) for help in setting up the regular expression.
  2. You will now need to choose the CSV file that you downloaded from the Upload History page in the image loader.
  3. Click the Process Output File
  4. If you have followed the steps correctly and encountered no errors, Symbiota will extract the catalog number from the filename; check to see if a record with that catalog number exists within the Symbiota data portal; if a record does exist with that catalog number, the image will be linked to the record; if a record does not exist with the that catalog number, an empty record with only the image and catalog number will be created.

 

If any further assistance is needed, don’t hesitate to contact Cat Chapman at cac469@nau.edu.

Comments are closed