The URLScraperWithUIDProvider is
an extention of the URLScraper provided in the base product. This
extension extracts the URL from the channel definition and appends the
uid to the URL. The uid is appened to the URL using "?uid=uidValue" -
however, if there is already a "?" in the URL then the uid is appended
Download the tar ball: URLScraperWithUIDProvider.tar
and un-tar it into a know directory.
In this directory you will find a deploy shell script. You must edit
this script to change the values to appropriate values for your
system. Also the java compiler must be available for the script
to build the class file
Once the script has been edited do "sh deploy".
Next log in to the Admin Console and create a new channel using the
URLScraperWithUIDProvider provider. Put the URL for the page that you
are calling in the URL field.Then place the channel into a container
and you are done!
this code has a problem if the URL that is defined in the channel
contains spaces. You can replace the spaces with %20 and all if fine!