Friday, July 26, 2013

No DB access? Use REST

The problem.

   There's an arbitrary sized image in a database which needed to be made square, sized into 4 sizes, and have a border to color match the picture ( like iTunes albums ) .  Also, do it in batches so as not to cause unneeded load on the database.  Then a minor detail of there's no direct sqlnet access to the db.

  Here's the current result.  The original picture was an now quite square at 490x604



The result is much more uniform






The list of work.

  Subsetting the list to be done was the simplest part.  A RESTful service defined which ties who has images to what to do produced this list.  This was my test on myself but it can obvious be opened up to do any subset or lower the pagination and process one page at a time.



The original image.

  There's not much to say here since I covered how to get an image out over REST here.


The processing.

  I read a lot of what people were doing to color match in various ways.  The actual code is in java and I can post it if anyone is interested.  However, here's what I basically did.

      Step 1,  Processes EVERY pixel and find the most common Hue ( HSV
      Step 2,  Rewalk the pixels and find every one of that Hue and get the most common Saturation and Brightness
      Step 3,  I found a Java Library named Scalr to do the resize
      Step 4,  Center the resized image and pad to make it square using the computed HSV as the padding color.

  Sounds simple but too me a while to figure the way to do this.  For this test image, it took ~300ms to process and make all the resized versions.  There's probably better/faster ways to do this but this seems to be working for me.
  
    For the image of me above the dominate color happened to be black which is why there are black bars.  However to prove it works, here's another example.  The color match is on the left and right and I put red lines around the seem on the right.



Putting it back into the Database.


  Now back to using REST.  Now there's 3 images that have been processed and need be updated into the database.  Most of what I've shown has been GET requests of the Apex Listener.  This will require a POST URI handler.  The URI template shows I'll post in the person email address and the size of the image.  This is so I know which column to place the updated image into.



The part not in the template anywhere is where the image will be.  The Apex Listener makes an automatic bind named :body.  This is the body of the file upload.  The other thing the listener does is print out anything in the htp.p buffer.  Which is why calling F on this post works.  I'm using htp.p in this block simply for debugging progress to see what's going where.


declare 
  p_id number;
  img BLOB;
begin
  /* bring the LOB to a local variable */
  img := :body;

  /* get the id */
  select person_id into p_id
    from people p
    where email = :email;

  /* print the size for debugging */
  htp.p('\n******************');
  htp.p('Lob size:'|| dbms_lob.getlength(img));

  /* based on which size stuff the image into different columns */
  if ( :size = 80 ) then 

     htp.p('Updating 80');
     update  photos
        set scaled_photo = img
      where person_id = p_id;
  elsif ( :size = 180 ) then 

    htp.p('Updating 180');
     update  photos
        set thumb_photo = img
      where person_id = p_id;
  elsif ( :size = 436 ) then

    htp.p('Updating 436'); 
      update  photos
         set large_thumb_photo = img
       where person_id = p_id;
  end if;

  htp.p('******************');

exception when others then
  htp.prn(SQLERRM);
end;



Putting it all together.
   The actual java code that runs it all is not that interesting.  I'm using Jackson to process the JSON of the people's images to process.  There's a few libraries out there for JSON processing this one just seemed easy for me to pick up.
  The posting of the images back is using Apache's HttpClient.  Here's that code.


 HttpClient httpclient = new DefaultHttpClient();
    try {
      // Create the POST client
      HttpPost httppost = new HttpPost("https://myhost.mydomain.com/pls/apex/rest/images/testPost?email="+ email+"&size="+size);
     
      // The image scaling is done locally to a file then posted
      InputStreamEntity reqEntity = new InputStreamEntity(new FileInputStream(file), -1);
       
      // set the POST to be binary content
      reqEntity.setContentType("binary/octet-stream");
      reqEntity.setChunked(true);
      httppost.setEntity(reqEntity);

      System.out.println("executing request " + httppost.getRequestLine());
      HttpResponse response = httpclient.execute(httppost);
      HttpEntity resEntity = response.getEntity();


      System.out.println("----------------------------------------");
      System.out.println(response.getStatusLine());
      String line = null;
      // now print the text from the htp.p debug messages.      
      
      BufferedReader in = new BufferedReader(new InputStreamReader(resEntity.getContent()));
      while( ( line = in.readLine() ) != null ){
        System.out.println(line);
      }
    } finally {
      // shut it down
      httpclient.getConnectionManager().shutdown();
    }