Programmatic interaction with Facebook data

The last time I interacted with Facebook using a program (2011), it was fairly straightforward, and perhaps it was too simple, it was abused.  As a result, in my recent experiments, I come to find out that, they have made it so complicated (my feeling, no offence to Facebook here), that its quite hard, if not impossible to do genuine development work using Facebook API, let alone achieve something sinister.

I am a self-anointed tech junky, and therefore no one should be surprised that i took up this challenge couple of hours back to build a brand new routine to get data in/out from/to facebook using a popular and open source Data integration tool, called Pentaho 

There were two triggers, one, I had not done some core debugging based research using something new. Second, a friend of mine is building this ecommerce site, www.freemall.in for which the marketing efforts would need a simpler interface to push details to the site, instead of some human going up there and doing all.  So, this programming interface becomes the first step of the future plugin for his ecommerce venture.

As I already indicated, programming with facebook's current API is a relatively hard task, at least for couple of iterations, before you get the hang of it.  So, I started with the tutorial at pentaho's site - http://www.pentahoevalcenter.com/data-integration/advanced-data-integration/facebook/ which turns out to be based on the old Facebook API and prompted me to look elsewhere for the right way of working.

Then comes the big thing - the oAuth authentication protocol employed by Facebook, get a token of this type, then of this type. go to this site, achieve this, copy this and take it there.. Its all so much that I had to go back to google and see if there is something simpler.. and yes, god bless google, there was someone like me who had gone to all the trouble and then of documenting it... God bless his soul for this - 
http://stackoverflow.com/questions/12168452/long-lasting-fb-access-token-for-server-to-pull-fb-page-info

After this, I went ahead with the pentaho tutorial, which did work out nice and fine for me.

I was able to post couple of messages to my test page using the application that i built for this test case.

hope this collection of links at a single place helps others.

availability of native drivers !!!

As I mentioned earlier, I was busy experimenting with a columnar database, InfoBright.

The experience till now is mixed, in some scenarios, the data loading is very good, as much as 40k rows per second, whereas through other channels its a poorer 500 rows per second. When I go with their built in loader, its lightening fast, but when I try from Pentaho or Informatica, its measly.

Apparantly, the drivers and the compatibility of the third party tools do play a role in attaining the performance.

The fact that they dont have any native driver published, is a huge bottleneck. Informatica dont even have a native connector for MySql, the more known cousin of InfoBright (being open source, the core engine of InfoBright is based on MySql only).

One thing came out of this experience for sure, the liking for open source got better and better. During the hunt to see the reason for slow performance from Pentaho, I even tried and managed to get the source code of the plugin (transformation) that pentaho uses for Infobright. Its pure java and it was a very powerful and awakening feeling to see the code of the item. I felt as if I have the power, the choice to make a difference, make it better. :)


I am currently exploring other options, one of them includes creating a dump file (read : csv) of the data, and then launching the command line tool to load the data into target db. I dont like it, but lets see if there is any better (read:faster) way around...