Tuesday, November 9, 2010

Busy Bee, That's me

Many months have passed since my last post. I really attribute that to the fact that I have been quite a busy bee. It has been nearly a year since I started my new job at Contextweb. It is a great company in the online advertising space and they are definitely keeping me busy. Which is a good thing :)
I have also been quite busy launching the brand-new gubinart.com website for a great New York based artist, Mikhail Gubin, and at the same time helping my girlfriend jump-start her own entrepreneur spirit by launching the nychubbub.com, a site devoted to people around New York City; for newcomers and native New Yorkers.

The great thing about the company is that it is quite established and has a lot of interesting work. I got a little taste of everything pretty much in my short tenure here.

Started working as part of the API team developing RESTful APIs for our .NET Portal web-apps to consume. The portals are used by our Publishers and AdNetworks. In reality the APIs could be later exposed to other business for their consumption as well. Which is actually what happened with one of the components that I've worked that deals with our contextualizing technology.

Six months later I was moved to the Platform team and got a glimpse of how we do our ad-serving. That wasn't on my own behest, but rather because there was a shortage of resources there that needed to be filled quickly. That only lasted about a sprint and half, before I was yet again asked to contribute to the Data team.

To be truthful, when I was first interviewing at Contextweb and realized that they use Hadoop, I was truly excited. That is one technology that I was very curious and have already read a book about. That whetted my appetite and I just wanted to get my hands dirty with this new emerging great piece of technology. This was quite a leap from working with more web-centric projects as I've done in the past. Now I began to deal with huge amounts of data, reporting, troubleshooting bugs in a distributed environment. This was quite different and the development cycle is very different as well. What I mean by that is that when you traditionally create apps, you can see the output almost instantaneously by running unit tests or some manual tests etc. In other words, the development cycle is pretty quick. With Hadoop though, you end up doing lots of context-switching since the jobs may take minutes and sometimes hours to run. Of course we have a suite of unit tests (that test the core) and functional tests that test our individual jobs, but still, for the most part the development life-cycle becomes much longer. This takes time getting used to and gets you thinking about possible issues up-front so as to avoid wasting precious time on the cycles.

Contextweb was very accommodating in terms of getting me up-to-speed in the new Data team. They sent me on a training course with Cloudera, which definitely helped to fill the voids of my self-taught knowledge about Hadoop. We also hold monthly NYC Hadoop meetups and those have been fundamental in learning about emerging eco-system surrounding Hadoop.

Ok, this one has been long overdue. The first version was written by my young self about ten years ago. I used what I knew best at the time which was classic ASP, MS Access DB, and I created a VB6 admin app that was used to manage the content. I've lost the software (Visual Studio) and interest in continuing the development using those ancient technologies. And instead I began the process of redesigning the site some years ago. I began with developing it in Grails, then I lost the source and my passion was sparked by all of the Ruby hype. So i began a Rails version utilizing the ActiveScaffold framework. And this was actually nearly complete when I decided to trash it and move my source code from bitbucket to github. Which I have to admit was pretty pointless. The whole domain and the admin section were pretty much completed. I'm not sure why I did that and still kick myself.

Anyhow, after a couple of years of vegetation in terms of this project, I was prompted to kick-start it again. The reason was that my parents switched to a mac and could no longer maintain the website (remember that the admin app was written for Windows utilizing VB6). I had to create a web-enabled admin section. As I've mentioned, I did not want to use classic ASP, even if it killed me.

I have been dabbling with Grails and it has matured quite a bit in those years. Many kind-hearted souls have contributed quite a large number of plugins. And so I decided to just stick to what I know best. Which is groovy and grails for web development.
I truly began the development sometime around mid-August. It didn't take that long to get something to show my client (my father) and we went from there, utilizing quick agile cycles to get the website to what he liked in a matter of weeks.
It tooks me around two or three weeks to figure out the deployment to a cloud server @ rackspace. After banging my head against the wall trying to do everything properly, using Tomcat and Apache, I gave up and decided to dumb everything down.
Instead, I went with Nginx and using the built-in grails container (I believe it is embedded Tomcat). I was able to get everything going after a while, doing some tuning of the memory utilization (I only have 512MB). And I set up YAJSW (Yet Another Java Service Wrapper) http://yajsw.sourceforge.net/ to reboot the service anytime it went down.
I am going to make a post about the deployment process separately.

This must have been one of the smoothest processes that I've experienced. I have never really dipped into using PHP, or any of the frameworks before, because I didn't really need to.
However, this time I was still quite occuppied trying to tie the loose ends with gubinart.com and needed something up fairly quickly.
I looked at Drupal and then found Joomla, which seems easier to manage. Since the person managing the site isn't that technical, that was a key feature for me.
I set up the site in less than a day :)
Of course doing tweaks to the styling and adding other features added a week or two more of work, overall the experience was quite shocking.
It is quite amazing how many extensions/plugins exist for this popular PHP CMS framework. There is anything I could think of available for free or for a nominal fee. I was able to setup the popular Disqus commenting system, the social media sharing plugin, polls, you name it.

So as you can see, I have been quite busy lately and looking forward to sharing some more technical juicy details.

No comments: