Tuesday, November 9, 2010

Busy Bee, That's me

Many months have passed since my last post. I really attribute that to the fact that I have been quite a busy bee. It has been nearly a year since I started my new job at Contextweb. It is a great company in the online advertising space and they are definitely keeping me busy. Which is a good thing :)
I have also been quite busy launching the brand-new gubinart.com website for a great New York based artist, Mikhail Gubin, and at the same time helping my girlfriend jump-start her own entrepreneur spirit by launching the nychubbub.com, a site devoted to people around New York City; for newcomers and native New Yorkers.


Contextweb
www.contextweb.com
The great thing about the company is that it is quite established and has a lot of interesting work. I got a little taste of everything pretty much in my short tenure here.

Started working as part of the API team developing RESTful APIs for our .NET Portal web-apps to consume. The portals are used by our Publishers and AdNetworks. In reality the APIs could be later exposed to other business for their consumption as well. Which is actually what happened with one of the components that I've worked that deals with our contextualizing technology.

Six months later I was moved to the Platform team and got a glimpse of how we do our ad-serving. That wasn't on my own behest, but rather because there was a shortage of resources there that needed to be filled quickly. That only lasted about a sprint and half, before I was yet again asked to contribute to the Data team.

To be truthful, when I was first interviewing at Contextweb and realized that they use Hadoop, I was truly excited. That is one technology that I was very curious and have already read a book about. That whetted my appetite and I just wanted to get my hands dirty with this new emerging great piece of technology. This was quite a leap from working with more web-centric projects as I've done in the past. Now I began to deal with huge amounts of data, reporting, troubleshooting bugs in a distributed environment. This was quite different and the development cycle is very different as well. What I mean by that is that when you traditionally create apps, you can see the output almost instantaneously by running unit tests or some manual tests etc. In other words, the development cycle is pretty quick. With Hadoop though, you end up doing lots of context-switching since the jobs may take minutes and sometimes hours to run. Of course we have a suite of unit tests (that test the core) and functional tests that test our individual jobs, but still, for the most part the development life-cycle becomes much longer. This takes time getting used to and gets you thinking about possible issues up-front so as to avoid wasting precious time on the cycles.

Contextweb was very accommodating in terms of getting me up-to-speed in the new Data team. They sent me on a training course with Cloudera, which definitely helped to fill the voids of my self-taught knowledge about Hadoop. We also hold monthly NYC Hadoop meetups and those have been fundamental in learning about emerging eco-system surrounding Hadoop.

Gubinart
Ok, this one has been long overdue. The first version was written by my young self about ten years ago. I used what I knew best at the time which was classic ASP, MS Access DB, and I created a VB6 admin app that was used to manage the content. I've lost the software (Visual Studio) and interest in continuing the development using those ancient technologies. And instead I began the process of redesigning the site some years ago. I began with developing it in Grails, then I lost the source and my passion was sparked by all of the Ruby hype. So i began a Rails version utilizing the ActiveScaffold framework. And this was actually nearly complete when I decided to trash it and move my source code from bitbucket to github. Which I have to admit was pretty pointless. The whole domain and the admin section were pretty much completed. I'm not sure why I did that and still kick myself.

Anyhow, after a couple of years of vegetation in terms of this project, I was prompted to kick-start it again. The reason was that my parents switched to a mac and could no longer maintain the website (remember that the admin app was written for Windows utilizing VB6). I had to create a web-enabled admin section. As I've mentioned, I did not want to use classic ASP, even if it killed me.

I have been dabbling with Grails and it has matured quite a bit in those years. Many kind-hearted souls have contributed quite a large number of plugins. And so I decided to just stick to what I know best. Which is groovy and grails for web development.
I truly began the development sometime around mid-August. It didn't take that long to get something to show my client (my father) and we went from there, utilizing quick agile cycles to get the website to what he liked in a matter of weeks.
It tooks me around two or three weeks to figure out the deployment to a cloud server @ rackspace. After banging my head against the wall trying to do everything properly, using Tomcat and Apache, I gave up and decided to dumb everything down.
Instead, I went with Nginx and using the built-in grails container (I believe it is embedded Tomcat). I was able to get everything going after a while, doing some tuning of the memory utilization (I only have 512MB). And I set up YAJSW (Yet Another Java Service Wrapper) http://yajsw.sourceforge.net/ to reboot the service anytime it went down.
I am going to make a post about the deployment process separately.


NYCHubbub
This must have been one of the smoothest processes that I've experienced. I have never really dipped into using PHP, or any of the frameworks before, because I didn't really need to.
However, this time I was still quite occuppied trying to tie the loose ends with gubinart.com and needed something up fairly quickly.
I looked at Drupal and then found Joomla, which seems easier to manage. Since the person managing the site isn't that technical, that was a key feature for me.
I set up the site in less than a day :)
Of course doing tweaks to the styling and adding other features added a week or two more of work, overall the experience was quite shocking.
It is quite amazing how many extensions/plugins exist for this popular PHP CMS framework. There is anything I could think of available for free or for a nominal fee. I was able to setup the popular Disqus commenting system, the social media sharing plugin, polls, you name it.

So as you can see, I have been quite busy lately and looking forward to sharing some more technical juicy details.

Monday, April 26, 2010

What's is going to happen to Java?

This has kind of been discussed by some lately but I think many people are still wondering but don't ask. Java has been around for quite some time now and has infiltrated many a businesses with its simple syntax yet powerful features. A lot of developers still do java and companies have invested heavily into Java, so I feel like it is here to stay. It's definitely starting to lose its luster lately for a few reasons.
One being the state of affairs with Java's creator and a long time guardian of its innovation.
The other being the multiple-processors becoming the norm in computers.

The former by itself has been quite a pain point of the Java community, even before the take-over. JCP has been relatively slow in keeping up with people's needs. Some of the other languages emerging are giving Java a run for its money in terms of the features and paradigms that they offer.

The ruby community (rubyflow.com, rubyinside.com) has been gaining a lot of momentum in the last few years and I really like a lot of the stuff that is going on there. A lot of innovation, the community is really growing and contributing to its growth. It is evolving at a more rapid pace than any other community as of late.

Erlang picked up interest lately because of its distributed nature, fault-tolerance and high-availability, which go hand-in-hand when dealing with communication type of systems.

There are definitely some interesting languages emerging on the JVM itself (JRuby of course being one).
We have Clojure, Scala, Groovy, Fantom to name a few.

From what I see Java is going to be slowly being superceeded by some of these. I know Groovy is definitely being used quite a bit already and the fact that now we have a statically typed version of it called Groovy++, is going to pique quite a bit of interest.

Scala is a really interesting for me personally, because it in a unique position of being a hybrid language. It is both Object-oriented and functional at the same time. Its speed is nearly identical to Java itself.
The akka framework being developed on top of Scala is the most interesting framework for me these days.

Clojure is really interesting too. It is nearly pure functional programming language and its built-in STM gives it a unique feature that not many functional languages have (I don't know if any do actually).

Fantom is one of the new kids on the block, yes I say that even when some of these other languages haven't even been around longer than a few years.
It brings an interesting aspect where the code written in Fantom can compilre for both the CLR (.Net) and JVM. An interesting way to bridge the gap between the two systems.

I have been reading up on all of these languages because I think they will all be players in the game. In fact they already are. Like I said though, I Java is here to stay.
Look at COBOL (I really don't want to compare it to COBOL), but still, if you look at COBOL, it is still around how many years later?

We live in interesting times and I don't think Java developers have to worry about job security, but I do think the circumstances over the last couple of years have spawned some really interesting languages, that people should check out or at least be aware of. If you're a Java developer, I would highly recommend learning at least one of the new languages, as learning is always good for you and it also opens you up to new interesting points of views and paradigms that you may have never even been aware existed.

Monday, January 18, 2010

My humble opinion on what is OSGi's purpose

There was a recent post on theserverside regarding Spring's decision to move spring dm server to Eclipse. There were a lot of posts that I thought sounded like people do not really understand what OSGi is really for and what are its benefits outside of the embedded realm.

www.theserverside.com post:
http://www.theserverside.com/news/thread.tss?thread_id=59183

Here is what I wrote there:

The responses suggest that many people don't consider OSGi a trivial concept.
And at first it isn't, but after reading a bit about it, it actually makes a lot of sense.
The primary issue that enterprise (as opposed to embedded) OSGi tries to solve is the JVM class loading issues.
Secondly, it is the separation of concerns.

As far as the JVM class-loading issue is concerned, I'm sure everyone that has worked with Java to a degree has run into JVM classpath issues, whether its related to having the same package class names, or having multiple jars with slight version variations loaded into the JVM memory.

OSGi allows the developer to specify which exact packages should be loaded into the JVM and even allows multiple JARs with let's say different versions to be loaded at the same time.

However, when you import you are given the option to specify which version of the package that you want to import. This provides much more granularity than just specifying the versions of the JAR files themselves.

When you create a jar, it is good practice to specify the versions of all of the packages that you are exporting as well.


As far as the second issue (separation of concerns), OSGi makes you start thinking about your code in terms of modules. A module should do something very specific.
As an example, you may have one module that has a specific JDBC driver for your database. Another one that deals with DAO layer. Another module that deals with exposing your business logic via REST or SOAP.

Furthermore, a good practice is to split each of your modules into an all interface bundle and one that is all implementation. That way you can swap out your implementation at any time with another bundle at runtime, without compromising your contracts defined in the interface bundle.


As far as the complexity of dealing with the Manifest file, that isn't the case. You actually don't need to change your manifest file directly.

As Stuart mentioned, if you use maven, the felix's maven bundler plugin will help you to manage your relationships of the bundles and packages for you.
You still need to provide which packages you want to import and export.

And another great feature of OSGi is that it allows you to swap out a piece of your functionality without bringing down the server.
In conventional containers the entire web application has to be restarted. With OSGi, nothing is down and only a piece of your code is actually refreshed with newer code.

Thursday, August 27, 2009

Using Groovy to gather a file path list

Just thought I'd share a good use case of Groovy as a basic script.

I've created this script in a need to speed up one of our legacy manual processes of gathering files to be released to another environment (QA, Staging and Prod).
Basically, when we used CVS, we would need to list out all of the files needed to be pushed manually. What this script does is allows you to specify the base directory in which it is to begin recursing. You can also specify the prefix to use.
For example, on another computer the paths where the CVS is located may be different.
This script also skips the CVS and SVN type files. And it switches the slashes from Windows (back-slash) to Unix style (forward-slash).



def outputMatch = { prependString, file ->
def fileName = file.path
fileName = fileName.replaceAll(/^\./, "");
fileName = fileName.replaceAll("\\\\", "/")
def prependStringStartPos = fileName.indexOf(prependString)
fileName.substring(prependStringStartPos, fileName.length())
}

def baseDir = null
def prependString = ""
def lookupString = null

if (args.length > 0 && args[0] != null && !args[0].equals("")){
// i.e. C:\dev\cvsrepo\devel\fr_site\frdart
baseDirectory = args[0]
}

if (args.length > 1 && args[1] != null && !args[1].equals("")){
// i.e. fr_site/frdart
prependString = args[1]
}

if (args.length > 2 && args[2] != null && !args[2].equals("")){
// i.e. frdart
lookupString = args[2]
}

def count = 0

if (baseDirectory == null)
baseDirectory = "."

new File(baseDirectory).eachFileRecurse{file ->

if (file.isFile() &&
!(file.parent.toLowerCase().endsWith("cvs") || file.parent.toLowerCase().endsWith("svn"))){
if ((lookupString != null && file.name.contains(lookupString))
|| lookupString == null){
println outputMatch(prependString, file)
count++
}
}
}

println "\nNumber of Files=" + count

Mockito and the Builder Pattern for speeding up fixtures in Unit Tests

Mockito is a great mocking library on the JVM and I love the terseness of it and all of the functionality that it offers just like EasyMock.

Problem Description
In one of my projects at my current job, I have been using it to mock out classes in my unit tests of some Freemarker templates. Technically, these are more of integration tests, but since the library that it uses to do the generation of the content is well unit-tested itself, I consider this to be in effect unit tests.

So, I came up with using the Builder pattern in conjunction with mockito to simplify the creation of fixtures. We have a complex object that consists of many other objects and in order to test the templates thoroughly, we needed to constructs many different variants of this object. So using the Agile approach I began creating many of these variants by using helper methods. I kept on refactoring until every hierarchical piece was eventually wrapped up in a separate method.

Just as an example of what I am talking about, I will create an imaginary hierarchical structure that mimics my company's object.

Domain
Let's say that the one object that we are talking about is an object that represents a country.
A country is made up of many states.
A state is made up of many cities.
Let's just keep it simple like this for a moment.
And we will keep track of the population total at each level and the full name of the leader at that post (president, governor, mayor) and the name of the entitity.


Classes

Country.java

package com.mzgubin.mockito_builder_example;

import java.util.List;
import java.util.ArrayList;

public class Country {

private String name;
private String leaderFullName;
private long population = 0;
private List states;

public void setName(String name){
this.name = name;
}

public String getName(){
return name;
}

public void setLeaderFullName(String leaderFullname){
this.leaderFullName = leaderFullName;
}

public String getLeaderFullName(){
return leaderFullName;
}

public void setPopulation(long population){
this.population = population;
}

public long getPopulation(){
return population;
}

public void setStates(List states){
this.states = states;
}

public List getStates(){
return states;
}
}


State.java

package com.mzgubin.mockito_builder_example;

import java.util.List;
import java.util.ArrayList;

public class State {

private String name;
private String leaderFullName;
private long population = 0;
private List cities;

public void setName(String name){
this.name = name;
}

public String getName(){
return name;
}

public void setLeaderFullName(String leaderFullname){
this.leaderFullName = leaderFullName;
}

public String getLeaderFullName(){
return leaderFullName;
}

public void setPopulation(long population){
this.population = population;
}

public long getPopulation(){
return population;
}

public void setCities(List cities){
this.cities = cities;
}

public List getCities(){
return cities;
}
}


City.java

package com.mzgubin.mockito_builder_example;

public class City {

private String name;
private String leaderFullName;
private long population = 0;

public void setName(String name){
this.name = name;
}

public String getName(){
return name;
}

public void setLeaderFullName(String leaderFullname){
this.leaderFullName = leaderFullName;
}

public String getLeaderFullName(){
return leaderFullName;
}

public void setPopulation(long population){
this.population = population;
}

public long getPopulation(){
return population;
}
}


Mocking
Before using the Builder pattern, I've had to create all of the mock objects manually.
(I am excluding imports etc, just because I want to concentrate on the important code)



package com.mzgubin.mockito_builder_example;

import java.util.List;
import java.util.ArrayList;

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

public class Helper {
public City getMockedCity(){
City city = mock(City.class);
when(city.getName()).thenReturn("New York City");
when(city.getPopulation()).thenReturn(10);
when(city.getLeaderFullName()).thenReturn("Michael Bloomberg");

return city;
}

public State getMockedState(){
List city = new ArrayList();

City city = getMockedCity();
cities.add(city);

State state = mock(State.class);
when(state.getName()).thenReturn("New York");
when(state.getPopulation()).thenReturn(100);
when(state.getLeaderFullName()).thenReturn("David Paterson");
when(state.getCities()).thenReturn(cities);

return state;
}

public Country getMockedCountry(){
State state = getMockedState();
List states = new ArrayList();
states.add(state);
Country country = mock(Country.class)

when(country.getPopulation()).thenReturn(1000);
when(country.getName()).thenReturn("United States");
when(country.getLeaderFullName()).thenReturn("Barrack Obama");
when(country.getStates()).thenReturn(states);
}
}


This is quite an easy example, but if you have things that are more complex, you can imagine, this can become quite tedious to maintain as the code/tests change.

Enter the Builder Pattern
The idea really came from Groovy's builders and I initially wanted to create the builders in Groovy. I did not have much time to investigate using Groovy's builders, so instead I chose to go with Java since I am much more familiar with it.
I created a single Builder per class in the following format:


package com.mzgubin.mockito_builder_example;

import org.apache.log4j.Logger;
import java.util.List;
import java.util.ArrayList;

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

public class CountryBuilder {

private final static Logger log = Logger.getLogger(CountryBuilder.class);

private String name;
private String leaderFullName;
private long population;
private List states;
private Country country;

public CountryBuilder() {
country = mock(Country.class);
}

public CountryBuilder(Country country) {
if (country != null) {
this.country = country;
populateFields(country);
} else {
country = mock(Country.class);
}
}

private void populateFields(Country country) {
this.name= country.getName();
this.leaderFullName = country.getLeaderFullName();
this.population= country.getPopulation();
this.states = country.getStates();
}

public String toString(){
StringBuilder sb = new StringBuilder();
sb.append("Country:\n");
sb.append("name=" + name).append("\n");
sb.append("leaderFullName=" + leaderFullName).append("\n");
sb.append("population=" + population).append("\n");
sb.append("states=" + states.size()).append("\n");

return sb.toString();
}

public Country build(){
if (states == null){
states = new ArrayList();
}

when(country.getName()).thenReturn(name);
when(country.getLeaderFullName()).thenReturn(leaderFullName);
when(country.getPopulation()).thenReturn(population);
when(country.getStates()).thenReturn(states);

dump();

return country;
}

public void dump(){
log.debug(toString());
}

public CountryBuilder name(String name){
this.name = name;
return this;
}

public CountryBuilder leaderFullName(String leaderFullName){
this.leaderFullName = leaderFullName;
return this;
}

public CountryBuilder population(long population){
this.population = population;
return this;
}

public CountryBuilder states(List states){
this.states = states;
}

public CountryBuilder state(State state) throws Exception {
if (this.states == null){
this.states = new ArrayList();
}
this.states.add(state);
return this;
}

public CountryBuilder state(StateBuilder stateBuilder) throws Exception {
this.state(stateBuilder.build());
}
}


And I am not going to list the other two Builder implementations of State and City, as they are nearly identical to this implementation.

And now ladies and gentlemen, viola!
So now we no longer really need to mess with Mockito's implementation too much when we setup the fixtures, except when the Builders themselves need to be augmented.

We can now build a Country fixture object like so:


Country country = new CountryBuilder()
.name("United States")
.fullLeaderName("Barrack Obama")
.population(1000)
.state(
new StateBuilder()
.name("New York")
.fullLeaderName("David Paterson")
.population(100)
.city(
new CityBuilder()
.name("New York City")
.fullLeaderName("Michael Bloomberg")
.population(10)
) // end of city
) // end of state
).build(); // end of country

Now, that is nice and clean!

Maybe next time I will show how to do the same utilizing Groovy's Builder pattern!

Just beware that I did not test this code, but hopefully it will compile. I will try and run it later to make sure it at least compiles. If you find any mistakes, please let me know.

Saturday, June 27, 2009

My pitch at trying to get my company to use Groovy alongside Java

This is a blog I've previously posted on my other blog page.

It was turned down since it was deemed too young of a language and not enough people know it as a whole.
What are your thoughts on this matter?
How would you pitch Groovy to your company if you had to, and would you?

Here is the link to the presentation that I've shown:
http://www.slideshare.net/mzgubin/groovy-finesse-1655111

Rethinking relational database backed domain mindsets

Yes there are issues with relational databases, but they have been have been a developers' bread and butter.

Most of every issue has been solved in some form or another.
The data duplication and inconsistency is usually solved by normalization.
The impedance mismatch has been solved with ORM layer.
The latest problem though of trying to distribute the database is becoming an issue though. The distribution part is emerging as companies are trying to scale out and utilize more hardware or VMs to solve the mounting pressures of growing number of hits.

So what is the solution to this problem? Well, it appears that relational databases are being overtaken by the key/value, or map-reduce type of databases, like: Amazon's Dynamo, Google's BigTable; and the open-source types like: BerkeleyDB, CouchDB, HBase, Hypertable, Tokyo Cabinet, Project Voldemort, Redis, MongoDB and the list goes on.

Object Databases (db4o) and hierarchial databases like Neo4j are in a different category.


These provide a way to not only distribute your databases, but also provide speed and efficiency that is needed. You also get eventual consistency.

There is a draw-back to this though. The drawback is that we need to rewire the way we think about domain design. There are no more relations per se and the eventual consistency means there is a chance for inconsistency?

So, how do you go about switching your relational database type of mindset to this one? Do you need to rethink the way you design the structure of your domains?
Maybe this just needs some type of a eureka effect to take place to get this.
Does anybody else have these types of questions?