Tuesday, May 22, 2012

Why I'm looking at Erlang over Scala

For the past few months I've been trying to learn a bit about Scala. It is a complex language full of nuance. How a Map can function like a Perl associative array is a very interesting feat of reflection, or compile time interface addition. But behind all of its complexity, something dark has been nagging at me. I've not been able to place my concern until using RabbitMQ in Java. But now I think I'll move on past the language to something else.
The pebble stuck in my craw is the dual nature of Scala: objective and functional. Like most dualism, I have a hard time understanding it. But beyond that, I think that the dualism plus Scala's desire to leverage existing Java libraries are inherently problematic from a daily development perspective.
The dualism is troublesome because it gives a safe veneer of functional to a seedy underbelly of mutable objects. Sure this function is passable, says Scala, but inside there be side effects. This is even more troubling when one couples older, non thread-safe Java code to a functional looking Scala snippet. Now we've lost our scalablity. We have to worry about state because our old Java 1.4 library is unsafe. But since we're thinking in Scala, this old way of life is possibly, probably, lost on us. Now we get to deal with random threading issues like in days of yore.
This is the threat to a Scala project: old busted Java getting coupled somewhere along the line with our nice, new pseudo-functional system. Our vals vs vars are undone by one call to some open source project that didn't worry enough about threading to even warn the user that the object is not thread safe. All it takes is one rouge developer in the project to totally undo the whole thing like this (and we probably all know that one guy; that guy who adds a random JAR to the project, but it's not really compatible with the other version of some other JAR already in the project, but you don't find out till run time 3 days after going to prod).
As a result of my choking on this problem, I've looked around for alternatives. Clojure looked good. Except that, even as a LISP, it suffers from the same cross-compatible Java calling ability that Scala provides. Side effects are possible, even when I don't mean it.
Fortunately, I've been working with RabbitMQ for the past month. As part of my trials I created random messages to dead-letter drops to test performance. With about 24 M of memory I can send 5K messages a second (about 1k per message) to the queue. When I attach a listener to the other end, I can sustain that type of throughput in my meager 256M Oracle Linux VM I've got hosting RabbitMQ. This wonderful little broker is written in Erlang.
Now I'm a pretty great cynic when it comes to technology. I don't drink the kool-aid (I never got into Ruby on Rails because I don't believe that Active Record solves all my problems) very easily and when I do, I don't drink much. But Rabbit got me thinking. With REST (some kool-aid, but not too much since in my mind it's autumn when it comes to web communication) and functional I think there could be a great opportunity for scaling. Couple this with Backbone.js or Ember.js and a whole new world of fast, user-friendly enterprise and consumer apps open up.
As a result of my mild dreams I'm looking at Erlang. Rabbit does well with it. It minimizes side effects to its actors' internal state. It doesn't scare me as much as Haskell (I'm looking at you Haskell fans on HN, damn Haskell fans you scary!). It doesn't run on the JVM and so can't bring in old, busted JARs! All totaled, I think that this will bring a pretty good opportunity to me.
Time will tell, but that's my reason from looking beyond Scala to something more functional. Please let me know your thoughts.



Thanks,
JPD