Planet Trapexit - Erlang/OTP News

Erlang Software Framework: EIF on AWS and an AMI for Erlang

2008-07-02T19:20:55+00:00

I got an email this morning that made me realize how long it had been since I had updated this site. I've been a little bit distracted since I got my book deal. Originally I submitted a proposal to write a book on Erlang, but through a series of events ...

Trapexit News Feed: Trapexit News :: New Cookbook Entry on Tracing!

2008-07-02T11:51:04+00:00

Author: francesco
Subject: New Cookbook Entry on Tracing!
Posted: Wed Jul 02, 2008 11:40 am (GMT 0)
Topic Replies: 0

Marstein has kindly contrinuted a new page to the Erlang Cookbook on tracing processes. You can view his contribution here... Or you can add your own recepie in trapexit's wiki based Erlang Cookbook.
_________________
--
http://www.erlang-consulting.com

21st Century Code Works: Notes from Erlang Exchange

2008-07-02T04:09:00+00:00

I was at the Erlang Exchange in London last week, which I really enjoyed. It was really great to see Joe Armstrong, and to see the faces behind the projects, e.g. Claes Wikström (Yaws, Bluetail, Kreditor, Tail-f), Steve Vinoski, Eric Merrit and Martin Logan (Erlware), Matthias Radestock (LShift & RabbitMQ), Mickaël Rémond (ejabberd) etc. etc.

Something that Joe said reminded me of a comment that Keith made about 6 months ago. I think this is something that gets lost in the hype around Erlang and multicore computing:

Erlang was designed to program fault tolerant systems.

It was not designed to program multicore computers. It was designed 15-20 years ago when multicore-everywhere was not even on the horizon. As a side-effect, it is a good language for multicore systems, since the only way to actually achieve fault tolerance is to have independant processes (read here for some more on this).

I think it's important for us as to remember to advocate this, and instead of marketing Erlang/OTP as "the application system for multicore", we should be marketing it as "the application system for fault tolerance". Oh, and by the way "this is one of the few systems that will actually use those cores that are coming your way".

The second point that I have been pondering, was brought on my Gordon Guthrie's talk about Erlang/OTP vs Google Apps as an application engine. At this moment, there aren't many application systems to choose from, Erlang/OTP, Google Apps and Amazon EC2 come to mind. In the context of the talk, an Application System (A/S) would be something that takes care of a lot of the non-functional requirements of you system, e.g. reliability, scalability (+distribution) etc.

For some businesses, using Google Apps or EC2 would be a good fit, but then you're tied into the value chain of that business, and in some instances you just cannot (for legal and other reasons) put your data on servers you don't own. If you go the OTP route you have full control over your application(s), but there's a lot of extra infrastructure that you will have to supply yourself.

If you start thinking of Erlang/OTP in this way, you realise that the terms should be reversed and it should actually be OTP/Erlang. The platform is actually the important thing. The platform is the thing that gives you the reliability, scalability, distribution and hot code swapping etc. that you require. Erlang becomes this awesome language that you use to build things on the platform. Looking at things from a different perspective can lead to some interesting insights.

If I'm standing at this vantage point, heated discussions like the recent meme-storm in the blogosphere of "Erlang vs Scala" become much less relevant. Is there a platform that gives you reliability, scalability, distribution, hot code replacement that uses Scala as a language? No? Oh. What a pity...

Trapexit's Erlang Blog Filter: Convenience Over Correctness

2008-07-02T02:42:58+00:00

My latest Internet Computing column, “Convenience Over Correctness,” (PDF) is now available. It continues the exploration of problems with RPC-oriented distributed programming approaches that I’ve been writing about in each of my three prior columns this year, as well as in columns from years gone by and in the erlang-questions mailing list.

For years we’ve known RPC and its descendants to be fundamentally flawed, yet many still willingly use the approach. Why? I believe the reason is simply convenience. Regardless of RPC’s well-understood problems, many developers continue to go down the RPC-oriented path because it conveniently fits the abstractions of the popular general-purpose programming languages they limit themselves to using. Making a function or method call to a remote or distributed function, object, or service appear just like any other function or method call allows such developers to stay within the comfortable confines of their language. Those who choose this approach essentially decide that developer convenience and comfort is more important than dealing with hard distribution issues like latency, concurrency, reliability, scalability, and partial failure.

Is this convenience for the developer the right thing to focus on? I really, really don’t think it is. There are ways of developing robust distributed applications that don’t require code-generation toolkits, piles of special code annotations, or brittle enterprisey frameworks. Perhaps the wonderful programming language renaissance we’re currently experiencing will help us to finally see the light and put tired old broken abstractions like RPC permanently out to pasture.

LShift on Erlang: RabbitMQ XMPP gateway released

2008-07-01T11:31:37+00:00

I’m pleased to announce that our XMPP gateway for exposing a RabbitMQ instance to the global XMPP network has been released (documentation, browse or check out code, download snapshot).

Update: Because it depends on a newer release of RabbitMQ than 1.3.0, you will also need to check out the server and codegen code from our public mercurial repositories, or download them as snapshots: server, codegen.

The mod_rabbitmq module implements an ejabberd extension module which gateways AMQP (as implemented by RabbitMQ) to XMPP.

By bridging between the two systems, we benefit from:

XMPP’s internet-scale addressing and federation
XMPP’s presence model
AMQP’s store-and-forward capability
AMQP’s routing and filtering (using exchanges and bindings)

The current implementation is a very simple mapping between the two systems. Its simplicity keeps the code short, but only exposes a subset of AMQP features to the XMPP network, and vice versa.

Yariv's Blog: Numenta, AI, and the Web

2008-06-30T04:26:22+00:00

Last week, I attended the Numenta workshop. I didn’t know much about Numenta before I went. My friend’s excitement about the technology Numenta is building piqued my curiosity, so I decided to check it out. It seemed that almost everyone else in the conference had read Jeff Hawkins’s On Intelligence and at least experimented with Numeta’s tools, so I felt like a real n00b. I’m happy I went, though, because I learned about some interesting ideas and technologies.

Jeff Hawkins, Numenta’s founder, has been fascinated with the workings of the brain throughout his career, but only two decades into it, after he founded Palm and Handspring, was he able to devote his efforts to artificial intelligence. In On Intelligence, Hawkins discusses his theories on the brain’s functions in detail. Numenta, a company he founded with Dileep George and Donna Dubinsky, aims to put these ideas to work in commercial and research applications.

Numenta is a platform company. The platform they develop is NuPIC (Numenta Platform for Intelligent Computing), a software toolkit essentially for building pattern classifiers. The fundamental concept behind NuPIC is called HTM (Hierarchical Temporal Memory). It postulates that the cortex learns to recognize patterns using a combination of two basic algorithms: hierarchical belief propagation, and the detection of invariants in a sequence of transformations in time. I won’t get into what this all means because there’s plenty of documentation on the Numenta website. I recommend browsing it if you find this interesting.

HTM is not just theory. Although NuPIC is in a very early stage, companies are applying NuPIC to a wide range of problems, including vision, voice recognition, finance, motion recognition (recognizing motion capture data to detect if a person is walking, running, sitting, etc) and games. This is just a small subset of its potential uses.

Using Nupic in its current state isn’t trivial. It provides the building blocks for HTM pattern classifiers, but application developers still have to do a good deal of work to tune the parameters of their HTM (How many nodes? How many levels in the hierarchy? How much training data to use? How many categories? What transformations to apply to the input over time to train the system?) to their problem domain. Also, some important features haven’t been implemented yet. For example, although NuPIC can be pretty effective at classifying images that contain a single object against a plain background (with enough training), it isn’t designed to recognize objects in images with noisy backgrounds or with multiple objects. (The problem of how to identify interesting objects in a scene is called the “attention” problem. To solve it you need to have a mechanism by which the top nodes could send feedback down to the bottom nodes. Hawkins said Numenta will tackle it in a future release.)

One reason I find Numenta so interesting is that I believe that NuPIC, or something like it, will play a role in the evolution of the Web. The current generation of web applications (Web 2.0 apps, if you will) is effective at aggregating massive amounts of data in different verticals (pictures, videos, bookmarks, status messages, paintings), slicing and dicing it in different ways, searching it, and displaying it in an organized fashion. Mashups provide additional context for the data gathered in the different silos of the web (Kosmix is a good example), but they don’t add any real “intelligence” to the mix, i.e. they don’t extract new knowledge from the data they aggregate. Numenta’s technology could be used to implement a new layer of intelligence on top of existing services by training it to recognize spacial and temporal pattens in the data they’ve collected. For example, imagine a Flickr API that let you submit an image and Flickr would tell you what the objects in the image are and where the picture was taken. Or a Facebook API for identifying the people in a picture. Or a Skype API for recognizing the speaker from a voice sample (creepy, I know). Or a HotOrNot API for automatically classifying the hotness of a person (ok, bad example :) ). Or a YouTube API for identifying the objects and events in a video clip. Or a icanhascheezburger API for automatically classifying the LOLness of a cat (well… maybe not :) ).

If this happens, maybe some day a mashup of these web services will be used to build something that resembles real AI. If (when?) someone manages to build a real-life WALL*E (great movie!), I think there’s a good chance its HTMs will be trained on the vast amounts of data gathered on the web.

Interesting times ahead. Stay tuned.

Web 2.0 and beyond: Erlang Exchange 2008

Roberto Saccon — 2008-06-30T02:53:58+00:00

The last two days I have spent at the Erlang Exchange Conference. It was great to met other Erlang hackers in person I just knew from their blog, emails or mailing list. Below the slides from my talk:

Programming in the 21st Century: Want to Write a Compiler? Just Read These Two Papers.

James Hague — 2008-06-29T06:00:00+00:00

Imagine you don't know anything about programming, and you want learn how to do it. You take a look at Amazon.com, and there's a highly recommended set of books by Knute or something with a promising title, The Art of Computer Programming, so you buy them. Now imagine that it's more than just a poor choice, but that all the books on programming are at written at that level.

That's the situation with books about writing compilers.

It's not that they're bad books, they're just too broadly scoped, and the authors present so much information that it's hard to know where to begin. Some books are better than others, but there are still the thick chapters about converting regular expressions into executable state machines and different types of grammars and so on. After slogging through it all you will have undoubtedly expanded your knowledge, but you're no closer to actually writing a working compiler.

Not surprisingly, the opaqueness of these books has led to the myth that compilers are hard to write.

The best source for breaking this myth is Jack Crenshaw's series, Let's Write a Compiler!, which started in 1988. This is one of those gems of technical writing where what's assumed to be a complex topic ends up being suitable for a first year programming class. He focuses on compilers of the Turbo Pascal class: single pass, parsing and code generation are intermingled, and only the most basic of optimizations are applied to the resulting code. The original tutorials used Pascal as the implementation language, but there's a C version out there, too. If you're truly adventurous, Marcel Hendrix has done a Forth translation (and as Forth is an interactive language, it's easier to experiment with and understand than the C or Pascal sources).

As good as it is, Crenshaw's series has two major omissions. The first is that there's no internal representation of the program at all. That is, no abstract syntax tree. It is indeed possible to bypass this step if you're willing to give up flexibility, but the main reason it's not in the tutorials is because manipulating trees in Pascal is out of sync with the simplicity of the rest of the code he presents. If you're working in a higher level language--Python, Ruby, Erlang, Haskell, Lisp--then this worry goes away. It's trivially easy to create and manipulate tree-like representations of data. Indeed, this is what Lisp, Erlang, and Haskell were designed for.

That brings me to A Nanopass Framework for Compiler Education [PDF] by Sarkar, Waddell, and Dybvig. The details of this paper aren't quite as important as the general concept: a compiler is nothing more than a series of transformations of the internal representation of a program. The authors promote using dozens or hundreds of compiler passes, each being as simple as possible. Don't combine transformations; keep them separate. The framework mentioned in the title is a way of specifying the inputs and outputs for each pass. The code is in Scheme, which is dynamically typed, so data is validated at runtime.

After writing a compiler or two, then go ahead and plunk down the cash for the infamous Dragon Book or one of the alternatives. Maybe. Or you might not need them at all.

Yariv's Blog: Twoorl Goes Multilingual

2008-06-29T03:24:47+00:00

Since its launch, Twoorl users have helped translate it to Spanish, German, French, Korean, Polish, Portuguese (Brazilian) and Russian. This is an awesome contribution from the Twoorl community. Big thanks to everyone who contributed a translation!

If you’re fluent in a language that Twoorl hasn’t been translated into and you’d like to contribute a translation for it, obtain the file twoorl_eng.erl, translate the english strings, and email me the modified file. (Please make sure the file is encoded in UTF-8.) If you’re familiar with Git, you can also clone the repository, make the changes, and send me a message through GitHub to pull your updates. Thanks in advance!

Namsisi: The Erlang Exchange

2008-06-26T14:44:23+00:00

I’m currently attending the Erlang Exchange conference in London.

So far I’ve attended two presentations in the main track before lunch and two presentations in the Tools & Gadgets track. I’m going to give a brief summary below. All the talks are being streamed on Yaika.

Main Track

Joe Armstrong on Software: Erlang & SMP

This is the talk of Joe Armstrong which kind of aligns with his book. It is quite basic and more aimed at Erlang beginners that experts. It contains a few interesting bits however, mostly the stuff about Symmetric Multi-Processing (SMP) and multi core processors. Not specifically Erlang related but at least message passing oriented, or as Joe calls it, Concurrency Oriented Programming (COP). All in all, “old stuff” (in a good sense) with some really interesting insights.

Introducing Erlang to Motorola: The Journey to Success

This talk was given by Nicholas Gunder and Torben Hoffman (which I enjoyed the OTP course together with) from Motorola (which interesting enough is now using Erlang, the language developed by their main competitor Ericsson). It was more of a manager talk than a technical talk, although they sometimes go all the way down to the hardware level, for example when talking about the Gumstix computers they are using.

This was a really nice talk, Nicholas and Torben are really funny guys. It gave a good insight into how they pushed Erlang from being a basically non-existant technology inside Motorola to a technology about which the managers ask “is there’s any other projects we could apply this to?” Their main conclusion was, I think, “have something finished to show.” That means, whenever pushing Erlang to management, show them something working. This is among other things a terrific illustrator on how productive Erlang is, which also compared to C is one of the selling points.

Tools & Gadgets Track

Erlang D-Trace

Given by Garry Bulmer, the talk was about intergrating Erlang with the OS X/Unix tracing facility DTrace. DTrace basically lets you trace on “anything” in the operating system and user space programs. You can trace on functions, accumulate data etc using your own DTrace scripts. Erlang already has an extremely powerful tracing and debugging facility. This integration aims to give a bigger picture on how your Erlang application and the rest of the OS and other applications interact and perform. Unfortunately due to GPL issues, DTrace is not available on Linux, only on OS X and other Unix variants.

What Bulmer and his team did, was to extend the Erlang VM with DTrace hooks so that tracing can be done on process creating and other Erlang primitives. The goal is to become a good complement to the internal Erlang tracing and debuggnig facilities.

Building Web Applications in Erlang

A talk about Erlang and web development given by Michal Slask and Xingdong Bian from Erlang Training & Consulting (disclaimer: I work for ETC). It was about the new web development platform that ETC is releasing. It takes its template philosophy from JSP, so that with tags such as list and choose you can call backend Erlang modules expanding to dynamic HTML code. All in a Model View Controller (MVC) fashion.

More about this soon so keep an eye on the Erlang Training & Consulting website and the Erlang mailing list!

Erlang & Ajax Web

Roberto Saccon gived a talk on integrating Ajax technologies with Erlang. The talk focused heavily on Coment and Roberts implementation in Erlang, ErlyComet. The talk is focusing on the implemantion and the results, both mistakes and successes. It is like a overview of both Ajax (for the Erlang community) and a collection of do’s and don’ts when using Ajax/Comet with Erlang. All in all, a web 2.0 presentation with an Erlang flavour.

The Rest of the Conference

This is it so far, maybe more posts will come later. Today it is just the Faxien & Sinan presentation I’m planning to attend.

EazyErl!: Parsing Binaries with erlang, lamers inside

rolphin — 2008-06-26T13:25:39+00:00

It's seems that there's a really high expectation on "parsing binaries" with erlang and reach others languages performance. For me this is a complete nonsense.
What's the meaning of "parsing a binary", a binary is not a text, it's a binary, a sequence of bytes...
A sequence must be defined by its length. You must know before reading anything, the size that'll be needed store what's coming.
Every crap software you can find has always prefered to use strcpy instead of memcpy. Whatever the language you use, you MUST know the size of what you're working with, this is not an advice this is mandatory.

From the post above, you can find that the only delimiter seems to be "\r\n". So if someone sends you 4Gb of data not ending with "\r\n" you'll keep reading it... (and of course blow your memory because this was not supposed to be)

While working at low level with C and flex scanners, I've always ask me this question: "What's the max size of the element I can accept ?". This simple question helps me build software that don't break with a simple 'perl print Ax60000' trick...

So is HTTP badly designed, because delimiters are "\r\n" and headers can spread on multiple lines ? The answer is absolutely YES.
Was'it difficult to build something more secure, using prefixed elements with their size ? The answer is absolutely NO ! (take ajp13 for example...)

Now that erlang is becoming more and more popular, lamers are lurking in the erlang direction. This is life, but will the erlang mailling list suffer from this ? The answer is yes :/

Someone with knowledge must not try to resolve someone's else problem, he must help him by asking the good question. (do you know the size a priori ?)

Why parsing binaries in java is faster than erlang ? Who cares, since parsing binaries is of course stupid !
Parsing real world protocol with erlang is lightning fast, both for writing and for executing. So teach lamers how to build real protocols and don't try help them with some trickery.

That's my rant for today :)

Process-one Blogs: Talk at Erlang Exchange Conference, London, UK on June 27th

2008-06-25T13:44:00+00:00

I will be talking in a London conference about ejabberd XMPP application server and Tsung benchmarking (HTTP and XMPP).

I will be at Erlang Exchange next friday (27th June, 2008), to talk about:

ejabberd for web 2.0 development: We have designed ejabberd as an XMPP application server that happens to be ideal to build web 2.0 applications and social networks. The talk will rely on two majors use cases: building a large scale community chat system (Major League of Baseball) and building a Twitter-like microblogging platform on top of ejabberd.
Tsung tutorial: This Tsung tutorial will present the fundamental concepts underlying Tsung architecture and will provide the required knowledge to write relevant benchmark scenarios. You will then learn how to create a scenario to benchmark an example web application and how to analyse the results of the benchmark.

I would be very glad to meet existing or future ejabberd and Tsung users next friday

More details: My talks and the conference programme.

Joe Armstrong: Invasion of Privacy

Joe Armstrong — 2008-06-24T13:53:06+00:00

On 18 June the Swedish Parliament passed a law giving sweeping new powers to the FRA (Swedish Defense Radio Establishment) allowing them to wiretap people in Sweden through phone conversations, email, text messages and more. All people in Sweden using electronic communication can have their communication monitored despite the fact that they are not suspected of committing any crime. In my view

Hypothetical Labs: Oh Frabjous Day!

kevin — 2008-06-24T10:16:09+00:00

(With humble apologies to Lewis Carroll, my favorite purveyor of kid-safe absurdity)

I’ve finally wrestled frabjous into a form approximating acceptable quality — you can check it out on GitHub here.

So, what is frabjous? It’s a wrapper around Erlang’s parse transforms with some of the sharp edges filed off. Frabjous insulates developers from the tedium of parse transforms in two ways.

First, it interfaces directly with the compiler and handles all of the low-level details around iterating over the AST and other parse transform muck. Frabjous knows all about multi-pass transforms and provides a persistent state mechanism for parse transformers so they can remember where they’ve been and what they’re doing.

Second, frabjous defines a new behavior, ast_transformer, to use when you want to write a frabjous-based parse transformer. Why is this a good thing? Well, ast_transformer defines a number of helpful functions like start/1 and is_complete/1 which model allow a transformer to track its state and implement multiple passes. ast_transformer also defines a standardized calling convention for transforming AST nodes using the function transform/3. No more inventing your own calling convention each time you write a parse transformer.

Frabjous integrates into the Erlang compiler via a header file named “frabjous.hrl“. Just use include_lib to include the header and your code is now frabjous-enabled. Wiring up AST transformers is done using the frabjous attribute.

A picture is worth a thousand words so let’s see what a source file which uses frabjous and its AST transformers looks like. This same is taken from frabjous/examples/src/pt_test.erl.

-module(pt_test).

-define(SERVER, ?MODULE).

-include_lib("frabjous/include/frabjous.hrl").

-frabjous([{remote_call, [{debug, true}]}]).

-behaviour(gen_server).

The example includes frabjous.hrl which “turns on” frabjous. It then goes on to configure a single frabjous AST transformer named remote_call and runs it in debug mode. When frabjous AST transformers are run in debug mode they emit before and after snapshots of the AST. This is indispensable when you’re debugging a transformer, either on its own or when it’s part of a chain of transformers. Yep, frabjous understands how to chain a series of transformers together.

The remote_call transformer takes attribute entries like this:

-call_gen_server([{server, ?SERVER}, {name, add}, {vars, [x,y]}]).

and turns them into functions like this, complete with matching export statements:

add(X, Y) -> gen_server:call(?SERVER, {add, X, Y}).

Why? Because I got really, really, really tired of writing the same message sending wrapper function over and over each time I write a gen_server.

These are still early days for frabjous so the code is a bit rough. There are several things I want to improve soon including:

Unit tests (!!!)
Teaching frabjous how to handle more than the first level of the AST nodes
Adding more functions to frabjous_helper to make AST node generation easier

If you’re interested in frabjous, please check it out and let me know what you think!

Dukes of Erl: Tokyocabinet and Mnesia

Paul Mineiro — 2008-06-23T22:25:59+00:00

As Daisy has already indicated, it is now possible to plug arbitrary storage strategies into Mnesia. For those who are familiar with mnesia_access, this is different; mnesia_access only covers reads and writes, not schema manipulations, and has other deficiencies that it render it useless for adding a new storage type in practice (what mnesia_access is great for is changing the semantics of mnesia operations, e.g., mnesia_frag). This project lets you make tables that are essentially indistinguishable from the built-in mnesia table types (ram_copies, disc_copies, disc_only_copies).

Anyway our goal was to get a good on-disk ordered_set table type, since we've found ordered_sets very useful for the kinds of problems we're solving but we're tired of being limited by memory. After looking around for a while Tokyocabinet emerged as our favorite for underlying implementation. We considered BDB and libmysql, but Tokyocabinet seemed simple and faster, and had a more accommodating license. So we ported Tokyocabinet to Erlang and then used the above storage API to connect to Mnesia.

As a side benefit Tokyocabinet might also be preferred to dets even for set-type applications because of the lack of file size limit and high performance. Tokyocabinet actually has a set-type storage strategy that we'd like to define an Erlang Term Store for, but as of this post, the set-type store doesn't support cursor positioning based upon a key, which makes the implementation of next tedious (although not impossible). So I'm waiting on the author to add that call; if that happens, we could have a nicer on-disk set-type table as well.

Everything is available on google code: mnesiaex (storage API) and tcerl (erlang port of Tokyocabinet).

Also Daisy (aka Joel Reymont) is really nice to work with.

Hypothetical Labs: I Wrote What?

kevin — 2008-06-22T12:59:47+00:00

Late last evening I was finally able to carve out some time to work on frabjous. Frabjous, for those who missed out on my only other post about it, is a parse transform framework for Erlang aimed at making writing parse transforms easier.

But that’s not important right now.

The important part is I was ready to release it last night and decided against it. It’s got unit tests, examples, and nice descriptive comments in all the public functions (and some of the private ones, too). But the code is, well, utter crap.

I wrote the original core of frabjous the day I flew out to Engine Yard’s offices. I figured with a day of air travel and my laptop I could get a lot of hacking done. I did. A lot of hacking but not much designing. The up side is I understand parse transforms really well. The down side is the code is way less organized and efficient than it could be.

Plain and simple, the code isn’t pretty. I have a simple benchmark for code prettiness which works equally well for OOP and functional languages. If I have to stare at the code for more than a minute or two to understand a given function, method, or class, then the code isn’t pretty. That’s it. Think of it as inscrutability quotient. The cleaner the code, aka pretty, the less time I have to spend understanding it. The longer I have to stare before I understand it the less pretty the code becomes. I wrote frabjous and I had several multi-minute “WTF?!” moments.

So, frabjous may or may not be released today. But when it is released it’ll be pretty even if i have to go medieval on it.

EazyErl!: Quick Tip, list join

rolphin — 2008-06-20T10:20:00+00:00

Another "lists:join" :


Join = fun([X|Rest], D) -> 
  [ X | [ [D,E] || E <- Rest ] ] 
  end.

Usage:


io:format("~s~n", [Join(["a", "b", "cde", "h", "k", "lm"], $,)]).
a,b,cde,h,k,lm
ok

Process-one News: Major League of Baseball teams up with ProcessOne to deploy large scale chat system

2008-06-20T08:24:00+00:00

The MLB.com site offers a wealth of information and services, including Game Day, a flash-based audio application that enables fans to follow every pitch as it happens.

MLB.com wanted to make its web site as ‘sticky’ as possible so that fans would want to come to the site, get involved and stay for a long period of time. It therefore decided to develop a new instant messaging facility to enhance the features of Game Day. At first, the organisation had some concerns about the project. A previous attempt to develop instant messaging had been unsuccessful because the solution implemented had been neither sufficiently scalable to meet demand nor visually exciting enough. MLB.com therefore knew that the success of the new venture would depend heavily on its choice of platform. The organisation had to find an instant messaging server that would give it both the flexibility to innovate and the confidence to grow.

In the previous year, MLB.com’s Game Day application had attracted over 85 million users. MLB.com knew how many concurrent visitors it attracted to its site on game days and it knew what its average year-on-year growth was. It also knew how many fans had used its earlier instant messaging service. Using these figures as the basis for some solid statistical analysis, it anticipated that demand for its new chat facility would not only be huge – but would also grow steadily over time.

Providing quality support

MLB.com interviewed a lot of potential partners for the project and carried out practical product evaluations. However, the solutions that it initially tested failed to provide the flexibility and scalability that it needed. Then, MLB.com met ProcessOne at a conference in California. “We were immediately impressed by ProcessOne’s offering,” says Joe Choti, CTO of MLB.com. “ProcessOne was head and shoulders above everyone else.”

ProcessOne provides high performance instant messaging servers that are based on ejabberd, an open source technology that is mainly written in the Erlang programming language. As it runs in a cluster mode, ejabberd is highly robust and can be easily expanded in size, making it ideal for large-scale deployments. MLB.com was familiar with ejabberd and quickly identified that the ProcessOne Instant Messaging Server could meet all of its requirements.

“ejabberd is a full featured jabber server compared to the other products that are in the market,” explains Christian Gough, system administrator at MLB.com. “The ProcessOne Instant Messaging Server offered every feature that we desired. By leveraging the scalability of Erlang, the solution allows us to easily expand and adapt the server whenever we want to in the future.”

The CEO of ProcessOne travelled to New York to work directly with MLB.com’s internal team and specify the precise requirements of the solution. “His help was extremely timely and useful,” says Gough. “The quality of support that we have received from ProcessOne has been very, very good.”

Creating an innovative service

For MLB.com, the visual concept of the solution was just as important as the underlying technology. The organisation wanted to create a Game Day chat facility that would give fans the look and feel of experiencing a live game with other fans. It therefore based its new instant messaging service on a virtual ballpark. Fans can visit different sections of the online ballpark during a game to join different chat sessions on different topics with different groups of people.

In one part of the virtual ballpark, fans of the home team can be discussing the state of the pitch or the decisions of the coach. In other parts of the ballpark, fans from opposing teams can get together to argue the merits of different players. “This is by-appointment chat,” explains Choti. “The ballpark opens when the game begins and closes when the game ends. An automated session manager feature manages the chat rooms and ushers users out at the end of a game, just as they would be ushered out of a real ballpark.”

As part of the project, ProcessOne developed a customised module for MLB.com to allow its ejabberd-based instant messaging server to interact with the company’s existing web services and authenticate users against its user database. “Users can now log in once to MLB.com and use a range of different services, including the chat facility, without having to log in again to a separate system,” explains Gough.

The entire instant messaging solution, including the customised module, was deployed in a very short timeframe. “ProcessOne took us from conception to deployment in just three weeks,” recalls Choti.

Meeting requirements by 110%

MLB.com is delighted with its new chat service. Over the course of each season, it now facilitates different instant messaging sessions at as many as 25,000 venues. This Game Day chat facility is helping to attract more users to the site and is encouraging visitors to stay on the site longer. This gives the company more opportunities to cross-sell and up-sell from its range of merchandise and services. In addition, MLB.com anticipates that the chat service will also help it to attract higher value advertisers to its site.

The feedback from users has been very positive. “Users absolutely love it,” says Choti. “We believe that we now offer one of the most exciting chat facilities available today.”

The ProcessOne Instant Messaging Server is performing very strongly. “The use of Erlang and clustering in the solution architecture not only ensures scalability, but also adds to the solution’s stability and flexibility,” says Gough. “The ProcessOne Instant Messaging Server offers a plethora of features and we have been able to easily extend it to meet our unique requirements.”

“The ProcessOne Instant Messaging Server has met our requirements 110 per cent,” adds Choti. “From the very first day that we brought the server online, we have not had a single problem with it. It’s a champ!”

Now that its Game Day chat facility is up and running, MLB.com doesn’t plan to stand still. Every year it tweaks and enhances its web site with new features. “We are very confident that the ProcessOne platform is scalable and robust enough to support us in whatever direction we go,” says Choti.

He concludes: “Our success has been based, not only on the technical solution provided by ProcessOne, but also on the personal support provided by ProcessOne. ProcessOne is not a vendor; it’s a partner and that’s a very important distinction.”

References

Download the case study (PDF): MLB.com scores a home run with instant messaging from ProcessOne

Click to access more case studies

Hypothetical Labs: Episode 4 Errata

kevin — 2008-06-19T17:27:41+00:00

See here for the gory details.

*sigh*

A hearty thanks to Jon Gretar for finding it. I’d much rather know about a bug and fix it than let it languish in public.

Also, if you haven’t told me what you’d like to see in the next episodes, please drop a comment on here. Right now I’m leaning towards doing a RESTian API via Yaws or Mochiweb (haven’t decided which), followed by unit testing, and then OTP. But there’s still time to make your case! Comment today!

Hypothetical Labs: Erlang By Example: Episode 4

kevin — 2008-06-18T18:48:49+00:00

Mike Clark and his merry band of busy hamsters have published episode 4 in my screencast series Erlang By Example. The episode is the longest so far, clocking in at an epic 39 minutes. I chose mnesia as the episode’s focus and used it to implement store and forwarding for messages in the chat system. If you’re interested in using mnesia, wondering how to write qlc queries, or just curious what a non-trivial Erlang application looks like, I recommend checking this episode out.

I’m also soliciting comments on what topic I should cover next in the series. The choices and their expected durations are, in no particular order:

Using OTP (2-3 episodes)
Adding a REST endpoint using either Mochiweb or Yaws (2 episodes)
Writing chat clients in another language like Python or Ruby (2-3 episodes)
Clarification: This is meant to illustrate how to interface other languages to Erlang via Erlang’s ports or other libraries such as erlectricity
Unit testing and dialyzer (1 episode)
Insert the burning topic I missed here

Please comment on this post and let me know what you’d like to see next!