Erlang/OTP News feeds
Welcome to Planet Trapexit. On this page, you will find the latest entries from all the news feeds currently in our database. If you know feeds that would be suitable here (that is, that relate to Erlang/OTP in any way, please add them here.
February 01, 2010
Erlang Factory News
New confirmed speakers for the Erlang Factory SF Bay Area
We have new confirmed speakers: Bjarne Däcker (Manager of the CSLab at Ericsson - the birthplace of Erlang), Sean Cribbs (Neotoma creator and web developer), Chad DePue (Entrepreneur and creator of ErlangInside.com), Sadek Drobi (Consultant and programming languages Evangelist at Valtech), Rob King (Erlang Enthusiast in DVLabs), Todd Lipcon (Software Engineer in Cloudera), Martin Logan (Erlang author and Faxien and Sinan committer), Eric Merritt (Erlang author and Erlware Committer), Dean Wampler (Scala fanboi and co-author of "Programming Scala"), Kenny Stone(Trading Systems Expert), Jack Moffit (XMPP expert and co-founder of Collecta), Thomas Arts (Professor and co-founder of QuviQ AB) and John Hughes (Inventor of QuickCheck and Erlanger of the year).
LShift on Erlang
RabbitMQ-shovel: Message Relocation Equipment
In several applications, it’s very useful to be able to take messages out of one RabbitMQ broker, and insert them into another. Many people on our mailing list have being asking for such a shovel, and we’ve recently been able to devote some time to writing one. This takes the form of a plugin for Rabbit, and whilst it hasn’t been through QA just yet, we’re announcing it so people who would like to play and even suggest further features for inclusion can do so sooner rather than later.
The shovel is written on top of the Erlang client. It supports both direct and network connections to nodes, SSL support, the ability to declare resources on nodes it connects to, basic round-robinrabbit balancing of both source and destination nodes, and allows you to configure many parameters controlling how messages are consumed from the source, and how they’re published to the destination. Multiple shovels can be specified, their statuses queried, and shovels can repeatedly reconnect to nodes in the event of failure.
The plugin is available from http://hg.rabbitmq.com/rabbitmq-shovel/, and is released under the MPL v1.1. There is a README included which contains full documentation. This is replicated below.
RabbitMQ-shovel
Introduction
This is a plug-in for RabbitMQ that shovels messages from a queue on one broker to an exchange on another broker. The two brokers may be the same. The plug-in allows several shovels to be specified at the same time. Each shovel may have a number of source and destination brokers specified, and one of each is chosen whenever the shovel attempts to make a connection: this permits simple round-rabbit load balancing.
Resources can be declared upon connection to both the source and destination brokers, and parameters can be specified for both the reception and publishing of messages.
Requirements
Currently, you must build the server from source, under branch bug16653. You must also have checked out the rabbitmq-public-umbrella hg repository, and have the rabbitmq-erlang-client built. From scratch, the following commands should build RabbitMQ with the shovel plug-in:
hg clone http://hg.rabbitmq.com/rabbitmq-public-umbrella cd rabbitmq-public-umbrella hg clone http://hg.rabbitmq.com/rabbitmq-codegen hg clone http://hg.rabbitmq.com/rabbitmq-erlang-client hg clone http://hg.rabbitmq.com/rabbitmq-server hg clone http://hg.rabbitmq.com/rabbitmq-shovel cd rabbitmq-server hg up -C bug16653 make -j mkdir -p plugins cd plugins ln -s ../../rabbitmq-erlang-client ln -s ../../rabbitmq-shovel cd ../../rabbitmq-erlang-client make cd ../rabbitmq-shovel make cd ../rabbitmq-server ./scripts/rabbitmq-activate-plugins make cleandb run
Configuration
The RabbitMQ configuration file specifies the shovel
configurations. This exists by default, in
/etc/rabbitmq/rabbitmq.config under Linux systems,
%RABBITMQ_BASE%\rabbitmq.config under Windows or somewhere else under
OS X. This file configures both RabbitMQ-server and all the plugins
installed in it. It is an Erlang-syntax file of the form:
[{section1, [section1-config]},
{section2, [section2-config]},
…
{sectionN, [sectionN-config]}
].
thus a list of tuples, where the left element of each tuple names the applications being configured. Don’t forget the last element of the list doesn’t have a trailing comma, and don’t forget the full-stop is needed after closing the list. Hence if you configure RabbitMQ-server and the RabbitMQ-shovel, then the configuration file may have a structure like this:
[{rabbit, [configuration-for-RabbitMQ-server]},
{rabbit-shovel, [configuration-for-RabbitMQ-shovel]}
].
A full example of the shovel configuration is:
{rabbit_shovel,
[{shovels,
[{my_first_shovel,
[{sources, [{brokers,
["amqp://fred:secret@host1.domain/my_vhost",
"amqp://john:secret@host2.domain/my_vhost"
]},
{declarations,
['queue.declare',
{'queue.bind',
[{exchange, "my_exchange">>},
{queue, >}]}
]}]},
{destinations, [{broker, "amqp://"},
{declarations,
[{'exchange.declare',
[{exchange, "my_exchange">>},
{type, "direct">>},
durable]}
]}]},
{queue, >},
{qos, 10},
{auto_ack, false},
{tx_size, 0},
{delivery_mode, keep},
{publish_fields, [{exchange, "my_exchange">>},
{routing_key, "from_shovel">>}]},
{reconnect, 5}
]}
]
}]
}
Firstly, all shovels are named. Here we have one shovel, called
‘my_first_shovel’. We can have multiple shovels if you wish. Every
shovel must have all sub-fields specified: sources, destinations, qos,
auto_ack, delivery_mode, publish_fields, reconnect.
Sources and Destinations
Sources and destinations specify respectively where messages are
fetched from and delivered too. One of ‘broker’ and ‘brokers’ must be
specified, and ‘broker’ is simply shorthand for when only one broker
needs specifying. Using ‘brokers’ allows a list of brokers to be
specified: whenever the connection to a broker is lost, another one is
chosen at random from the list and a connection attempt is made to
that. The syntax for broker URIs is:
amqp://username:password@host:port/vhost
If username or password are omitted, the default values of guest and
guest are used. If the vhost is omitted, the default value of / is
used. If the host is omitted, then the plugin uses the “direct”
connection internally rather than a network connection: this means it
connects to the RabbitMQ-server node on which it is running without
going through the network stack. This is much more efficient. If port
is omitted then the default value is used (5672 or 5671 if SSL is
used).
SSL is implemented, for which additional parameters are needed:
amqps://username:password@host:port/vhost?cacertfile=/path/to/cacert.pem&certfile=/path/to/certfile.pem&keyfile=/path/to/keyfile.pem&verify=verifyOption&fail_if_no_peer_cert=failOption
(note, this is a single line)
All five parameters (3 paths: cacertfile, certfile and keyfile; 2
options: verify, fail_if_no_peer_cert) must be specified. See the SSL
guide at http://www.rabbitmq.com/ssl.html#configure-erlang for details
of SSL in RabbitMQ in general and specifically for the Erlang client
(on which the shovel is built).
Note that SSL cannot be used with the direct connection (i.e. a host
must be specified when using SSL), and that it is preferable to use
the non-SSL direct connection when connecting to the same node that’s
running the shovel.
Resource Declarations
Both sources and destinations can have an optional ‘declarations’
clause. The value of this is a list, consisting of AMQP Methods. If
default values are sufficient, then the method name alone can be
specified - e.g. ‘queue.declare’. If parameters need to be set then
the method should be given as a tuple, with the right hand side a
proplist specifying which fields need altering from their default
values. E.g:
{'exchange.declare',[{exchange, "my_exchange">>},
{type, "direct">>},
durable]},
One very useful feature here is the Most-Recently-Declared-Queue
feature, in which RabbitMQ remembers the name of the most recently
declared queue. This means that you can declare a private queue, and
then bind it to exchanges without ever needing to know its name.
queue :: binary
This parameter specifies the name of the queue on the source brokers
to consume from. This queue must exist. Use the resource declarations
to create the queue (or ensure it exists) first. Note again that the
Most-Recently-Declared-Queue feature can be used here, thus an
anonymous queue can be used: use <<>> to indicate the
Most-Recently-Declared-Queue.
qos :: non-negative-integer
The shovel consumes from a queue. The QoS controls how many messages
are sent to the shovel in advance of the message the shovel is
currently processing.
auto_ack :: boolean
Setting this to ‘true’ turns on the no_ack flag when subscribing to
the source queue.
tx_size :: non-negative-integer
When set to 0, transactions are not used. Other values make publishes
transactional, with a commit every N messages. In lieu of the auto-ack
option, when transactions are not used, messages are acknowledged to
the source immediately after every publish. When transactions are
used, acks are only issued to the source on receipt of the commit-ok
message from the destination. This can thus be used to guarantee that
messages are only acknowledged (and thus forgotten about by the source
broker) when they are guaranteed to have been received by the
destination broker.
delivery_mode :: ‘keep’ | 0 | 2
This affects the delivery_mode field when publishing to the
destination. A value of ‘keep’ means that the same delivery_mode
should be used as when the message was originally published to the
source broker. 0 and 2 override the original setting.
publish_fields
This is a list of tuples which override fields in the publish method
when publishing to the destination. This can be used to direct
messages to a particular exchange on the destination, for example, or
change the routing key. By default, the routing key of the message as
it is received by the shovel is passed through, but this can be
overridden as necessary.
reconnect :: non-negative-integer
When an error occurs, the shovel will disconnect from both the source
and destination broker immediately. This will force uncommitted
transactions at the destination to be rolled back, and delivered but
unacknowledged messages from the source to be requeued. The shovel
will then try connecting again. If this is unsuccessful, then it’s not
a good idea for the shovel to very quickly and repeatedly try to
reconnect. The value specified here is the number of seconds to wait
between each connection attempt.
Note that if set to 0, the shovel will never try to reconnect: it’ll
stop after the first error.
Obtaining shovel statuses
From the broker Erlang prompt, call
rabbit_shovel_status:status(). This will return a list, with one row
for each configured shovel. Each row has three fields: the shovel
name, the shovel status, and the timestamp (a local calendar time of
{{YYYY,MM,DD},{HH,MM,SS}}). There are 3 possible statuses:
’starting’: The shovel is starting up, connecting and creating resources.
‘running’: The shovel is up and running, shovelling messages.
{’terminated’, Reason}: Something’s gone wrong. The Reason should give a further indication of where the fault lies.
by matthew at February 01, 2010 12:13 PM
January 31, 2010
Simon Willison's Weblog
Hot Code Loading in Node.js
Hot Code Loading in Node.js. Blaine Cook’s patch for Node.js that enables Erlang-style hot code loading, so you can switch out your application logic without restarting the server or affecting existing requests. This could make deploying new versions of Node applications trivial. I’d love to see a Node hosting service that allows you to simply upload a script file and have it execute on the Web.
January 30, 2010
Trapexit's Erlang Blog Filter
Erlang Factory SF Bay Area 2010
Interested in Erlang? You might consider attending Erlang Factory SF Bay Area 2010. Below is a message that Francesco Cesarini, conference organizer and co-author of the most excellent book Erlang Programming, sent to the erlang-questions list yesterday providing more information about the conference, especially pointing out that the Very Early Bird registration deadline is tomorrow (Sunday January 31). Hope to see you there!
Hi All,
a note to say that we are almost done with the programme for the 2010 SF Bay Area Erlang Factory. This year, we are lucky to have keynote speakers such as Joe Armstrong, Bjarne Dacker, Kenneth Lundin and Steve Vinoski. They will be giving four of the 35 scheduled talks on the 25th and 26th of March in the San Francisco Bay Area. The almost complete programme is available here:
http://erlang-factory.com/conference/SFBay2010/programme
The conference will be preceded by three days of University courses taught by experts such as Simon Thompson, John Hughes, Thomas Arts, Henry Nystrom and Kevin Smith. Come and learn Erlang, OTP, QuickCheck or Web Development with Erlang. More information on the courses are here:
http://erlang-factory.com/conference/SFBay2010/university
This *Sunday* (January 31st) is the deadline for the very early bird deadline. Register by Sunday night and save $400 on the on-site registration price.
The conference hotel and venue is the SF Airport Hilton, a short BART / Caltrain ride from SF and the Valley. We have secured a very competitive price of US$109 per room and night at the conference hotel, this being one of the reasons for us choosing it. The other is the lower price of the venue, allowing us to pass on the savings to the delegates through a higher very early bird discount. We are planning an ErlLounge open to everyone who can’t make the two days on the 25th, and hope we will be able to surpass last year’s success. If you have thoughts or questions, you are welcome to drop me a line.
Hope to see you all there!
Francesco
–
http://www.erlang-solutions.com
by steve at January 30, 2010 09:29 PM
Erlang Inside
Erlang Factory SF Bay 2010 – Francesco Cesarini on the Conference and Old-School vs New-School Erlangers
The Erlang Factory’s 2010 conference is March 25th and 26th in San Francisco, with the university three days before, starting the 22nd. The conference is at the Hilton San Francisco Airport, with three tracks, each on a different theme. The number of tracks gives attendees an unusually broad set of choices for talks. When attending [...]
by Chad DePue at January 30, 2010 06:19 PM
Programming in the 21st Century
What to do About Erlang's Records?
The second most common complaint about Erlang, right after confusion about commas and semicolons as separators, is about records. Gotta give those complainers some credit, because they've got taste. Statically defined records are out of place in a highly dynamic language. There have been various proposals over the years, including Richard O'Keefe's abstract syntax patterns and Joe Armstrong's structs. Getting one of those implemented needs the solid support of the Erlang system maintainers, and it's understandably difficult to commit to such a sweeping change to the language. So what are the alternatives to records that can be used right now?
To clarify, I'm really talking about smallish, purely functional dictionaries. For large amounts of data there's already the gb_trees module, plus several others with similar purposes.
In Python, a technique I often use is to return a small dictionary with a couple of named values in it. I could use a tuple, but a dictionary removes the need to worry about order. This is straightforward in Erlang, too:
fun(length) -> 46; (width) -> 17; (color) -> sea_green end.Getting the value corresponding to a key is easy enough:
Result(color)This is handy, but only in certain situations. One shortcoming is that there's no way to iterate through the keys. Well, there's this idea:
fun(keys) -> [length, width, color]; (length) -> 46; (width) -> 17; (color) -> sea_green end.Now there's a way to get a list of keys, but there's room for error: each key appears twice in the code. The second issue is there's no simple way to take one dictionary and create a new one with a value added or removed. This road is becoming messy to go down, so here's more data-driven representation:
[{length, 46}, {width, 17}, {color, sea_green}]
That's just a list of key/value pairs, which is searchable via the fast, written-in-C function lists:keyfind. New values can be appended to the head of the list, and there are other functions in the lists module for deleting and replacing values. Iteration is also easy: it's just a list.
We still haven't bettered records in all ways. A big win for records, and this is something few purely functional data structures handle well, is the ability to create a new version where multiple keys get different values. For example, start with the above list and create this:
[{length, 200}, {width, 1400}, {color, sea_green}]
If we knew that only those three keys were allowed, fine, but that's cheating. The whole point of dictionaries is that we can put all sorts of stuff in there, and it doesn't change how the dictionary is manipulated. The general solution is to delete all the keys that should have new values, then insert the new key/value pairs at the head of the list. Or step through the list and see if the current key is one that has a new value and replace it. These are not linear algorithms, unfortunately. And you've got the same problem if you want to change multiple values in a gb_tree at the same time.
What I've been using, and I admit that this isn't perfect, is the key/value list approach, but forcing the lists to be sorted.This allows the original list and a list of changes to be merged together in linear time. The downside is that I have to remember to keep a literal list in sorted order (or write a parse transform to do this for me).
There's still one more feature of records that can't be emulated: extracting / comparing values using Erlang's standard pattern matching capabilities. It's not a terrible omission, but there's no way to dodge this one: it needs compiler and runtime system support.
by James Hague at January 30, 2010 06:00 AM
January 29, 2010
Programming in the 21st Century
Nothing Like a Little Bit of Magic
Like so many other people, I was enthralled by the iPad introduction. I haven't held or even seen an iPad in person yet, but that video hit me on a number of levels. It's a combination of brand new hardware--almost dramatically so--and uses for it that are coming from a completely different line of thinking. I realized it's been a long time since I felt that way about the introduction of a new computer.
I remember the first time I tried a black and white 128K Mac in a retail store. A mouse! Really tiny pixels! Pull-down menus! Graphics and text mixed together! And the only demo program was what made the whole experience click: MacPaint.
I remember when the Atari 520ST was announced. Half a megabyte of memory! Staggering amounts of power for less than $1000! A Mac-like interface but in full color! Some of the demos were simple slideshows of 16-color 320x200 images, done with a program called NeoChrome, but I had never seen anything like them before.
I remember when the Amiga debuted that same year. Real multitasking! Digitized sound! Stereo! Hardware for moving around big bitmaps instead of just tiny sprites! Images showing thousands of colors at once! Just the bouncing ball demo was outside what I expected to ever see on a computer. And there was a flight-sim with filled polygon graphics. Behind the scenes it was the fancy hardware enabling it all, but it was the optimism and feeling of new possibilities that fueled the excitement.
I remember when the Macintosh II came out, with 24-bit color and impossibly high display resolutions for the time. It seemed like a supercomputer on a desk, the kind of thing that only high-end graphics researchers would have previously had access to.
PCs never hit me so unexpectedly and all at once, but there were a few years when 3D hardware started appearing where it felt like the old rules had been thrown out and imagining the future was more important than looking back on the same set of ideas.
Am I going to buy an iPad? I don't know yet. I never bought most of the systems listed above. But I am glad I've been experiencing that old optimism caused by a mix of hardware and software that suddenly invalidates many of the old, comfortable rules and opens up territory that hasn't been endlessly trod upon.
by James Hague at January 29, 2010 06:00 AM
January 28, 2010
Process-one Blogs
Google Wave XMPP notification gateway
At ProcessOne, we have developed an experimental notification Google Wave XMPP gateway.
I wrote it because, in the current state of Wave usage, my contacts always had to write me an XMPP message or send me an email to explain that I have content available.
I felt this was for me a large deterrent for using it and quite paradoxical in a real-time web world.
To solve this problem and benefit from the largely used XMPP protocol for notification, I wrote a Google Wave gateway. For now, it is quite raw because it relies on GWT client protocol and lacks any mechanism from Google to do delegated authentication. However, we are going to implement a more standard client protocol and delegated authentication as Google makes those features available.
Anyway, here is a small demo of how to use it to give you an idea of the user experience.
It is deployed on talkr.IM server, so you can give it a try.
Enjoy!
by Mickaël Rémond at January 28, 2010 10:00 AM
January 27, 2010
Erlang Factory News
Very Early-Bird discount ends this weekend!
Don't miss out Very Early-Bird discount for the Erlang Factory SF Bay Area 2010. Register by midnight this Sunday, 31st January 2010 and save $400!
The discount applies to the conference and to the Erlang University courses: Erlang Express, Erlang OTP Express, QuickCheck for Erlang developers and Web Programming with Erlang.
Process-one Blogs
Talkr.IM service update: Google Wave gateway
The Talkr.IM XMPP/Jabber service will undergo today a service update, adding a gateway to Google Wave.
ProcessOne's public XMPP service Talkr.IM will receive a new feature for all users: a Google Wave gateway. This gateways will enable Talkr.IM XMPP users to be notified of modified Waves.
Our Google Wave gateway will notify the subscribed users of all the changes in a Wave, by providing an direct link to a maximized Wave window. The other windows (Navigation, Contacts, Search) will be minimized.
We are planning a service interruption at:
- 10:00 UTC
- 11:00 Paris
- 13:00 Moscow
- 02:00 Los Angeles
- 05:00 New York
- 19:00 Tokyo
We will remind our online users a few minutes before.
Update: The GWave transport is online, feel free to test and play with it! Warning, this is an alpha software, you my meet bugs.
by Nicolas Vérité at January 27, 2010 10:30 AM
January 25, 2010
Caoyuan's Blog
Progress of Migrating AIOTrade to Scala #2
My next step is to push AIOTrade to another level of UE, with hotkey for symbols, indicators etc, with better UI.
So far, my Scala adventure went almost smoothly, I'm thinking about the data exchange between client and data source, maybe the actors from Scala, LiftWeb or AKKA will bring some fresh atmosphere.
by dcaoyuan at January 25, 2010 10:17 AM
Programming in the 21st Century
Flickr as a Business Simulator
Flickr came along exactly when I needed it.
In 2004, I knew I was too immersed in technical subjects, and Flickr motivated me to get back into photography as a change of pace. I loved taking photos when I was in college (mostly of the set-up variety with a couple of friends), but I hardly touched a camera for the next decade. When I first found out about Flickr, not long after it launched, the combination of having a new camera and a potential audience provided me with a rare level of inspiration. I remember walking around downtown Champaign on June 1, 2004, spending two hours entirely focused on taking photos. I didn't have a plan, I didn't have a preferred subject; I just made things up as I went.
This is one of my favorites from that day:
Flickr was pretty raw back then. You could comment on photos, but there wasn't the concept of favoriting a good shot or the automated interestingness ranking. As those systems went into a place, it was easier to get feedback about the popularity of photos. Why did people like this photo but not this other one? How does that user manage to get dozens of comments per shot?
It took me a while to recognize some of the thought patterns and feelings that I had once I started paying attention to the feedback enabled by Flickr. They were reminiscent of feelings I had when I was an independent developer. I was rediscovering lessons which I had, at great expense, learned earlier. Now I can, and will, recount some of these lessons, but that in itself isn't very useful or exciting. Anyone can recite pithy business knowledge, and anyone can ignore it too, because it's hard to accept advice without it being grounded in personal experience. The important part is that you can experience these lessons firsthand by using Flickr.
Create an account and give yourself a tough goal, such as getting 50,000 photostream views in six months or getting 500 photos flagged as favorites. And now it's a business simulator. You're creating a product--a pool of photographs--which is released into the wild and judged by people you don't control. The six month restriction simulates how long you can survive on your savings. Just like a real business, the results have a lot to do with the effort you put forth. But it's not a simple translation of effort into success; it's trickier than that.
Now some of the lessons.
You don't get bonus points for being the small guy. It sounds so appealing to be the indie that's getting by on a shoestring. Maybe some customers will be attracted to that and want to stick to the man by supporting you. On Flickr you're on the same playing field as pros with thousands of dollars worth of equipment and twenty years' experience. You can still stand out, but don't fool yourself into thinking that your lack of resources that's used an excuse for lower quality is going to be seen as an endearing advantage.
While quality is important, keep the technical details behind the scenes. Just as no one really cares what language your application is written in, no one really cares what lens you took a photograph with or what filter you used. Be wary of getting too into the tech instead of the end result.
What you think people want might not be what people want. This one is tough. Are you absorbed in things that you think are important but are irrelevant, or even turn-offs, to your potential audience? This is the kind of thing that a good record producer would step in and deal with ("Just stop with the ten minute solos, okay?"), but it can be difficult to come to these realizations on your own, especially if you're seeing the problems as selling points.
Don't fixate on why you think some people are undeservedly successful. All it does it pull you away from improving your own photos/products as you pour energy into being bitter. Your personal idea of taste doesn't apply to everyone else. There may be other factors at work that you don't understand. Just let it go or it will drag you down.
But don't take my word for it. Just spend a few months in the simulator.
by James Hague at January 25, 2010 06:00 AM
January 22, 2010
LShift on Erlang
Plugin exchange types for RabbitMQ
An obvious extension point for an AMQP broker is the addition of new types of exchange. An exchange type essentially represents an algorithm for dispatching messages to queues, usually based on the message’s routing key, given how the queues are bound to the exchange — it’s a message routing algorithm.
At a minimum, supporting new exchange types requires only some scaffolding to plug in to (an exchange type registry) and a hook for routing messages. However, this wouldn’t support some more interesting use cases, and in particular it didn’t support our motivating use case. Exchange types that want to keep their own state need to be initialised, and be notified about other lifecycle events.
The branch bug22169 of RabbitMQ supports plugin exchange types, by providing a behaviour for exchange type modules to implement, and an exchange type registry to map a module to a type (i.e., what the client supplies in the type field of exchange.declare).
The behaviour requires exported hooks for validating exchange declarations, creating exchanges, recovering durable exchanges, publishing to an exchange (this is where the routing comes in), maintaining bindings, and deleting an exchange. RabbitMQ continues to maintain the database of exchanges and bindings, and calls the hooks after it’s done its own bookkeeping.
For simplicity, the hooks are not called atomically with the bookkeeping; so, it is possible for instance to publish to a new exchange for which the hook has not completed. However, provided there are no asynchronous operations in the hook implementation, the hook will have completed by the time the OK message is sent to the client. This is no more racey than AMQP itself, with consistency at the channel level where operations follow a single thread of control.
Here’s an example that simply io:formats things as they happen:
-module(rabbit_exchange_type_debug).
-include("rabbit.hrl").
-behaviour(rabbit_exchange_behaviour).
-export([description/0, publish/2]).
-export([validate/1, create/1, recover/2, delete/2, add_binding/2, delete_binding/2]).
-export([register_debug_types/0]).
-include(”rabbit_exchange_behaviour_spec.hrl”).
-rabbit_boot_step({debug_exchange_types,
[{description, "debugging exchange types"},
{mfa, {?MODULE, register_debug_types, []}},
{post, rabbit_exchange_type},
{pre, exchange_recovery}]}).
description() ->
[{name, <<"debug">>},
{description, <<"Debugging exchange">>}].
backing_module(#exchange{ type = Type }) ->
%% Presume that Type is EITHER one of the standard types –
%% i.e,. that we have been registered this module as direct,
%% topic, fanout or match — or, for testing purposes, it’s
%% registered (as in the boot steps above) as debug_direct, etc.
Type1 = case atom_to_list(Type) of
“x-debug-” ++ T -> T;
T -> T
end,
list_to_atom(”rabbit_exchange_type_” ++ Type1).
publish(Exchange, Delivery) ->
io:format(”Publish ~p to ~p~n”, [Delivery, Exchange]),
Module = backing_module(Exchange),
Module:publish(Exchange, Delivery).
validate(X) ->
io:format(”Validate ~p~n”, [X]),
(backing_module(X)):validate(X).
create(X) ->
io:format(”Create ~p~n”, [X]),
(backing_module(X)):create(X).
recover(X, Bs) ->
io:format(”Recover ~p with bindings ~p~n”, [X, Bs]),
(backing_module(X)):recover(X, Bs).
delete(X, Bs) ->
io:format(”Delete ~p with bindings ~p~n”, [X, Bs]),
(backing_module(X)):delete(X, Bs).
add_binding(X, B) ->
io:format(”Add binding ~p to ~p~n”, [B, X]),
(backing_module(X)):add_binding(X, B).
delete_binding(X, B) ->
io:format(”Delete binding ~p from ~p~n”, [B, X]),
(backing_module(X)):delete_binding(X, B).
register_debug_types() ->
lists:foreach(
fun (T) ->
rabbit_exchange_type:register(T, ?MODULE)
end,
[<<"x-debug-direct">>,
<<"x-debug-topic">>,
<<"x-debug-fanout">>,
<<"x-debug-headers">>]).
Bit by important bit:
-behaviour(rabbit_exchange_behaviour).
-export([description/0, publish/2]).
-export([validate/1, create/1, recover/2, delete/2, add_binding/2, delete_binding/2]).
rabbit_exchange_behaviour specifies these exported callbacks.
-include("rabbit_exchange_behaviour_spec.hrl").
This include has the specs for each of the exported functions, if you’re using specs.
-rabbit_boot_step({debug_exchange_types,
[{description, "debugging exchange types"},
{mfa, {?MODULE, register_debug_types, []}},
{post, rabbit_exchange_type},
{pre, exchange_recovery}]}).
This uses the new boot sequence mechanism to register the exchange type during boot. The pre and post steps (”enables” and “requires” respectively — they are due to be renamed) say that the function given as mfa above must be run after the exchange type registry is available, but before any exchanges are recovered.
publish(Exchange, Delivery) ->
io:format("Publish ~p to ~p~n", [Delivery, Exchange]),
Module = backing_module(Exchange),
Module:publish(Exchange, Delivery).
This exchange type simply delegates to a “backing” exchange type.
register_debug_types() ->
lists:foreach(
fun (T) ->
rabbit_exchange_type:register(T, ?MODULE)
end,
[<<"x-debug-direct">>,
<<"x-debug-topic">>,
<<"x-debug-fanout">>,
<<"x-debug-headers">>]).
rabbit_exchange_type maintains a registry of type to module; because of this indirection, we can register this module as many different types, then check the declared type of the exchange in our hook to see which type we’re expected to be. Note that the AMQP specification requires the “x-” prefix for non-standard exchange types.
This should reach default branch soon after RabbitMQ 1.7.1 is released. Until then, if you want to play, you’ll have to
rabbitmq-server$ hg update -C bug22169
You can drop modules straight into src, but they are better packaged
as plugins — follow the drill at the plugin development
guide
(and note that your plugin may only need to be a library application).
by mikeb at January 22, 2010 02:38 PM
Process-one Blogs
Talkr.IM service update: gateways
The Talkr.IM XMPP/Jabber service will undergo today a service update, adding gateways to legacy IM systems.
ProcessOne's XMPP service Talkr.IM will receive new features for all users: IM gateways. These gateways will enable XMPP users to join legacy IM systems with their existing accounts. This will seamlessly merge the legacy IM contact lists to the main XMPP contact list ("roster"). Of course the main features are presence and chat, but these gateways also enable the status exchange, as well as the avatar, and much more.
We had already deployed an IRC gateway, the new gateways (or "transport" in XMPP terms) are the following:
- AIM
- ICQ
- MSN/WLM
- Yahoo!
This adds IM contact points to your addressbook, and enable the inclusion of millions of legacy IM users to XMPP users.
We are planning a service interruption at:
- 13:00 UTC
- 14:00 Paris
- 16:00 Moscow
- 05:00 Los Angeles
- 08:00 New York
- 22:00 Tokyo
We will remind our online users a few minutes before.
Update: our service update is done, you can now register your legacy IM accounts and play with them.
by Nicolas Vérité at January 22, 2010 10:10 AM
January 18, 2010
Process-one Blogs
[ANN] ejabberd 2.1.2 bugfix release
We are pleased to announce ejabberd 2.1.2, which contains several bugfixes over last month's maintenance release.
Brief summary of changes:
- Fix SASL PLAIN authentication message for RFC4616 compliance
- Fix support for old Erlang/OTP R10 and R11
- If server start fails due to config file problem, display some lines and stop node
- PubSub and PEP: several improvements and bugfixes
- WebAdmin: fix offline message displaying
- When server stops with new stop_kindly command: inform users, clients, MUC
Check the Release Notes for a more complete list of changes:
http://www.process-one.net/en/ejabberd/release_notes/release_note_ejabberd_2.1.2
If you upgrade from ejabberd 2.0.5 or older, read carefully the release notes of ejabberd 2.1.0 too, because there were several changes in the installation path and the configuration options.
The list of solved tickets since previous version is available on ProcessOne bug tracker:
http://redir.process-one.net/ejabberd-2.1.2
ejabberd 2.1.2 is available as source code package and binary installers for Linux 32 bits, 64 bits, Mac OS X Intel, and Windows:
http://www.process-one.net/en/ejabberd/downloads
by Jérôme Sautret at January 18, 2010 04:03 PM
ejabberd@jabber.ru
ejabberd 2.1.2 - Second bugfix release
ejabberd 2.1.2 has been released. It contains several bugfixes over last month's maintenance release.
Brief summary of changes:
- Fix SASL PLAIN authentication message for RFC4616 compliance
- Fix support for old Erlang/OTP R10 and R11
- If server start fails due to config file problem, display some lines
and stop node - PubSub and PEP: several improvements and bugfixes
- WebAdmin: fix offline message displaying
- When server stops with new stop_kindly command: inform users, clients, MUC
by badlop at January 18, 2010 03:07 PM
Erlang Eclipse IDE
erlide 0.7.3
A new release of erlide is available, with following (user visible) changes from 0.7.0:
* 468-split-erlang-code-so-that-compiler-backend-can-use-R11
* 487-configure-report-directory'
* fix #480: import wizard doesn't descend in subdirectories
* 490-External-Files-project-closed'
* 257--type---navigate-from-spec-to-declared-type'
* 476-open-module-dialog--allow-inexact-matches'
* 444 - text selected randomly in editor
* 472-launch--extra-arguments-to-runtime-don-t-get-used
* 469-editor---erlang-reconciler-still-broken'
* ignore errors when deleting beam files
* erlang templates, with syntax highlighted preview, indentation
* use binary for noparse:initial_text parameter
* Fix timeout in light_scan_string, send bin instead of string
* send to console action, useful when debugging and testing
* fixed -opaque and external type refs
* add option to monitor ide backend
* improve backend launching: send environment; make node status local
Enjoy!
/Vlad
by vladdu@users.sourceforge.net (Vlad Dumitrescu) at January 18, 2010 11:03 AM
January 17, 2010
Caoyuan's Blog
Progress of Migrating AIOTrade to Scala
Well, I've done most parts of migrating AIOTrade to Scala, not all features return yet. I gain lots of experiences of inter-op between Scala and Java, since AIOTrade has to be integrated into an existed Java framework NetBeans Platform. And also, whole project is now managed by Maven instead of Ant, which reduces lots of pain of dependencies upon crossing sub-projects.
This project is now hosted on kenai.com, you can check out the code to get an overview of how to integrated Maven + Scala + NetBeans Modules. Of course, all were done with NetBeans Scala plugin.
LOC of this project so far:
$ ./cloc.pl --read-lang-def=lang_defs.txt ~/myprjs/aiotrade.kn/opensource/
677 text files.
617 unique files.
154 files ignored.
http://cloc.sourceforge.net v 1.08 T=3.0 s (167.7 files/s, 21373.7 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code scale 3rd gen. equiv
-------------------------------------------------------------------------------
Scala 353 7981 16301 27180 x 1.36 = 36964.80
Java 43 1148 833 6946 x 1.36 = 9446.56
XML 104 231 389 2414 x 1.90 = 4586.60
Bourne Shell 2 81 81 488 x 3.81 = 1859.28
HTML 1 7 15 26 x 1.90 = 49.40
-------------------------------------------------------------------------------
SUM: 503 9448 17619 37054 x 1.43 = 52906.64
-------------------------------------------------------------------------------
A screen snapshot:
by dcaoyuan at January 17, 2010 10:24 PM
January 16, 2010
Programming in the 21st Century
No Comment
I received a few emails after last time along the lines of "Oh. Perl. Homebrew CMS. That's why you don't allow people to post comments. Well, no, but it was definitely a conscious decision. The Web 2.0 answer is that I'm outsourcing comments to reddit and Hacker News. The real reason is this:
The negativity of online technical discussions makes me bitter, and even though I'm sometimes drawn to them I need to stay away.
To be fair, this isn't true only of technical discussions. Back when I was on Usenet, I took refuge from geeky bickering in a group about cooking...only to find people arguing the merits of Miracle Whip versus mayonnaise. Put enough people together and there are sure to be complaints and conflicting personal agendas. But with smart, technically-oriented people, I'd expect there to be more sharing of real experiences, but that's often not the case.
Here's a lesson I learned very early on after I started working full-time as a programmer (and that's a peculiar sentence for me to read, as I no longer program for a living). I'd be looking at some code at my desk, and it made no sense. Why would anyone write it like this? There's an obvious and cleaner way to approach the same problem. So I'd go down the hall to the person who wrote it in the first place and start asking questions...and find out that I didn't have the whole picture, the problem was messier than it first appeared, and there were perfectly valid reasons for the code being that way. This happened again and again. Sometimes I did find a real flaw, but even then it may have only occurred with data that wasn't actually possible (because, for example, it was filtered by another part of the system). Talking face to face changed everything, because they could draw diagrams, pull out specs, and give concrete examples.
I think that initial knee-jerk "I've been looking at this for ten seconds and now let me explain the critical flaws" reaction is a common one among people with engineering mindsets. And that's not a good thing. I've seen this again and again, from people putting down programming languages for silly, superficial reasons (Perl's sigils, Python's enforced indentation), to ridiculous off-the-cuff put downs of new products (such as the predictions of doom in the Slashdot announcement of the original iPod in 2001).
The online community that I've had the most overwhelmingly positive experience with is the photo-sharing site Flickr. I'll talk about Flickr again and again, because it played a big part in getting me out of some ruts, and I've seen more great photographs over the last five years than I would have seen in ten lifetimes otherwise. I know that if you dig around you can find tedious rants from equipment collectors, but I do a good job of avoiding those. I don't think I've ever seen real negativity in photo comments other than suggestions for different crops or the occasional technical criticisms. There are so many good photos to see that there's no reason to waste time with ones the don't appeal to me. That's supported by only allowing up-voting of photos (by adding a shot to your favorites); there's no way to formally register dislike.
Flickr gets my time, but most of the programming discussion sites don't.
by James Hague at January 16, 2010 06:00 AM
January 14, 2010
Dukes of Erl
Minor Erlang Interface Tricks
erlrc requires that you drop a file whose name is your node name and whose contents is the node cookie into a particular directory so that the packaging system can find running Erlang VMs and ask them to do hot-code upgrades. It's not that hard, but I figured I would put some shell scripts into the erlrc google code project demonstrating how to do this and other minor tricks that make it a little nicer to talk to an Erlang VM from the (non-Erlang) command line.
All of these scripts are driven by a POSIX shell syntax configuration file describing an Erlang VM, which can be pretty short. Here's one I'm using right now for my personal stuff:
% cat /usr/share/myerlnode/myerlnode.rc
node_name_file='/etc/erlrc.d/nodes/erlang'
run_extra_args='+A 64 -noshell -noinput -s crypto -s mnesia -eval "case erlrc_boot:boot () of ok -> ok; _ -> init:stop () end" >erlang.out 2>erlang.err &'
node_name_file (the only required config setting) defines what the node name will be (via the basename). This config file drives four scripts:
- erlstart-run-erlang: starts an Erlang VM. doesn't do that much, really: creates the node file for erlrc, and sets the heart command in case you want to use heart.
- erlstart-remsh: starts a remote shell on an Erlang VM.
- erlstart-eval: takes the argument, evals it on an Erlang VM, and prints the result to standard out. very useful shell script glue for maintenance scripts.
- erlstart-etop: runs etop on an Erlang VM.
% erlstart-eval 'application:which_applications()'erlstart-run-erlang is fairly low level, so for an init script I would do something like the following: first, a little stub installed into /etc/rc.d
[{anwhereos,"Rest API for time series persistence and retrieval.","0.1.0"},
{drurlyjsclientsrv,"Serve the drurly jsclient from the drurly server.",
"0.0.1"},
{drurly,"Social sharing server.","2.2.0"},
{mcedemo,"TinyMCE + Nitrogen demo.","1.3.0"},
{inets,"INETS CXC 138 49","5.0.12"},
{mochiweb,"MochiWeb is an Erlang library for building lightweight HTTP servers.",
"0.2009.05.26"},
{nitrogen,"Nitrogen web framework for Erlang.","0.2009.05.12.3"},
{nitromce,"A Nitrogen element which corresponds to a TinyMCE editor instance.",
"4.0.1"},
{sgte,"String template language for Erlang.","0.7.1"},
{signzor,"Erlang library to generate signed printable encodings.","0.0.1"},
{tcerl,"Erlang driver for tokyocabinet.","1.3.1h"},
{webmachine,"An Erlang REST framework.","0.2009.09.24b"},
{erlrc,"Extensible application management.","0.2.3"},
{mnesia,"Mnesia storage API extensions.","4.4.7.6.1"},
{crypto,"CRYPTO version 1","1.5.3"},
{sasl,"SASL CXC 138 11","2.1.5.4"},
{stdlib,"ERTS CXC 138 10","1.15.5"},
{kernel,"ERTS CXC 138 10","2.12.5"}]
#! /bin/sh
# chkconfig: 2345 20 80
# description: Control my personal Erlang node.
exec myerlnodectl "$@"
and then the actual guts installed as myerlnodectl
#! /bin/sh
eval_with_main_node () \
{
erl -name myerlnodetmp$$ \
-hidden \
-setcookie "$cookie" \
-noshell -noinput \
-eval "MainNode = list_to_atom (\"$1\"), $2" \
-s erlang halt
}
get_hostname () \
{
erl -name myerlnodetmp$$ -setcookie $$ -noshell -noinput -eval '
[ Host ] = tl (string:tokens (atom_to_list (node ()), "@")),
io:format ("~s~n", [ Host ])
' -s init stop
}
id=`basename "$0"`
if test -d /root
then
HOME=${HOME-/root}
else if test -d /var/root
then
HOME=${HOME-/var/root}
fi
fi
export HOME
ERLSTART_CONFIG_FILE=${ERLSTART_CONFIG_FILE-/usr/share/myerlnode/myerlnode.rc}
export ERLSTART_CONFIG_FILE
. "$ERLSTART_CONFIG_FILE"
cookie=${cookie-turg}
user=${user-erlang}
hostname=${hostname-`get_hostname`}
node_name=`basename "$node_name_file"`
full_name="$node_name@$hostname"
shutdown_file=${shutdown_file-/var/run/myerlnode.shutting_down}
ERL_CRASH_DUMP=${ERL_CRASH_DUMP-/dev/null}
export ERL_CRASH_DUMP
case ${1-"status"} in
start)
test "`id -u`" -eq 0 || exec sudo $0 "$@"
printf "Starting Erlang... "
if test -f "$node_name_file" && \
test true = "`erlstart-eval 'true' 2>/dev/null`" 2>/dev/null
then
echo "already started."
exit 0
fi
pid=`eval_with_main_node "$full_name" '
io:format ("~p", [
case rpc:call (MainNode, os, getpid, []) of
{ badrpc, _ } -> undefined;
Pid -> list_to_integer (Pid)
end ])'`
test "$pid" -gt 0 2>/dev/null && {
test -f "$shutdown_file" && {
oldpid=`cat "$shutdown_file"`
test "$pid" -eq "$oldpid" && {
echo "shutdown in progress (pid = '$pid')." 1>&2
exit 1
}
}
}
rm -f "$shutdown_file"
# check for -s shell support
su -l -s /bin/sh $user -c true >/dev/null 2>/dev/null
if test $? = 0
then
dashs="-s /bin/sh"
else
dashs=""
fi
${niceness+ nice -n $niceness} \
su -l $dashs "$user" -c \
"env cookie=\"$cookie\" erlstart-run-erlang \"$ERLSTART_CONFIG_FILE\""
eval_with_main_node "$full_name" \
"Wait = fun (_, 0) ->
failed;
(Cont, Max) ->
case net_adm:ping (MainNode) of
pong ->
ok;
pang ->
timer:sleep (100),
Cont (Cont, Max - 1)
end
end,
DontTellMe = 100,
ok = Wait (Wait, DontTellMe)" || {
echo "" 1>&2
echo "$id: could not connect to node after 10 seconds" 1>&2
exit 1
}
eval_with_main_node "$full_name" \
"Wait = fun (_, 0) ->
failed;
(Cont, Max) ->
case rpc:call (MainNode, init, get_status, []) of
{ started, _ } ->
ok;
{ starting, _ } ->
timer:sleep (100),
Cont (Cont, Max - 1);
{ Status, _ } ->
{ failed, Status }
end
end,
DontTellMe = 100,
ok = Wait (Wait, DontTellMe)" || {
echo "" 1>&2
echo "$id: node did not boot after 10 seconds" 1>&2
exit 1
}
echo "done."
;;
stop)
test "`id -u`" -eq 0 || exec sudo $0 "$@"
printf "Stopping Erlang... "
if test ! -f "$node_name_file" || \
test true != "`erlstart-eval 'true' 2>/dev/null`" 2>/dev/null
then
echo "not running."
exit 0
fi
pid=`erlstart-eval 'os:getpid ()' 2>/dev/null`
test "$pid" -gt 0 2>/dev/null && {
printf '%s' $pid > "$shutdown_file"
chown $user:$user "$shutdown_file"
}
erlstart-eval 'init:stop ()' >/dev/null 2>/dev/null
eval_with_main_node "$full_name" \
"Wait = fun (_, 0) ->
failed;
(Cont, Max) ->
case net_adm:ping (MainNode) of
pong ->
timer:sleep (100),
Cont (Cont, Max - 1);
pang ->
ok
end
end,
DontTellMe = 100,
ok = Wait (Wait, DontTellMe)" || {
echo "" 1>&2
echo "$id: node still responsive after 100 seconds" 1>&2
exit 1
}
rm -f "$node_name_file"
echo "done."
;;
status)
if test -f "$node_name_file" && \
test true = "`erlstart-eval 'true' 2>/dev/null`" 2>/dev/null
then
echo "Erlang is running"
else
echo "Erlang is not running"
fi
;;
*)
echo "$id: unknown command $1" 1>&2
exit 1
esac
exit 0
Hopefully you found that inspirational.
by Paul Mineiro (noreply@blogger.com) at January 14, 2010 12:41 PM
January 11, 2010
Erlang Training and Consulting
- News
14 December 2009: Erlang Training and Consulting's 10th Birthday – it's time to change our name!
2009 has been a huge year for us. Not only have we had an incredible
year assisting companies on six continents with their use of Erlang but
we have also organised 4 conferences, 10 Erlang User Group Meetings
and started a KTP project in e-learning with the University
of Kent. To add to all that - it’s also our 10th Birthday!
It all began 10 years ago on 13th December 1999,
when our founder Francesco Cesarini decided to set up the first
non-Scandinavian company to offer Erlang/OTP training and consultancy
services. At that time, Erlang wasn't known outside of the 'Ericsson
world'. Over the years the business grew dramatically. We have expanded
geographically, with offices now in London (UK), Uppsala (Sweden) and
Krakow (Poland) and now we employ 40 employees worldwide. The services
on offer have also expanded significantly. We are now a one-stop shop
for all Erlang needs, including:
- Erlang training at all levels
- Certification in Erlang
- Provision of experienced Erlang contractors
- Erlang consulting services
- In-house systems development
- 24/7 support
- Assistance
with building Erlang teams.
Great service, a customer-focus and skilled and experienced staff have been the basis of our growth over the past ten years. This will not change. However in 2010 we will do it with a new name! In order to better describe the breadth of our service offering, on 1st January 2010 we will become Erlang Solutions Ltd.
09 December 2009: Marcus Taylor at Strengthen Your Business
More than 50 representatives from the business world attended a special networking event at the University of Kent’s Medway campus. Called Strengthen Your Business, the free session on 25 November gave business leaders the chance to explore how their companies can team up with the University, and benefit from its commercial expertise and resources.
A variety of business people who had successfully worked with the University on student placements or Knowledge Transfer Partnerships spoke of their experiences, while others outlined the opportunities for collaboration in graduate recruitment and continuing professional development schemes. There was also the chance for individuals to network after the main talks.
Marcus Taylor, Chief Executive for London-based Erlang Training and Consulting, spoke of the strategic help his company had received with Knowledge Transfer Partnerships – a government-funded initiative that helps businesses develop their competitiveness, productivity and performance. ‘I’m not an experienced bid writer, but the University has a wealth of experience in this area and supported us wonderfully throughout the whole process – so much so that we’re thinking of applying for our fourth project,’ he said.
James Corbin, Placement Officer for the University of Kent, said it was a golden opportunity for businesses to find out about the resources on their own doorstep. 'The University is host to a wealth of knowledge that can often provide businesses with the solutions they seek,’ he said. ‘We’re convinced that firms who work with the University can gain a competitive edge.’
The event followed hot on the heels of the launch of the University’s ICE initiative, which took place at the Canterbury Innovation Centre on 17 November. ICE stands for innovation, creativity and enterprise, and is a project that aims to connect local, regional, national and international businesses to the University of Kent and to each other.
The Strengthen Your Business event at Medway was run in partnership with Thames Gateway Chamber of Commerce.
07 December 2009: Erlang Factory 2010 Dates Announced!!
The dates of the Erlang Factory in the SF Bay Area and London have been announced.
- SF Bay Area: 25-26 March 2010
- London: 10-11 June 2010
30 November 2009: Erlang Training and Consulting soon Erlang Solutions will be present at ACCU 2010
Francesco Cesarini (founder of Erlang Training and Consulting, soon Erlang Solutions Ltd.) and Ulf Wiger (CTO of Erlang Training and Consulting, soon Erlang Solutions Ltd.) will be speaking at ACCU 2010. Francesco's talk on Styling your Architecture in an Evolving Concurrent World and Ulf's on Message-Passing Concurrency in Erlang, have both been accepted by the conference committee.
ACCU 2010 will take place from 14th to 17th April 2010 at the Barcelo Oxford Hotel, in Oxford, UK. The programme of the conference will feature a special track on software testing, and sessions on concurrent and distributed systems, C++, Java,agile development and Erlang!
Meet us at ACCU 2010!
23 November 2009: The Largest Erlang Open Source Project Directory!
Trapexit has announced its own Erlang open source crawler. It will crawl the web, gathering searchable information on all open source Erlang projects from all the major repositories. Trapexit will allow discussing these projects in the forum, adding documentation in the Trapexit wiki or rating the various contributions. To view the project index repository, visit http://projects.trapexit.org.
19 November 2009: Learn Erlang from experts
Do you want to learn about the basic, sequential and concurrent aspects of the Erlang programming language? And would you like to be taught by the expert, author of the 'Erlang Programming' book?
If the answer is yes, then we have something which may interest you:
On Wednesday, 3rd February 2010 in London Francesco Cesarini, the author of 'Erlang Programming' also the founder and CSO of Erlang Training and Consulting Ltd. will give a tutorial on Practical Erlang Programming.
You will learn the basics of how to read, write and structure Erlang programmes. The goal of tutorial is a hands-on introduction to the theory and concepts behind sequential and concurrent Erlang programming, explaining the Erlang syntax, semantics and concurrency model. We conclude with an overview of the error handling mechanisms used to build fault tolerant systems with five nines availability.
The target audience are software developers and engineers with an interest in server side applications and massively concurrent systems.
To learn more about the tutorial, please go here and if you want to book it click here.
The tutorial will take place in the Caesar Room, Imperial Hotel, Russell Square, London WC1B 5BB.
See you there!
19 November 2009: ETC at the 11th Symposium on Trends in Functional Programming
Erlang Training and Consulting Ltd. sponsors the 11th Symposium on Trends in Functional Programming and sits on the programme committee.
The symposium is an international forum for researchers with interests in all aspects of functional programming. It embraces a broad view of current and future trends in functional programming and aspires to be a lively environment for presenting the latest research and applications.
The 11th Symposium on Trends in Functional Programming will be held at the University of Oklahoma, on campus in Norman, Oklahoma, May 17-19, 2010.
12 November 2009: Take our Erlang Certification Survey!
Erlang Training and Consulting Ltd in conjunction with the University of Kent intends to launch a pilot version of an e-certification platform for Erlang in the near future.
To assist in this process, we have launched a survey on Erlang Certification. The objective of the study is to better understand the expectations of users, in order to ensure a well-focused product.
We would appreciate a few minutes of your time in completing this survey. To take the survey, please go to:
http://surveys.erlang-consulting.com/index.php?sid
The results of the survey will be shared with third parties, in publications and as part of research into Certification. When doing so, all results will be completely anonymised and any individual-identifying information removed.
Thank you for your time and cooperation!
28 October 2009: Erlang Web 1.4 has been released!
We are happy to announce the next official release of the Erlang Web framework. The latest 1.4 version is full of new features and enhancements, like integration with EWGI, new template language support: ErlyDTL, Erlang Web Test Suite application bundled, multiple level request dictionary support.
Apart from the above, there are also some changes worth mentioning:
* new wpart introduced: wpart:comment
* embedding default values in wparts from templates (patch submitted by Elena Bobrova)
* top-down template inclusion feature
* experimental ejabberd-flavoured e_hook introduced
* upgraded to Yaws 1.85
13 October 2009: Ready for the Erlang courses in London?
Have you always wanted to learn more about Erlang, but never knew how to get started?
We have a special offer, which may encourage you. If you sign up now for a five-day Erlang By Example or Erlang Open Telecom Platform courses in London you will get the Early-Bird Discount of £245 off the standard price! Don't miss out and register now.
Erlang Training and Consulting
- Jobs
Software Engineer - Distributed Systems Engineering, San Mateo, CA, USA
Responsibilities:
* participate in the research and design for state-of-the-art fault-tolerant management frameworks
* own the design process, implementation, and verification of framework components leveraging standard software engineering methodologies
* maintain and improve existing infrastructure components
Requirements:
* a passion for building great software
* extensive experience in developing highly scalable distributed systems
* in-depth understanding of High Availability concepts, e.g., replication, consensus protocols
* strong system development skills
* good knowledge of parallel programming paradigms
* advanced degree in CS, or equivalent
* solid coding skills in Erlang/OTP
* team player; capable of conducting independent research
* result-driven, self-motivated, self-starter
* not afraid to take on hard technical challenges
* excellent communication skills
Experience in any of the following fields is a definite plus:
* database internals
* experience with PostgreSQL/Greenplum Database
* enterprise software development
Principal, IT Architecture, Financial Services, Menlo Park, California (USA)
Our client is looking for dynamic, highly energized risk-takers who are ready to join a fast-paced, world leader in the financial services industry. They move at the speed of the Internet. Their work culture fosters an incredible exposure to innovative ideas and challenges that create experience and opportunity. Many have risen through the organization to become industry leaders, in both the financial services world and in the community at large. Our client made up of some of the brightest and most talented people in the industry - are you ready to join the industry leader?
1. Essential Responsibilities:
- Architect, Service Oriented Architecture
- Design and Develop solutions in the are of SOA, Distributed Computing and Software Packaging and Release methodologies.
- Assist in day-to-day developer support for Services and Packaging frameworks
- Write code in Java, C and Erlang
- Evaluation of state of the art Open Source frameworks
- Present Solutions, and work with other technology managers to craft solutions
- 5+ yrs in-depth C or C++ experience
- Strong Distributed Computing knowledge
- Experience with Web services, SOAP, WSDL/XSD, Service Orchestration, etc.
- Experience in scripting languages, such as Ruby, Python, or Perl.
- Experience with AJAX technologies.
- In-depth experience with Linux, Unix.
- Experience with Agile Methods, including Test Driven Development.
- Experience with Erlang or Functional Programming
BS degree, preferably in Engineering, or Electrical Engineering, but
Computer Science is acceptable
Senior Erlang Developer, London, UK
Gambit Research is a West London software company developing market
information and trading software for the betting industry.
We are looking to hire an experienced Erlang developer to work on extending and
improving an existing trading system. This role will involve working as part
of a small team of Erlang developers and working closely with Python developers
and the trading team.
You should have experience of developing reliable, fault tolerant
applications in
Erlang. Experience with UNIX, networking, PostgreSQL and Python would also be
highly desirable.
- Starting salary £30,000 plus, depending on experience.
- 25 days holiday per year.
- Discretionary bonus scheme.
- Flexible working hours.
- Relaxed and informal working environment.
Systems Administrator, London, UK
We are seeking a competent Systems Administrator to develop our business and development environment.
1. The requirement
The ideal applicant will need to have experience of setting up an office infrastructure for a software development company where there are hosted components and office based components. We also have offices in Krakow and Uppsala that should have access to the infrastructure. Our goal is to phase out Microsoft and move completely over to Open Office and Open Source offerings so somebody who understands and believes in this approach is hugely desirable.
The following are a broad list of the technologies and application we are either using, or intend to use and which the incumbent would be expected, in the longer term, to manage and improve.
OS: Windows XP and Vista, All flavours of Linux, UNIX.
DB's: mySQL, Postgresql, Berkley DB
Collaborative environments: MS Exchange and Zimbra
Office suites: MS Office and Open Office,
Set up of VPNs
Set up and maintenance of Firewalls
Wireless Networks
PBX's: Nortel BCM50
Wiki's
Single Sign on
File share and Backups
Process for signing up new users and removing leavers
Policies for access privileges
Security and encryption
Experience of CRM systems like SugarCRM or Salesforce and Trouble Ticketing systems like Bugzilla or Trac would be desirable
Knowledge of email and IM protocols are useful
Experience of dealing with service providers and selecting best ADSL, SDSL services is also a plus as well as SLA's
Selection and recommendations for hardware purchases Ability to work alone and under pressure
A hands on and "I can do it" attitude
Confident person
A creative thinker
2. The company offers:
A supportive and friendly working atmosphere together with the opportunity to progress.
Salary: 22,000GBP - 28,000GBP + Discretionary Bonus
Benefits: Private Health Insurance, Life Protection, 20 days holiday, Bike Scheme, Stakeholder Pension Scheme.
Training and advancement is on offer once the successful candidate has proven themselves within the company as a dedicated member of the team.
Only applicants who have a legal right to work within the UK will be considered.
January 06, 2010
Ulf Wiger
QCon 2010 Concurrency Track
I will be hosting the “Concurrency Challenge” track at QCon 2010. The speakers list is now ready with all titles and abstracts in place.
I am quite pleased with the mix and the dignity of the speakers. One of the things we wanted to highlight is that there is no one-size-fits-all concurrency model, and it’s time for us to start being a little bit more specific about what kind of concurrency we have in mind.
Each setting brings its own set of challenges, and in this track, you can expect to get a good orientation from some brilliant people, who know how to tell a story and are not afraid of offering their opinion.
by Ulf Wiger at January 06, 2010 04:01 PM
January 05, 2010
Erlang Factory News
Erlang Factory SF Bay Area 2010 update
We have new confirmed speakers: Nick Gerakines (Author of "Facebook Application Development" and "Erlang Web Application Development"), Kresten Krab Thorup (CTO of Trifork and Creator of Erjang), Cliff Moon (Dynomite creator and committer), Kenji Rikitake (Expert researcher in network security), Kevin Smith (Pragmatic Programmer author), Dave Smith (software engineer at Basho) and Joe Williams (Developer at Cloudant and creator of Erlang Memcached client).
If you want to be among them and give a presentation at the Erlang Factory, you have time until 31st January to submit your talk for review by the Conference committee.
Last but not least, registration for the Erlang Factory SF Bay Area 2010 is now open. Sign-up before 30th January 2010 and SAVE $400 with the Early-Bird offer!
Keep a look out and follow us on Twitter @erlangfactory
January 04, 2010
Process-one Blogs
ProcessOne, a year in review
At ProcessOne, we wanted to wish you all a very happy 2010 year. To give you a good sense of all what has been accomplished, we have written this small review of the 2009 year at ProcessOne. We hope you will enjoy the reading.
Software
First, we have released ejabberd 2.0.3 to 2.0.5, and we finally made it with ejabberd 2.1 (with its bugfix 2.1.1). It has received much scalability improvements, as well as a big PubSub boost. Even more changes have made it an even more rock-solid and featureful real-time server. A STUN server has been added in ejabberd, for multimedia purposes. The version 3.0 is in the works, based on the opensource exmpp library, freshly released this year also.
On the desktop, OneChannel, the PubSub client working on AIR, has been released, optionally using the xmpp-sandbox.org preconfigured real-time feeds.
Our gateway packaging is stable, scalable and works pretty well, covering Microsoft Live Messenger, Yahoo! IM, AIM and ICQ, XMPP, and also non-IM systems, like Twitter and Wave gateway.
On the mobile front, OneTeam for iPhone, the mobile XMPP client, has been released in versions 2.1, 3.0, 3.1, and 3.2, and we believe it is the best XMPP client out there, with room for improvements and innovations in the coming year.
OneWeb, the extension for Firefox, has been released as an alpha product, with a Fennec port (Firefox mobile). It basically adds data sync and remote control between browsers, demoing the power of XMPP in the browser.
Our supervision console TeamLeader has had many improvements in 2009 and is now clean and efficient, being deployed on large customer projets.
Our XMPP desktop client OneTeam for Firefox has become a strong client and should become a major component in our offering in 2010.
We also are offering a module for the APNS, or Apple Push Notifications Service, for the iPhones.
Announced
We have announced and demoed the OneTeam Media Server, our Flash server implementation, acting as a media relay and much more, for multimedia applications.
The ProcessOne Wave Server is in the work, gaining features and improving fast. A demo on Tkabber has been done, showing very basic client/server interactions, like create a wavelet, join and edit it, the basics of collaborative real-time editing of a text document.
Our OCS Bridge enables a real federation of ejabberd servers with the Microsoft's IM+VoIP corporate servers, while this one only painfully offers gateways to XMPP since this year.
Services
We have given birth to a number of services:
- We have opened the ProcessOne Labs, showing all our non-production ready, but promising products.
- Hosted.IM offers for XMPP hosting services for small businesses.
- IMstore enables customers to buy products and services, the fast and easy way, on the web via a simple form.
- The APNS, or Apple Push Notifications Services, runs on our infrastructure for OneTeam for iPhone clients using our OneTeam.IM and Talkr.IM services.
- Tweet.IM is a XMPP to Twitter gateway, offering nice features like delivery modes.
- xmpp-sandbox.org is the XMPP service for developers.
- Talkr.IM is our XMPP service for the masses.
Events, Talks & Articles
Here are the places where we have been, and what has been said about us:
- In December, we have organized the Sea Beyond event, for real-time communications.
- The concept of "Web Augmented Reality" came to birth.
- A Strategic Guide to Instant Messaging and Security has been published.
- Facebook has been caught working on XMPP with ejabberd.
- id Software's QuakeLive online game is using ejabberd for its ingame chat infrastructure.
- Mickaël Rémond went to Mobile World Congress, Erlang Factory in San Jose and London, XMPP Summit in Brussel and Palo Alto, Erlang User Conference in Stockholm, Le Web in Paris.
- We have been covered by Datamonitor and Giga OM.
- The series "Scalable XMPP bots" has been published, promoting the use of exmpp (I, II, III).
Roadmap 2010
In 2010, our focus will be mainly on the following points:
- Hosted.IM will be offering new simple and extendable services.
- OneTeam for Firefox will be released, with WAR (Web Augmented Reality), and much more innovative features.
- ejabberd will become more industrial, with 3.0 getting to production ready state, better TeamLeader support, good test suite coverage and more modules on the IMstore.
- Talkr.IM will offer the seamless migration from/to another XMPP service.
Conclusion
It has been a busy year, with many achievements, much of which we cannot talk about, or we have just forgotten. But the coming year 2010 will be even more exciting.
by Nicolas Vérité at January 04, 2010 01:00 PM
January 03, 2010
Me Dev, You Jane
Erlang Quick Tip: The user_default module
If you are anything like me you spend a lot of your time in the erlang shell debuging and testing your code. And some commands are used more than others but they may be a bit on the long side and you may make regular errors while typing it. One of these wonderful functions is make:all([load]). that will compile using your Emakefile and then magically reload the newly compiled modules into the running shell. What I reglarly do is to type load instead of [load].
Enter the user_default module. Any function defined in the user defined user_default module or in the Erlang defined shell_default module will be allowed to run without typing in the module name. One of these functions that you may use often is the c(module\_name). function. First start by creating a directory for your utility modules. I like having it in the ~/.ebin/ diretory. Now add that direcory to the code path by opening or creating the file ~/.erlang and inserting this line:
code:add_pathz("/Users/username/.ebin").
Create your user_default.erl file and but in the following:
-module (user_default).
-export ([sync/0]).
%% Compiles all files in Emakefile and load into current shell.
sync() ->
make:all([load]).
Then compile it with erlc user_default.erl and start an Erlang shell. Now inside a project you can run sync(). and it will recompile and load you project.
For those interested I will put up my ~/.ebin code up at github.com/JonGretar/erlang_user_utilities. Please fork and make your own changes and share with the world.
by Jón Grétar Borgþórsson at January 03, 2010 03:09 PM
Caoyuan's Blog
Jetty + FastCGI Servlet + Trac = blogtrader.net
I ported jfastcgi to Scala and fixed some bugs. Now my web site is configured running under Jetty with the modified fastcgi servlet as gateway to a Trac fastcgi daemon.
I encountered an issue (fixed later) with Trac fcgi daemon though, which may cause the daemon died silently, but the fastcgi servlet will restart it automatically when this happens.
All source code can be found via Browse Source
by dcaoyuan at January 03, 2010 10:00 AM
January 01, 2010
Joe's Blog
Fun with the CouchDB _changes feed and RabbitMQ.
I was recently introduced to yajl-ruby, ruby bindings to the C based yajl json parsing/encoding libraries. After discovering that it can parse HTTP streams it seemed like it would be a perfect fit for use with CouchDB. A while back I wrote some code to push update notifications to RabbitMQ and a commenter mentioned using the _changes feed instead. Combining the _changes feed and yajl-ruby’s HttpStream seemed like a good way to do it.
The _changes feed is a running list of all the documents that have changed in a database listed in order by sequence number. This is similar to update notifications but gives more information such as the document IDs and is HTTP based (with multiple feed styles) rather than stdout. Additionally you can create design document filters which can be specified as a query parameter to give you only the parts of the feed you want. All in all _changes is a pretty powerful feature.
Now for the fun stuff, the code. There are a few dependencies I used to do this, specifically focused on making it fast. As such I used EventMachine based libraries for AMQP and HTTP requests. The first bit of code takes the _changes feed for the “test” database, parses the feed, uses the document ID to request that document and publish it to the queue. One key item to note is that this code requires the latest yajl-ruby from github to run properly. Additionally, this works nicely with feed=continuous so it grabs the documents as they are changed without a need for polling.
Note that there is a variable for since, this allows you to start from a specific sequence number so you can skip over old changes.
The next bit of code works from the other side of the queue. It subscribes to the queue, parses the JSON, performs some operations on it and puts the results back into another CouchDB database called “results”.
What could it be used for? My first thought is some sort of parallel computation, boot up a few dozen EC2 nodes and start dumping data into CouchDB. Have all those nodes pop messages off the queue, process them and dump the results back into Couch. Legitimately one could chain these together to process the results again. The queue ends up being a simple job management system with the EC2 nodes popping new messages as they finish processing them. With a little bit of work, features and the right use case I think could be a pretty powerful system.
Check out the code, my other projects and follow me on twitter @williamsjoe.
[edit: made a slight improvement to changes_sub.rb on 20100107]
by joe at January 01, 2010 11:14 PM
Programming in the 21st Century
The Recovering Programmer
I wrote the first draft of this in 2007, and I thought the title would be the name of my blog. But I realized I had a backlog of more tech-heavy topics that I wanted to get out of my system. I think I've finally done that, so I'm going back to the original entry I planned to write.
When I was a kid, I thought I'd be a cartoonist--I was always drawing--or a novelist. Something artistic. When I became obsessed with video games in the 1980s, I saw game design as being in the same vein as cartooning and writing: one person creating something entirely on his own. I learned to program 8-bit computers so I could implement games of my own design. Eventually, slowly, the programming overtook the design. I got a degree in computer science. I worked on some stuff that looks almost impossible now, like commercial games of over 100K lines of assembly language (and later I became possibly the only person to ever write a game entirely in PowerPC assembly language).
Somewhere along the lines, I realized I was looking at everything backward, from an implementation point of view, not from the perspective of a finished application, not considering the user first. And I realized that the inherent bitterness and negativity of programming arguments and technical defensiveness on the web were making me bitter and negative. I've consciously tried to rewind, to go back to when programming was a tool for implementing my visions, not its own end. I've found that Alan Cooper is right, in that a tech-first view promotes scarcity thinking (that is, making perceived memory and performance issues be the primary concerns) and dismissing good ideas because of obscure boundary cases. And now programming seems less frustrating than it once did.
I still like to implement my own ideas, especially in fun languages like Erlang and Perl. I'm glad I can program, because personal programming in the small is fertile ground and tremendously useful. For starters, this entire site is generated by 269 lines of commented Perl, including the archives and the atom feed (and those 269 lines also include some HTML templates). Why? Because it was pleasant and easy, and I don't have to fight with the formatting and configuration issues of other software. Writing concise to-the-purpose solutions is a primary reason for programming in the twenty-first century.
If blogs had phases, then this would be the second phase of mine. I'm not entirely sure what Phase 2 will consist of, but I'll figure that out. Happy 2010!
by James Hague at January 01, 2010 06:00 AM
December 31, 2009
Anders Conbere's Journal
Atom feeds for igor... difficult
The first problem was actually reading the atom specs which I had neglected to do. Most of my previous work with atom had been in either creating elements used in various locations (Google data is all passed as Atom elements, as well as moments of injecting atom elements into xmpp streams). So I cobbled together what I thought was a decent first attempt at a Jinja template for building the atom file, published it and redirected my browser there... only to see nothing.
The XML was being generated, but not parsed.
The problems left were:
- writing out dates in RFC:3339 format
- building correct id's for elements
for RFC 3339 I found a nice python library in the public domain for producing the correct format from a datetime object. To you Henry Precheur I am quite grateful.
Generating correct id's was a little more tricky. First I tried generating uuid's but upon reading Mark Pilgrims post on atom element id's I ended up writing a small function to generate tag: urns based on the time when a post was published (I'm not sure this is a /good/ solution given that I'm working with version control systems and you might legitimately have many articles published at the same time).
In the end I'm fairly content with the result, it required a morning's effort, more time than I had wanted to devote to atom publishing, but still successful.
December 30, 2009
Erlang Inside
Mustache – GitHub’s framework agnostic templating ported to Erlang
Mustache is an Erlang port of the Ruby framework of the same name, originally written by Chris Wanstrath, one of the founders of GitHub. A powerful feature of this type of framework is the ability to write a ‘template’ that you could use in a Rails application and then reuse in an Erlang application with [...]
by Chad DePue at December 30, 2009 02:00 PM
Caoyuan's Blog
Importing Old Blogs Almost Done
I've almost done importing old blogs from Roller db to this Trac based one, except image/resource links.
The pretty url of this web site is also working now.
by dcaoyuan at December 30, 2009 10:11 AM
Programming in the 21st Century
Follow-up to "Functional Programming Doesn't Work"
Not surprisingly, Functional Programming Doesn't Work (and what to do about it) started some lively discussion. There were two interesting "you're crazy" camps:
The first mistakenly thought that I was proposing fixing problems via a judicious use of randomly updated global variables, so every program turns into potential fodder for the "Daily WTF."
The second, and really, the folks in this camp need to put some effort into being less predictable, was that I'm completely misunderstanding the nature of functional programming, and if I did understand it then I'd realize the true importance of keeping things pure.
My real position is this: 100% pure functional programing doesn't work. Even 98% pure functional programming doesn't work. But if the slider between functional purity and 1980s BASIC-style imperative messiness is kicked down a few notches--say to 85%--then it really does work. You get all the advantages of functional programming, but without the extreme mental effort and unmaintainability that increases as you get closer and closer to perfectly pure.
That 100% purity doesn't work should only be news to a couple of isolated idealists. Of the millions of non-trivial programs ever written--every application, every game, every embedded system--there are, what, maybe six that are written in a purely functional style? Don't push me or I'll disallow compilers for functional languages from that list, and then it's all hopeless.
"Functional Programming Doesn't Work" was intended to be optimistic. It does work, but you have to ease up on hardliner positions in order to get the benefits.
by James Hague at December 30, 2009 06:00 AM
December 29, 2009
CR Erlang Project
Looking for Volunteers
We need to build a startup team for this company. If you would like to do the challenging development work in the following areas, join us:
RAIC
Databases
Networking
Operating Systems
Autonomic Systems
At this stage we are looking for voluntary contributions with equity in the company offered for the top contributions. If you are interested, you will have to join us at the CR forum (you will need to register).
December 28, 2009
Caoyuan's Blog
New Web Site for blogtrader.net
I just re-launched blogtrader.net, which is now based on trac, but on glassfish V3.0 (moved to Jetty 7.0 later) via a custom CGIServlet. It's a headache to get a pretty url for trac, I may re-explore it later.
All old blogs should be imported back here, but not now.
by dcaoyuan at December 28, 2009 06:10 PM
Erlang Inside
The State of Erlang Web Middleware – EWGI and SimpleBridge
In the interest of publishing more often and covering smaller topics, lets review web-server-agnostic middleware for Erlang.
The current standard for Erlang web application middleware is EWGI, inspired by Python’s PEP 333 and providing similar functionality as Ruby’s Rack. All major Erlang web servers support EWGI, and most web frameworks.
Created by Rusty Klophaus, SimpleBridge “takes the pain out of coding to multiple Erlang web servers by creating a standardized interface.” It is used by the soon to be released new version of The Nitrogen Web Framework. According to Rusty, SimpleBridge has some improvements over EWGI, such as a smaller code base; more easily extendable (Takes about 150 lines to add support for a new HTTP server, vs. ~350 for EWGI; support for multipart file uploads, with size limits and handle-able errors; static file support; more specific interface functions for getting and setting cookies.
However, based on my experience with Rack, Multipart file uploads, static file support, and anything but basic cookie manipulation are all best handled by a web server such as nginx or a reverse proxy like HAProxy anyway. Perhaps it is better that EWGI tries to do less in this case…
Let’s look at a simple EWGI example with MochiWeb (shamelessly pulled from the docs)…
First we startup mochiweb calling the loop/1 function. Mochiweb will call our loop/1 function whenever a request is received.
In loop/1 we setup a call to our simple implementation of the EWGI interface.
Now the EWGI implementation is called with a tuple that aways starts with ewgi_context, and contains a Request and a Response. This code returns a 5-tuple response (further details in the documentation).
EWGI solves the problem of creating a standard interface to middleware components. It has become the standard, and the power of such standards lies in their implementations’ ubiquity. Making your own hit web framework at home in your spare time has never been easier.
by Chad DePue at December 28, 2009 03:34 PM
Caoyuan's Blog
About Caoyuan's Blog
Caoyuan's blogs about future.
For more information about this site see Wiki
by dcaoyuan at December 28, 2009 11:19 AM
Programming in the 21st Century
Functional Programming Doesn't Work (and what to do about it)
Read suddenly and in isolation, this may be easy to misinterpret, so I suggest first reading some past articles which have led to this point:
Admitting that Functional Programming Can Be Awkward
Follow-up to "Admitting that Functional Programming Can Be Awkward"
Back to the Basics of Functional Programming
Purely Functional Retrogames
Puzzle Languages
How I Learned to Stop Worrying and Love Erlang's Process Dictionary
After spending a long time in the functional programming world, and using Erlang as my go-to language for tricky problems, I've finally concluded that purely functional programming isn't worth it. It's not a failure because of soft issues such as marketing, but because the further you go down the purely functional road the more mental overhead is involved in writing complex programs. That sounds like a description of programming in general--problems get much more difficult to solve as they increase in scope--but it's much lower-level and specific than that. The kicker is that what's often a tremendous puzzle in Erlang (or Haskell) turns into straightforward code in Python or Perl or even C.
Imagine you've implemented a large program in a purely functional way. All the data is properly threaded in and out of functions, and there are no truly destructive updates to speak of. Now pick the two lowest-level and most isolated functions in the entire codebase. They're used all over the place, but are never called from the same modules. Now make these dependent on each other: function A behaves differently depending on the number of times function B has been called and vice-versa.
In C, this is easy! It can be done quickly and cleanly by adding some global variables. In purely functional code, this is somewhere between a major rearchitecting of the data flow and hopeless.
A second example: It's a common compilation technique for C and other imperative languages to convert programs to single-assignment form. That is, where variables are initialized and never changed. It's easy to mechanically convert a series of destructive updates into what's essentially pure code. Here's a simple statement:
if (a > 0) {
a++;
}
In single-assignment form a new variable is introduced to avoid modifying an existing variable, and the result is rather Erlangy:
if (a > 0) {
a1 = a + 1;
} else {
a1 = a;
}
The latter is cleaner in that you know variables won't change. They're not variables at all, but names for values. But writing the latter directly can be awkward. Depending on where you are in the code, the current value of whatever "a" represents has different names. Inserting a statement in the middle requires inventing new names for things, and you need to make sure you're referencing the right version. (There's more room for error now: you don't just say "a," but the name of the value you want in the current chain of calculations.)
In both of these examples imperative code is actually an optimization of the functional code. You could pass a global state in and out of every function in your program, but why not make that implicit? You could go through the pain of trying to write in single-assignment form directly, but as there's a mechanical translation from one to the other, why not use the form that's easier to write in?
At this point I should make it clear: functional programming is useful and important. Remember, it was developed as a way to make code easier to reason about and to avoid "spaghetti memory updates." The line between "imperative" and "functional" is blurry. If a Haskell program contains a BASIC-like domain specific language which is also written in Haskell, is the overall program functional or imperative? Does it matter?
For me, what has worked out is to go down the purely functional path as much as possible, but fall back on imperative techniques when too much code pressure has built up. Some cases of this are well-known and accepted, such as random number generation (where the seed is modified behind the scenes), and most any kind of I/O (where the position in the file is managed for you).
Learning how to find similar pressure relief valves in your own code takes practice.
One bit of advice I can offer is that going for the obvious solution of moving core data structures from functional to imperative code may not be the best approach. In the Pac-Man example from Purely Functional Retrogames, it's completely doable to write that old game in a purely functional style. The dependencies can be worked out; the data flow isn't really that bad. It still may be a messy endeavor, with lots of little bits of data to keep track of, and selectively moving parts out of the purely functional world will result in more manageable code. Now the obvious target is either the state of Pac-Man himself or the ghosts, but those are part of the core data flow of the program. Make those globally accessible and modifiable and all of a sudden a large part of the code has shifted from imperative to functional...and that wasn't the goal.
A better approach is to look for small, stateful, bits of data that get used in a variety of places, not just on the main data flow path. A good candidate in this example is the current game time (a.k.a. the number of elapsed frames). There's a clear precedent that time/date functions, such as Erlang's now(), cover up a bit of state, and that's what makes them useful. Another possibility is the score. It's a simple value that gets updated in a variety of situations. Making it a true global counter removes a whole layer of data threading, and it's simple: just have a function to add to the score counter and another function to retrieve the current value. No reason to add extra complexity just to dodge having a single global variable, something that a C / Python / Lua / Ruby programmer wouldn't even blink at.
(Also see the follow-up).
by James Hague at December 28, 2009 06:00 AM
December 25, 2009
Process-one Blogs
Merry Christmas and Happy New Year !
Year 2009 has been for us full of opportunities and allowed us to launch new promising software, like OneChannel the first XMPP publish and subscribe client, and to demonstrate our ability to innovate.
We still have lots of new features, software and improvements in the work: Oneteam Media Server, ProcessOne Wave server, new XMPP client, new online service portfolio, and many other surprises.
Year 2010 is going to be a great year for ProcessOne.
Thank you all for your ongoing support. Let's build the realtime web together !
by Mickaël Rémond at December 25, 2009 09:30 AM
December 23, 2009
Dukes of Erl
erlrc and rpm
Two of the Dukes now work at OpenX, which is starting to dip its toe into the Erlang waters. They use CentOS so they agreed to fund porting framewerk and erlrc to rpm; previously I'd only used them with deb.
A bit of background: erlrc is a set of Erlang modules and shell scripts that are designed to be integrated into packaging hooks so that installation, upgrade, or removal of a package causes corresponding hot-code activity to happen inside registered Erlang VMs on the box. Since erlrc was designed to easily integrate with multiple package managers, getting it to work with rpm was mostly about me understanding rpm's package hooks model. The result is an experience like this,
% sudo yum -q -y install lyet
erlrc-start: Starting 'lyet': (erlang) started
% sudo yum -q -y remove lyet
erlrc-stop: Stopping 'lyet': (erlang) unloaded
i.e., installing an rpm causes a running Erlang VM on the box to hot-code load the new modules and start the associated application, and removing the rpm causes the associated application to be stopped and the corresponding modules to be hot-code unloaded.
If you use fw-template-erlang than the appropriate packaging hooks are added for you automatically, both for deb and now rpm. However even manual creation of rpm spec files is pretty easy:
- erlrc-stop is called in %preun if the installation count indicates removal
- erlrc-upgrade is called in %preun if the installation count indicates upgrade
- erlrc-start is called in %posttrans
by Paul Mineiro (noreply@blogger.com) at December 23, 2009 03:12 PM
ejabberd@jabber.ru
ejabberd 2.1.1 - First bugfix release
There is a new bugfix release in the 2.1.x line: ejabberd 2.1.1. It contains several important bugfixes over last month's major release. None of the fixes is critical, but you are encouraged to update if you are running ejabberd 2.1.0.
List of bugfixes:
- Fix connection if starttls_required and zlib are set
- S2S: fix allow_host/2 on subdomains. added hook s2s_allow_host
- MUC: Add support for serving a Unique Room Name
- MUC: Route vCard request to the occupant bare JID
by badlop at December 23, 2009 03:09 PM
Process-one Blogs
[ANN] exmpp 0.9.2 new release
We are pleased to announce a new release of exmpp.
exmpp is a XMPP library written in Erlang and released under the Erlang Public License. It helps the development of XMPP entities by providing functions to build and interpret XML stanzas. Once compiled, check the generated HTML documentation.
List of main changes since the previous release six months ago:
- Adding support for stream error stanzas in exmpp_session when logged in
- Autodetect compilation parameters for Mac OS X Snow Leopard
- Component support for exmpp, and some fixes
- Fix segmentation fault with exmpp and expat
- New functions exmpp_client_pubsub:discover_nodes/1 and 2
- Stringprep Bidi checking doesn't reject strings with RandALCat and LCat characters
- queryns was not defined for IQ packets in received_packet record
The API of exmpp 0.9.2 is considered to be stable, so there aren't any
major changes expected in the API before reaching 1.0.0.
exmpp home page:
http://exmpp.org/
Download exmpp 0.9.2 source code package from:
http://download.process-one.net/exmpp/
You can also check the ProcessOne Labs page:
http://www.process-one.net/en/labs/
by Jérôme Sautret at December 23, 2009 02:56 PM
[ANN] ejabberd 2.1.1 bugfix release
We are pleased to announce the bugfix release ejabberd 2.1.1, which was published on the 17th of December. ejabberd 2.1.1 contains several important bugfixes over last month's major release. None of the fixes is critical, but you are encouraged to update if you are running ejabberd 2.1.0.
Short list of bugfixes:
- Fix connection if starttls_required and zlib are set
- S2S: fix allow_host/2 on subdomains. added hook s2s_allow_host
- MUC: Add support for serving a Unique Room Name
- MUC: Route vCard request to the occupant bare JID
- MUC: Support converting one-to-one chat to MUC
- PubSub: Receive same last published PEP items at reconnect if several resources online
- PubSub: Typo in mod_pubsub_odbc breaks Service Discovery and more
- Web: Fix memory and port leak when TLS is enabled in HTTP
- WebAdmin: report correct last activity with odbc backends
- Change captcha.sh to not depend on bash
- Generate main XML file also when exporting only a vhost
- Fix last newline in ejabberdctl result
- Guide: fix -setcookie, mod_pubsub_odbc host, content_types
Check the Release Notes for a more complete list of changes:
http://www.process-one.net/en/ejabberd/release_notes/release_note_ejabberd_2.1.1
If you upgrade from ejabberd 2.0.5 or older, read carefully the release notes, because there were several changes in the installation path and the configuration options.
The list of solved tickets since 2.1.0 is available on ProcessOne bug tracker:
http://redir.process-one.net/ejabberd-2.1.1
ejabberd 2.1.1 is available as source code package and binary installers for Linux 32 bits, 64 bits, Mac OS X Intel, and Windows:
http://www.process-one.net/en/ejabberd/downloads
by Jérôme Sautret at December 23, 2009 02:40 PM
December 22, 2009
Process-one Blogs
ProcessOne launches Talkr.IM: a new open, free public XMPP service
Talkr.IM is the new XMPP service provided by ProcessOne. It is an open and free service for all audiences worldwide.
What is talkr.IM?
In a few words, Talkr.IM offers scalable and robust presence and chat services for all standard XMPP clients of your choice. Just create an account now, by filling the registration form, and you will be able to login and start chats. It is fully interoperable with XMPP services such as Google Talk, Nokia Ovi, LiveJournal, Yandex Online, jabber.org, jabber.ru, jabbim.cz, jabberes.org, and jabber.fr for example.
Why talkr.IM?
Although this is a plain good XMPP service like others are providing, we want to provide a service that is professionally managed on a daily basis, with a strong service level, responsiveness and uptime.
We also emphasize on privacy and freedom. Privacy is there to ensure that your data are safe with us and will not be abused or transfered. We would like the service to be as much as a safe as possible. Freedom put the emphasize on using as much as free software as possible and ensure data portability. The service runs on free software and the data are yours. We plan to provide a migration feature. You will be able to import your data from another server, but we are also working on the reverse, so that you are able to export them and take them to another server, maybe your own server at some point. That's your data, that's your life and its important for you to keep the control on it.
Instant messaging conveys the value of the pionneer of Internet. One of the first simple messaging client was talk on various Unix flavour in the 1970's. It was free software and was a new way to communicate. Jabber and XMPP after that has been the heir of this tradition of freedom. Talkr.IM wants, like many other XMPP public servers to be a testimonial to this spirit: be free and open minded.
Why another XMPP hosting service ?
There are XMPP services backed by big companies like Google Talk, Nokia Ovi, LiveJournal or Yandex Online. There are also the valuable and much respected XMPP services administered by groups of volunteers and enthusiats, like jabber.org, jabber.ru, jabbim.cz, jabberes.org, and jabber.fr.
In none of these services, we had the opportunity to fully help, as much as we could, and really be implicated into the service administration and optimization.
With Talkr.IM, it is intended to provide a strong service to all users, which we can fully support. It is not intended to compete with any of the above mentionned XMPP services, as it is fully compliant and interoperated, in the pure tradition of the fair-play of the open Internet.
We believe we can achieve these goals since we are the makers of ejabberd. This work will of course benefit the product, in terms even more robustness, features, scalability and security.
Services
That said, besides the simple presence and IM, we also offer more services:
- conference will let you create and join multi-party chats with friends, family and colleagues (Multi user chat)
- irc is the gateway to the IRC networks that will enable you to join channels you wish from your XMPP client
- pubsub and PEP enables you to publish information for yourself and receive information from your contacts such as your mood, activity, geolocation, or the titles of the music you are listening to
- users is the directory that enables you to search for friends
The Talkr.IM service will also enhance the user experience in the future, by adding more services.
Feedback
Please feel free to express your needs for the Talkr.IM service and its evolution, as we are listening to and value your feedback.
Please join Talkr.IM, and contribute to the forums.
by Nicolas Vérité at December 22, 2009 03:47 PM
December 21, 2009
LShift on Erlang
Merry Christmas: Toke — Tokyo Cabinet driver for Erlang
Tokyo Cabinet is a rather excellent key-value store, with the ability to write to disk in a sane way (i.e. not just repeatedly dumping the same data over and over again), operate in bounded memory, and go really fast. I like it a lot, and there’s a likelihood that there’ll be a RabbitMQ plugin fairly soon that’ll use Tokyo Cabinet to improve the new persister yet further. Toke is an Erlang linked-in driver that allows you to use Tokyo Cabinet from Erlang.
There is already a Tokyo Cabinet driver for Erlang, tcerl, however, I couldn’t make it work: even after fixing the C so that it compiles (I hit this bug), I still couldn’t make it work. Inspecting the code, I get the feeling it’s bit-rotted - the Tokyo Cabinet API has moved on, and tcerl hasn’t kept up.
The other issue with tcerl is that it’s not a linked-in driver. Erlang allows two different types of drivers: the first are external C programs — these have a main() and run in their own process. Communication is done by stdin/stdout. These are a bit safer because if they crash they don’t take out the Erlang VM, but they’re never going to be blazingly fast. Toke, on the other hand, is a fully linked-in driver. It dynamically links with the Erlang VM, exists in the same address space and goes as fast as it possibly can (using the Erlang driver callbacks which avoid all copying of data passed from the Erlang). My tests show it’s about three times slower driving Tokyo Cabinet from Erlang via Toke, than driving it natively through C (which is quite good: some googling suggests both the Ruby and Python bindings to Tokyo Cabinet are rather slower). Toke is also about twice as slow as the Erlang ets module, which is in-memory only.
Toke only implements the Tokyo Cabinet hash table (tchdb*) functions, and doesn’t even support all of those: I only wrapped exactly what I needed. You’ll want to read the documentation for Tokyo Cabinet for these. The functions implemented are as follows (refer to the Tokyo Cabinet documentation to explain these further):
- toke_drv:new/1 — Set up the driver with a new TCHDB object.
- toke_drv:delete/1 — Destroy the driver’s TCHDB object.
- toke_drv:tune/5 — Tune the driver’s TCHDB Object.
- toke_drv:set_cache/2 — Set the number of records to cache.
- toke_drv:set_xm_size/2 — Set the extra amount of memory mapped in.
- toke_drv:set_df_unit/2 — Set the steps between auto defrag.
- toke_drv:open/3 — Open a db.
- toke_drv:close/1 — Close an open db.
- toke_drv:insert/3 — Insert. If the key already exists, value is updated.
- toke_drv:insert_new/3 — Insert new. If the key already exists, the old value is silently kept.
- toke_drv:insert_concat/3 — Concatenate the supplied value with an existing value for this key.
- toke_drv:insert_async/3 — Asynchronously insert. If the key already exists, value is updated.
- toke_drv:delete/2 — Delete a key from the db.
- toke_drv:get/2 — Fetch a key from the db. Returns ‘not_found’ on occasion.
- toke_drv:fold/3 — Fold over every value in the db. This internally uses the iteration functions. It’s just wrapped up as a fold to make it appear more functional.
- toke_drv:stop/1 — Stop the driver and close the port.
You should be able to use Mercurial to clone it:
# hg clone http://hg.opensource.lshift.net/toke/Make sure you have Tokyo Cabinet installed (ideally from source. If you’re using a package, make sure you have the development headers available. If you do compile from source, make sure you
ldconfig to make your system pick up the new library once it’s installed.). Then it should just be a case of make. There’s also a make run target that starts up an Erlang shell with the paths set up correctly for testing Toke:
toke# make run erl -pa ebin +K true +A30 Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:30] [hipe] [kernel-poll:true] Eshell V5.7.4 (abort with ^G) 1> toke_test:test(). passed 2>
Toke is licensed under the MPL. As ever, feedback is very welcome, as are patches!
by matthew at December 21, 2009 05:01 PM
December 18, 2009
RedHotErlang
Websockets and Yaws
<p>For years I've wanted to start tech blogging, however not having my own RSS/blogging software always put me off. After all, what lame webserver hacker am I, blogging on someone elses RSS software. Now, <br /> finally my friend Tobbe Tornquist did ...
by klacke at December 18, 2009 10:17 PM
Joe's Blog
Best Music of 2009.
I do this every year and figured 2009 should be no different. Here are my picks for the best albums and EPs of the year in no specific order.
Best albums:
Wavves – Wavves
Telefon Tel Aviv – Immolate Yourself
Riceboy Sleeps – Riceboy Sleeps
Dirty Projectors – Bitte Orca
Black Moth Super Rainbow – Eating Us
Animal Collective – Merriweather Post Pavilion
Japandriods – Post-Nothing
People Under the Stairs – Carried Away
Fuck Buttons – Tarot Sport
Russian Circles – Geneva
Best EPs:
Extra Life/Nat Baldwin – A Split
LITE – Turns Red
Abe Vigoda – Reviver
STATS – Marooned
by joe at December 18, 2009 06:12 PM
Process-one Blogs
Sea Beyond event summary
This thursday 17th of December, ProcessOne has held an event in Paris, France, around real-time communications. Here are the bits.
As announced and programmed, we held our event "Sea Beyond", on this thursday 17th of december. It took place at Le Before, an art gallery in the center of Paris. Usually, we have snow on the french capital only once a year... And the weather chose that special day, to cover the streets with the white jacket.
We had almost 40 attendants during the whole day, either for the XMPP Sandbox part for developpers, or the Lighthouse part for presentations.
We had a wide range of people, from pure players to operators, including manufacturers and freelancers, and even the press. Let's quote a few, like Nokia, Erlang Consulting, Yoono, af83, Ohm Force, StudiVZ, Meetic, Tandberg, Mozilla Foundation Europe, Orange, INRIA, Yoono, ...
During this real-time communications event, there has been a real sandbox as well as a real (tiny) lighthouse. There was also real pizzas for lunch, as well as real Champagne at the end of the day.
The first part of the day should have been a hacking session, at least this was what we had planned. But people took a lot of interest into workshops about PubSub and the Jingle Nodes. People then talked to each other about real-time technologies, and the future of internet and telecom. We have demoed our products, like the OneTeam client that we hope to release soon, also our supervision console TeamLeader. Mathieu Barcikowski also demonstrated Yoono to some people.
Then we reorganized the room for the presentations. We started with the lightning talks, with Philippe Sultan from INRIA, Asterisk developer and book author, who has presented the Asterisk module for Jingle. Nicolas Vérité from ProcessOne Project Manager, then presented the Talkr.IM XMPP service soon to be launched. Jérôme Sautret, ProcessOne CTO, did a summary of Sea Beyond XMPP sandbox. And Mathieu Barcikowski from Yoono, finished the session by demonstrating to the whole audience this time, the Yoono Firefox extension.
The presentations started with Thiago Camargo from Nimbuzz, who introduced his concept of Jingle Nodes, a network of XMPP peers for relay and relay discovery. This presentation has impressed quite a lot of people: the technology, as well as the ease of the Thiago on these subjects. Christophe Romain from ProcessOne the detailed the PubSub use at BBC, for their LiveText over IP in their radios channels in the browser. The session has been closed by Mickaël Rémond from ProcessOne, who talked and demoed our alpha products implementing the Wave real-time communication system from Google.
Then we had a cocktail to conclude the day, with once again ver interesting and enriching talks.
We believe people enjoyed the all day. We hope they enjoyed it as much as we did!
Thank you to everyone who joined and helped made this event a big success!
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
