Planet Trapexit - Erlang/OTP News

REALITY.SYS: Immer dieses Händewaschen…

Sat, 04 Jun 2011 08:23:13 +0000

Angesichts der, ähnlich SARS und Schweinepest, herbeigetrommelten Seuche “EHEC”, muss man sich ernsthaft die Frage stellen, ob wir in Sachen Hygiene nicht immer noch im Mittelalter stecken.

Hallo Leute! Sich die Hände richtig zu waschen ist eine Sache, die man normalerweise als Kleinkind gelernt haben muss!

Und die Betonung, dass man nach einem Toilettenbesuch die Hände waschen sollte, ist mehr als lächerlich. Es ist erschreckend, traurig und zeugt nur von unserer sterilen Lebensweise in dieser angeblich fortschrittlichen Welt. Ja, sterile Lebensweise, als Bezeichnung für unser Abgekoppelt-Sein von elementarsten menschlichen Ritualformen wie richtiges Waschen des eigenen Körpers, richtiger Umgang mit fremden Sachen (z.B. indem wir ausgestellte Waren nicht ohne Weiteres betatschen sollten!) etc.

Wie blamabel muss es denn für uns sein, wenn man uns über die Medien darum bitten muss, sich die Hände richtig zu wachen und das Gemüse bzw. Fleisch vor dem Verzehr ebenfalls gewaschen zu haben.

Aber nein, am leichtesten ist es doch, diesen blöden Südeuropäern alles in die Schuhe zu schieben. Ja, genau, die Spanier waren’s, diese gemeinen Kerle. Sie haben uns all das eingebrockt und siehe da: wir müssen uns jetzt die Hände waschen!

Process-one Blogs: “Google Social”: A paradox of its own

Thu, 02 Jun 2011 12:59:07 +0000

Yesterday was a sad day for the Internet, as an open and decentralized content repository.

At the same moment, Twitter launch its follow button and Google extend its +1 button for websites. They both are attempting to catch up with Facebook success with the "like button".

Of course, it feels ridiculous for content producers and consumers to expect to place more buttons on their pages in the hope to attract more traffic. The escalation in the battle for embedding buttons in web pages and the battle to get the users clicks is leading nowhere.

I would rather however focus on Google move, the paradox and possibly the trap it could become for the company. I have read numerous times that Google initiative in "social" area had always been a failure. Even Eric Schmidt admitted at D9 conference that he had messed up on Google Social initiative.

In my opinion, the main reason for failure is because social, as defined by today's market, is the complete opposite of Google model and view of the online world.

"Like buttons" are replacements for the HTML links that authors used to place on their webpage to talk about topics they liked and point to relevant content. But as HTML links were purely open and available for search engine web spiders, the end result of the "like button" action is closed and proprietary. Facebook and Twitter are one of the biggest threat for Google: they are cutting Google supply of their raw material: the data to analyze, both content and relations (links).

Google reactions yesterday has been to launch its own initiative, its own "like button" to gather its own data. However, doing so Google enter the trap of the closed Internet, in which they cannot prosper.

Google's model has been to compete on algorithm, not data. They have beaten all other search engine using the same data set than anyone could use. They hire smart scientists to produce clever mathematical solution to the world data analysis problem. Algorithm competition is their reason to exist: organizing the world data, not owning it.

Launching a "like button" is admitting that they were wrong in their core vision. Can the world be organized automatically by algorithm, from news to search ? It is an impossible equation to solve for them without deeper changes in Google's vision. In other words, the social graph that Facebook wants to build is actually what the web is about, and should be at the heart of the open web. Copying that closed vision of the web is a mistake.

What happen today looks like Yahoo!'s revenge (even if Yahoo! company is now out of the game): curation is the return of directories of content, with a larger number of curators and ranking depending on the proximity of the curators to yourself.

Google simply cannot win with this definition of the "social web". If social is about building competing subsets of Internet, defined by their user base, Google will decline on search, their core business. They cannot remain the entry point on the Internet for a large number of users, because they could not be able to bring them on every place you might want to go on the web.

The solution for Google requires to change the view of the social web, not to build their own clone of social network. Google social should be about putting people at the heart of an open Internet, where companies keeps on competing on algorithms. They already have valuable tools to rely on: google reader can share publicly what you like to read, blogger to publish your thought. They can build "Google People" around them.

I think that Google should keep on producing tools to make easy to share publicly what people would like to share, from status, to picture, though and links. This should be published in a well defined and documented way that will lead to a more open web. To be frank, such standards are emerging in many initiatives (webfinger, salmon, foaf, ostatus, opensocial and many others). Google promotes or even design several of those web protocols.

However, Google +1, until it leads to publicly available data on user profile, is a move in the wrong direction for Google. Let's hope for the Internet that their next move will open the data and bring the competition back to algorithms.

Credits: Photo by Blaise Alleyne

erlang.org RSS Feed: R14B03 released

Wed, 25 May 2011 16:00:00 +0000

Erlang/OTP R14B03 has been released as planned on May 25:th 2011. It is the third R14 service release.

See the release notes in the readme file

Download the new release from the download page.

Highlights:

Diameter is a brand new application in this release. The application support the diameter protocol specified in RFC 3588 and is intended to provide an Authentication, Authorization and Accounting (AAA) framework for applications.
The documentation for stdlib and kernel now uses type specifications from the source modules which should guarantee that the documentation and code are consistent with regard to the type information.

Erlang Factory News: Countdown to the Erlang Factory London!

Wed, 25 May 2011 09:01:32 +0000

There are only two weeks left to the Erlang Factory London. If you haven't registered yet, you can still do this by clicking here.

We have confirmed speakers including Robert Virding and Mike Williams (Inventors of Erlang), Tony Falco (COO of Basho Technologies), John Hughes (Inventor of QuickCheck) and Eric Merritt (Author of Erlang and OTP in Action). Many more are listed on the Speakers Page.

You are welcome to follow @erlangfactory on Twitter.

Programming in the 21st Century: Constantly Create

Sun, 22 May 2011 06:00:00 +0000

When I wrote Flickr as a Business Simulator, I was thinking purely about making a product--photos--and getting immediate feedback from a real audience. Seeing how much effort it takes to build-up a following. Learning if what you think people will like and what they actually like are the same thing.

It works just as well for learning what it's like to be in any kind of creative profession, such as an author of fiction or a recording artist.

Go look at music reviews on Amazon, and you'll see people puzzling over why a band's latest release doesn't have the spark of their earlier material, pointing out filler songs on albums, complaining about inconsistency between tracks. Sometimes the criticisms are empty, but there's often a ring of truth. There's an underlying question of why. How could a songwriter or band release material that isn't always at the pinnacle of perfection?

After years of posting photos to Flickr, I get it. I'm just going along, taking my odd photographs, when all of a sudden one resonates and breaks through and I watch the view numbers jump way up. Then I've got pressure: How can I follow that up? Sometimes I do, with a couple of winners in a row, but inevitably I can't stay at that level. Sometimes I take a break, not posting shots for a month or more, and then I lose all momentum.

When I'm at a low point, when I devolve into taking pictures of mundane subjects, pictures I know aren't good, I think about how I'm ever going to get out of that rut. Inevitably I do, though it's often a surprise when I go from a forgettable photo one day to something inspired the next.

The key for me is to keep going, to keep taking and posting photos. If I get all perfectionist then there's too much pressure, and I start second-guessing myself. If I give up when my quality drops off, then that's not solving anything. The steady progress of continual output, whether good or bad output, is part of the overall creative process.

Erlang Factory News: Early Bird Rate - Book before 22 May to save £100

Wed, 18 May 2011 09:16:37 +0000

Book your place now at the Erlang Factory London 2011 at the Early Bird Rate of £495 for the conference and £1295 for the University and conference. This is a saving of £100 off the standard price of the conference!Only valid until the 22 May!

Speakers so far include , Kostis Sagonas, the leader of the HIPE Team at Uppsala University, Marcus Kern, CTO at MIG, Steve Vinoski, distributed systems expert and Scott Lystig Fritchie, senior software engineer at Basho Technologies. Many more speakers are still to be announced.

Book now to secure this fantastic rate!

Trapexit's Erlang Blog Filter: More Erlang Web Server Benchmarking

Wed, 18 May 2011 05:54:45 +0000

In my previous blog entry I questioned the value of most web server benchmarking, particularly as related to Erlang. Typical benchmarks are misleading, inaccurate, and poorly executed. Perhaps worse, the intent of publishing them seems to be to assert that the fastest web server (at least according to the tests performed) is of course also the best web server. You’d think the flaws of this fallacy would be so obvious that nobody would fall for it, but think again: watching the delicious “erlang” tag over the past few days revealed the benchmarks my blog post referred to to be one of the most bookmarked Erlang-related pages during that timeframe.

Not surprisingly, though, it looks like I’m not the only one bothered by poor benchmarking practices. Over on his blog, Mark Nottingham just published a brilliant set of rules for HTTP load testing. It’s quite instructive to take your favorite set of published web server benchmarks and see just how many of Mark’s rules they violate.

Like I hinted last time, if you want benchmarks, you are best off by far if you run them yourself. That way, their relevance to the problems you’re addressing will be much more likely, and you can run them in a similar, or even the same, environment on which you plan to deploy. You can also gear the benchmarks to much more closely resemble your applications and the loads you require them to handle. Doing the benchmarking work yourself will give you valuable hands-on experience with the servers and frameworks you’re considering, allowing you to get a feel for important factors such as feature completeness and correctness, ease of development, flexibility, and ease of deployment and runtime management/monitoring, none of which can be gauged by someone else’s performance benchmarks. Finally, by doing your own benchmarking you can also help ensure the validity and usefulness of your results by following Mark’s load testing rules.

Erlang Inside: Motivated Reasoning and Erlang vs Python vs Node

Mon, 16 May 2011 15:51:08 +0000

A comparison between Misultin, Mochiweb, Cowboy, Node.JS, and Tornado. Tests such as this one tend to focus on either raw parsing speed or total number of concurrent connections, and which is more important depends on the ultimate application. Or they perfectly tune the tester’s favorite framework and use a stock configuration for the other competitors. [...]

Erlang Inside: Erlang in Haskell

Mon, 16 May 2011 03:49:18 +0000

Haskellers who want to experiment with the concepts of Erlang in Haskell can now do so with an experimental Erlang-like distributed computing framework called ‘remote’. It’s quite basic but covers concurrency, selective receive of messages, local and global registration of processes, and trapping exits. Related Microsoft Research Paper.

Scattered Thoughts: mist

Wed, 11 May 2011 09:40:59 +0000

I have a new secret project. It's called Mist. The goal is to make it easy to build, use, distribute and maintain small-scale p2p apps. By small-scale I mean apps where each invocation has at most a few dozen users. You would use Mist to build the next etherpad, not the next bittorrent.

The main inspiration behind Mist is Sugar. Sugar supports seamless sharing of activities (applications) across local mesh networks or the internet, even if some of the peers don't even have the activity installed. Where Sugar is focused on providing a working system here and now, I am more interested in how distributed programming can be made easier in the future. I also want to build applications that I can use myself so my initial target is Ubuntu/Gnome.

Starting from September, I'm going to be doing just enough freelance work to support full time work on Mist. I'm aware that this is a huge project and probably doomed to failure so there a few key principles that I hope will keep me on track. Wherever possible I will reuse existing code and protocols. Mist will be built out of small components each of which are useful by themselves. Where practical Mist will be compatible with Sugar, so that any improvements can be folded back into the Sugar codebase. Finally, Mist is a prototype. The goal will be to get something up and running to experiment with different ways of building distributed apps.

So without further ado here is a misty outline of what Mist might one day look like:

Seamless connectivity, serverless presence and activity discovery
- Mostly borrowed from Sugar
- Apps communicate over Telepathy tubes
- Use Sugar presence service for activity discovery
- LAN / ad-hoc networks supported by Telepathy Salut and Avahi
- Centralized communication supported using Telepathy Gabble to talk to Jabber servers
- Decentralized communication supported using a Telepathy plugin for erl-telehash
  - Use telehash peers for ICE
  - Use link-local XMMP to talk
  - Use telehash taps to implement Avahi compatible mDNS
- Use pgp for authentication and end-to-end encryption
Securely portable applications
- Use an existing system. Some options which will need to be researched:
  - Bitfrost is specific to the XO laptop and probably too hard to port
  - Browsers provide well-tested sandboxes and mobile code but limit access to hardware. Perhaps ChromeOS or Firefox Chromeless?
  - Java apps can possibly be installed securely. Looking at the Android source may be instructive.
Development tools for distributed apps
- Want to experiment with Bloom. I have some ideas for static analysis of bloom programs that might be fun.
- Expose libraries/APIs for common patterns eg leader election, operational transform
- Support serverless syncing of contacts, bookmarks, mail etc using DHT with enforced reciprocal storage (pretty sure I've read a paper on this somewhere)

The immediate plan for the next few months (during which I'm still working full time) is:

Get erl-telehash working (currently has no test suite and doesn't support taps, _ring/_line or rate limiting)
Get ICE working between telehash peers
Write a Telepathy backend using erl-telehash and link-local XMPP
Extend Empathy to use the telehash Telepathy backend

If you are wondering where Mist gets its name from - it's not the cloud.

Trapexit's Erlang Blog Filter: Erlang Web Server Benchmarking

Tue, 10 May 2011 02:01:01 +0000

Over on his blog, Roberto Ostinelli published “A comparison between Misultin, Mochiweb, Cowboy, NodeJS and Tornadoweb.” I was going to write a reply comment there, but it got pretty long so I decided to publish it here instead. I’m going to ignore the non-Erlang web servers it discusses and focus entirely on Erlang. I’m not trying to really pick specifically on Roberto here, but rather I decided to finally write something I’ve been meaning to write for awhile now about Erlang web servers and benchmarking.

First, I second the request made by one commenter for including Yaws in the measurements. Roberto, if you need help with the code or setup, just let me know. If one insists on writing these kinds of benchmarks, which as you’ll learn if you read this whole entry is something I question, the least he or she could do is include Yaws since it’s the granddaddy of all Erlang web servers.

Based on the benchmark code Roberto published, I wrote the following simple Yaws module to conform to the problem statement and registered it in my Yaws configuration as a “/” appmod:

-module(yaws_bench).
-export([out/1]). out(Arg) -> [{status, 200}, {content, "text/xml", case yaws_api:queryvar(Arg, "value") of {ok, Value} -> ["<http_test><value>", Value, "</value></http_test>"]; _ -> "<http_test><error>no value specified</error></http_test>" end}].

I then measured it on my Ubuntu 10.10 two-core system using Roberto’s published httperf command against the misultin and Mochiweb code he published, and found that Yaws definitely holds its own, even though it’s a full-featured web server and does not claim to be just a lightweight library offering (sometimes partial) HTTP support as some frameworks do. For some tests Yaws outperforms misultin, and for others it doesn’t. This is interesting, considering that neither Klacke nor I have made any attempts at performance improvements in Yaws recently.

Second, the benchmarks do not compare apples to apples. Both Mochiweb and Yaws, for example, produce replies that are larger in size than misultin’s replies, primarily because they both include Server and Date headers. As I’ve learned from years of helping maintain Yaws, date calculations can noticeably and surprisingly impact Erlang web server performance, yet simply leaving Date headers out isn’t an option for real-world apps since HTTP 1.1 pretty much requires them (section 13.2.3 of RFC 2616 states, “HTTP/1.1 requires origin servers to send a Date header, if possible, with every response, giving the time at which the response was generated…”). Caches use Date headers for several reasons, for example in the absence of cache-control headers to help heuristically calculate content expiration. Even ignoring the date calculation requirements, just creating and delivering larger replies due to the presence of the Server header will negatively impact any comparisons based on request/second measurements.

Third, the benchmarking approach includes no application “think time.” How many real-world apps just blast request after request down a connection without any intervening time to handle replies? If the goal is to measure something akin to real-world apps, then the benchmarks should at least be using something like httperf’s --wsess option to simulate client think time. And unfortunately doing that is hard to get right for generic benchmarks, since different client apps will have different think times.

On a related note, what exactly is the goal of these benchmarks? To imply that faster is better? That’s unfortunately a commonly-held fallacy. Given that the blog entry states that the target is dynamic applications, then consider the fact that the performance of a real-world dynamic application is often dominated by something other than the web server — perhaps some back-end service from which page data is being fetched, for example. A real-world setup greatly concerned with performance is likely to have nginx out in front, probably with a local cache, to handle fast-path requests, shunting only those requests it can’t fulfill off to the slower back-end server. Such benchmarking games are therefore often misguided as far as real-world dynamic apps are concerned because they end up measuring something that isn’t even in the critical path in a real setup.

I don’t agree with Kyle Drake’s comment on Roberto’s blog about code ugliness, since the Erlang code posted there is very clear and would look like “garbage” only to someone who doesn’t know the language. But I do agree with the sentiment, which is that for dynamic apps, what often matters is what kind of code, and how much, you have to write and maintain to support your app. Given that Erlang web servers tend to make use of the underlying Erlang/OTP facilities for HTTP parsing and socket handling, then all things considered you’re just not going to get a huge variation in performance among them, assuming they’re written halfway decently. What matters for dynamic apps are the stability of the web server/library and the programming model it offers. These are what Roberto should really be benchmarking, but of course that’s basically impossible since stability would take a long time to prove, and programming model is a matter of taste that can’t be conveniently measured using artificial benchmarking tools. This reminds me of one of my old columns on this very issue as applied to enterprise middleware, entitled “The Performance Presumption” (PDF); the short version is that people often measure performance simply because performance is relatively easy to measure. The lesson is that you shouldn’t rely on generic benchmarks, but rather you should take the time to create specific benchmarks that mimic the app you want to develop, and base your decisions on the results of that exercise.

On top of all that, I don’t really understand the desire to keep writing new Erlang web frameworks for performance reasons. As I stated earlier, if a framework uses Erlang’s built-in packet decoding and socket handling, it won’t perform a great deal better than any other Erlang web framework. OTOH, if someone writes a new framework with the hope of providing a really nice new programming model — webmachine is a fantastic example of this — then they shouldn’t be “proving” how good the programming model is by trying to show how fast it is. Ever seen webmachine being advertised via performance benchmarks? Neither have I.

Let’s face it, the Erlang web development community isn’t large enough to support numerous web servers and frameworks. I’m sure some will disagree, but publishing artificial benchmarks designed to “prove” which is best IMO results mostly in just fragmenting the community. If you really have an itch to write a fast Erlang web server, you’d help the community much more by contributing to an existing one, including the Erlang inets web server included in Erlang/OTP and now powering the Erlang website. For Yaws, Klacke and I often take patches and suggestions from our users, and we gladly welcome solid contributions intended to improve Yaws performance. If you’re just dying to show off your chops, note that improving performance in a long-lived and highly stable codebase like Yaws without breaking anyone’s code is far more challenging than writing another new server that basically doesn’t differ much from what already exists.

Or perhaps better yet, contribute to the Erlang core. IMO the next major performance improvements in Erlang web servers will come not from minor tweaks in handling binaries or such things, but rather via radical improvements in the Erlang TCP driver or even from developing a whole new HTTP-specific driver. Unlike a war of artificial benchmarks among Erlang web servers, these approaches have a great chance to improve the lot of all Erlang web systems.

erlang.org RSS Feed: ACM SIGPLAN Erlang Workshop in Tokyo on September 23, 2011

Fri, 06 May 2011 23:08:27 +0000

ACM SIGPLAN Erlang Workshop

The Tenth ACM SIGPLAN Erlang Workshop will take place in Tokyo, Japan, on September 23, 2011. Please see the call for papers here .

Erlang Factory News: Book your place at the Erlang Factory London Now and save £100!

Wed, 04 May 2011 10:31:39 +0000

Book now to secure this fantastic rate!

Anders Conbere's Journal: Learning to Program is Easy

Wed, 04 May 2011 01:42:12 +0000

Years ago I spent a summer teaching science classes to kids aged 6 to 12 at the Oregon Museum of Science and Industry. This program wasn't particularly well managed, mostly a disaster, but quite a bit of fun. The most fun I had was helping teach a class on game programming to 5th and 6th graders. The class used a pretty nice implementation of Logo called MicroWorlds which benefited from having a simple GUI and IDE for the kids to use. Maybe today we'd use something like Scratch or EToys. Either way the class began with a simple introduction to making things happen in this virtual world, and some of the basics of the command structure. And within minutes there was a volley of hands that shot into the air each with it's very own bug.

That might not sounds like much, but by the time you've engaged someone to the point that they've typed in some commands and arrived at an unexpected occurrence... you've basically got them hook line and sinker. So that day and some of the next was often spent detailing the use of variables, the intricacies of syntax, and what rules exist and why they make their life easier. By the end of the week we would have flies narrowly dodging venus fly traps, and robots that had to defeat monsters.

I'm not saying every kid produced a work of genius, and some had more difficulties than others. But the nature of programming does not have to be a complex and difficult venture. Simply mapping input to a behavior gets you pretty far. This is a big part of the idea of Scheme. How do we build a language that is simple enough that our english majors can accomplish something in it, but powerful enough that our computer science students can develop complex ideas with it. The point is that most people can be taught to create at least rudimentary programs in just a few hours of tutorial.

And this really brings me to the reason I'm writing this post. And that is, that while programming might be easy, making applications, architecting large projects, putting all the various pieces together in a way that gives you the flexibility to accomplish what you might need to, but the structure to finishing what you have to, is amazingly difficult. And two different skills. You can teach someone to use python like a calculator, and to write functions that help them manage their bills. But the ability to take that knowledge and expand upon it to write a personal finance applications is much more difficult to teach.

Some of this is abstraction. Abstraction is difficult for humans, and I say this knowing that of all the creatures we've ever encountered we're likely the most capable abstractors in existence. But while that might be true, or brains are wired to work inside a rather constrained and physical world. There's a test that's often mentioned in literature where they ask the participants to accomplish a mathematical task using variable names, and then ask it again relating it to cards. In the first case most participants answer incorrectly, while in the next they do it with ease. The great difficult of mathematics as well as programming arrises from the depths of abstraction.

Just like Mathematics might be the deep study one abstract layer after another, complex application design mirrors this. While in mathematics at the bottom might be something like Category Theory from which derives Algebra from which derive the infinity of number systems from which derive Real and Complex numbers from which derive Real Analysis and then the Calculuses and on down the chain. A Complex application might have a User Interface which abstracts a Programable Interface which calls down into a Data Layer that then talks to a Persistence Layer that talks to the Operating System on down to the bare metal. Most of the extremely competent application architects I know might be considered a software version of a "tall, thin man". Someone who is familiar with all the ramifications of his design from the top layers to the cost at the bottom layers.

What I'm trying to say is that Programming might be easy. But the understanding of both the ramifications of the current environment (what can be done with the current language and tools) and the cost of the various layers below it, is hard. And that teaching THAT is much more difficult task than teaching someone how to use a fancy calculator. Books like SICP attempt to accomplish this task by dealing with abstraction in it's own right, by merging programing and mathematics and enabling a kind of cross pollination there of. But even that can leave a student with little understanding of how to put all the pieces together. I write this because my focus since college has been into the art of programming. And I'm beginning to understand where the limitations of my studies have been, and where I'm headed. And my new goal of trying to find more people to look at my code and find new ways of expressing those problems.

Erlang Factory News: Only 5 Places left at the Erlang Factory London!

Tue, 03 May 2011 15:02:00 +0000

There are only 5 chances left to book your place at the Erlang Factory London 2011 at the Very Early Bird Rate of £395 for the conference and £1195 for the University and conference. This is a saving of £200 off the standard price of the conference!When they are gone, they are gone!

Book now to secure this fantastic rate!

erlang.org RSS Feed: The Erlang Factory London is back!

Tue, 03 May 2011 09:10:31 +0000

The Erlang Factory London is back! The dates you need for your diary are 6th, 7th and 8th June for the Erlang University courses and 9th and 10th June for the Erlang Factory Conference.

There are 10 places left at the very Early bird rate of £395 which is a saving of £200! Book now to get your place!

Programming in the 21st Century: Impressed by Slow Code

Sat, 30 Apr 2011 06:00:00 +0000

At one time I was interested in--even enthralled by--low-level optimization.

Beautiful and clever tricks abound. Got a function call followed by a return statement? Replace the pair with a single jump instruction. Once you've realized that "load effective address" operations are actually doing math, then they can subsume short sequences of adds and shifts. On processors with fast "count leading zero bits" instructions, entire loops can be replaced with a couple of lines of linear code.

I spent a long time doing that before I realized it was a mechanical process.

I don't necessarily mean mechanical in the "a good compiler can do the same thing" sense, but that it's a raw engineering problem to take a function and make it faster. Take a simple routine that potentially loops through a lot of data, like a case insensitive string comparison. The first step is to get as many instructions out of the loop as possible. See if what remains can be rephrased using fewer or more efficient instructions. Can any of the calculations be replaced with a small table? Is there a way to process multiple elements at the same time using vector instructions?

The truth is that there's no magic in taking a well-understood, working function, analyzing it, and rewriting it in a way that involves doing slightly or even dramatically less work at run-time. If I ended up with a routine that was a bottleneck, I know I could take the time to make it faster. Or someone else could. Or if it was small enough I could post it to an assembly language programming forum and come back in a couple of days when the dust settled.

What's much more interesting is speeding up something complex, a program where all the time isn't going into a couple of obvious hotspots.

All of a sudden, that view through the low-level magnifying glass is misleading. Yes, that's clearly an N-squared algorithm right there, but it may not matter at all. (It might only get called with with low values of N, for example.) This loop here contains many extraneous instructions, but that's hardly a big picture view. None of this helps with understanding the overall data flow, how much computation is really being done, and where the potential for simplification lies.

Working at that level, it makes sense to use a language that keeps you from thinking about exactly how your code maps to the underlying hardware. It can take a bit of faith to set aside deeply ingrained instincts about performance and concerns with low-level benchmarks, but I've seen Python programs that ended up faster than C. I've seen complex programs running under the Erlang virtual machine that are done executing before my finger is off the return key.

And that's what's impressive: code that is so easy to label as slow upon first glance, code containing functions that can--in isolation--be definitively proven to be dozens or hundreds of times slower than what's possible on a given CPU, and yet the overall program is decidedly one of high performance.

(If you liked this, you might enjoy Timidity Does Not Convince.)

Learn You Some Erlang: Building an Application With OTP

Fri, 29 Apr 2011 12:00:00 +0000

This chapter lets us make practical use of the OTP behaviours seen so far. We do this by writing a process pool application that will let us handle resources and tasks. We explore ideas behind process trees, onion layer theories for processes and general views of how OTP can be used to write software.

Erlang Factory News: Erlang Factory London Open for Registration - Save £200

Wed, 20 Apr 2011 09:16:52 +0000

Be one of the first 50 people to book your place at the Erlang Factory London 2011 at the Very Early Bird Rate of £395 for the conference and £1195 for the University and conference. This is a saving of £200 off the standard price of the conference!

Book now to secure this fantastic rate!

Programming in the 21st Century: Follow the Vibrancy

Wed, 20 Apr 2011 06:00:00 +0000

Back in 1999 or 2000, I started reading a now-defunct Linux game news site. I thought the combination of enthusiastic people wanting to write video games and the excitement surrounding both Linux and open source would result in a vibrant, creative community.

Instead there were endless emulators and uninspired rewrites of stale old games.

I could theorize about why there was such a lack of spark, a lack of motivation to create anything distinctive and exciting. Perhaps most of the projects were intended to fulfill coding itches, not personal visions. I don't know. But I lost interest, and I stopped following that site

When I wanted to modernize my programming skills, I took a long look at Lisp. It's a beautiful and powerful language, but I was put off by the community. It was a justifiably smug community, yes, but it was an empty smugness. Where were the people using this amazing technology to build impressive applications? Why was everyone so touchy and defensive? That doesn't directly point at the language being flawed--not by any means--but it seemed an indication that something wasn't right, that maybe there was a reason that people driven to push boundaries and create new experiences weren't drawn to the tremendous purported advantages of Lisp. So I moved on.

Vibrancy is an indicator of worthwhile technology. If people are excited, if there's a community of developers more concerned with building things than advocating or justifying, then that's a good place to be. "Worthwhile" may not mean the best or fastest, but I'll take enthusiasm and creativity over either of those.

(If you liked this, you might enjoy The Pure Tech Side is the Dark Side.)

Scattered Thoughts: telehash: router

Tue, 19 Apr 2011 18:24:35 +0000

Now that we have all the necessary datastructures we can build the router itself. Most of the routing table logic is handled by the bit_tree and bucket modules. The router just ties these together and handles I/O.

Before actually running the routing table the router has to find out its own address, as it is seen from the outside world. It does this by sending +end signals to a list of known telehash nodes (eg telehash.org:42424).

record(bootstrap, { % the state of the router when bootstrapping
          timeout, % give up if no address received before this time
          addresses % list of addresses contacted to find out our address
         }).

bootstrap(Addresses, Timeout) ->
    ?INFO([bootstrapping]),
    State = #bootstrap{timeout=Timeout, addresses=Addresses},
    {ok, _Pid} = gen_server:start_link(?MODULE, State, []).
                          
init(State) ->
    switch:listen(),
    case State of
        #bootstrap{timeout=Timeout, addresses=Addresses} ->
            Telex = telex:end_signal(util:random_end()),
            lists:foreach(fun (Address) -> switch:send(Address, Telex) end, Addresses),
            erlang:send_after(Timeout, self(), giveup);
        #state{} ->
            ok
    end,
    {ok, State}.

Then we listen until we either get a reply with a _to field or run out of time.

handle_info({switch, {recv, From, Telex}}, #bootstrap{addresses=Addresses}=Bootstrap) ->
    % bootstrapping, waiting to receive a message telling us our own address
    case {lists:member(From, Addresses), telex:get(Telex, '_to')} of
        {true, {ok, Binary}} ->
            try util:to_end(util:binary_to_address(Binary)) of
                End ->
                    Self = util:to_bits(End),
                    Table = touched(From, Self, empty_table(Self)),
                    dialer:dial(End, [From], ?ROUTER_DIAL_TIMEOUT),
                    refresh(Self, Table),
                    ?INFO([bootstrap, finished, {self, Binary}, {from, From}]),
                    {noreply, #state{self=Self, pinged=sets:new(), table=Table}}
            catch
                _ ->
                    ?WARN([bootstrap, bad_self, {self, Binary}, {from, From}]),
                    {noreply, Bootstrap}
            end;
        _ ->
            {noreply, Bootstrap}
    end;

handle_info(giveup, #bootstrap{}=Bootstrap) ->
    % failed to bootstrap, die
    ?INFO([giveup, {state, Bootstrap}]),
    {stop, {shutdown, gaveup}, Bootstrap};

Once we know our own address we can fill in the state record and start managing the routing table.

-record(state, { % the state of the router in normal operation
          self, % the bits of the routers own end
          pinged, % set of addresses which have been pinged and not yet replied/timedout
          table % the routing table, a bit_tree containing buckets of nodes
         }).

One of the jobs of the router is to remove unresponsive nodes from the routing table. To check if a node is responsive we just a random +end signal and wait for a reply. If the node is unresponsive it gets marked as stale and we try to find a suitable replacement. The node won't actually be dropped from the table until a replacement is found - this prevents the table from getting flushed if our network connection goes down.

ping(To) ->
    Telex = telex:end_signal(util:random_end()),
    % do this in a message to self to avoid some awkward control flow
    self() ! {pinging, To},
    switch:send(To, Telex),
    erlang:send_after(?ROUTER_PING_TIMEOUT, self(), {timeout, Address}).

timedout(Address, Self, Table) ->
    bit_tree:update(
      fun (_Suffix, _Depth, _Gap, Bucket) ->
              case bucket:timedout(Address, Bucket) of
                  {node, Node, Update} ->
                      % try to touch this node, might be suitable replacement
                      ping(Node),
                      Update;
                  Update ->
                      Update
              end
      end,
      util:to_bits(Address),
      Self,
      Table
     ).

handle_info({pinging, Address}, #state{pinged=Pinged}=State) ->
    % do this in a message to self to avoid some awkward control flow
    ?INFO([recording_ping, {address, Address}]),
    Pinged2 = sets:add_element(Address, Pinged),
    {noreply, State#state{pinged=Pinged2}};
handle_info({timeout, Address}, #state{self=Self, pinged=Pinged, table=Table}=State) ->
    case lists:member(Address, Pinged) of
        true ->
            % ping timedout
            ?INFO([timeout, {address, Address}]),
            Table2 = timedout(Address, Self, Table),
            {ok, State#state{table=Table2}};
        false ->
            % address already replied
            {ok, State}
    end;

One of the rules of the router is that it should never pass on information about a node that it hasn't personally confirmed to exist. Once we receive a message from a node we know that it exists (later we will implement _ring/_line to protect against address spoofing):

touched(Address, Self, Table) ->
    bit_tree:update(
      fun (Suffix, _Depth, Gap, Bucket) ->
              May_split = (Gap < ?K), % !!! or (Depth < ?ROUTER_TABLE_EXPANSION)
              bucket:touched(Address, Suffix, now(), Bucket, May_split)
      end,
      util:to_bits(Address),
      Self,
      Table
     ).

On receiving a .see command we record all the contained addresses as potential nodes and ping them to try to confirm their existence.

seen(Address, Self, Table) ->
    bit_tree:update(
      fun (Suffix, _Depth, _Gap, Bucket) ->
              case bucket:seen(Address, Suffix, now(), Bucket) of
                  {node, Node, Update} ->
                      % check if this node is stale
                      ping(Node),
                      Update;
                  Update ->
                      Update
              end
      end,
      util:to_bits(Address),
      Self,
      Table
     ).

On receiving a +end signal we reply with a .see command containing the nearest K addresses which we have confirmed to exist.

see(To, End, Table) ->
    Telex = telex:see_command(nearest(?K, End, Table)),
    switch:send(To, Telex).

nearest(N, End, Table) when N>=0 ->
    Bits = util:to_bits(End),
    iter:take(
      N,
      iter:flatten(
        iter:map(
          fun ({_Prefix, Bucket}) -> bucket:by_dist(End, Bucket) end,
          bit_tree:iter(Bits, Table)))).

On receiving a message we have handle the above three cases, which gets a little ugly.

handle_info({switch, {recv, From, Telex}}, #state{self=Self, pinged=Pinged, table=Table}=State) ->
    % this counts as a reply
    Pinged2 = sets:del_element(From, Pinged),
    % touched the sender
    % !!! eventually will check _line here
    ?INFO([touched, {node, From}]),
    Table2 = touched(From, Self, Table),
    % maybe seen some nodes
    Table3 =
        case telex:get(Telex, '.see') of
            {ok, Binaries} ->
                try [util:binary_to_address(Bin) || Bin <- Binaries] of
                    Addresses ->
                        ?INFO([seen, {nodes, Addresses}, {from, From}]),
                        lists:foldl(fun (Address, Table_acc) -> seen(Address, Self, Table_acc) end, Table2, Addresses)
                catch
                    _ ->
                        ?INFO([bad_seen, {nodes, Binaries}, {from, From}]),
                        Table2
                end;
            _ ->
                Table2
        end,
    % maybe send some nodes back
    case telex:get(Telex, '+end') of
        {ok, Hex} ->
            try util:hex_to_end(Hex) of
                End ->
                    ?INFO([see, {'end', End}, {from, From}]),
                    see(From, End, Table3)
            catch
                _ ->
                    ?WARN([bad_see, {'end', Hex}, {from, From}])
            end;
        _ ->
            ok
    end,
    {noreply, State#state{pinged=Pinged2, table=Table2}};

The last responsibility of the router is to periodically refresh buckets which haven't recently seen any activity.

handle_info(refresh, #state{self=Self, table=Table}=State) ->
    ?INFO([refreshing_table]),
    refresh(Self, Table),
    {noreply, State};
handle_info({dialed, _, _}, State) ->
    % response from a bucket refresh, we don't care
    {noreply, State};

dialed(Address, Self, Table) ->
    bit_tree:update(
      fun (_Suffix, _Depth, _Gap, Bucket) ->
              bucket:dialed(now(), Bucket)
      end,
      util:to_bits(Address),
      Self,
      Table
     ).

needs_refresh(Bucket, Now) ->
    case bucket:last_dialed(Bucket) of
        never ->
            true;
        Last ->
            (timer:now_diff(Now, Last) div 1000) < ?ROUTER_REFRESH_TIME
    end.

refresh(Self, Table) ->
    Now = now(),
    iter:foreach(
      fun ({Prefix, Bucket}) ->
              case needs_refresh(Bucket, Now) of
                  true ->
                      ?INFO([refreshing_bucket, {prefix, Prefix}, {bucket, Bucket}]),
                      To = util:random_end(Prefix),
                      From = nearest(?K, To, Table),
                      dialer:dial(To, From, ?ROUTER_DIAL_TIMEOUT);
                  false ->
                      ok
              end
      end,
      bit_tree:iter(Self, Table)
     ),
    erlang:send_after(?ROUTER_REFRESH_TIME, self(), refresh),
    ok.

That's it. As usual the (untested) code is in the repo. The next post will probably deal with taps.

Scattered Thoughts: telehash: gen_event woes

Tue, 19 Apr 2011 17:33:57 +0000

I ran into some tricky bugs caused by a misconception I had about gen_event. Since this is not explicitly stated in the gen_event documentation I will say it here: gen_event does NOT spawn individual processes for each handler. Each handler is run sequentially in the event manager process.

Now obviously the documentation is not at fault here. I assumed that each handler got its own process solely because the callbacks resembled gen_server. However, a little googling reveals that several other people made the same mistake so I thought it was worth mentioning.

Here is how I found this out. I was working on the router implementation for telehash. When I tested the bootstrapping algorithm everything looked fine until the first dial, after which nothing else happened. Straight away I suspected a bug in the dialer, but repeating the exact same call in the console worked fine. After a few deadends I opened pman to look for anything suspicious but couldn't find the dialer process (because it doesn't exist, it's an event handler). I assumed that it was somehow crashing silently and wasted an hour or so reading and rereading the code and stepping through various calls in the debugger. No matter what I tried the dialer worked absolutely perfectly unless it was called by the router.

Eventually I noticed that the switch_event process was blocking inside a receive and the whole thing unravelled. The dialer is an event handler so when started it calls:

gen_event:add_handler(switch_event, dialer, State)

which is a synchronous call to the switch_event process. The router event handler is running inside the switch_event process so when the router tries to dial it deadlocks.

The moral of this story is RTFM.

This is easily fixed by changing

dialer:dial(End, [Address])

spawn(fun () -> dialer:dial(End, [Address]) end)

but there were more problems. Most of the event handlers used erlang:send_after to handle timeouts but since they all run in the same process they all receive each others timeouts. Also, every event handler is run sequentially so the switch_event process becomes a huge bottleneck.

The solution I settled on was to change each event handler into a gen_server and write a simple event handler that just forwards events to its owner. By using gen_event:add_sup_handler and listening for event handler exits we can keep the two in sync.

Erlang Factory News: Erlang Factory SF Bay Area 2011 - A Great Success