Planet Trapexit - Erlang/OTP News

Caoyuan's Blog: How the Functional Ability of Scala Comparing to Haskell - an Example

2010-03-15T21:06:29+00:00

About one and half year ago, it was my first time to consider Scala seriously, I wrote a blog about the syntax example of Scala, Erlang and Haskell .

With more experience of Scala, I'd like to know how about the functional ability of Scala comparing to Haskell. I picked up Paul R. Brown's perpubplat blog engine again, and tried to port more code from Haskell to Scala, and here's the code example of Entry.scala in Scala comparing to Brown's original Entry.hs:

Original Haskell code piece

-- | Data structures for an item (post or comment) and the
-- overall structure in terms of parents and children.
module Blog.Model.Entry where
import qualified Blog.FrontEnd.Urls as U
import Utilities
import qualified Blog.Constants as C
import Maybe
import List ( sortBy, isPrefixOf, intersperse)
import qualified Data.Map as M
import Data.Map ( (!) )
type ISO8601DatetimeString = String
type XhtmlString = String
-- | Overall data model for the runtime.  
data Model = Model { -- | 
                     by_permatitle :: M.Map String Item,
                     by_int_id :: M.Map Int Item,
                     child_map :: M.Map Int [Int],
                     all_items :: [Item],
                     next_id :: Int }
empty :: Model
empty = Model M.empty M.empty M.empty [] 0
data Kind = Post | Comment | Trackback
            deriving (Show, Read, Eq)
build_model :: [Item] -> Model
build_model [] = empty
build_model items = Model (map_by permatitle sorted_items)
                    bid
                    (build_child_map sorted_items)
                    (sorted_items)
                    (n+1)
    where
      sorted_items = sort_by_created_reverse items
      bid = (map_by internal_id sorted_items)
      n = fst . M.findMax $ bid
build_child_map :: [Item] -> M.Map Int [Int]
build_child_map i = build_child_map_ (M.fromList $ (map (\x -> (internal_id x,[])) i)) i
-- Constructed to take advantage of the input being in sorted order.
build_child_map_ :: M.Map Int [Int] -> [Item] -> M.Map Int [Int]
build_child_map_ m [] = m
build_child_map_ m (i:is) = if (parent i == Nothing) then
                                build_child_map_ m is
                            else
                                build_child_map_ (M.insertWith (++) (unwrap $ parent i) [internal_id i] m) is
-- | Insert an item, presuming that all of its data other than
-- internal identifier have been correctly set.
insert :: Model -> Item -> (Item,Model)
insert m i = (i', m { by_permatitle = M.insert (permatitle i') i' $ by_permatitle m
                    , by_int_id = M.insert n i' $ by_int_id m
                    , child_map = M.insert (internal_id i') [] $
                                  case parent i of
                                    Nothing ->
                                        child_map m
                                    (Just p_id) ->
                                        M.insert p_id (insert_comment_ m (item_by_id m p_id) i') $ child_map m
                    , all_items = insert_ after (all_items m) i'
                    , next_id = n + 1 } )
    where
      n = next_id m
      i' = i { internal_id = n }
insert_comment_ :: Model -> Item -> Item -> [Int]
insert_comment_ m p c = map internal_id (insert_ before (children m p) c)
insert_ :: (Item -> Item -> Bool) -> [Item] -> Item -> [Item]
insert_ _ [] y = [y]
insert_ o s@(x:xs) y = if (x `o` y) then
                           (x:(insert_ o xs y))
                       else
                           (y:s)
after :: Item -> Item -> Bool
after a b = (created a) > (created b)
before :: Item -> Item -> Bool
before a b = (created a) < (created b)
-- | Apply a structure-preserving function, i.e., one that does not
-- change parent/child relationships or ids, to a specific item.
alter :: (Item -> Item) -> Model -> Item -> IO Model
alter f m i = do { ts <- now
                 ; let i' = (f i) { updated = ts }
                 ; return $ m { by_permatitle = M.insert (permatitle i') i' $ by_permatitle m
                              , by_int_id = M.insert (internal_id i') i' $ by_int_id m
                              , child_map = if (parent i == Nothing) then
                                                child_map m
                                            else
                                                M.insert p_id resort_siblings $ child_map m
                              , all_items = insert_ after all_but i' } }
    where
      not_i = \item -> (internal_id item) /= (internal_id i)
      all_but = filter not_i $ all_items m
      p_id = unwrap $ parent i
      p = item_by_id m p_id
      resort_siblings = map internal_id (insert_ before (filter not_i $ children m p) i)
cloak :: Model -> Item -> IO Model
cloak = alter (\i -> i { visible = False })
uncloak :: Model -> Item -> IO Model
uncloak = alter (\i -> i { visible = True })
permatitle_exists :: Model -> String -> Bool
permatitle_exists = (flip M.member) . by_permatitle
max_id :: Model -> Int
max_id = fst . M.findMax . by_int_id
post_by_permatitle :: Model -> String -> Item
post_by_permatitle = (!) . by_permatitle
maybe_post_by_permatitle :: Model -> String -> Maybe Item
maybe_post_by_permatitle = (flip M.lookup) . by_permatitle
item_by_id :: Model -> Int -> Item
item_by_id = (!) . by_int_id
children :: Model -> Item -> [Item]
children m i = map (item_by_id m) ((child_map m) ! (internal_id i))
unwrap :: Maybe a -> a
unwrap (Just x) = x
unwrap Nothing = error "Can't unwrap nothing!"
data Author = Author { name :: String,
                       uri :: Maybe String,
                       email :: Maybe String,
                       show_email :: Bool
                     }
              deriving ( Show,Read,Eq )
-- | General purpose runtime data structure for holding a post or
-- comment.  For a comment, a number of the fields will be ignored
-- (e.g., comments and tags) until/if the presentation and syndication
-- system gets fancier.
data Item = Item { -- | an internal unique number for this post
                   internal_id :: Int,
                   -- | the kind of item that this represents
                   kind :: Kind,
                   -- | the title of the post, as it should be rendered on
                   -- the web or inserted in an Atom feed; this should be a
                   -- valid XHTML fragment.
                   title :: XhtmlString,
                   -- | the summary of the post, as it should be rendered on
                   -- the web or intersted into an Atom feed; this should be
                   -- a valid XHTML fragment.
                   summary :: Maybe XhtmlString,
                   -- | the body of the post as an XHTML fragment.  This
                   -- will be wrapped in an XHTML @<div>@ when rendered on
                   -- the web or in a feed.
                   body :: XhtmlString,
                   -- | tags for the post, if any, expected to be in
                   -- alphabetical order and consisting of letters, digits,
                   -- dashes, and/or underscores.
                   tags :: [String],
                   -- | a generated UID for the post; this is expected to be
                   -- suitable for use as an Atom GUID.  The expectation is
                   -- that it will be supplied by the implementation when
                   -- the post is ingested.
                   uid :: String,
                   -- | a permanent title for the item, consisting of only
                   -- lowercase letters, digits, and dashes.
                   permatitle :: String,
                   -- | the timestamp, as an ISO8601 datetime, when the post
                   -- came into being.  This is never blank and would be
                   -- supplied by the implementation when the post is
                   -- ingested.
                   created :: ISO8601DatetimeString,
                   -- | the timestamp, as an ISO8601 datetime, when the post
                   -- was updated.  Initially, this is equal to the value of
                   -- the 'created' field.
                   updated :: ISO8601DatetimeString,
                   -- | the author of the post, expected to be hardwired to
                   -- the author of the blog
                   author :: Author,
                   -- | whether or not the item is to be displayed.
                   visible :: Bool,
                   -- | this item's parent, if any.
                   parent :: Maybe Int
                 }
            deriving ( Show, Read, Eq )
-- | Compute a permalink for the item relative to the supplied base URL.
permalink :: Model
          -> Item -- ^ the item
          -> String
permalink m i = U.post (relative_url m i)
relative_url :: Model -> Item -> String
relative_url m = _form_permalink . (ancestors m)
_form_permalink :: [Item] -> String
_form_permalink [] = ""
_form_permalink [i] =  let s = permatitle i in
                       if (kind i == Post) then
                           "/" ++ s
                       else
                           "#" ++ s
_form_permalink (i:is) = if (kind i == Post) then
                            ("/" ++ permatitle i) ++ (_form_permalink is)
                        else
                            (_form_permalink is)
ancestor_path :: Model -> Item -> String
ancestor_path m i = concat . (intersperse "/") . (map permatitle) $ ancestors m i
ancestors :: Model -> Item -> [Item]
ancestors m i = ancestors_ m [] (Just $ internal_id i)
ancestors_ :: Model -> [Item] -> Maybe Int -> [Item]
ancestors_ _ is Nothing = is
ancestors_ m is (Just i) = ancestors_ m (i':is) (parent i')
    where
      i' = item_by_id m i
lastUpdated :: [Item] -> ISO8601DatetimeString
lastUpdated ps = maximum (map updated ps)
drop_invisible :: [Item] -> [Item]
drop_invisible = filter visible
sort_by_created :: [Item] -> [Item]
sort_by_created = sortBy created_sort
created_sort :: Item -> Item -> Ordering
created_sort a b = compare (created a) (created b)
sort_by_created_reverse :: [Item] -> [Item]
sort_by_created_reverse = sortBy created_sort_reverse
created_sort_reverse :: Item -> Item -> Ordering
created_sort_reverse a b = compare (created b) (created a)
-- | Filter a list of items according to a date fragment
date_fragment_filter_ :: ISO8601DatetimeString -> [Item] -> [Item]
date_fragment_filter_ s = filter ((s `isPrefixOf`) . created)
-- | Filter a list of posts for those made in a specific year.
year_filter :: Int -- ^ year
            -> [Item] -> [Item]
year_filter y = date_fragment_filter_ $ show y
-- | Filter a list of posts for those made in a specific month.              
month_filter :: Int -- ^ year
             -> Int -- ^ month
             -> [Item] -> [Item]
month_filter y m | (0 < m) && (m < 13) = date_fragment_filter_ ((show y) ++ (pad_ m))
                 | otherwise = const []
-- | Filter a list of posts for those made on a specific day
day_filter :: Int -- ^ year
           -> Int -- ^ month
           -> Int -- ^ day
           -> [Item] -> [Item]
day_filter y m d = date_fragment_filter_ ((show y) ++ (pad_ m) ++ (pad_ d))
-- | Utility function to zero pad months and days in date expressions.
pad_ :: Int -> String
pad_ i | i < 10 = "-0" ++ (show i)
       | otherwise = ('-':(show i))
-- to do: make this faster using the sortedness.
tags_filter :: [String] -> [Item] -> [Item]
tags_filter t p = foldl (flip ($)) p (map tag_filter t)
tag_filter :: String -> [Item] -> [Item]
tag_filter t = filter ((t `elem`) . tags)
plink_filterf :: String -> Item -> Bool
plink_filterf = flip $ (==) . permatitle
plink_filter :: String -> [Item] -> [Item]
plink_filter = filter . plink_filterf
ymd_plink_finder :: Int -> Int -> Int -> String -> [Item] -> [Item]
ymd_plink_finder y m d t = (plink_filter t) . (day_filter y m d)
all_posts :: Model -> [Item]
all_posts = (filter (\x -> Post == kind x)) . all_items
all_comments :: Model -> [Item]
all_comments = (filter (\x -> Comment == kind x)) . all_items
flatten :: Model -> [Item] -> [Item]
flatten m = flatten_ (children m)
flatten_ :: (a -> [a]) -> [a] -> [a]
flatten_ _ [] = []
flatten_ f (i:is) = (i:(flatten_ f (f i))) ++ (flatten_ f is)
concat_comments :: Model -> [Item] -> [Item]
concat_comments m = (foldr (++) []) . (map $ children m)
(</>) :: String -> String -> String
s </> t = s ++ ('/':t)
to_string :: Item -> String
to_string i = concat [metadata i, "\n", body_block i, "\n", summary_block i]
metadata :: Item -> String
metadata i = unlines $ apply i [ ("internal_id",show . internal_id),
                                 ("parent", show . parent),
                                 ("title",title),
                                 ("tags",show_no_quotes . tags),
                                 ("permatitle",permatitle),
                                 ("kind",show . kind),
                                 ("uid",uid),
                                 ("created",created),
                                 ("updated",updated),
                                 ("author",show . author),
                                 ("visible",show . visible) ]
show_no_quotes :: [String] -> String
show_no_quotes = concat . (intersperse ", ")
apply :: Item -> [(String,(Item -> String))] -> [String]
apply _ [] = []
apply i (x:xs) = ((concat [fst x, ": ", (snd x) i]) : (apply i xs))
body_block :: Item -> String
body_block i = concat ["--- START BODY ---\n",
                       (body i),
                       "\n--- END BODY ---\n"]
summary_block :: Item -> String
summary_block i | summary i == Nothing = ""
                | otherwise = concat ["--- START SUMMARY ---\n",
                                      (unwrap $ summary i),
                                      "\n--- END SUMMARY ---\n"]
default_author :: Author
default_author = Author C.author_name C.author_uri C.author_email True

In Scala:

package org.aiotrade.blog.model
import java.util.Calendar
import org.aiotrade.blog.{Constants => C}
object Entry {
  type XhtmlString = String
  class Model (
    var by_permatitle: Map[String, Item],
    var by_int_id: Map[Int, Item],
    var child_map: Map[Int, List[Int]],
    var all_items: List[Item],
    var next_id: Int
  ) {
    // call by name
    def apply(block: => Unit) = {block; this}
  }
  abstract class Kind //deriving (Show, Read, Eq)
  case object Post extends Kind
  case object Comment extends Kind
  case object Trackback extends Kind
  class Author (
    var name: String,
    var uri : Option[String] = None,
    var email: Option[String] = None,
    var show_email: Boolean = false
  )
  /** General purpose runtime data structure for holding a post or
   * comment.  For a comment, a number of the fields will be ignored
   * (e.g., comments and tags) until/if the presentation and syndication
   * system gets fancier.
   */
  class Item {
    // an internal unique number for this post
    var internalId: Int = _
    // the kind of item that this represents
    var kind: Kind = _
    // the title of the post, as it should be rendered on
    // the web or inserted in an Atom feed; this should be a
    // valid XHTML fragment.
    var title: XhtmlString = _
    // the summary of the post, as it should be rendered on
    // the web or intersted into an Atom feed; this should be
    // a valid XHTML fragment.
    var summary: Option[XhtmlString] = None
    // the body of the post as an XHTML fragment.  This
    // will be wrapped in an XHTML @<div>@ when rendered on
    // the web or in a feed.
    var body: XhtmlString = _
    // tags for the post, if any, expected to be in
    // alphabetical order and consisting of letters, digits,
    // dashes, and/or underscores.
    var tags: List[String] = _
    // a generated UID for the post; this is expected to be
    // suitable for use as an Atom GUID.  The expectation is
    // that it will be supplied by the implementation when
    // the post is ingested.
    var uid: String = _
    // a permanent title for the item, consisting of only
    // lowercase letters, digits, and dashes.
    var permatitle: String = _
    // the timestamp, as an ISO8601 datetime, when the post
    // came into being.  This is never blank and would be
    // supplied by the implementation when the post is
    // ingested.
    var created: Long = _
    // the timestamp, as an ISO8601 datetime, when the post
    // was updated.  Initially, this is equal to the value of
    // the 'created' field.
    var updated: Long = _
    //the author of the post, expected to be hardwired to
    // the author of the blog
    var author: Author = _
    //whether or not the item is to be displayed.
    var visible: Boolean = _
    //this item's parent, if any.
    var parent: Option[Int] = None
    def apply(block: Item => Unit) = {block(this); this}
  }
  def empty = new Model(Map(), Map(), Map(), Nil, 0)
  def build_model(is: List[Item]) = is match {
    case Nil => empty
    case _ =>
      val sortedIs = sortByCreatedReverse(is)
      val bid = Map() ++ sortedIs.map{x => (x.internalId -> x)}
      val n = bid.keySet.max
      new Model(Map() ++ sortedIs.map{x => (x.permatitle -> x)},
                bid,
                buildChildMap(sortedIs),
                sortedIs,
                n + 1)
  }
  def buildChildMap(is: List[Item]) = buildChildMap_(Map() ++ is.map(_.internalId -> Nil), is)
  def buildChildMap_(map: Map[Int, List[Int]], is: List[Item]) =
    map ++ {
      for (i <- is if i.parent.isDefined) yield {
        // pid, cids definitions go into body // it's more efficient.
        val pid = i.parent.get
        val cids = map.getOrElse(pid, Nil)
        pid -> (i.internalId :: cids)
      }
    }
  def sortByCreatedReverse(is: List[Item]) = is.sortWith{ _.created < _.created }
  /** Insert an item, presuming that all of its data other than
   internal identifier have been correctly set.
   */
  def insert(m: Model, i: Item): (Item, Model) = {
    val n = m.next_id
    i.internalId = n
    (i, m {
        m.by_permatitle += (i.permatitle -> i)
        m.by_int_id += (n -> i)
        m.child_map = (i.parent match {
            case None => m.child_map
            case Some(p_id) => m.child_map + (p_id -> (insert_comment_(m, item_by_id(m)(p_id), i)))
          })
        m.all_items = insert_(after, m.all_items, i)
        m.next_id = n + 1 }
    )
  }
  def insert_comment_(m: Model, p: Item, c: Item): List[Int] =
    insert_(before, children(m)(p), c) map (_.internalId)
  def insert_(o: (Item, Item) => Boolean, is: List[Item], y: Item): List[Item] = is match {
    case Nil => List(y)
    case x :: xs => if (o(x, y)) x :: insert_(o, xs, y) else (y :: is)
  }
  def after (a: Item, b: Item): Boolean = a.created > b.created
  def before(a: Item, b: Item): Boolean = a.created < b.created
  /**
   * Apply a structure-preserving function, i.e., one that does not
   * change parent/child relationships or ids, to a specific item.
   */
  def alter(f: (Item => Item), m: Model, i: Item) = {// -> IO Model
    val not_i = (item: Item) => item.internalId != i.internalId
    val all_but = m.all_items filter not_i
    val p_id = unwrap (i.parent)
    val p = item_by_id(m)(p_id)
    val resort_siblings = insert_(before, children(m)(p) filter not_i, i) map (_.internalId)
    val ts = System.currentTimeMillis
    val i1 = f(i) {_.updated = ts}
    m {
      m.by_permatitle += (i1.permatitle -> i1)
      m.by_int_id += (i1.internalId -> i1)
      m.child_map = i.parent match {
        case None => m.child_map
        case _ => m.child_map + (p_id -> resort_siblings)
      }
      m.all_items = insert_(after, all_but, i1)
    }
  }
  def cloak(m: Model, i: Item) = // -> IO Model
    alter (i => i {_.visible = false}, m, i)
  def uncloak(m: Model, i: Item) = // -> IO Model
    alter (i => i {_.visible = true}, m, i)
  def permatitle_exists(m: Model, p: String): Boolean =
    m.by_permatitle.contains(p)
  def max_id(m: Model): Int =
    m.by_int_id.keySet.max
  def post_by_permatitle(m: Model, p: String): Item =
    m.by_permatitle(p)
  def maybe_post_by_permatitle(m: Model, p: String): Option[Item] =
    m.by_permatitle.get(p)
  def item_by_id(m: Model)(id: Int): Item = m.by_int_id(id)
  def children(m: Model)(i: Item): List[Item] =
    m.child_map(i.internalId) map (item_by_id(m))
  def unwrap[T](a: Option[T]): T = a match {
    case Some(x) => x
    case None => error("Can't unwrap none!")
  }
  def relative_url(m: Model, i: Item): String =
    _form_permalink(ancestors(m, i))
  def _form_permalink(is: List[Item]): String = is match {
    case Nil => ""
    case i :: Nil =>
      val s = i.permatitle
      if (i.kind == Post)
        "/" + s
      else
        "#" + s
    case i :: is =>
      if (i.kind == Post)
        ("/" + i.permatitle) + _form_permalink(is)
      else
        _form_permalink(is)
  }
  def ancestors(m: Model, i: Item): List[Item] =
    ancestors_(m, Nil, Some(i.internalId))
  def ancestors_(m: Model, is: List[Item], i_? : Option[Int]): List[Item] = i_? match {
    case None => is
    case Some(i) =>
      val i1 = item_by_id(m)(i)
      ancestors_(m, i1 :: is, i1.parent)
  }
  def lastUpdated(ps: List[Item]): Long =
    ps map (_.updated) max
  def drop_invisible(is: List[Item]): List[Item] =
    is filter (_.visible)
  def sort_by_created(is: List[Item]): List[Item] =
    is sortWith created_sort _
  def created_sort(a: Item, b: Item) =
    a.created < b.created
  def sort_by_created_reverse(is: List[Item]): List[Item] =
    is sortWith created_sort_reverse _
  def created_sort_reverse(a: Item, b: Item) =
    b.created < a.created
  def date_fragment_filter_(is: List[Item], ts: Int*) = {
    val cal = Calendar.getInstance
    ts match {
      case Seq(y, m, d) => is filter {i => pad_(cal, i.created) match {
            case (`y`, `m`, `d`) => true
            case _ => false
          }
        }
      case Seq(y, m) => is filter {i => pad_(cal, i.created) match {
            case (`y`, `m`, _) => true
            case _ => false
          }
        }
      case Seq(y) => is filter {i => pad_(cal, i.created) match {
            case (`y`, _, _) => true
            case _ => false
          }
        }
    }
  }
  def year_filter(y: Int)(is: List[Item]): List[Item] =
    date_fragment_filter_(is, y)
  def month_filter(y: Int, m: Int)(is: List[Item]): List[Item] =
    date_fragment_filter_(is, y, m)
  def day_filter(y: Int, m: Int, d: Int)(is: List[Item]): List[Item] =
    date_fragment_filter_(is, y, m, d)
  def pad_(cal: Calendar, t: Long): (Int, Int, Int) = {
    cal.setTimeInMillis(t)
    import Calendar._
    (cal.get(YEAR), cal.get(MONTH) + 1, cal.get(DAY_OF_MONTH))
  }
  def tags_filter(ts: List[String])(is: List[Item]): List[Item] =
    is filter (i => (false /: ts) {_ || i.tags.contains(_)})
  def tag_filter(t: String)(is: List[Item]): List[Item] =
    is filter (_.tags.contains(t))
  def plink_filterf(p: String)(i: Item): Boolean =
    i.permatitle == p
  def plink_filter(p: String)(is: List[Item]): List[Item] =
    is filter plink_filterf(p)
  def ymd_plink_finder(y: Int, m: Int, d: Int, p: String)(is: List[Item]): List[Item] =
    plink_filter(p) (day_filter(y, m, d) (is))
  def </> (s: String, t: String): String =
    s + '/' + t
  def all_posts(m: Model): List[Item] = m.all_items filter (_.kind == Post)
  def all_comments(m: Model): List[Item] = m.all_items filter (_.kind == Comment)
  def flatten(m: Model, is: List[Item]): List[Item] =
    flatten_(children(m), is)
  def flatten_[T](f: T => List[T], is: List[T]): List[T] = is match {
    case Nil => Nil
    case i :: is => i :: flatten_(f, f(i)) ::: flatten_(f, is)
  }
  def concat_comments(m: Model, is: List[Item]): List[Item] =
    ((is map children(m)) :\ List[Item]())(_ ::: _)
  def metadata(i: Item): String =
    apply(i, List(("internal_id", show(_.internalId)),
                  ("parent",      show(_.parent)),
                  ("title",       _.title),
                  ("tags",        show_no_quotes(_.tags)),
                  ("permatitle",  _.permatitle),
                  ("kind",        show(_.kind)),
                  ("uid",         _.uid),
                  ("created",     show(_.created)),
                  ("updated",     show(_.updated)),
                  ("author",      show(_.author)),
                  ("visible",     show(_.visible)))
    ) mkString "\n"
  // curring
  def show(f: Item => Any)(i: Item): String =
    f(i) toString
  def show_no_quotes(f: Item => List[String])(i: Item): String =
    f(i) mkString ", "
  def apply(i: Item, xs: List[(String, Item => String)]): List[String] = xs match {
    case Nil => Nil
    case x :: xs => (x._1 + ": " + x._2(i)) :: apply(i, xs)
  }
  def body_block(i: Item): String =
    "--- START BODY ---\n" + i.body + "\n--- END BODY ---\n"
  def summary_block(i: Item): String = i.summary match {
    case None => ""
    case Some(x) => "--- START SUMMARY ---\n" + x + "\n--- END SUMMARY ---\n"
  }
  val default_author = new Author(C.author_name, C.author_uri, C.author_email, true)
}

Damien Katz: O'Reilly's CouchDB: The Definitive Guide for $9.99 Today Only!

Damien Katz — 2010-03-15T19:00:39+00:00

CouchDB: The Definitive Guide

Use this discount code: DDCDB

Trapexit's Erlang Blog Filter: New Column on Webmachine

2010-03-15T06:19:06+00:00

“The Functional Web” column is finally back, this time with a column about Webmachine co-authored with Justin Sheehy. The column title is Developing RESTful Web Services with Webmachine, and you can follow that link to retrieve the PDF.

Webmachine is a highly innovative web application framework, and it can teach you a great deal about the specifics of HTTP and the details of REST. It’s also written in Erlang, which continues to be my favorite programming language of all time because of its incredible practicality, utility, and elegance.

My column hiatus was due to extreme startup workload, which for better or worse is showing no sign of letting up anytime soon. But it’s nice to get the column back on track for the March/April Internet Computing issue, and one of my goals is to avoid missing any more issues this year. Many thanks to Justin for his contribution to this issue of the column.

ejabberd@jabber.ru: ejabberd 2.1.3 - Third bugfix release

2010-03-12T16:28:03+00:00

ejabberd 2.1.3 has been released. It contains many bugfixes and some improvements.

This is a short list of changes:

New ejabberd_c2s option: max_fsm_queue
ejabberdctl: Support concurrent connections with bound connection names
Cross-domain HTTP-Bind support (EJAB-1168)
Hibernate http-bind process after handling a request

Erlang Factory News: New Speaker for the Erlang Factory SF Bay Area 2010

2010-03-12T12:45:55+00:00

We would like to announce Geoff Cant as a Speaker for the Erlang Factory SF Bay Area. Geoff will give a talk on "Enet: TCP/IP in Pure(ish) Erlang".

To see all Speakers please click here. The programme of the conference is available here.

Reminder: The Erlang Factory in SF Bay Area is just over a week away! Do not miss out on what will be the biggest gathering of Erlang expertise in the USA, register today!

Process-one Blogs: OneTeam XMPP client entering private alpha

2010-03-11T14:00:21+00:00

Our OneTeam XMPP client for Linux, Mac OS X, and Windows is entering a private alpha period.

OneTeam is a multiplatform XMPP client for Linux, Mac OS X, and Windows. We have slowly worked on it during long months (and even years, it is being developed since 2006) but never had enough time and resources to release it properly.

The time to launch OneTeam in the wild and interoperable internet is soon arriving. We believe we have achieved our goal to deliver some real enterprise-class user experience. As a proof, we are using it inhouse on a daily basis as a productivity tool, without any major glitch.

For the record, OneTeam works as a Firefox extension, as well as a standalone application, on the three major platforms, thanks to the XUL technology. OneTeam has a nice dashboard-based interface, and handy user interactions improvements everywhere. This XMPP client supports a wide range of features, including chat and presence of course, but also Multi-User Chat (groupchat), message threading, history browsing, etc. It also features Jingle voice calls, as well as the exclusive Jingle Nodes feature demoed at FOSDEM, that works much like Skype. On the more technical side, there is the XML console, as well as a very interesting remote debugging console.

So, we are now releasing OneTeam, but in alpha quality for now, to the curious ones, and those willing to help and test. Please feel free to contact us, if you want to join the alpha testing, and report issues. Please e-mail nverite at process-one.net for a OneTeam ride.

Damien Katz: Win a Nexus One from Couchio

Damien Katz — 2010-03-11T01:58:29+00:00

Port CouchDB to an Android app and get our phone! Details.

Programming in the 21st Century: Eleven Years of Erlang

James Hague — 2010-03-10T06:00:00+00:00

I've written about how I started using Erlang. A good question is why, after eleven years, am I still using it?

For the record, I do use other languages. I enjoy writing Python code, and I've taught other people how to use Python. This website is statically generated by a Perl program that I had fun writing. And I dabble in various languages of the month which have cropped up. (Another website I used to maintain was generated by a script that I kept reimplementing. It started out written in Perl, but transitioned through at least REBOL, J, and Erlang before I was through.)

One of the two big reasons I've stuck with Erlang is because of its simplicity. The functional core of Erlang can and has been described in a couple of short chapters. Knowledge of four data types--numbers, atoms, lists, tuples--is enough for most programming problems. Binaries and funs can be tackled later. This simplicity is good, because the difficult part of Erlang and any mostly-functional language is in learning to write code without destructive updates. The language itself shouldn't pour complexity on top of that.

There are many possibilities for extending Erlang with new data types, with an alternative to records being high on the list. Should strings be split off from lists into a distinct entity? What about arrays of floats, so there's no need to box each value? How about a "machine integer" type that's represented without tagging and that doesn't get automatically promoted to an arbitrarily sized "big number" when needed?

All of those additional types are optimizations. Lists work just fine as strings, but even the most naive implementation of strings as unicode arrays would take half the memory of the equivalent lists, and that's powerful enticement. When Knuth warned of premature optimization, I like to think he wasn't talking so much about obfuscating code in the process of micro-optimizing for speed, but he was pointing out that code is made faster by specializing it. The process of specialization reduces your options, and you end up with a solution that's more focused and at the same time more brittle. You don't want to do that until you really need to.

It may be an overreaction to my years of optimization-focused programming, but I like the philosophy of making the Erlang system fast without just caving in and providing C-style abilities. I know how to write low-level C. And now I know how to write good high-level functional code. If I had been presented with a menu of optimization-oriented data types in Erlang, that might never have happened. I'd be writing C in the guise of Erlang.

The second reason I'm still using Erlang is because I understand it. I don't mean I know how to code in it, I mean I get it all the way down. I know more or less what transformations are applied by the compiler and the BEAM loader. I know how the BEAM virtual machine works. And unlike most languages, Erlang holds together as a full system. You could decide to ditch all existing C compilers and CPUs and start over completely, and Erlang could serve as a foundation for this new world of computing. The ECOMP project (warning: PowerPoint) proved that an FPGA running the Erlang VM directly gives impressive results.

Let me zoom in on one specific detail of the Erlang runtime. If you take an arbitrary piece of data in a language of the Lua or Python family, at the lowest-level it ends up wrapped inside a C struct. There's a type field, maybe a reference count, and because it's a heap allocated block of memory there's other hidden overhead that comes along with any dynamic allocation (such as the size of the block). Lua is unabashedly reliant on malloc-like heap management for just about everything.

Erlang memory handling is much more basic. There's a block of memory per process, and it grows from bottom to top until full. Most data objects aren't wrapped in structs. A tuple, for example, is one cell of data for the length followed by the number of cells in the tuple. The system identifies it as a tuple by tagging the pointer to the tuple. You know the memory used for a tuple is always 1 + N, period. Were I trying to optimize data representation by hand, with the caveat that type info needs to be included, it would be tough to do significantly better.

I'm sure some people are correctly pointing out that this is how most Lisp and Scheme systems have worked since those languages were developed. There's nothing preventing an imperative language from using the same methods (and indeed this is sometimes the case).

Erlang takes this further by having a separate block of memory for each process, so when the block gets full only that particular block needs to be garbage collected. If it's a 64K block, it's takes microseconds to collect, as compared to potentially traversing a heap containing the hundreds of megabytes of data in the full running system. Disallowing destructive updates allows some nice optimizations in the garbage collector, because pointers are guaranteed to reference older objects (this is sometimes called a "unidirectional heap"). Together these are much simpler than building a real-time garbage collector that can survive under the pressure of giant heaps.

Would I use Erlang for everything? Of course not. Erlang is clearly a bad match for some types of programming. It would be silly to force-fit Erlang into the iPhone, for example, with Apple promoting Objective C as the one true way. But it's the best mix of power and simplicity that I've come across.

Erlang Inside: Zotonic destroys WordPress and rethinks the CMS with Erlang

2010-03-09T18:50:24+00:00

A chat with Marc Worrell, Lead Architect of Zotonic - a new Content Management System written entirely in Erlang.

Damien Katz: CouchDB Case Study: Assay Depot

Damien Katz — 2010-03-06T23:40:35+00:00

Apache CouchDB Case Study: Assay Depot

...replication give this ability that would have been much more difficult to accomplish in MySQL. We could have done this in a relational manner but it would have been very challenging, very custom. CouchDB give a lot of the benefits for free.

CouchDB has changed the way I think about developing web applications

Programming in the 21st Century: It Made Sense in 1978

James Hague — 2010-03-04T06:00:00+00:00

Whenever I see this list of memory cell sizes, it strikes me as antiquated:

BYTE = 8 bits
WORD = 16 bits
LONG = 32 bits

Those names were standard for both the Intel x86 and Motorola 68000 families of processors, and it's easy to see where they came from. "Word" isn't synonymous with a 16-bit value; it refers to the fundamental data size that a computer architecture is built to operate upon. On a 16-bit CPU like the 8086, a word is naturally 16-bits.

Now it's 2010, and it's silly to think of a 16-bit value as a basic enough unit of data to get to the designation "word." "Long" is similarly out of place, as 32-bit microprocessors have been around for over 25 years, and yet the standard memory cell size is still labeled in a way that makes it sound abnormally large.

The PowerPC folks got this right back in the early 1990s with this nomenclature:

BYTE = 8 bits
HALFWORD = 16 bits
WORD = 32 bits

That made sense in 1991, and it's still rational today. (64-bit is now common, but the jump isn't nearly as critical as it was the last time memory cell size doubled. The PowerPC name for "64-bits" is "doubleword.")

Occasionally you need to reevaluate your assumptions and not just cling to something because it's always been that way.

Process-one Blogs: Talkr.IM XMPP/Jabber server gets Apple Push notifications

2010-03-03T21:00:16+00:00

The free public XMPP/Jabber server Talkr.IM gets Apple Push notifications for iPhone.

The ProcessOne's free, open, public XMPP/Jabber server Talkr.IM is gaining a new feature: Apple Push notifications. This well enable users to get notified of offline messages when they arrive, as well as simulate a continued XMPP session on iPhone devices that still do not implement multitasking.

For the latter feature, since the iPhone still does not accept to run applications in the background, if you want to use another application, you will have to shutdown your XMPP client and thus disconnect from Talkr.IM. With the Push feature enabled on your XMPP client (if this one supports Apple Push notifications), you can tell your contacts you are still online when you are using other applications. You will then automatically receive notifications of new messages. And when you re-open your XMPP client, you will receive these messages.

OneTeam for iPhone has the Apple Push feature enabled: you will be able to use it on you Talkr.IM account. Just use you Jabber ID, like username@talkr.im, and for a simple configuration, go to the "Settings" tab, then "Push settings".

The Talkr.IM XMPP server will stop at 10:00am CET. This means:

01:00am Los Angeles
04:00am New York
12:00 Moscow
18:00 Tokyo

UPDATE: the Apple Push module is installed on Talkr.IM, and working fine. You can use it on OneTeam for iPhone. You can run a APNS module on your own ejabberd server with IMpush.

Erlware: Erlware on R13B04

Martin J. Logan — 2010-03-03T18:15:13+00:00

Erlware tools are now running on R13B04 for Mac OSX Leopard (Intel and PowerMac)Mac OSX Snow LeopardLinux i686Linux x86_64To install the 'erl' shell and start playing use 'faxien install-release erl'Note* escript still needs an update - contributions welcome.

Process-one Blogs: ejabberd tip: simple health check

2010-03-03T16:30:28+00:00

An ejabberd health check mechanism might be useful in your deployments, but using the HTTP file server might be overkill...

Some people want to do regular health checks on ejabberd, which is always a good idea. Doing a health check over XMPP might not be the solution, here is at least two reasons:

with a pure ad-hoc solution like a cron script, you might need an XMPP library you might not want to install on your servers
monitoring systems might not talk XMPP at all...

So people first think of the internal ejabberd HTTP file server, which then they have to configure through a specific listener and the right module... possibly with logs, they would have to manage (rotate, parse, clean, etc.).

Sometimes people just don't think there is already an HTTP listener on their internal BOSH Connection Manager. You would argue that BOSH uses the HTTP POST method. But the ejabberd BOSH Connection Manager replies to simple GET:

wget http://myserver.net/http-bind

And ejabberd will answer:

ejabberd mod_http_bind v1.2

An implementation of XMPP over BOSH (XEP-0206)

This web page is only informative. To use HTTP-Bind you need a Jabber/XMPP client that supports it.

Erlang Factory News: Countdown to the Erlang Factory SF Bay Area

2010-03-03T15:46:07+00:00

There are only 3 weeks left to the biggest Erlang gathering in the USA. We have 43 speakers giving 35 talks in 6 tracks! The programme of the conference can be viewed here. If you haven't registered yet, you can still do this by clicking here.

'I'm attending', 'I'm speaking' and 'See you there' banners are now available here. You can add them to your blog, Twitter page or any website. You are also welcome to follow @erlangfactory on Twitter.

Tragically L33T: The Lizard Brain, the Dip, and other Godinisms

2010-03-03T08:00:00+00:00

Seth Godin is a really smart guy. More importantly, he takes his ideas and does something with them.

I came across Seth in an interview he did with Merlin Mann. He talked a lot about fear and the lizard brain: the part of our brain that is only hungry, scared, selfish and horny. Its the part of our brains that ruled in high school.

It will also sabotage us if we let it.

Whenever we get close to completing something big or interesting or important to us, the lizard brain senses a threat. “What if we fail?” “Ridicule is painful!” “Better to go along in life quietly, then do something that might get us noticed.”

The lizard brain tells us to quit when success is near. Seth mentioned in his interview that he listens to his lizard brain… and then does the opposite. If his lizard brain is quiet, he probably isn’t doing anything important. If his lizard brain is screaming bloody murder, he knows he is on the right track.

I picked up a couple of Seth’s books, including Tribes, The Dip, and his new book, Linchpin. As I come across more interesting tidbits I will share them.

Programming in the 21st Century: Dehumidifiers, Gravy, and Coding

James Hague — 2010-03-03T06:00:00+00:00

For a few months I did freelance humor writing. Greeting cards, cartoon captions, that sort of thing. My sole income was from the following slogan, which ended up on a button:

Once I've gathered enough information for the almighty Zontaar, I'm outta here!

Sitting down and cranking out dozens of funny lines was hard. Harder than I expected. I gave it up because it was too draining (and because I wasn't making any money, but I digress).

Periodically I decide I want to boost my creativity. I carry around a notebook and write down conversations, lists, brainstormed ideas, randomness. I recently found one of these notebooks, so I can give some actual samples of its contents. Below half a page of "Luxury Housing Developments in Central Illinois Farmland" (e.g., Arctic Highlands), there's a long list titled "Ridiculous Things." Here are a few:

salads
spackle
key fobs
wine tastings
mulch
hair scrunchies
asphalt
Fry Daddy^TM
cinder blocks
relish
Frito Pie
aeration shoes

Okay, okay, I'll stop. But you get the idea.

As with the humor writing, I remember this taking lots of effort, and it took real focus to keep going. Did this improve my creativity? I'd like to think so. It certainly got me thinking in new directions and about different topics. It also made me realize something fundamental: technical creativity, such as optimizing code or thinking up clever engineering solutions, is completely different from the "normal" creativity that goes into writing stories or taking photos.

Years ago, I followed the development of an indie game. This was back when writing 3D games for non-accelerated VGA cards was cutting edge. The author was astounding in his coding brilliance. He kept pulling out trick after trick, and he wasn't shy about posting key routines for others to use. Eventually the game was released...and promptly forgotten. It may have been a technical masterpiece, but it was terrible as game, completely unplayable.

I still like a good solution to a programming problem. I still like figuring out how to rewrite a function with half the code. But technical creativity is only one form of creativity.

RedHotErlang: Returning a file with Nitrogen

2010-03-02T19:41:01+00:00

Let's say I want to create a file download link in my Nitrogen application. First I create a src/pages/web_file.erl file, and fill it with some code: <pre><code>-module(web_file). -export([main/0, title/0, body/0, event/1]). -include...

Erlware: How to create and Erlware repository with Portius

Martin J. Logan — 2010-03-02T07:31:12+00:00

Erlware tools, primarily Faxien, pull Erlang packages (Applications and Releases) from a repository. The Portius release makes it trivial for any group, company, or individual to host their own repository for their private resources. This video shows you how to use Portius to create an Erlware Erlang/OTP Package repository in just a few minutes.

Damien Katz: BBC and CouchDB

Damien Katz — 2010-03-01T22:11:06+00:00

Vaguely interesting KV/#couchdb stat from the BBC - 3.3 billion requests handled since last summer, running at about 150-170 million per day

http://twitter.com/endafarrell/status/9820160677

LShift on Erlang: Memory matters - even in Erlang

2010-02-28T21:42:43+00:00

Some time ago we got an interesting bug report for RabbitMQ. Surprisingly, unlike other complex bugs, this one is easy to describe:

At some point basic.get suddenly starts being very slow - about 9 times slower!

Basic.get doesn’t do anything complex - it just pops a message from a queue. This behaviour was quite unexpected. Our initial tests confirmed that we have a problem when a queue contains thousands of elements:

queue_length: 90001  basic_get 3333 times took: 1421.250ms
queue_length: 83335  basic_get 3333 times took: 1576.664ms
queue_length: 60004  basic_get 3333 times took: 1403.086ms
queue_length: 53338  basic_get 3333 times took: 9659.434ms [ look at that! ]
queue_length: 50005  basic_get 3333 times took: 9885.598ms
queue_length: 46672  basic_get 3333 times took: 8562.136ms

Let me repeat that. Usually popping a message from a queue takes Xms. At some point, it slows down to 9*Xms.

It turned out that the problem is with the queue:len() function, which is executed during the basic.get. Actually, queue:len() calls only erlang:length() builtin. At some point it switches to the “slow” mode.

Erlang:length() is a builtin that iterates through a linked list and counts it’s length. It’s complexity is O(N), where N is the length of the list. This function is implemented in the VM so it’s expected to be very, very fast.

The problem is not with erlang:length() being slow. It’s about being unpredictably slow. Let’s take a look at Erlang interpreter source code (erl_bif_guard.c:erts_gc_length_1). Here’s the main loop for erlang:length():

i=0
while (is_list(list)) {
    i++;
    list = CDR(list_val(list));
}

It does nothing unusual - it just iterates through list elements. However, recompiling Erlang with some debugging information confirms that the problem is indeed here:

clock_gettime(CLOCK_REALTIME, &t0);
while (is_list(list)) {
    i++;
    list = CDR(list_val(list));
}
clock_gettime(CLOCK_REALTIME, &t1);
td_ms = TIMESPEC_NSEC_SUBTRACT(t1, t0) / 1000000.0;
if (i > 200000 || td_ms > 2.0) {
    fprintf(stderr, "gc_length_1(%p)=%i %.3fms\n\r", reg[live], i, td_ms);
}

gc_length_1(0x7f4dbfa7fc19)=499999 2.221ms
gc_length_1(0x7f4dbfa7fc19)=499999 2.197ms
gc_length_1(0x7f4dbfa7fc19)=499999 2.208ms
(hibernation)
gc_length_1(0x7f4db0572049)=499999 13.793ms
gc_length_1(0x7f4db0572049)=499999 12.806ms
gc_length_1(0x7f4db0572049)=499999 12.531ms

This confirms Matthias’ initial guess - the slowdown starts after Erlang process hibernation.

For those who aren’t Erlang experts: Hibernation is an operation that compacts an Erlang process. It does aggressive garbage collection and reduces the memory footprint of a process to absolute minimum.

The intended result of hibernation is recovering free memory from the process. However its side effect is a new memory layout of objects allocated on the heap.

Ah, how could I have forgotten! The memory is nowadays slow! What happens, is that before hibernation list elements are aligned differently, more dense. Whereas after hibernation they are sparse. It’s easy to test it - let’s count the average distance between pointers to list elements:

gc_length_1(0x7f5c626fbc19)=499999 2.229ms avg=16.000 dev=0.023
gc_length_1(0x7f5c626fbc19)=499999 3.349ms avg=16.000 dev=0.023
gc_length_1(0x7f5c626fbc19)=499999 3.345ms avg=16.000 dev=0.023
(hibernation)
gc_length_1(0x7f5c61f7d049)=499999 13.800ms avg=136.000 dev=0.266
gc_length_1(0x7f5c61f7d049)=499999 12.726ms avg=136.000 dev=0.266
gc_length_1(0x7f5c61f7d049)=499999 12.367ms avg=136.000 dev=0.266

Confirmed! Standard deviation is surprisingly small, so we can read the numbers as:

Before hibernation list elements are aligned exactly one after another, values are somewhere else.
After hibernation list elements are interleaved with values.

This behavior does make sense. In most cases when you traverse the list, you actually do something with the values. After hibernation, when you access list item, the value will be already loaded to the CPU cache.

Knowing the mechanism, it’s easy to write a test case that reproduces the problem.

The average distance between pointers in my case is constant - the standard deviation is negligible. This information has a practical implication - we can “predict” where the next pointer will be. Let’s use that information to “fix” the Erlang VM by prefetching memory!

while (is_list(list)) {
    i++;
    list2 = CDR(list_val(list));
    __builtin_prefetch((char*)list2 + 128*((long)list2-(long)list));
    list = list2;
}

Test script running on original Erlang VM:

length: 300001  avg:0.888792ms dev:0.061587ms
length: 300001  avg:0.881030ms dev:0.040961ms
length: 300001  avg:0.875158ms dev:0.019436ms
hibernate
length: 300001  avg:14.861762ms dev:0.150635ms
length: 300001  avg:14.833733ms dev:0.017405ms
length: 300001  avg:14.884861ms dev:0.220119ms

Patched Erlang VM:

length: 300001  avg:0.742822ms dev:0.029322ms
length: 300001  avg:0.739149ms dev:0.012897ms
length: 300001  avg:0.739465ms dev:0.014417ms
hibernate
length: 300001  avg:7.543693ms dev:0.284355ms
length: 300001  avg:7.342802ms dev:0.330158ms
length: 300001  avg:7.265960ms dev:0.053176ms

The test runs only a tiny bit faster for the “fast” case (dense conses) and twice as fast for the “slow” case (sparse conses).

Should this patch be merged into mainline Erlang? Not really. I have set the prefetch multiplier value to 128 and I don’t even know if it’s optimal. This was only an experiment. But it was fun to see how low-level system architecture can affect high-level applications.

Tragically L33T: Moving my Blog to Jekyll

2010-02-28T08:00:00+00:00

In the past I have not blogged very often. In fact I seem to blog less often than Wordpress releases a security patch. This was making me nervous and, combined with the issues of writing posts offline at events like NFJS, I decided a change was in order.

Enter Jekyll, the static page blog generator behind Github Pages. So far the workflow of managing text files in a Git repository is working well for me. Not being able to leave well enough alone I created a Rakefile to manage certain tasks like create a tagcloud for the sidebar, creating tag specific pages listing posts, and creating a draft post.

Creating a draft post was pretty straightforward:

desc 'create a new post in draft mode'
task :new => [:require_input] do
  title = ask("Title: ")
  filename = title.downcase.gsub(/[^a-z0-9]/,"-")
  template=File.read "lib/post_template.markdown"
  File.open("_drafts/#{filename}.markdown", 'w+') do |f| 
    f << template.gsub(/POST_TITLE/, title)
  end
  sh "git add _drafts/#{filename}.markdown"
end

Publishing a draft to the blog will consist of a git mv of the draft file to the _posts directory with the data appended to the filename.

As for comments I have switched over to Disqus, which allowed me to import my Wordpress comments and link to them on my Jekyll blog.

RedHotErlang: Nurturing creativity and the passionate programmer

2010-02-27T09:11:02+00:00

I happened to read an article in the Harvard Business Review the other day. The article was about how Pixar was fostering creativity among their employees. I realized that I've been nurturing my creativity for years by spending a <br...

Erlware: Create an Erlang Mochiweb Web Application in Minutes Video

Martin J. Logan — 2010-02-25T08:01:29+00:00

This video describes how to create a web application in Erlang/OTP with Mochiweb. This is done very quickly leveraging common OTP standards and the Erlware toolchain.This was made possible by leveraging the work to OTPify Mochiweb done at the last Chicago Erlang User Group code sprint.

Chicago Erlang User Group: Erlang Development Cycles Presentations

Barry Nicholson — 2010-02-24T13:54:00+00:00

Here's the video from the Development Cycles Talk given on February 17, 2010. Lots of great ideas in here!Development Cycles Part 1 tools used can be found at http://code.google.com/p/faxienthe github erlware accountand sinan can be found with faxien by running "faxien install-release sinan"Also, check out http://ecb.sourceforge.net/ for the emacs IDE mode and http://cedet.sourceforge.net/ and

Erlang Training and Consulting
- News: 02 February 2010: Erlang Solutions at QCon London 2010

2010-02-24T10:57:16+00:00

Erlang Solutions Ltd. sponsors QCon London 2010, hosts the track, gives the talks and organises Erlang User Group meeting there.

Qcon is an annual London enterprise software development conference designed for team leads, architects and project management. It gathers Java, .NET, Ruby, SOA, Agile, Erlang and architecture communities.

Erlang Solutions will be present at QCon on Friday, 12th March 2010, when Ulf Wiger (CTO of Erlang Solutions) will be hosting the Concurrency Challenge track. He will also give an introductory talk The Concurrency Challenge at 10:20 and a presentation on Death by accidental complexity at 4:30 pm.

Also, Francesco Cesarini (founder of Erlang Solutions) will be giving two tutorials: "Practical Erlang Programming" on Monday, 8th March and "Erlang/OTP System Principles" on Tuesday, 9th March. These are an all- day events, will start at 9:00 and finish at 16:00.

A day before, on 11th March 2010 from 18:30 until 20:30 Erlang Solutions organises special free London Erlang User Group Meeting at QCon. Everyone is welcome, even if you do not participate in QCon London 2010, just register here.There will be four talks:

Francesco Cesarini presents "Erlang community around the world"
Ulf Wiger talks about "Erlang in the Clouds"
Justin Sheehy will present "Introduction to RIAK" &
Joe Armstrong talks about "Erlang Libraries"

QCON 2010 will be held in The Queen Elizabeth II Conference Centre, in London from 8th until 12th March 2010.

See you there!

Tutorials: March 8-9, 2010

Conference: March 10-12, 2010

When registering for the QCon, use the Discount Code "erlangug" and save £50 off the price!

Erlang Training and Consulting
- News: 22 February 2010: Call for Summer 2010 Internships!

2010-02-24T10:57:16+00:00

Erlang Solutions regularly provides summer internships for qualified IT students. Many former interns have said that it was the best learning experience they have ever had. We are currently interviewing for this year's summer internships.

We are located in the heart of London. See contact us section. Applicants from all over the world are encouraged to seek the positions (we have 9 nationalities represented in our offices) but to apply, you have to be eligible to work in the EU.

The Internships run from June 1 until the end of August, and can continue through September, depending upon student availability and interest.

For more information and to apply, please visit our Jobs section here.

Kevin Scaldeferri's Weblog: Alpha Male Programmers Are Driving Women Out

2010-02-24T10:57:04+00:00

Yesterday, DHH made an argument that alpha male programmers aren’t keeping women out of the tech field. I’m of the opinion that he’s wrong and that his argument is flawed, and in a moment I’ll explain why, but let me get a few things out of the way so they don’t distract from the rest of my argument.

First, I don’t think that “alpha males” or the “rockstar” mentality are the only causes of under-representation of women in technology. As far as I can tell, the causes are many, varied, generally difficult to deal with, and in one or two cases may even be valid reasons why we might ultimately accept some degree of imbalance in the field. Second, this is not strictly a gender issue. Some men are also driven away by this behavior, although I think women are more likely to be; and also some “alpha males” happen to be female. Third, this is not a personal attack on DHH or any other individual, although some people might read parts of it in that way. But my goal here is that the range of individuals who find themselves uncomfortably reflected in what I say, but don’t simply reject it all out of hand, might view that discomfort as an opportunity for personal growth. Finally, I am certainly not claiming that I am perfect in this regard. I’ve made mistakes in the past; I will make them again. I simply hope that my friends will be kind enough to point out my mistakes to me, so that I can try to do better.

Okay, now that that’s all out of the way...

I first claim that DHH is wrong. My proof is empirical and anecdotal, but fortunately for me, I’m on the side of this debate that gets to use those techniques. I.e., I’m asserting existence of a phenomenon, rather than non-existence. I know numerous women who have been driven away by alpha male behavior. In some cases, they simply moved to a different team, or a different company. In other cases, they switched to non-programmer roles. And some left the industry entirely. I know this because they told me. They described specific individuals and specific behaviors which drove them away.

With some frequency in these debates, male programmers will claim that they don’t know any women who have left the field for this reason (or who have experienced sexism in the field, or who were offended by the example under discussion, or even just the milder claim that DHH makes, that no one has any idea what to do). I can explain this in only one of two ways: either they don’t know any women in the field to begin with or don’t talk with them beyond the minimal professional requirements, or women are not telling them because they are part of the problem. Perhaps it would be more effective if these women directly confronted the people causing the problem, but the fact of the matter is that most people, men and women, dislike conflict. We’re much more comfortable griping to a sympathetic friend than to the cause of our unhappiness. So consider me something akin to an anonymizing proxy. Without revealing names or too many of the specifics, please trust me when I say that almost every woman in the field experiences and complains about this.

Now I also said that DHH’s argument is flawed, and I will spend the rest of this post pointing out the various flaws I see.

DHH claims alpha males cannot be a problem in programming because the average male programmer is “meek, tame, and introverted” compared to other fields. First off, “alpha males” are by definition not average; they are the most dominant individuals of a group. And, it may even be possible that the general meekness or introversion of programmers makes it easier for a small set of individuals to dominate the interaction, rather than reaching a condition of detente between a group of uniformly assertive individuals. Second, presumably DHH does not interact with these people from other fields in a professional context. A point repeatedly stressed in this recent “pr0n star” controversy is that it’s not an issue of people being anti-sex or anti-porn; it’s about what’s appropriate in the workplace, or in a professional context. Standards for behavior in a social context are different. Third, he speaks in terms of whether these other men are more or less “R-rated”. This is not the point. Women are just as “R-rated” as men. They curse. They talk about sex (often more explicitly than men). The issue is not about whether we say or do adult things, it’s about whether we respect each other as human beings and whether we understand the societal norms of what is and is not appropriate in particular contexts. In fact, in this regard, I’ll defend DHH. Saying “fuck” (in the exclamatory, non-sexual usage) in the course of a technical presentation is not problematic in this day and age within the technology community. I think most of us swear freely in the course of struggling with a nasty bug or production problem. This is a normative expression of frustration within our community, and it does not oppress or disrespect other members of the community. (As far as I know. It’s possible that people just aren’t telling me that it upsets them when I curse at my monitor. If that’s the case, I hope someone will tell me.) Finally, DHH observes that these other fields have a more even mix of men and women. What he misses is that when the distribution is relatively equal it is generally easier and more comfortable for men to be men and women to be women. It is perhaps counterintuitive, but environments which are heavily skewed call for greater sensitivity to gender or other cultural differences simply because it is so easy to unintentionally create an oppressive or exclusionary atmosphere.

In the final paragraphs of his post, DHH suggests that somehow by respecting women we are squashing some other sort of “edge” and diversity in the community. I’m a little puzzled by what he means by this, and I’m sort of afraid that he thinks that being a heterosexual male who likes to look at scantily-clad women (or who openly admits as much) is somehow “edgy”. It’s not. By definition, hetero males like women; and it’s well established that men tend to be visually oriented. Pointing out that you fall in this category does not make you “diverse”, it makes you a completely typical representative of your gender and orientation. No one needs to be reminded of it.

Moreover, it might be true that maximal gains are had by pushing the edges (although I don’t think that one should naively assume that analogy from physics or economics applies to social endeavors), but for this to be work there has to be negative feedback when boundaries are crossed. If the edge-walkers want society to accept their behavior, they must be prepared to apologize, to make reparations, and to correct their course when they go over the line. This is the difference between a trend-setter and a sociopath.

There’s quite a bit more that I could say on this issue, but I fear this may be becoming too long already, and I think it’s probably best to focus only on the arguments presented in this particular post at the moment. To summarize things in a couple of sentences, the phenomenon of women being discouraged by alpha male behavior is real. You merely need to talk, and listen, to women in the field to verify this. (But you might have to earn some trust first.) Comparisons with men in other fields in non-professional settings do not have much relevance to the matter at hand. Claims that respecting the feelings and experiences of a minority group is damaging to the community overall are extraordinary and require extraordinary support. Being a thought leader and being offensive are two very different things.

It’s really quite discouraging that so much of this discussion still seems mired in the question of whether a problem even exists, or whether it is desirable and possible to address the problem. This lack of acceptance leads both to the explicit refusals to acknowledge the validity of the complaints of the offended, as well as the phenomena of false apologies and insincere claims that “I would help if only I knew how (and if it doesn’t require any actual change of behavior on my part)”. Male programmers need to pull their heads out of the sand. The evidence, both hard statistical data and anecdotal, is overwhelming. It also is not hard to find advice about what can be done differently. The hard part is moving from a vague desire for diversity and balance to serious, meaningful, sometimes painful self-examination and commitment to change and improvement. It’s not easy to admit flaws in yourself, to acknowledge when you’ve hurt another person, or to make a true apology. Change doesn’t happen overnight or simply because you say that you want it to. It takes work, but it’s an important part of being a human being and being a member of a community.

Comments

Kevin Scaldeferri's Weblog: Speeding up a Rails request by 150ms by changing 1 line

2010-02-24T10:57:04+00:00

We’re pretty obsessed with performance at Gilt Groupe. You can get a taste for what we’re dealing with, and how we’re dealing with it, from our recent presentation at RailsConf.

One of the techniques we’re using is to precompute what certain high-volume pages will look like at a given time in the future, and store the result as static HTML that we serve to the actual users at that time. For ease of initial development, and because there’s still a fair bit of business logic involved in determining which version of a particular page to serve, this was done inside our normal controller actions which look for a static file to serve, before falling back to generating it dynamically.

We’re now running on Rails 2.3 and, of course, Rails Metal is the new hotness in 2.3. I spent the last couple days looking into how much improvement in static file serving we would see by moving it into the Metal layer. Based on most of what I’ve read, I expected we might shave off a couple milliseconds. This expectation turned out to be dramatically wrong.

Metal components operate outside the realm of the usual Rails timing and logging components, so you don’t get any internal measurements of page performance. Instead, I fired up ab to measure the serving times externally. What I found for the page I was benchmarking was that the Metal implementation took about 5ms. The old controller action took 170ms. But, wait... the Rails logs were only reporting 8ms for that action. Something was fishy.

I started inserting timers at various places in the Rails stack, trying to figure out where the other 160ms was going. A little bit was routing logic and other miscellaneous overhead, but even setting a timer around the very entry points into the Rails request serving path, I was only seeing 15ms being spent. This was getting really puzzling, because at this point where a Rack response is returned to the web server, I expected things to look identical between Metal and ActionController. However, looking more closely at the response objects I discovered the critical difference. The Metal response returns an [String], while the controller returned an ActionController::Response.

I went into the Rails source and found the each method for ActionController::Response. Here it is:

   def each(&callback)
     if @body.respond_to?(:call)
       @writer = lambda { |x| callback.call(x) }
       @body.call(self, self)
     elsif @body.is_a?(String)
       @body.each_line(&callback)
     else
       @body.each(&callback)
     end

     @writer = callback
     @block.call(self) if @block
   end

The critical line is the case where the body is a String. The code iterates over each line in the response. Each line is written individually to the network socket. In the case of the particular page I was looking at, that was 1300 writes. Ouch.

To confirm this was the problem, I changed that line to

      yield @body

With the whole body being sent in a single write, ab reported 15ms per request, right in line with what I measured inside Rails.

1 line changed. 150ms gained. Not too bad.

This sort of performance pessimization we uncovered is particularly insidious because it’s completely invisible to all the usual Rails monitoring tools. It doesn’t show up in your logged response time; you won’t see it in NewRelic or TuneUp. The only way you’re going to find out about it is by running an external benchmarking tool. Of course, this is always a good idea, but it’s easy to forget to do it, because the tools that work inside the Rails ecosystem are so nice. But the lesson here is, if you’re working on performance optimizations, make sure to always get a second opinion.

Comments

Trapexit News Feed: Trapexit News :: Erlang Open Source crawler!

2010-02-24T10:56:53+00:00

Author: admin
Subject: Erlang Open Source crawler!
Posted: Thu Nov 19, 2009 4:22 pm (GMT 0)
Topic Replies: 0

Trapexit is proud to announce its own Erlang open source crawler. We crawl the web for you, gathering searchable information on all open source Erlang projects from all the major repositories. You can discuss these projects in the forum, add documentation in the trapexit wiki or rate the various contributions. To view our new project index repository, visit http://projects.trapexit.org.

Trapexit News Feed: Trapexit News :: Navigating The Erlang Source!

2010-02-24T10:56:53+00:00

Author: francesco
Subject: Navigating The Erlang Source!
Posted: Thu Aug 20, 2009 2:24 pm (GMT 0)
Topic Replies: 0

While trapexit user zghst was looking at the erlang source, there was quite a bit of information about it he felt was missing or hard to find. As a result, he started a wiki page to help new people like him find their way around it: A Guide To The Erlang Source. Why not give him a hand in improving it?

Erlang Factory News: Early Bird Saver - only two weeks left!

2010-02-24T10:56:23+00:00

You only have two weeks left to take advantage of our Early Bird special offers. Sign-up for the Erlang Factory before 28th February 2010 and SAVE $200 off the standard price!

When registering, you can also sign up for the Erlang University courses and choose from: Erlang Express, Erlang OTP Express, QuickCheck for Erlang developers and Web Programming with Erlang. The discount applies to the courses as well.

Don't miss out - book your place today!

Erlang Factory News: Book your hotel room for the Erlang Factory!

2010-02-24T10:56:23+00:00

The 2010 Erlang Factory will be held at the Hilton San Francisco Airport. You can now book your hotel room there for the GREAT CONFERENCE RATE! For rooms with a Kingsize bed, delegates will pay only $109.00 per night – room only, with FREE PARKING and FREE WI-FI in the rooms for delegates.

To book for the special Erlang Factory rate click HERE.

RiboComments/Erlang: Generating prime numbers with Erlang and Java

2010-02-24T10:56:23+00:00

During my work with the course-ware of a new Erlang course, I experimented with one of the programming assignments to compare the threading performance of Erlang versus Java. The assignment is one of the classical programs from teaching concurrent programming: How to generate prime numbers using a pipeline of sieve ...

RiboComments/Erlang: The critical section problem in Erlang

2010-02-24T10:56:23+00:00

The programming language Erlang is based on micro-threads and asynchronous message passing. There is a (naive) belief that critical section problems cannot arise in languages based solely on message passing. The justification for this stand-point is the absence of mutex synchronization primitives, which is absolutely essential in shared-data based concurrent ...

Erlang Training and Consulting
- Jobs: Junior and senior superhackers wanted, Sweden

2010-02-24T10:55:33+00:00

Facebook, Google and Amazon have with great success been using new technologies and tools to build high performance and reliable services on the Internet. Now our client is looking for great programmers who want to take these technologies even further. These positions are especially interesting to those of you wishing to join an exceptional development team with some of the worlds most renown Erlang programmers.

The position

Our client is looking for new colleagues who can help develop the system which is the core of our client services that serve millions of consumers around Scandinavia. You will have the opportunity to learn Erlang - a functional and concurrent programming language which makes it easy to program in mulitcore and distributed environments.

As a developer your work will help our client reach 1,5 million consumers and support the 4000 e-stores that daily work in the system. Your code will analyse and process transactions for several billion kronor per year in a complex and distributed environment.

In your work you will cooperate closely with the rest of the development department in an atmosphere of performance and entrepreneurship. If you have the ambition to grow and learn then we have the tools and desire to help you do so. We have chosen to create a development department that stimulates every developer and gives them great personal freedom. We develop using the Scrum agile methodology with two week iterations, or sprints, and each team has great freedom to organise it's own work.

Our client is looking for those of you who are very talented programmers with a love of writing high-quality code have a strong desire to get things done want to work with the latest technologies within web (e.g. Erlang, Ajax and Google-wave) are creative and innovative are analytical can work independently are interested in usability have an academic qualification, preferably within Computer Science or a similar subject love to program in functional languages (e.g. Erlang, Haskell, Scheme) or modern high level languages such as Scala, Ruby and Python.

We look positively on candidates with experience of: GUI-development, internet banks, distributed databases, Scrum, electronic payment methods, statistical analysis, Erlang, XML, HTML, Yaws, OTP, Mnesia.

To qualify for the position you are expected to pass several tests most of which are designed to test your ability as a programmer. A typical test question could be "write a function which compares two strings containing persons full names and checks to see if they are identical except for the order of the names".

Erlang Training and Consulting
- Jobs: Undergraduate Summer Interns, London, UK

2010-02-24T10:55:33+00:00

Erlang Solutions is currently interviewing for its summer internships. As in previous years, we are looking for bright, dedicated, motivated, curious and self-driven individuals in their second to fourth year of studies the field of Computer Science. Exposure to Erlang and Unix derivatives (preferably Linux) are a must. The right candidates will be working with exciting new projects (SMS, Telecom, E-Commerce, Instant Messaging, Banking) with some of the best Erlang developers around. And as a bonus, you will be in the heart of London. Applicants from all over are encouraged to seek the positions (We have 9 nationalities represented in our offices), but to apply, you have to be eligible to work in the EU. Let us have your CVs and a covering letter, letting us know about yourselves, your hobby projects, and why you want to work with us at Erlang Solutions. We need your application at the latest on the 30th of April.

Erlang Announce List: Erlang announce mailing list :: Erlang/OTP now at Github

2010-02-24T10:55:04+00:00

Author: Anonymous
Subject: Erlang/OTP now at Github
Posted: Wed Nov 25, 2009 2:14 pm (GMT 0)
Topic Replies: 0

The official git repository for Erlang/OTP can now be found at Github:

http://github.com/erlang/otp

Build instructions can be found at:

http://wiki.github.com/erlang/otp

We plan to add more wiki pages with additional information in the
near future.

--
Björn Gustavsson, Erlang/OTP, Ericsson AB

________________________________________________________________
erlang-announce mailing list. See http://www.erlang.org/faq.html
erlang-announce (at) erlang.org

Post received from mailinglist

Erlang Announce List: Erlang announce mailing list :: Erlang/OTP R13B03 has been released

2010-02-24T10:55:04+00:00

Author: Anonymous
Subject: Erlang/OTP R13B03 has been released
Posted: Wed Nov 25, 2009 1:56 pm (GMT 0)
Topic Replies: 0

Bug fix release : otp_src_R13B03
Build date : 2009-11-23

This is R13B03, the third maintenance release for the R13B major release.

You can find the README file for the release at

http://www.erlang.org/download/otp_src_R13B03.readme

The source distribution and binary distribution for Windows can be
downloaded from

http://www.erlang.org/download/otp_src_R13B03.tar.gz
http://www.erlang.org/download/otp_win32_R13B03.exe

The distribution can also be downloaded using the BitTorrent
protocol. Use the following torrent files to download the source
distribution and binary distribution for Windows:

http://www.erlang.org/download/otp_src_R13B03.tar.gz.torrent
http://www.erlang.org/download/otp_win32_R13B03.exe.torrent

Note: To unpack the TAR archive you need a GNU TAR compatible program.

For installation instructions please read the README file that is part
of the distribution.

The on-line documentation can be found at: http://www.erlang.org/doc/
You can also download the complete HTML documentation or the Unix manual files

http://www.erlang.org/download/otp_doc_html_R13B03.tar.gz
http://www.erlang.org/download/otp_doc_man_R13B03.tar.gz

We also want to thank those that sent us patches, suggestions and bug
reports,

The OTP Team

--
Björn Gustavsson, Erlang/OTP, Ericsson AB

________________________________________________________________
erlang-announce mailing list. See http://www.erlang.org/faq.html
erlang-announce (at) erlang.org

Post received from mailinglist

EazyErl!: Filtering lines efficiently

rolphin — 2010-02-24T09:00:02+00:00

Whenever you are dealing with log lines or that you're program is filtering data you always have to handle 'escaping' efficiently.

While developing a log module using a gen_event, I needed to escape simple quotes.
Sometime thoses quotes were already escaped...

I've found this regexp to handle gracefully the case:

re:replace( Bin, "(?<!\\\\)'", "\\\\'", [ global ] ).

Not so easy to read, and because of the various backslashes, this regexp needed some tests
before being fully usable.
Basically this regexp only filter simple quote when they're not already escaped...

Now that I've my filter for quotes and I need a filter for newline characters.
Everytime you find some newlines in your log files, you can be sure that many tools already in your network will not treat them efficiently, worse this may break everything after ...

So I came across this regexp:

re:replace( Bin, "[\\n\\r]+", " ", [ global ] ).

Finally having two filter functions for every line, I wanted to be able to add or remove easily any function.
You can reference functions with this notation:

fun filterquotes/1

Or a list of functions:

[ fun filterquotes/1, fun filternewlines/1 ]

With this notation and the famous 'lists:foldl', I can filter a line with many functions quite nicely:

% Data is the accumulator, but we don't change it :)
        lists:foldl( fun( Fun, Data ) ->
                         Fun(Data) 
        end, Bin, [ fun filterquotes/1, fun filternewline/1 ]).

Here's the code:

filter( Bin ) when is_atom(Bin) ->
        Bin;
filter( Bin ) when is_integer(Bin) ->
        Bin;
filter( Bin ) ->
        lists:foldl( fun( Fun, Data ) ->
                         Fun(Data) 
        end, Bin, [ fun filterquotes/1, fun filternewline/1 ]).

filterquotes(Bin) ->
        re:replace( Bin, "(?<!\\\\)'", "\\\\'", [ global ] ).

filternewline( Bin ) ->
        re:replace( Bin, "[\\n\\r]+", " ", [ global ] ).

EazyErl!: Reading an openssl .priv.key file and extracting the key

rolphin — 2010-02-23T22:23:26+00:00

Extracting the private key from a .priv.key file is simple.
The private key is encrypted using a AES-128 with your passphrase.

The initial vector is also stored in the file, you can extract it directly from the first line:

get_salt( "Salted__", Salt:8/binary, Rest/binary>> ) ->
        {Salt, Rest}.

The last part is handled by some md5() of your passphrase and the initial vector:

Key = crypto:md5([ Password, Salt ]),
        IV = crypto:md5([ Key, Password, Salt ]),
        crypto:aes_cbc_128_decrypt( Key, IV, Rest).

Now the full module:

-module(priv_key).

-compile(export_all).

priv_key_file( File, Password ) ->
        {ok, Bin} = file:read_file(File),
        {Salt, Rest} = get_salt(Bin),
        Key = crypto:md5([ Password, Salt ]),
        IV = crypto:md5([ Key, Password, Salt ]),
        crypto:aes_cbc_128_decrypt( Key, IV, Rest).
        
get_salt( "Salted__", Salt:8/binary, Rest/binary>> ) ->
        {Salt, Rest}.

Nicolas Charpentier's blog: Code availability on git-hub

2010-02-21T08:54:32+00:00

Bored to switch code between my private subversion repository and git-hub for my open-source projects, I decided to use exclusively git-hub for them. My trac wiki pages are also moved to git-hub for improve the documentation of those projects. Selenium-RC Erlang binding Rake tasks to build erlang code Small mock library for erlang (Please don’t use mock to test [...]

Damien Katz: About Me

Damien Katz — 2010-02-20T04:53:36+00:00

All about me. Me me me me me.

Vitals:
Birth date - October 24, 1973
Height - 6'1"
Eyes - brown
Hair - dark brown
Complexion - fair and freckly
Brain - wrinkly
Spouse - Laura Ann Katz (maiden name Toenjes)
Kids - Gwendolyn, Roseanna and Zack
Religion - unapologetically atheist

I live in Piedmont CA.

I am the creator of CouchDB.

I am the CEO of Couchio.

My good points:
I am very devoted, loving and supportive of my wife and daughters. I have a very positive attitude and am energetic. I am smart and a fast learner, especially when it comes to math, science and engineering stuff. I love solving really hard problems. I am a hard worker, and I am very passionate about my work. I am very honest. I like to make lots of jokes, sometimes they are even funny. I take good care of myself and exercise regularly. I am athletic. I read constantly (always non-fiction). I have good eye for aesthetics. I am good with my hands and like to make things. I like to bake. I generally try not to take life too seriously.

My bad points:
I am egotistical, judgmental and sometimes have a bad temper (but I'm working on those things). I am vain and self conscious. I don't take very good care of my teeth (but I do floss regularly: once every 4 months, whether I need it or not). I am messy and disorganized. I have a short attention span in meetings and lectures. I take my work too seriously. I am brutally honest (but I'm convinced it's not really a bad point). My self-created list of bad points is oddly short, hmmmm.

My weird points:
While I am pretty athletic (strong, fast, can jump, etc), I'm physically uncoordinated and hence suck at nearly every sport (except basketball, I'm decent at that). I am somewhat shy around new people, sometimes it comes off as being as being a snob. I spend waaay too much time in front of the computer, but that's the way I like it. I drink lots of diet soda, lots. I can't whistle. Sometimes my feet smell like hot buttered popcorn. I once was the proprietor of liquidpoop.com -- the internet's largest repository of diarrhea-related poetry. Although I'm vain, my wardrobe mostly consists of stained, freebie t-shirts and a few pairs of jeans.

Damien Katz: Chris demos CouchApps

Damien Katz — 2010-02-20T01:25:35+00:00

Here is a Chris Anderson screencast demoing some new CouchApp jQuery plugins. He's a smooth criminal.

Damien Katz: Migrating Notes/Domino to CouchDB

Damien Katz — 2010-02-19T23:17:26+00:00

I've been talking to some IT shops who are migrating away from Lotus Notes and Domino. More than wanting to use something else, it seems the reasons they are migrating away is that people in management just don't like Lotus Notes.

The problem these IT shops face is they don't have many good options to migrate to. Sharepoint works for very generic collaborative activities, but not so much for custom business apps that are so pervasive in large Notes installs. That doesn't mean people don't try.

So these IT shops are interested in migrating to Apache CouchDB for obvious reasons. CouchDB is largely inspired by the Lotus Notes backend. CouchDB has a document database, peer based replication, views, full text indexing add-ons, security, and HTTP client access. It also has a very active and growing open source community.

So in theory, migrating to CouchDB from Notes/Domino should be easier than migrating to any other technology. But what we don't yet have is many tools, documentation and examples for migrating from Notes to CouchDB.

Anyone out there have experience with this type of thing for Notes? Anyone interested in helping us develop the tools and documentation for migrating to CouchDB? There is a lot of business opportunity here, and we are looking for partners and VARs to help customers with tools and migrations. If interested, have some ideas or feedback, feel free to email me at damien@couch.io.

Process-one Blogs: Jingle Nodes on Talkr.IM

2010-02-18T14:00:00+00:00

A Jingle Nodes relay has been installed on the Talkr.IM XMPP service.

The public XMPP server Talkr.IM has received its own Jingle Nodes relay on: xmpp:jn.talkr.im. You can browse the Talkr.IM services via your Service Discovery interface in your XMPP client.

This new exclusive feature on Talkr.IM XMPP server enables Jingle calls by providing a public service for media relaying. Basically, this means that XMPP clients which can handle Jingle Nodes, will be able to join other Jingle Nodes enabled clients to establish voice calls even when both are hidden behind network barriers (like NAT, for the techies).

One many entities of the XMPP federated network (clients, servers) will have Jingle Nodes enabled, this will be possible to establish calls, in a totally P2P manner, much like Skype, but with much more features like the control of who can use your relay. Read the Jingle Nodes website and the Jingle Nodes XEP to know more.

This Talkr.IM Jingle Nodes relay is one of the very first step to a simpler Jingle user experience worldwide.

Visit the Talkr.IM website, and create your own Talkr.IM XMPP account freely: https://www.talkr.im/signup.

Process-one Blogs: OneTeam for iPhone in version 3.2.5, with Facebook chat

2010-02-17T16:08:11+00:00

OneTeam for iPhone has been published on the AppStore in version 3.2.5, bringing bugfixes, and Facebook chat based on XMPP.

The version 3.2.5 of our XMPP client OneTeam for iPhone has hit the shelves of the Apple's application store. It is recommended that you upgrade it.

OneTeam 3.2.4 for iPhone had already fixed bug related to Multi-User Chats (also known as groupchats): in some cases, OneTeam used to crash on joining password-protected and members-only MUC.

OneTeam 3.2.5 for iPhone now allows OneTeam users to connect to the XMPP interface to Facebook chat. You can now connect with Jabber-ID set your "your_username@chat.facebook.com", and password set to your Facebook password.

Be careful, Facebook's XMPP chat interface does not offer MUC services, nor is federated, which means you still cannot chat with your friends on Talkr.IM or Google's Talk/GMail, but only with Facebook friends.

Check out OneTeam for iPhone on our website, as well as Apple's AppStore.

Chicago Erlang User Group: OTP Packaging Video 12/16/2009

Barry Nicholson — 2010-02-16T15:36:00+00:00

Here's the video from the December 16, 2009 Erlang/OTP Packaging talk given to the Chicago Erlang Users Group by Martin Logan.Part 1 Part 2 Part 3

ProTest news: QuickCheck for testing C programs

2010-02-16T00:00:00+00:00

Quviq developed an extension to QuickCheck that enables easy testing of C code from the Erlang shell and therewith enable the use of QuickCheck on C code.

This library is now evaluated on C libraries in the GNU package as well as on some real automotive software.

ProTest news: Message Sequences Charts help formulating properties

2010-02-16T00:00:00+00:00

Quviq developed a new library to express relationships between events that happen in a system. This allows the testers to specify things like "this message should be delivered if the receiver was logged in around the time the message was sent". Message sequence charts are often used to give examples of possible ways in which messages may arrive. The newly developed library allows to specify a number of examples as one general property on message sequences.

The library was successfully used in a case study at ProcessOne and a paper describing this case has been submitted for publication.

ProTest news: PhD thesis with part on property based testing

2010-02-16T00:00:00+00:00

Laura Castro from the University of A Coruna finished her PhD thesis, which is to be defended on April 15, 2010. In her thesis she describes advantages of using functional programming for the development of software systems. Part of the thesis is devoted to testing these systems and property based testing plays an important rôle.

Damien Katz: Wanted: Hosting/Infrastructure Engineer

Damien Katz — 2010-02-12T00:18:26+00:00

We are looking to hire someone with strong hosting or infrastructure experience to help us develop a CouchDB hosting platform.

Experience with CouchDB is nice but not necessary. We most want someone who is passionate about infrastructure with large scale hosting experience and with open source contributions.

We offer competitive pay, stock options, great health benefits and 6 weeks vacation, and an opportunity to change the world.

Email inquiries to damien@couch.io

Chicago Erlang User Group: Chicago Erlang User Group Meeting Februrary 17th

Jordan Wilberding — 2010-02-11T12:59:00+00:00

The time and location:We are confirmed for Wednesday February 17th at 6:00PM. Orbitz has agreed again to let us use their office space for the meetup. We need all the names of the people that will attend so we can place them on the security list. Please email to RSVP at martinjlogan -at- gmail.The address for the talk is 500 W. Madison St, Chicago IL (the Olgilvie transportation center). Come up

Damien Katz: Thoughts on an Open Source Company

Damien Katz — 2010-02-10T19:20:13+00:00

I wrote this almost a year ago. It's something we want Couchio live by, to build a company we all want to be a part of.

Our companies mission to make the world a better place through open source software.

People are hired based on contributions to open source. Code, documentation, advocacy, community help, legal help, etc. People who work to make the world better for everyone through open source are the people we hire.

Every employee has an equity stake in the company. The decision to hire will be partially based on "do we think giving this person this % of the company is worth it to us all" Bad hiring decisions impact us all, so we are all motivated to hire those who will add value.

How do we make money? By providing hosting and services. Rock solid apps, hosting and service. We give away our software, but we provide services people want and need at a healthy profit.

If the company is in the shitter, never have layoffs, decrease pay % across the board. If we have to drop pay below levels that can support people, we can't be a company.

Stress kills. Kills motivation, creativity, and in the long run it literally kills people. Most people want to do good. A few try really hard to do good. We make sure people know once they are hired, they never have to worry about their job. We will try hard to not be a source of stress to our employees.

Never fire. We constantly emphasize you can't get fired for lack of productivity or being stupid or foolish. You can get fired for ethical reasons. We should strive for openness and honesty in all matters.

Never ever ever have a hiring binge. We hire based on contributions to open source. Always. When we have the resources, we hire the best contributors. Otherwise we don't hire.

We don't want people who tend to ask "How can I contribute? What should I do?". We want people who identify for themselves what needs to be done and how they can contribute, and then do it. Our employees want to contribute to make things better, not for a paycheck, not for validation from a manager or even the community. They do it because they think it will make things better.

We don't have managers. Everyone who we hire is already a productive contributor on their own. People can take leadership and mentoring roles, but these roles are granted by those being led and mentored. No one ever has control over someone else's time.

We don't hire for positions like HR, sales, etc. We either outsource, or a person we have already hired for their contributions to open source fills the position.

We don't encourage or discourage project or code ownership. In the case of disagreements, working code always has a place. We encourage forks internally and externally. There can be no reals rules beyond this guidance. We acknowledge this is messy and imperfect, and will always be source of friction and disagreement.

How do we ensure people are productive and don't goof off? We don't.

We don't measure productivity? Smart people can always appear to be productive while goofing off, making it seem they are working hard when wasting time, causing stress for themselves and often for others who want to be seen to be strong contributors. We don't monitor people's productively. We let people goof off if they want. It's okay to goof off. I'm goofing off right now as write this.

Then why do our people work?

Our people are hired based on contributions to open source, their contributions make the world a better place. Our company mission is to do that, and we are filled with people who've already done that. We all want it to succeed and make a profit, so we can make ever greater contributions to the world. People contribute how they see fit: By working on profitable projects, by developing new projects, by doing work that will never make a profit but makes the world better for us all. Sometimes it's by taking some time off to recharge so you can contribute later.

If the company becomes unprofitable, or sick, it's everyone's responsibility to make it healthy. No one has to ask permission how to make it better, how to make things profitable, they just do it.

Already this has been put to the test, as we were presented an eminently qualified hire with a very positive energy. The problem was no open source involvement or contributions. I wanted to make an exception but I realized if we really want to give this idea of an open source company shot, we can't compromise already on the 4th hire we make. So I had to say no, which was hard.

I don't yet know if this will work in the long run, but for now I want us to give it a real shot and see what happens.

Programming in the 21st Century: Optimizing for Fan Noise

James Hague — 2010-02-10T06:00:00+00:00

The first money I ever earned, outside of getting an allowance, was writing assembly language games for an 8-bit home computer magazine called ANALOG Computing. Those games ended up as pages of printed listings of lines like this:

1050 DATA 4CBC08A6A4BC7D09A20986B7B980
0995E895D4B99E099DC91C9DB51CA90095C0C8
CA10E8A20086A88E7D1D8E7E,608

A typical game could be 75 to 125+ of those lines (and those "three" lines above count as one; it's word-wrapped for a 40-column display). On the printed page they were a wall of hex digits. And people typed them in by hand--I typed them in by hand--in what can only be described as a painstaking process. Just try reading that data to yourself and typing it into a text editor. Go ahead: 4C-BC-08-A6...

Typos were easy to make. That's the purpose of the "608" at the end of the line. It's a checksum verified by a separate "correctness checker" utility.

There was a strong incentive for the authors of these games to optimize their code. Not for speed, but to minimize the number of characters that people who bought the magazine had to type. Warning, 6502 code ahead! This:

   LDA #0
   TAY

was two fewer printed digits than this:

   LDA #0
   LDY #0

Across a 4K or 6K game, those savings mattered. Two characters here, four characters there, maybe the total line count could be reduced by four lines, six lines, ten lines. This had nothing to do with actual code performance. Even on a sub-2MHz processor those scattered few cycles were noise. But finding your place in the current line, saying "A6," then typing "A" and "6" took time. Measurable time. Time that was worth optimizing.

Most of the discussions I see about optimization are less concrete. It's always "speed" and "memory," but in the way someone with a big house and a good job says "I need more money." Optimization only matters if you're optimizing something where you can feel the difference, and you can't feel even thousands of bytes or nanoseconds. Optimizing for program understandability...I'll buy that, but it's more of an internal thing. There's one concern that really does matter these days, and it's not abstract in the least: power consumption.

It's more than just battery life. If a running program means I get an hour less work done before looking for a place to plug in, that's not horrible. The experience is the same, just shorter. But power consumption equals heat and that's what really matters to me: if the CPU load in my MacBook cranks up then it gets hot, and that causes the fan to spin up like a jet on the runway, which defeats the purpose of having a nice little notebook that I can bring places. I can't edit music tracks with a roaring fan like that, and it's not something I'd want next to me on the plane or one table over at the coffee shop. Of course it doesn't loudly whine like that most of the time, only when doing something that pushes the system hard.

What matters in 2010 is optimizing for fan noise.

If you're not buying this, take a look at Apple's stats about power consumption and thermal output of iMacs (which, remember, are systems where the CPU and fan are right there on your desk in the same enclosure as the monitor). There's a big difference in power consumption, and corresponding heat generated, between a CPU idling and at max load. That means it's the programs you are running which are directly responsible for both length of battery charge and how loudly the fan spins.

Obvious? Perhaps, but this is something that didn't occur with most popular 8-bit and 16-bit processors, because those chips never idled. They always ran flat-out all the time, even if just in a busy loop waiting for interrupts to hit. With the iMacs, there's a trend toward the difference between idle and max load increasing as the clock speed of the processor increases. The worst case is the early 2009 24-inch iMac: 387.3 BTU/h at idle, 710.3 BTU/h at max load, for a difference of 323 BTU/h. (For comparison, that difference is larger than the entire maximum thermal output of the 20-inch iMac CPU: 298.5 BTU/h.)

The utmost in processing speed, which once was the goal, now has a price associated with it. At the same time that manufacturers cite impressive benchmark numbers, there's also the implicit assumption that you don't really want to hit those numbers in the everyday use of a mobile computer. Get all those cores going all the time, including the vector floating point units, and you get rewarded with forty minutes of use on a full battery charge with the fan whooshing the whole time. And if you optimize your code purely for speed, you're getting what you asked for.

Realistically, is there anything you can do? Yes, but it means you have to break free from the mindset that all of a computer's power is there for the taking. Doubling the speed of a program by moving from one to four cores is a win if you're looking at the raw benchmark numbers, but an overall loss in terms of computation per watt. Ideas that sounded good in the days of CPU cycles being a free resource, such as anticipating a time-consuming task that the user might request and starting it in the background, are now questionable features. Ditto for persistent unnecessary animations.

Nanoseconds are abstract. The sound waves generated by poorly designed applications are not.

ejabberd@jabber.ru: ejabberd and exmpp source code are moved from SVN to Git

2010-02-09T16:15:00+00:00

After many months of planning, ejabberd and exmpp have been fully migrated to Git.

During the last 7 years, ejabberd source code was hosted at:

CVS at Jabber.Ru
CVS at JabberStudio.org
SVN at ProcessOne
Git preliminarly built with git-svn, at Github

Starting now, ejabberd source code is natively in Git, and hosted at:

The minimal instructions to start using it are mentioned in:
http://www.process-one.net/en/ejabberd/downloads

Damien Katz: First week in the new office

Damien Katz — 2010-02-09T00:40:32+00:00

Last week was our first week in our new office in Old Downtown Oakland. It's a really neat area with lots of restaurants and bars, and hardly any murders.

Oh yeah, we've changed our name to Couchio. Our new blog will be here http://blog.couch.io/ soon.

Our office:

Our office manager Claire:

Chris and Mikeal:

Claire and Jan:

Nitin:

Super Awesome Art by Julie Armbruster:

Me:

My Office:

So far we are really disorganized and discombobulated. But I'm are working on it! I even bought Management for Dummies. Things will be running smoothly in no time ;)

Also we are looking hard for someone to help us offer CouchDB support and hopefully build a whole support organization. Email me damien@couch.io if you are interested or know someone who is.

RedHotErlang: inotify for erlang

2010-02-06T16:44:06+00:00

inotify is a cool linux kernel service, that lets you subscribe to file system events. For example, if you are interested in reacting to changes in the file "~/.emacs", and since you are not a lamer you abhor polling, you would like to be able to ...

RedHotErlang: Dialyzer

2010-02-05T21:59:19+00:00

At times you sit there at the typical Erlang conferences and hear Kostis Sagonis speak of dialyzer, typer and all the funky tools. It sounds just so great. The downside is of course that all the tools require an effort on your part. You need to re...

Process-one Blogs: ProcessOne at FOSDEM and XMPP Summit

2010-02-04T17:19:01+00:00

ProcessOne will be present at FOSDEM and XMPP Summit.

This week-end, we will be joining the XMPP community at Brussels, for the FOSDEM, as well as the XMPP-specific events: XMPP hackfest and XMPP Summit.

The FOSDEM, or Free and open source software developers' european meeting, will be held on Saturday, as well as Sunday, at ULB Campus Solbosch, with myriads of FOSS people for different countries and profiles.
Before and after this event, the XMPP community will gather on Friday for the XMPP hackfest, and Monday for the XMPP Summit.

The XMPP hackfest is the place where coders can code and test interop, while the XMPP Summit is more oriented towards specifications improving.

ProcessOne will be there and demo three of our software products:

OneTeam, our Firefox extension, providing interesting features
OneTeam for iPhone, enabling the always-on XMPP experience to a pocket near you
IMGateways, our transports, especially our Twitter gateway, as well as our Google Wave gateway

Feel free to join, as we would like to show you our software, and we would like to have your opinion on these.

Planet Trapexit - Erlang/OTP News

Caoyuan's Blog: How the Functional Ability of Scala Comparing to Haskell - an Example

Original Haskell code piece

In Scala:

Damien Katz: O'Reilly's CouchDB: The Definitive Guide for $9.99 Today Only!

Trapexit's Erlang Blog Filter: New Column on Webmachine

ejabberd@jabber.ru: ejabberd 2.1.3 - Third bugfix release

Erlang Factory News: New Speaker for the Erlang Factory SF Bay Area 2010

Process-one Blogs: OneTeam XMPP client entering private alpha

Damien Katz: Win a Nexus One from Couchio

Programming in the 21st Century: Eleven Years of Erlang

Erlang Inside: Zotonic destroys WordPress and rethinks the CMS with Erlang

Damien Katz: CouchDB Case Study: Assay Depot

Programming in the 21st Century: It Made Sense in 1978

Process-one Blogs: Talkr.IM XMPP/Jabber server gets Apple Push notifications

Erlware: Erlware on R13B04

Process-one Blogs: ejabberd tip: simple health check

ejabberd mod_http_bind v1.2

Erlang Factory News: Countdown to the Erlang Factory SF Bay Area

Tragically L33T: The Lizard Brain, the Dip, and other Godinisms

Programming in the 21st Century: Dehumidifiers, Gravy, and Coding

RedHotErlang: Returning a file with Nitrogen

Erlware: How to create and Erlware repository with Portius

Damien Katz: BBC and CouchDB

LShift on Erlang: Memory matters - even in Erlang

Tragically L33T: Moving my Blog to Jekyll

RedHotErlang: Nurturing creativity and the passionate programmer

Erlware: Create an Erlang Mochiweb Web Application in Minutes Video

Chicago Erlang User Group: Erlang Development Cycles Presentations

Erlang Training and Consulting - News: 02 February 2010: Erlang Solutions at QCon London 2010

Erlang Training and Consulting - News: 22 February 2010: Call for Summer 2010 Internships!

Kevin Scaldeferri's Weblog: Alpha Male Programmers Are Driving Women Out

Kevin Scaldeferri's Weblog: Speeding up a Rails request by 150ms by changing 1 line

Trapexit News Feed: Trapexit News :: Erlang Open Source crawler!

Trapexit News Feed: Trapexit News :: Navigating The Erlang Source!

Erlang Factory News: Early Bird Saver - only two weeks left!

Erlang Factory News: Book your hotel room for the Erlang Factory!

RiboComments/Erlang: Generating prime numbers with Erlang and Java

RiboComments/Erlang: The critical section problem in Erlang

Erlang Training and Consulting - Jobs: Junior and senior superhackers wanted, Sweden

Erlang Training and Consulting - Jobs: Undergraduate Summer Interns, London, UK

Erlang Announce List: Erlang announce mailing list :: Erlang/OTP now at Github

Erlang Announce List: Erlang announce mailing list :: Erlang/OTP R13B03 has been released

EazyErl!: Filtering lines efficiently

EazyErl!: Reading an openssl .priv.key file and extracting the key

Nicolas Charpentier's blog: Code availability on git-hub

Damien Katz: About Me

Damien Katz: Chris demos CouchApps

Damien Katz: Migrating Notes/Domino to CouchDB

Process-one Blogs: Jingle Nodes on Talkr.IM

Process-one Blogs: OneTeam for iPhone in version 3.2.5, with Facebook chat

Chicago Erlang User Group: OTP Packaging Video 12/16/2009

ProTest news: QuickCheck for testing C programs

ProTest news: Message Sequences Charts help formulating properties

ProTest news: PhD thesis with part on property based testing

Damien Katz: Wanted: Hosting/Infrastructure Engineer

Chicago Erlang User Group: Chicago Erlang User Group Meeting Februrary 17th

Damien Katz: Thoughts on an Open Source Company

Programming in the 21st Century: Optimizing for Fan Noise

ejabberd@jabber.ru: ejabberd and exmpp source code are moved from SVN to Git

Damien Katz: First week in the new office

RedHotErlang: inotify for erlang

RedHotErlang: Dialyzer

Process-one Blogs: ProcessOne at FOSDEM and XMPP Summit

Erlang Training and Consulting
- News: 02 February 2010: Erlang Solutions at QCon London 2010

Erlang Training and Consulting
- News: 22 February 2010: Call for Summer 2010 Internships!

Erlang Training and Consulting
- Jobs: Junior and senior superhackers wanted, Sweden

Erlang Training and Consulting
- Jobs: Undergraduate Summer Interns, London, UK