Graham Now Hacking Scala at GrahamLea.com

This is just a quick post for people who've been following 'Graham Hacking Scala' to let you know that I am now blogging - often about Scala but sometimes about other stuff - at www.grahamlea.com Hope to see you there! Graham.

JavaScript and Scala: Good Parts and Bad

I've just finished reading "JavaScript: The Good Parts" by Douglas Crockford. It's a fantastic book, mostly because it's incredibly concise, elaborating only the details that experienced programmers should need to understand the true form of JavaScript. If you do ANY JavaScript programming and you haven't read it, I suggest you go out and get it straight away, and don't write a single line more of buggy JS code until you do.

What is 'this'?
Early on in the book, Crockford highlights one of JavaScript's oddest and most dangerous quirks: the late binding of 'this'. Basically, whenever you call a function, the value of 'this' within the function is determined by HOW you called the function!

To demonstrate: If object 'a' has a function called 'run()' and you invoke it like this:

a.run();

then the value of 'this' within 'run()' will be 'a'. JavaScript is a functional language, so functions can be passed around as values. If you instead invoked the function above using only a reference to the function value, like this:

var f = a.run;
f();

then the value of 'this' within 'run()' will be... the global variable pool! As you can imagine, that can lead to some pretty nasty and hard to track bugs.

Functional Limitations
 If you're a fan of functional programming, you might have tweaked straight away that this could place limits on the usefulness of functions as first class objects, and specifically higher-order functions (i.e. functions that take other functions in their parameters).

Here's an example of an object that doesn't work with higher-order functions:

var object1 = {
    total: 0,
    add: function(numbers) {
        var i;
        for (i = 0; i < numbers.length; i++) {
            this.total += numbers[i]
        }
        return this.total;
    }
};

var valueOf = function (f, p1) {
    return f(p1);
}

var numbers = [1, 2, 3, 4];

document.writeln(object1.add(numbers) + " / " + valueOf(object1.add, numbers));
// outputs: 10 / NaN

Obviously the reason this quirk causes confusion is because JavaScript has borrowed a keyword from other languages but uses it to mean something subtly different. Most OO languages use 'this' to represent a pointer to an object in which the function seemingly resides. Importantly, in OO/functional languages, when a function is used as a value, it takes with it a closure around 'this' that is used when invoking it. In JavaScript, however, the 'this' pointer just refers to the object on which the function was called, or if there was no object, just to the global variable pool.

This Naughtiness Explained
There is a reason for this weirdness. Because JavaScript is classless, the function doesn't actually belong to the object in which it resides. In fact, it's not hard to imagine situations where JavaScript's behaviour is what you'd expect. If I took a function from one object and placed it in another object (which I can do, because the function is also a value), I actually would expect 'this' to refer to the new object when I invoked it, and JavaScript provides just that capability:

var b = {};
b.run = a.run;
b.run(); // 'this' will refer to 'b'

Once you understand where JavaScript is coming from a bit better, it's not too hard to see the solution to this problem. 'this' doesn't provide a closure around the context where your function was defined, so if you want a closure around the context where your function was defined, then give your function a closure around that context, rather than assuming that 'this' will be the same context whenever the function is invoked. To demonstrate this by correcting the example above:

var object2 = (function () {
    var that = {};
    that.total =  0;
    that.add = function(numbers) {
        var i;
        for (i = 0; i < numbers.length; i++) {
            that.total += numbers[i]
        }
        return that.total;
    };
    return that;
})();

document.writeln(object2.add(numbers) + " / " + valueOf(object2.add, numbers));
// outputs: 10 / 20

Scala: The Bad Parts?
 Does any of this have anything to do with Scala? Well, in the details, no, but in the general sense there is a very important approach in Crockford's book. His suggested solution to the problem above is simple: don't use 'this' in JavaScript and don't use its accompanying feature, 'new'. In fact, he suggests not using LOTS of features of JavaScript. He dedicates 14 pages of appendix to listing them.

Crockford also spends a very un-indulgent two pages discussing the modern phenomenon of "feature-driven design", both in products and programming languages, explaining how "Features that offer value to a minority of users impose a cost on all users" and concluding the book with this pearler: "It would be nice if products and programming languages were designed to have only good parts."
It was those two lines that caused me to reflect on Scala. I started asking myself questions like: Are there too many features in Scala? Are there parts that aren't "good parts"? Are there features that very few people make use of but that cost everyone because of the complexity they add? From my own experience, the answer to all three of these is "yes".

Crockford explains that we all crave simplicity and that, when we aren't given it, we make it for ourselves by creating our own feature subset, and I realised that I've done this to some extent in the way I use Scala. So finally I began wondering how long it will be until someone publishes "Scala: The Good Parts", complete with appendices listing all the features that you should steer clear of for everyone's sake. What confusing or dangerous feature of Scala do you reckon will be at the top of the list?

Want to learn more?

From Amazon...

From Book Depository...

Updates to Scala/Spring/Hibernate/Maven Webapp

One of the projects I maintain is a public domain code base on GitHub called Scala-Spring-Hibernate-Maven-Webapp. The repository contains source code for kickstarting your own webapp project using the latest versions of Scala, Spring Web, Hibernate and Maven.

Project Updates
I've been committing some major changes over the last couple of weeks, including:

• Upgrading to Scala 2.9.1, Spring 3.1.0 and Hibernate 4.0.0 
• Adding to the existing REST-ful Create and Retrive examples some Update and Delete operations 
• Adding server-side form validation using JSR-303 annotations with Spring Web MVC 
• Creating examples of automated web testing using Selenium WebDriver and the PageObjects pattern

 If you're about to start your own project with these technologies, or are just keen to see how they all work together, you can get your own copy of the source code from here:
https://github.com/GrahamLea/scala-spring-hibernate-maven-webapp

Resolved Issues
I encountered a couple of time-consuming issues during the upgrades to the latest versions and I think it would be worth sharing how they were resolved in order to save some time for others googling for solutions to the same problems.

Spring Schema SAXParseException
After upgrading the versions of all the Spring Schemas in my Spring XML files, I started getting the following error while trying to start up the webapp:

org.xml.sax.SAXParseException: cos-all-limited.1.2: An ''all'' model group must appear in a particle with '{'min occurs'}'='{'max occurs'}'=1, and that particle must be part of a pair which constitutes the '{'content type'}' of a complex type definition.

At first this appears to be a bug in the Spring schema, but a bit more investigation revealed that it is, in fact, a bug in an old version of the Xerces XML parser library. In my case, an old version of the library was being pulled in through commons-dbcp. The solution I took was to explicitly add a dependency on the latest version of Xerces:

        <dependency>
            <groupId>xerces</groupId>
            <artifactId>xercesImpl</artifactId>
            <version>2.9.1</version>
            <scope>runtime</scope>
        </dependency>

Hibernate Function Not Supported
After upgrading Hibernate to 4.0.0, I started getting this error message whenever I tried to execute a query:

org.hibernate.exception.GenericJDBCException: This function is not supported
Caused by: java.sql.SQLException: This function is not supported

I don't remember whether HSQLDB appeared in the stack trace or not, but the solution to this was to upgrade the version of HSQLDB I was using from 1.8.0.10 to 2.2.6. If you get the same error and you're not using HSQLDB, I would suggest considering an upgrade of your JDBC driver or maybe even your database server.

"No Session found for current thread" 
Also after upgrading to Spring 3.1.0 and Hibernate 4.0.0, and after changing all my Hibernate 3 Spring beans to the corresponding Hibernate 4 versions, I started getting the old classic "No Session found for current thread" error when trying to view any page with a transaction. I thought this was very odd, because the traditional solution to this has always been to make sure you have an OpenSessionInViewInterceptor, but I already had one, defined like this:

    <bean class="org.springframework.web.servlet.mvc.annotation.DefaultAnnotationHandlerMapping">
        <property name="interceptors">
            <list><ref bean="openSessionInViewInterceptor"/></list>
        </property>
    </bean>

By pure chance, I'd seen a Spring documentation page earlier the same day where I'd noticed an alternative way of declaring interceptors using the 'mvc' namespace:

    <mvc:interceptors>
        <ref bean="openSessionInViewInterceptor"/>
    </mvc:interceptors\>

Replacing my old DefaultAnnotationHandlerMapping definition with the one above got my OpenSessionInViewInterceptor working again and solved the "No Session found for current thread".

No Appenders for Log4J
Finally, this one is a bit silly, but I spent a good half hour on it without finding any good help on the web so I thought it was worth documenting. At some point I started getting this error out of Log4J every time I started my webapp:

log4j:WARN No appenders could be found for logger

After all kinds of googling, experiments with winding back dependency upgrades and investigations into ClassLoaders, I eventually noticed that I just had a typo in the Log4J configuration file:

og4j.rootCategory=INFO,Console

Notice the 'l' missing at the start of that line? Ouch. So if you're getting this error message about no appenders and you think your log4j config is all set up properly, check whether the config for your root category is correct before investing too much time in breakpoints inside Log4J classes.

Want to learn more?
Here's some books you might find useful if you plan to go further with Spring, Hibernate, Scala or Maven:

From Amazon

From The Book Depository

Spring in Action - Craig Walls & Ryan Breidenbach (Manning)

Hibernate in Action - Gavin King & Christian Bauer (Manning)

Spring Recipes - Gary Mak (Apress)

Programming in Scala - Martin Odersky, Lex Spoon & Bill Venners (Artima)

Maven - A Developer's Notebook - Vincent Massol & Timothy M. O'Brien (O'Reilly)

SodaTest 0.2.1 Released

I've just released the second milestone of SodaTest, the spreadsheet-driven integration testing framework.

A significant change with this release is that SodaTest is now available from the Maven Central repository. You should be able to access the latest versions of all SodaTest artefacts from Maven Central right now as org.sodatest : sodatest-* : 0.2.1

As well as a couple of small bug fixes, this release incorporates a number a improvements based on feedback from use of SodaTest in corporate environments. The key enhancments were:

• Coercions can now be created in Java without any knowledge of Scala
• Built-in support was added for coercing strings into Java enums
• An implicit function is available for converting into a Coercion any function with the type: (String) => _
• A CurrencyAmountCoercion and CurrencyAmountCoercionForJava are now available for coercing a variety of financial value formats into your own custom Currency or Money class
• Reports no longer report a mismatch due to trailing empty cells in the actual output
• A bunch of usability fixes such as handling unusual situations, reporting better error messages, correct usage messages, more scaladoc and READMEs.

You can view the full list ofissues that were resolved as part of 0.2.1 on GitHub.

So, if you've been holding out from using SodaTest wondering if anyone is using it for real, whether it's going to be improved over time, and whether you should even bother trying it out, the answers are yes, yes and yes!

(And in case you're wondering why this release is 0.2.1 and not 0.2, it's because 0.2 got lost in the first, failed attempt to release from Sonatype to Maven Central.)

Will Typesafe Address Scala Criticisms?

If you've been paying attention to the Scala corner of the web over the last week, you'll no doubt have heard something about Yammer - an enterprise-focussed social network - switching from Scala back to Java.

If you've been on holidays and missed it all, basically what happened was that Coda Hale, the Infrastructure Architect at Yammer, wrote a private email to the Typesafe team giving loads of mostly critical feedback regarding Scala, but the email was subsequently leaked and became the talk of the town, among both Scala supporters, like David Pollak, and critics, such as Stephen Colebourne.


A Painfully Honest Critique

Firstly, let me summarise the main points in Coda's email:

* Yammer have decided to change their "basic infrastructure stack" from Scala to Java.

* The main reason is that Scala has "friction and complexity" that isn't offset by enough productivity gain.

* Some seemingly simple constructs in Scala, e.g. List, actually have very complex interfaces and behaviour that necessitate explaining a lot of concepts to beginners. The belief by some that this complexity can be ignored except by library authors has proven to be false in practice.

* The most vocal members of the Scala online community are generally very academic in their practice and discussion of Scala.

* Existing libraries seem to be being re-written just because people want to practice their use of advanced functional programming concepts, but these re-writes are then being recommended as the library of choice.

* The Yammer team eventually decided the most sensible way to deal with the Scala community was to ignore it.

* Learning Scala properly is important, but doing so is neither quick nor easy.

* Java is not going away, so the choice to use Scala is actually a choice to use Scala and Java, along with the interaction challenges that brings.

* The Yammer team had "endless issues" with SBT, and found themselves writing plugins to replicate Maven functionality.

* Switching to Maven solved most of their problems, but they found it has been marginalised by the Scala community, with poor support and constant encouragement to switch to SBT.

* The backwards-incompatibility of Scala major releases requires library authors to cross-compile,  which has resulted in library re-invention when authors are no longer committed to the project.

* The backwards-incompatibility also causes headaches for developers, which has resulted in many production Scala deployments not being upgraded to 2.9.

* Yammer found idiomatic Scala code performed very poorly. By eliminating for-comprehensions, avoiding Scala's collection library, avoiding closures and marking everything with private[this], they were able to achieve speed ups of 100x in some components.

So that was that. For what it's worth, Coda's email is also quite complimentary of Scala. I think he gives the impression that he likes it and would really like to keep using it, but the realities of making it work in a day-to-day production-deploying team have weighed them down so much that the benefits have been eclipsed.

A Frustratingly Measured Official Response

Not long after Coda's email became public, Typesafe published on their blog what is quite obviously a public response to the email, and it's that blog I'd like to take a closer look at today.

Here are the main points from the Typesafe blog:

* Typesafe are investing in Scala IDE support by contributing to the Scala IDE for Eclipse

* They are addressing the learning curve by developing training courses, but also provided references to some of the free resources on the web.

* 'scalac' is slow because it has more work to do than 'javac', but they are investigating using parallel and incremental techniques to improve speed, with a focus on improving sbt.

* If you don't want to (or can't) upgrade your Scala version, you can purchase commercial support from Typesafe to get access to long-term maintenance releases of previous versions.

* In order to reduce people's major-release upgrade issues, they are planning to fold a bunch of existing libraries into the Scala release .

* They take performance seriously, but are wary of engaging in premature optimisation.

* Typesafe are not just focused on the Scala language and core libraries but also on frameworks to help build applications more easily.

* They describe the Scala community using a couple of niceties as well as the words "opinionated" and "quirky".

* They say there's many places where Scala could be improved and they're hard at work doing so.

* They mention Yammer amongst a group of other companies with successful production deployments of Scala.



What Aren't They Telling Us?

What I think is most interesting about this response is the things it doesn't mention and the places where it's specific about what they're doing to solve issues.

Firstly, there's no mention of Maven. There is some commitment to improving compile times, but they seem to suggest that this will be for sbt users only. It has to be considered odd that Maven has been given so little attention by the chief proponents of Scala. One of the main advantages often cited for adopting Scala is its easy integration with Java, "so you can continue using all your favourite libraries". But the commitment to prolonging the life of people's Java API knowledge doesn't extend to the Java build chain. Should it? Why is Maven important? It's important because it works, and it's almost ubiquitous. It has had years and years of work poured into it and has been tested and improved by feedback from hundreds of thousands of users and plugin developers. It has a global ecosystem that is well understood. It has become a lingua franca; many people would know how to build their project with Maven but couldn't tell you how to build it with javac and jar. If you want to distribute your project to the world through Maven Central via Sonatype's Nexus repository, they've got fantastically detailed instructions for how to do so... with Maven. The Java-fluent world - which Typesafe want to convert into the Scala world - don't need another build tool. We need relatively minor improvements to Maven and its plugins to ensure we can keeping working without having to learn something completely new.

Secondly, Typesafe give no commitment to eliminating the backwards compatibility problems in future major releases. In fact, one of the parts of their workaround - the offering of paid support for people who want to continue using older releases - actually makes it in their interest to not solve the problem. The other half of their workaround - bringing certain libraries into the Scala release - will mean that, as a developer, your backwards compatibility issues will only be solved if your taste in libraries is similar to those of the Typesafe team. If you like a library that hasn't been deemed worthy of inclusion, you'll be left with all the same cross-compiling and "version-x-for-scala-y" dependency issues that you have now.

Thirdly, they seem to fob off Yammer's performance problems as an issue specific to their environment, suggesting that fixing such things for everybody would be "premature optimisation". I've heard this argument before: that it only makes sense for companies like Yammer and Twitter to spend time optimising tiny bits of code because it can save them millions across all their computers. But the same argument applies to small businesses just as well. If my business is growing rapidly, but I have limited cash flow, having a 100x performance increase is going to make a significant contribution to my bottom line if I can serve all of my customers on one machine instead of 100 machines. "Premature optimisation" is a phrase that should be reserved for application developers, not framework developers. Just optimise the hell out of it, please.

Finally: the community. While Typesafe dropped a small comment about the community in its response, its really not an issue that they can solve, though I do think it will greatly effect the measure of their success in the long term. Really, the comments in Coda's email should be a wake up call to the Scala community at large: a sizable Scala software development team, full of very smart people who've written some very scalable and successful software, found the Scala community to be so fractured and unhelpful that they eventually decided to ignore it and, I would assume, not be a part of it. That's not just a loss for the community, but also a warning, because for every developer that finds the community unhelpful and so decides to program Scala in his own microcosm, there will be numerous others who decide to not bother with Scala at all for the same reason. This alone could be a weighty enough issue to relegate Scala to a hobby language in the long term rather than seeing wide-spread adoption in the enterprise community.



Hope and Trepidation

When I originally read through Coda's email I was nodding my head in agreement and hoping that Typesafe would be listening carefully and would take it as an opportunity to shift their focus, maybe just temporarily, but long enough to knock over these major objections. Unfortunately, when I read through Typesafe's response, I didn't see any real hint that they plan to fix the problems that I find most annoying. In some cases I even saw excuses about why they won't be doing anything. I don't think "We're focusing on making sbt better" is  a response to "Why isn't there decent Maven support?". I know they've been meeting with influential Scala teams in the days since this response, I just hope they listen to what they've got to say.

So now I'm left feeling a little unsure about Scala. Will it continue to improve for me and people like me? People who like to use IDEA + Maven, not Eclipse + sbt, who are educated in creating applications rather than composing mathematical proofs? Or is the current state of the art in Scala, along with its broad range of minor but resonating annoyances, what I should expect to still be grappling with in five years time? Only time will tell whether Typesafe can navigate this wave of criticism and find a promised land of general popularity, but I think it is an indication that Scala's window of opportunity has entered the waning phase, and now is the critical time at which to capitalise on the extant interest.

What do YOU think?
Do you sympathise with the Yammer team's frustrations?
Are you confident that Typesafe will listen and respond to current criticisms with relevant and prompt action?

Top 10 Reasons Java Programs Envy Scala

I presented a talk at the 2nd meeting of the new ScalaSyd Meetup last night. I talked through the "Top 10 Reasons Java Programs Envy Scala" in an attempt to give Java developers a taste of some little things that could make them much more productive if they switch to Scala.

If you'd like to watch the Prezi, or listen to the talk (or both!), here are the links:

Presentation: Top 10 Reasons Java Programs Envy Scala (Graham Lea)

Audio Simulcast: Top 10 Reasons Java Programs Envy Scala (Graham Lea)

At the end of the talk, I mention that people who are tempted to learn Scala might want to look at my blog post, Graham's guide to learning Scala.

If you'd like to have a look at the code examples used in the talk you can view them on or clone them from GitHub.

Presenting at ScalaSyd, October 26

In two weeks' time I'll be presenting a talk at the recently-formed ScalaSyd Meetup. The title of the talk is "Top 10 Reasons that Java programs envy Scala"; the synopsis: "A whirlwind tour through 10 productivity-increasing features of Scala that aims to inspire Java developers to jump on the Scala bandwagon for their own coding pleasure."

There will be some other talks, Atlassian are providing the venue, pizza and drinks and JetBrains have donated some swag to dish out. It should be a good night so if you'll be in Sydney on Wednesday October 26, make a plan to drop in and make sure you RSVP on Meetup.com. Hope to see you there!

Code Writers vs Code Creators

It's occurred to me recently that one of the key turning points for a professional software developer is the realisation that you don't have to make do with the tools you've got - you can create your own tools. This is why 'The Pragmatic Programmer' recommends activities like learning scripting languages and using code generation.

What is your Goal?
As a software engineer, my key goal is to CREATE software. A lot of the time, creating will mean WRITING software (i.e. typing out a program), but that isn't always the case. Many pieces of code have large amounts of boilerplate, or contain elements that are repeated again and again across the code base. Usually this means that the code could have been derived from some description of the end result that was simpler than the code itself.  And wherever that is the case, the code is a candidate for being tool-aided or tool-generated.

What is a Tool?
Now, tools can take many forms.  Sometimes you just need to abstract a super class from some existing code, rather than copying code, and that base class becomes a tool for solving any similar problem. If you have a similar problem across a number of different applications, you might pull code out into a library that can be reused  across those code bases.

I'm using a pretty loose definition of 'tool' here. Basically I consider anything a tool if it has no function by itself, but can be used to create things which can perform a function. Base classes and libraries definitely fit this definition, but I hope that for the majority of developers this kind of reuse is bread and butter.

The kind of tool-creation I think we should aspire to is a little more high-level than this. At regular points in the creation of software, every programmer, and sometimes even whole teams, will come across problems where they feel awkward with the solution and with the process for creating it. So what I'm advocating is that developers and teams need to have the courage in these situations to put down the normal tools and say, "This isn't working. We need better tools for this task." Sometimes there will be existing tools for solving your problem, and sometimes you'll have to create the right tool yourself. But the key quality is to be on the lookout for situations where the productivity of your code creation is being hampered by something, and to be ready to say out loud, "let's take the time to fix this". The problem might be irrelevant boilerplate, repetetive details, or it may even be something not directly related to coding, like a randomly failing build or an automated process that sends you too many emails. Anything that's stealing time away from value creation and could potentially be semi-automated can become the target of tooling.

What's the Catch?
Of course, there's common sense to be used in deciding when to employ a tool or create one. If you're going to write your own tool, you need to make a really honest (i.e. pessimistic) estimation of the ROI (return on investment): Could the time it will take you to write the tool end up being longer than the time it will save you? Is this problem (and hence the tool to solve it) happening again and again or is it just a one off? Will the tool require ongoing maintenance that will significantly offset the gain it provides? You also need to have the discipline to create just the tool that YOU need, and to not try to solve the larger, generic set of problems that are like yours, but which you don't have. Remember: the purpose of creating a tool is to get you to the finish line faster, not to divert you from the race entirely because you're finding it uninteresting.

Are Your Eyes Open to Opportunities for Greater Productivity?
At the end of the day, though, I believe there will often come times for every developer where they find a problem that justifies the creation of a custom tool. If this hasn't been your experience, I'd like to challenge you to think about this: is the charge of your occupation to write lines of code as fast as possible, or to deliver working solutions as fast as possible? Assuming you're not paid by the line, you're really being paid to be a Code Creator, not just a Code Writer. That means you have a responsibility to use the right tools for the job, and hence a mandate to create them if they don't exist.

Want to learn more?

From Amazon...

From Book Depository...

How to Create a Webapp with Scala, Spring, Hibernate and Maven in Late 2011

Way back in the ancient history of January 2010, I wrote a walk-through of how to create a webapp with Scala, Spring Hibernate and Maven.

As with all things, technologies have progressed, as have other things, including the way code is shared online. So I decided it was time to update the code form that post and share it somewhere more modern.

You can now find my code for starting a Scala + Spring + Hibernate + Maven Webapp on GitHub.

At the time of writing, this project uses Scala 2.9.0-1, Spring 3.0.5.RELEASE and Hibernate 3.6.5.Final. I hope to keep it updated as new versions of these libraries are released, and also to add some more flesh to the example as time permits. (Ha!)

In you're looking more for a tutorial rather than just code, the original post still serves as a great walk-though of all the steps that went into creating the sample webapp, as well as showing lots of mistakes that I made that are probably quite common.

Want to learn more?
Here's some books you might find useful if you plan to go further with Spring, Hibernate, Scala or Maven:

From Amazon

From The Book Depository

Spring in Action - Craig Walls & Ryan Breidenbach (Manning)

Hibernate in Action - Gavin King & Christian Bauer (Manning)

Spring Recipes - Gary Mak (Apress)

Programming in Scala - Martin Odersky, Lex Spoon & Bill Venners (Artima)

Maven - A Developer's Notebook - Vincent Massol & Timothy M. O'Brien (O'Reilly)

What's the best way to transform a list with a cumulative function?

I've been doing this kind of thing a lot lately: I have a list of somethings. I want a new list based on that list, but it's not a straight one-to-one map() operation - each value in the resulting list is a function of the corresponding value and all values before it in the input list. (I'm sure there's a one-word name for this type of function, but I don't specialise in maths vocab.)

Example
As an example to discuss, imagine I have a list of numbers, and I want a list of the cumulative totals after each number:

 def main(args: Array[String]) {
  val numbers = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
  val cumulativeTotals = ... ?
  println(cumulativeTotals)
}

should output:

List(1, 3, 6, 10, 15, 21, 28, 36, 45, 55)

Funky Folds

So, until the other day, I've usualy been doing this with a foldLeft():

 private def cumulativeTotalFolded(numbers: List[Int]): Seq[Int] = {
  numbers.foldLeft((0, List[Int]()))((currentTotalAndCumulativeTotals, n) => {
    val (currentTotal, cumulativeTotals) = currentTotalAndCumulativeTotals
    (currentTotal + n, (currentTotal + n) :: cumulativeTotals)
  })._2.reverse
}

Now, I'm not holding that up as a good example of anything. Folds can be hard to understand at the best of times. A fold that passes around a Tuple2 as the accumulator value is not code that simply communicates to the next guy what's going on.

Stream Simplicity
So, after a particularly hairy instance of one of these the other night, I lay in bed trying to think of a better way. It struck me ('round about 1am) that Streams are a much more natural construct for calculating cumulative functions.

If you haven't come across Streams before, they're basically a way to define a collection recursively be providing the first value in the collection and a function that will calculate the next element in the collection (and, recursively, all the elements after it) as each subsequent element is requested by the client.

Streams are good for this problem because the iteration through the list is simple, while the "next" value usually has easy access to the "previous" value, so to speak. I've used this once or twice now and I like it a lot better.

For the problem above, my solution using a Steram looks like this:

 private def cumulativeTotalStream(numbers: List[Int], total: Int = 0): Stream[Int] = {
  numbers match {
    case head :: tail =>
      Stream.cons(total + head, cumulativeTotalStream(tail, total + head))
    case Nil => Stream.Empty
  }
}

(Note: to make this work with the println() above, you'll need to toList() the Stream.

Recursion Wrangling
There is, of course, another obvious way, which is to construct the list using a recursive function that passes two accumulators: one for the current total and one for the resulting List of totals that is accumulating:

@tailrec
private def cumulativeTotalRecursive(
 numbers: List[Int], currentTotal: Int = 0, cumulativeTotals: List[Int] = Nil): Seq[Int] = {

 numbers match {
   case head :: tail =>
     cumulativeTotalRecursive(tail, currentTotal + head, (currentTotal + head) :: cumulativeTotals)
   case Nil => cumulativeTotals.reverse
 }
}

There's nothing wrong with this solution, and it probably performs much better than the Stream version, but I feel a bit weird about passing so many parameters around for such a simple operation.

I could reduce the parameter count by getting the currentTotal from the accumulator list instead of passing it around:

 @tailrec
private def cumulativeTotalRecursive2(
  numbers: List[Int], cumulativeTotals: List[Int] = Nil): Seq[Int] = {

  (numbers, cumulativeTotals) match {
    case (number :: tail, Nil) => cumulativeTotalRecursive2(tail, List(number))
    case (number :: tail, lastTotal :: otherTotals) => 
      cumulativeTotalRecursive2(tail, (number + lastTotal) :: cumulativeTotals)
    case (Nil, _) => cumulativeTotals.reverse
  }
}

but then the function body ends up more complex than the one with an extra parameter, which isn't a good trade-off.

Mutability the key to Simplicity?
Finally, I thought about the possibility of solving it with a for comprehension. I realised quickly that I'd need a mutable variable, but the result is a very, very simple piece of code:

  private def cumulativeTotalComprehension(numbers: List[Int]): Seq[Int] = {
    var currentTotal = 0
    for (n <- numbers) yield {
      currentTotal += n
      currentTotal
    }
  }

I'm pretty sure this wouldn't look as pretty for a lot of the problems I've been tackling with the foldLeft() but, mind you, none of them looked very pretty either. Is this an okay solution? Do Scala afficionados going to vomit in disgust when they see the var keyword?

Your Turn
What I'd really like to know is whether there's an idiomatic way of doing this that I've just never come across.

That's all I can come up with at the moment. I'm sure there's other ways to do it. One that looks simple, takes a sinlge parameter and runs fast would be ideal. If you've got some good ideas of other ways to do this, please leave something in the comments!

Want to learn more?
If all of this has just made you think you might need to do a bit more study on what recursion, Streams or folding are, try one of these great books:

From Amazon...

From Book Depository...

Introducing SodaTest: Spreadsheet-Driven Integration Testing for Scala and Java

I'd like to reveal what I've been working on in my spare time for the last couple of months.

The Announcement

SodaTest: "Spreadsheet-Driven Integration Testing", is an open-source framework for creating Executable Requirements for Integration and Acceptance testing in Scala and Java.

The impetus for starting this project was to attempt to create a tool that improves on Ward Cunningham's Framework for Integration Testing, "FIT". As an 'Executable Requirements' testing tool, it can also be considered as an alternative to Fitnesse, Concordion, RSpec, Cucumber, JDave, JBehave, SpecFlow, and Thoughtworks' Twist.

The Background

We used FIT in anger when I first arrived at my current workplace over four years ago. Since then I've watched the team become first dissatisfied with it, then scathing and fearing of it, before abandoning development of new FIT tests in place of Integration Tests written in JUnit. Nevertheless, we all still felt that Executable Requirements were a Good Thing and made a couple of failed restarts in trying to get back on the wagon.

From my own perspective, many of our issues (but not all) were to do with the tool. I identified two main issues that I'd seen hurt people over and over: the input format and the programming model. So I set out to create something that solved these two problems while remaining fairly close to what I think is a great foundation in FIT.

The Result

The result is a tool I've called SodaTest. It uses spreadsheets, in the form of comma-separated value (CSV) files, as the input format. The contents of the spreadsheet are basically small individual tables, much like what would appear in a FIT test. There is a required but simple structure to the tables and a minimum of special words and symbols to provide some context to the parser.

I also aimed to keep the programming model as simple as possible. I tried to make sure there is only noe sensible way things could be done, so as not to confuse developers with options. I've gone to lengths to ensure developers of Fixtures will have to do a minimum of work by having the framework do a lot of boilerplate work for them. Lastly, I've structured and named everything in a way that I believe will guide developers in the right direction (by which I mean away from mistakes that I have made in the past while writing FIT tests).

I've also taken the time to add a little sugar here and there that I thought was missing from FIT, for example integration with JUnit and a more comprehensive set of built-in coercion strategies for converting strings into strong types.

I'm quite pleased with the result. (I wouldn't be telling people about it if I wasn't!) Yesterday I released version 0.1 of SodaTest and people should be able to use this first release of SodaTest to create tests that do almost everything that FIT could do, but with less effort in creating the tests, writing the Fixtures, and getting the whole lot executing in their environment.

You can find out more about the motivation for SodaTest and the features it includes by reading the README on GitHub.

While SodaTest is written almost entirely in Scala (and the most benefit will be gained by using Scala as the language for writing Fixtures), I've also written the sodatest-api-java extension that allows SodaTest Fixtures to be written in Java. There is one limitation where Scala is (currently) still needed, but I reckon 95% of Fixtures should be able to be written entirely in Java if that is something you care about.

The Next Steps

The next steps for SodaTest are clear to me: Dogfood, Broadcast, Feedback and Evolve.

I want to convince my team at work to start experimenting with Executable Requirements again using my home-grown tool; I'd love it if other people in the Scala and Java communities could download this tool and give it a little tryout during their next iteration or two; I want to hear Feedback from people about what's good about SodaTest, what needs more work and whether there's parts that are just plain horrible; and, if the feedback is positive enough to consider SodaTest a preliminary success, I want to continue improving it in the areas where it's still holding people back.

There is already a Roadmap of possible features to add, but now it's really time to get it in people's hands and find out from users what is the next most important thing it needs to do.

Try It Out!

If you're a Scala or Java software developer and Executable Requirements are either a passion of yours or something you've been wanting to try out, why don't you give SodaTest a try? You don't have to commit to it, just write a couple of tests with it, get them running, then passing, and send me some feedback. All feedback is useful, even if you think it sucks! (As long as you tell me why.)

To get started with SodaTest, I suggest you:

• Read the SodaTest Project README
• Use Git to clone and have a look at the examples included with the project
• Download SodaTest and write a test. If you're using Maven, you can use the repository and dependencies below, otherwise you can download JARs from GitHub.

<repositories>
   <repository>
       <id>sodatest-github-repo</id>
       <name>SodaTest repository on GitHub</name>
       <url>http://grahamlea.github.com/SodaTest/m2repository/ </url>
   </repository>
</repositories>
<dependencies>
   <dependency>
       <groupId>org.sodatest</groupId>
       <artifactId>sodatest-api</artifactId>
       <version>0.1</version>
       <scope>test</scope>
   </dependency>
   <dependency>
       <groupId>org.sodatest</groupId>
       <artifactId>sodatest-runtime</artifactId>
       <version>0.1</version>
       <scope>test</scope>
   </dependency>
</dependencies>

Explaining Scala to Your Wife Through Knitting

On Friday night I was sitting in my kitchen coding up some Scala. (Oh, what excitement a having small, sleeping child brings to Friday nights!)

My wife, who is a die-hard knitter, came into the room and, having seen me do little in the evenings for the last few years except code Scala, asked, "Is Scala just the bee's knees?"

My response, purely from reflex, was "No. CODING is the bee's knees!"

"Sure," she said (with a roll of the eyes, I suspect), "but is it like knitting something really interesting? Compared to Java, I mean?"

"Not quite," I said, "it's like if you had electric knitting needles that would let you start off the row and once you've done the interesting bit, the needles take over and do the rest of the row for you. So you'd just knit the interesting bits, not the dumb bits. With Java, though, you spend a lot of time writing dumb stuff."

"Right," she said, "that would be cool."

Yes, it would be cool. Sadly for knitters, electric knitting needles do not exist. Java programmers, on the other hand, have the freedom to use Scala if we want to stop doing dumb stuff.

How would you explain Scala to your wife?

Scala's Match is Not Switch++

Scala's match operator is a beautifully powerful beast. It can do some really sophisticated stuff with very little effort. However, if your introduction to 'match' was along the lines of, "It's like Java's 'switch', but better", then I want to offer a word of caution. I'll show an example where this thinking caused problems for me, briefly explain match and explain how I look at it now to prevent this confusion for myself.

A (Bad) Example
Some things can look quite straightforward, but at runtime act quite differently from how they look. You might call these bugs. Take the following snippet as an example. What do you think this prints out?

object TestingObject {
  def main(args: Array[String]) {
    val testValue = "Boris"
    val inputValue = "Natasha"
    List(inputValue) match {
      case List(testValue) => println("Match!")
      case _ => println("No match")
    }
  }
}

Because you're a smart alec and you know what makes for an interesting blog, you have probably correctly guessed that this prints out "Match!".

Let's try and get a little more information...

object TestingObject {
  def main(args: Array[String]) {
    val testValue = "Boris"
    val inputValue = "Natasha"
    List(inputValue) match {
      case List(testValue) => println("Match! testValue = " + testValue)
      case _ => println("No match")
    }
  }
}

The code now prints "Match! testValue = Natasha". This gives us a bit more insight into what's going on. Obviously the testValue that is defined in the case expression is not a reference to the original val testValue but a declaration of a new reference with a different scope.

My mistake when I wrote this code was to believe that everything that I put between case and => was an input to the matching algorithm, i.e. it was going into some == or .equals() that happens under the hood. This is true in Java's switch statement, where everything in the case must be a constant, but it's certainly not how things work in Scala's match.

New Perspective
I prefer now to think of match the other way around to how it was first introduced to me: I now consider the use of an explicit extractor object to be the normal case, and I think of any other form of case expression, such as matching against a constant value, as syntactic sugar for some built-in, implicit extractor.

When you use an extractor object, you have an object that defines an unapply() function which accepts an object of the type you are matching as a parameter. The unapply() function can either return a Boolean or an Option. If it returns a Boolean, then true means the argument is a match and false means that it wasn't. If it returns an Option, then None means that it wasn't a match, while a Some indicates it did match. Not only this, but the object contained in the Some becomes available to the expression on the right of the =>, assuming you give it a name in the case expression. I think this is the most significant difference between switch and match: In Scala, the case expression has outputs.

My problem in the example at the start was caused because I thought of List(testValue) in the case expression as an input, but it's not. It's actually the name of an Extractor object (List) and a name given to the output of that Extractor (testValue).

My new way of thinking is to pretend that everything in a Scala case is an extractor. So when I look at a case that is matching a plain old constant, I think to myself, "That's calling an extractor that returns true if the input value equals the constant. If I see case x: Int => I think, "That's calling an extractor that returns Some(x) if the object is an Int or None if it's not.

Re-Program Yourself
So, my suggestion is to rid yourself of any notion that Scala's match is like "switch on steroids". That view can lead to the false belief that match is a switch that can deal with all kinds of interesting, non-constant input values. In actual fact, the cases in match are all about Extractors, not inputs, and matching against constant values in the same way as switch is best thought of as just a nice little trick that the compiler does for you to hide the Extractor.

Want to Learn More?
If you suspect that a bit too much of your Scala knowledge may come from blogs that were comparing it with Java, reading one of these books might help:

From Amazon...

From Book Depository...

'Programming in Scala' book now FREE

If you're looking to learn Scala or looking to broaden your knowledge thereof, you'll be pleased to know that it's just been announced that the first edition of 'Programming in Scala' by Martin Odersky (the creator of Scala), Lex Spoon, and Bill Venners is now FREE online in HTML format.

Here's the announcement:
http://groups.google.com/group/scala-announce/browse_thread/thread/502c7d5a386e357f

Here's the online version:
http://www.artima.com/pins1ed/

If you want to grab a hard copy as well, you can get it here...

From Amazon...

From Book Depository...

Scala Pragmatism: Ignore CanBuildFrom

After a recent blog of mine about converting two lists to a map, a commentor wrote that they were...

trying to understand the code I read [in] the ScalaDoc for the zip() [function]:

def zip[A1 >: A,B,That](that: Iterable[B])(implicit bf: scala.collection.generic.CanBuildFrom[Repr,(A1, B),That]): That

would you please help us to read/understand this?

This is indeed curly stuff which, I'll have to admit, I wouldn't be able to fully explain without consulting a text book or two to get the terms right. Then you'd also have to read the same text books before you could understand my explanation.

If I can take some poetic license with the question, though, I think what the poster really wants to know is why the signature is so complex when the intent of the function is so simple. So instead of explaining the intricate details of what this signature means, I'm going to explain why, to make practical use of the zip() function, you don't need to understand the above signature, as well as a little bit about what the added complexity actually achieves.

Firstly: Why you don't need to bother too much.

If you look at the Scala API for List, you'll see that as well as the above signature for zip(), it also lists a "[use case]" signature for zip, which is simply this:

def zip [B] (that: Iterable[B]) : List[(A, B)]

The purpose of this 'Use Case' signature is to let you know you that, most of the time, you will be able to call zip() on a List and pass in just another List (or some other concrete subclass of Iterable) and get back a List of Tuple2s. Just like this:

scala> val list1 = List(1, 2, 3)
list1: List[Int] = List(1, 2, 3)

scala> val list2 = List('a', 'b', 'c')
list2: List[Char] = List(a, b, c)

scala> val result = list1.zip(list2)
result: List[(Int, Char)] = List((1,a), (2,b), (3,c))

That's the pragmatic part, and the most important part to understand. If you cast your eyes down the List scaladoc, you'll notice that there's lots of these use cases. (You should understand why this is the case once you understand why the extra complexity is there.)

To sum that up: to use the zip function, you just have to pass in an Iterable. You will rarely, if ever, need to know what CanBuildFrom is or what all those type parameters refer to.

Secondly: Why then have all that complexity in the signature, when I'm not using it?

Whoa! That's actually not true! The fact that we've called the function without explicitly providing that second parameter doesn't mean that you haven't used the extra complexity. In fact, you have used it, and it's provided you a small benefit, all without you realising. This is where a little bit of magic happens...

In understanding why that signature is so complex, the key is actually in the explanation above of what zip does. Note that I said that you can call zip() on a List and pass in just another List, and you will get back another List. This is actually a little bit amazing. It probably doesn't seem that amazing, but then you have to consider that the implementation of the zip function is not actually defined on the List class, but is defined in a trait that List inherits from called IterableLike. What is surprising here is that the IterableLike trait doesn't know anything about the List class, and yet its zip function is able to return an instance of List. (And remember that Scala's List is a concrete class, very different from Java's List interface.)

This is what the CanBuildFrom parameter is all about: it allows functions like zip to be defined once, high up in the collections API hierarchy, but then to also return a different type of result depending on the type of the collection on which the method is actually invoked. To put that in practical terms: if I call zip on a List, I'll get back a List of tuples, but if I call zip on a mutable.HashSet, I'll get back a mutable.HashSet of tuples. The same goes for map() and filter() and countless other functions. This is the magic: these functions don't know anything about these concrete types they are returning, but you haven't had to tell it about the concrete types either! To achieve the same result in Java without any casting, you would have to override the zip function in each and every concrete subclass of IterableLike from which you wished to return a specific collection type. (Edit: Actually, I don't think this is true. I think I've figured out in my head a way to do it that doesn't require casting or overriding.)

There is just one mystery left: why don't you have to pass in an instance of CanBuildFrom? As you can see, the parameter list containing this parameter is implicit. This means that, if this parameter list is not explicitly defined in any invocation, the Scala compiler will to attempt to find a variable or object that is in scope, and declared to be implicit, and also meets the type criteria of the parameter. So you don't have to provide a CanBuildFrom instance because scalac is tracking one down for you.

But where is it tracking it down from? All code has to come from somewhere, right? At first, I thought that there might be one CanBuildFrom instance in the Predef object that served the needs of nearly every collection. The real answer is a bit less exciting: each concrete subclass provides its own instance of CanBuildFrom in its companion object, though in many cases this definition is as simple as creating a new instance of GenericCanBuildFrom. I have to be honest and admit that I haven't yet figured out how this definition in the companion object makes its way into the zip function and other functions. While you unwittingly import the companion object whenever you import a collection type, you haven't imported all the companion object's functions into scope. Perhaps someone with a bit more patience for searching the Scala source code will nice enough to explain it in the comments.

So, to sum up:

• You don't need to understand CanBuildFrom in order to use functions that have one in their signature.
• Look for "[use case]" entries in the scaladoc for the collections API and use those as the basis for function calls unless the compiler seems to be telling you to do otherwise
• The advantage of the complex function signatures is that the definition can be written once, but can return many different concrete types of collection, without you having to tell it how to do that.

Want to learn more?

If you'd like to get a bit more detail about how Scala's collections and the related classes like CanBuildFrom operate under the covers, there is a series of pages on scala-lang describing some of the implementation details of Scala's new collections API.

And if you really want to know every last thing about why this signature looks the way it looks and how it works, you might want to read the academic Scala paper, 'Generics of a Higher Kind', but please don't post questions about that on my blog! ;)

If you just want to learn a bit more about Scala, try one of these:

From Amazon...

From Book Depository...

Scala == Effective Java ?

I started reading Joshua Bloch's Effective Javalast week. I'll have to admit that I haven't read it before, but only because I've been told by several people, "you already do most of what's in there anyway." Seeing as we tell all the new recruits to read it, I thought I should actually flip through it myself so I know what's in there.

Books of best practices are always written in relation to domains that have many possibilities for bad practices (choosing otherwise would make for a very short book). Reading the first chapter of Effective Java, I was amused as I realised that, if you're coding in Scala instead of Java, many of the book's recommendations are either unnecessary, because Scala doesn't permit the corollary bad practice, or built into the language of Scala, or made easier to implement than they are in Java. This isn't a criticism of the book, but an observation that the state of the art is moving on, and Java is being left behind.

From the first 25 items in the book, here are my notes on practices that either become easier to follow or unnecessary if you are using Scala:

Item 1: Consider static factory methods
One of the four advantages given for static factory methods is that construction of classes with type parameters is briefer through static methods because the type parameters can be inferred on a parameterised method call. Scala solves this one by inferring type parameters on constructors, but not only that. It infers LOTS of types, almost everywhere: fields, local variables, method return types, anonymous function parameters - all can usually have their type inferred from the context, meaning that Scala code has a lot less declarative noise, while still remaining statically type-checked.

Item 2: Consider the Builder pattern instead of long constructors
Joshua writes himself that the Builder pattern simulates the combination of named parameters and default parameters that are found in other languages, and Scala supports named and default parameters. What he means by "simulates" is that you need to write a whole extra Builder class in Java to get the nice Builder effect at the client site, whereas in Scala, you essentially get a Builder for free every time you write a constructor.

Item 3: Enforce Singletons
While code-enforced singletons have gone out of fashion a bit with the popularity of IoC containers, they're still around a lot and these rules are still important. Scala supports singletons at the language level, providing the 'object' keyword which, substituted for the word 'class' in a declaration, will compile a singleton object into your code instead of a class. The generated object follows all of Josh's recommendations, including deserializing to the same instance (as long as you use Scala's @serializable rather than extends java.io.Serializable)

Item 4: Enforce non-instantiability with private constructors
There's really only two cases where you want to do this: singletons (see above), and utility classes, which are really just singletons that have no state. Either way, Scala's 'object' keyword for creating singletons is the simple answer.

Item 5: Avoid auto-boxing
The recommended panacea for unwanted auto-boxing is to prefer primitives wherever possible. Now, Scala is a purely object-oriented language, which means there are no primitives, and all numbers, characters and booleans are represented at the language level as objects. However, Scala uses of these primitive wrapper objects compile to byte code which uses Java's primitives wherever possible, so this recommendation is implemented for you by the Scala compiler.

Items 7, 8 and 9: Overriding toString, hashCode and equals
If you're authoring an immutable data structure, declaring your Scala class as a case class will tell the compiler to automatically implement toString, hashCode and equals for you, along with an unapply() method that can be used in a pattern matching clause. (There are some disadvantages to using Scala's case classes, but I believe they work well for most situations.)

Item 11: Override clone() judicously
While Scala as a language doesn't provide an answer to this one, it's considered best-practice Scala to favour immutable types, with transformation being much favoured over mutation. Following this principle will reduce the need to ever use clone(), because immutable objects can be shared among many clients and threads without the shared-mutable-state worry that might cause you to consider cloning something.

Item 14: Use public accessors methods rather than public fields
While Scala appears to have public fields - and indeed to make fields public by default - in fact Scala implements Bertrand Myer's "Uniform Access Principle", with all access to fields (which are in fact all private) being made through accessor and mutator functions that have the same name as the field. In other words, the compiler writes get and set methods for you and everything that looks like field access in your code actually goes through these methods.

Item 15: Minimize mutability
As already mentioned, it's considered Scala best practice to shun mutable state as much as possible. One of Josh's four recommendations for decreasing mutability is to make values final wherever possible. All Scala fields and local variables must be preceded by either 'val', indicating immutability (of the reference), or 'var', indicating that the reference can change. (Function parameters are always vals, and hence final.) Forcing programmers to make this choice when each variable is declared encourages the practice of using a lot of immutability, compared to final which is an optional modifier and seen by many as extra noise in most declarations.

Item 18: Prefer interfaces to abstract classes
Scala has traits - abstract classes that essentially allow multiple inheritance of function definitions (as opposed to interfaces, which only inherit function declarations). The possibility of multiple inheritance discounts quite a few of the disadvantages Josh raises against using abstract classes in Java. Of course, it also introduces some new issues, and exacerbates some others, like those Josh lists in Item 16 about preferring composition+delegation over inheritance. Multiple inheritance is a double-edged sword, for sure.

Item 21: Use function pointers
Scala, as a functional language, has functions as first-class members of the language, with every function naturally being available as an object (same as what Josh calls a function pointer) should the need arise. It also supports anonymous, inline functions, which, if available in Java, could reduce current "function pointer" logic like this:
new Thread(new Runnable() { public void run() { System.println("Running"); } } );
down to something like this:
new Thread({System.println("Running")})

Item 23: Don't use raw generic types
Scala doesn't allow you to write code that uses raw generic types, even if those types are defined in Java: it just won't compile. For what it's worth, raw generic types are not a feature, but merely an artefact of backwards-compatibility. Scala, not trying to be backwards-compatible with Java 4, just doesn't need raw types and, as a result, is able to provide stricter type safety for type-parameterised classes and functions.

Item 25: Prefer Lists over Arrays
While you should probably still prefer Scala's List class to Arrays for most applications, Scala prevents the chief problem cited with Java's arrays. In Java, you can cast a String[] to an Object[] and then assign a Long to one of the entries. This compiles without error, but will fail at runtime. In Scala, however, arrays are represented in source code as a parameterized class, Array[T], where T is an invariant type parameter. This basically just means that you can't assign an Array[String] to an Array[Object], and so Scala prevents this problem at compile time rather than choking at runtime.

Scala == Effective Java ?
So I'm going to put this question out there:
If 'Effective Java' is considered essential reading, and the best practices in it are the de facto standard for writing good programs, shouldn't we all be giving serious consideration to switching to a language that is so very close to Java, but makes good programming even easier?

Want to learn more?

From Amazon...

From Book Depository...

How to convert two Scala Lists into a Map

Update:

Since I posted this 24 hours ago, no fewer than 13 lovely people have left comments to let me know that this is actually far, far easier than I thought. There is a function on the List class (since 2.8.0, I think someone said), inherited from TraversableOnce, called toMap() which, if the sequence contains Tuple2s, will make a Map from them.

So the ideal solution is simply:

(keys zip values) toMap

Now the question is: How come everyone else knows about this and I didn't? :(

A couple of people also mentioned that you don't need to convert a List into an Array to be able to pass it into a var args method (but you still need the funky noise). Handy!

Thanks, everyone.


Original Post...

Not sure if this will ever be of use to anyone else, but I thought I'd put it out there just in case.

Problem:
You have a List/Seq of keys and a List/Seq of values, with the key at index n in one list matching the value at index n in the other. But what you really want is to convert them into a Map.

Solution:

Map(keys.zip(values).toArray: _*)

Explanation:
The steps here are:

1. "Zip" the keys list with the values list, which creates a List of Tuple2 objects
2. Convert the list of Tuples to an Array of Tuples. This is necessary because the Map object, which extends MapFactory, doesn't have any apply() methods that deal with lists but only the one that accepts a var args parameter of Tuple2s.
3. Add this funky noise ": _*" which tells Scala to pass the Array as multiple varargs parameters, rather than trying to pass the whole Array as the first parameter.
   

Graham's Guide to Learning Scala

It's a pretty widely-accepted view that, as a programmer, learning new languages is a Good Idea (tm). Most people with more than one language under their belt would say that learning new languages broadens your mind in ways that will positively affect the way you work, even if you never use that language again.

With the Christmas holidays coming up and many people likely to take some time off work, this end of the year presents a great opportunity to take some time out from your week-to-week programming grind and do some learning.

With that in mind, I present "Graham's Guide to Learning Scala". There are many, many resources on the web for learning about Scala. In fact, I think there's probably too many! It would be quite easy to start in the wrong place and quickly get discouraged.

So this is not yet another resource to add to the pile. Rather, this is a guided course through what I believe are some of the best resources for learning Scala, and in an order that I think will help a complete newbie pick it up quickly but without feeling overwhelmed.

And, best of all, it has 9 Steps!

[Note: Many of the resources have a slant towards teaching programmers that know Java, but I imagine if you know any other popular OO language like C++, C#, Ruby, Python, Objective-C or Smalltalk, you shouldn't have a problem picking it up.]

Are you ready? Here goes...

Step 1: Lay the Foundation

Read 'Introduction to the Scala Language' on scala-lang.org

This single page is a succinct but dense description of Scala and will let you know what you're in for, and why you should keep going.


Step 2: Get the Tools

Download and install the latest Scala release

Download and install the latest IntelliJ IDEA *Community Edition* release

Open IDEA and install the 'Scala' plugin


Step 3: Hello, World!

Create a new IDEA project with a Scala module.
Create a new file called HelloWorld.scala and paste in the following code:

object HelloWorld {
  def main(args: Array[String]) {
    println("Hello, world!")
  }
}

Hit Ctrl-Shift-F10 to run it.
Congratulations! First Scala program done and dusted!


Step 4: Commute your Java skills to Scala

Read Daniel Spiewak's 'Scala for Java Refugees' series.

This should give you enough knowledge and experience with Scala syntax to be able to write Scala that does most of the things you can do in Java.

Make sure you enter the code examples into IDEA and try them out as you go. I suggest typing them out instead of copy-paste, because that's likely to give you exposure to common syntax mistakes and compiler errors.


Step 5: Open Your Eyes

Read through the 'Tour of Scala' on scala-lang.org

These short pages will give you some exposure to some of the more advanced features of Scala. Don't worry if you don't understand everything - just keep reading. You're not trying to learn how to use all these things, just to know that they're around.


Step 6: Learn the Basics, the Ins & the Outs

Read chapters 1 to 8 of Dean Wampler & Alex Payne's 'Programming Scala' (O'Reilly)

This is where the learning gets heavy. We're talking about more than half a book here. You'll learn everything from basic syntax (some of which you'll already know from the Refugees series) to Scala's more advanced patterns for object-oriented programming, an introduction to Scala's functional programming capabilities and some of the curly edges of Scala's type system.

Again, make sure you try out running the code samples in IDEA. I suggest you also have a play around with some of the concepts in your own little test classes. Any time you find yourself thinking "I wonder if I can use this feature to do this?", grab a keyboard and find out for yourself.


Step 7: Open the Dragon Scroll and Release your Scala Fu

Take a break from learning and do some hacking!

Think of a simple idea that you'd like to implement in Scala, then code it! (Seriously, I recommend keeping it REALLY simple.) If you can't think of something yourself, you can steal my idea of writing a function to perform word-wrapping.


Step 8: Take a Walk around the Grounds

Read the rest of 'Programming Scala' (chapters 9 to 14)

The rest of the book covers some of the peripheral libraries and concepts in Scala that contribute to making it a great language, like Agents for multi-threaded programming, "herding" XML input and output and building blocks for creating domain-specific languages, as well as reviewing some of the tools that have sprung up in the Scala ecosystem.


Step 9: Go Forth and Hack!

This is where the directed part of the journey ends, but hopefully it's only the first step.

What you do with Scala next is up to you, but by this point you know more than enough about Scala to start using it seriously at home or even at work, for fun, or for making some serious cash! If you do make some serious cash by hacking Scala, please remember how it all started and send a little monetary "Thank you" my way. ; )


Addendum: Getting Help

Perhaps it was a little presumptuous of me to effectively say "go away now and code". Chances are, you'll need some help as you keep experimenting, learning and doing more and more cool stuff with Scala.

The two best places that I've found to connect with other Scala users are the scala-user mailing list run by scala-lang.org and, for curly technical problems that you just can't figure out yourself, stackoverflow.com. Actually, reading through other people's Scala questions on Stack Overflow can also be a great way to pick up new ideas!


Want to learn more?

From Amazon...

From Book Depository...