Creating a simple simulation - The HarvesterSpec

In my previous post, I introduced the Producer actor. Now it's time to introduce the Harvester.

In this post I will define the specifications of the Harvester and show how to test the concurrency of the eventual implementation.

In order to do this, I will also dive into some of the cooler features of Scala, especially foreach, map, list-comprehensions and futures.

Game concepts

Harvesters are built close to or on top of Producers and are required to harvest and store the raw materials. Examples of harvesters include Mines, Farms etc.

The materials retrieved from the Producer are stored inside the Harvester, until the Harvester's capacity is reached. After that point, Harvesters no longer harvest actual goods.

Finally, Harvesters operate on a specific speed, implying that the act of harvesting does not immediately result in harvested goods. In terms of our simulation: multiple Harvest messages can be required before a Harvester actually produces something.

Implementation

Specification

Since I have a fully functional unit testing framework at my disposal, I started by creating the specification for my Harvester. For those who want to know: this approach seems to be called Behavior Driven Development (BDD). Whatever. Condensed, it looks like this:

"A Harvester with speed 1" should {
  "respond to Harvest messages"
  "harvest goods from a producer"
  "store the harvested goods"
  "no longer store goods when it's full"
  "no longer harvest goods when it's full"
  "be connected to a matching producer"
}
"A Harvester with speed higher than 1" should {
  "require a number of Harvest calls before actually harvesting"
}
"A simulation with Harvesters" should {
  "run with thousands of others Harvesters in parallel"
}

This is a lot more than my previous specification, and let's be honest: many of the associated code blocks are quite boring. So let's have a look at the interesting cases.

Timeouts

"respond to Harvest messages" in {
  producer = new Producer("Test Producer", stuff).start()
  harvester.connectTo(producer)
  harvester !? (100, Harvest) must be_==(Some(Harvested(stuff, 1)))
}

The call !? now takes a tuple as an argument: a timeout and the message to send to the harvester. If the harvester replies within the timeout, the value Some(...) will be returned. If the harvester takes too long, we get the value None.

Using a timeout ensures that my implementation can fail without causing my test to hang.

Mock Actors

"harvest goods from a producer" in {
  var producerCalled = false
  producer = actor {
    loop {
      react {
        case Produce => {
          producerCalled = true
          reply(Produced(stuff, 1))
        }
		case Stop => exit()
      }
    }
  }
  harvester.connectTo(mockProducer)

  harvester !? (100, Harvest) must be_==(Some(Harvested(stuff, 1)))
  producerCalled must beTrue
}

Did you see what I did here? I created an inline actor (indicated by the lower case actor construct), and used it to keep track of the calls to it. In other words, I've mocked my producer. And all of this without introducing new explicit classes or using extra mocking frameworks. In my opinion, that's cool.

Inline actors are started automatically; however, they do not exit() automatically, so note how it also responds to Stop messages.

Lists, lots of objects and futures

The final part is about testing the concurrency of the Harvesters-to-be. Let's have a look:

"A simulation with Harvesters" should {
  val max = 10000
  val ps = for (i <- 1 to max) yield new Producer("p" + i, "g")
  val hs = for (i <- 1 to max) yield new Harvester("h" + i, "g", 1, 1)
  val actors = ps ++ hs

  doBefore {
    // combine harvesters and producers
    (hs zip ps) foreach {case (h, p) => h,connectTo(p)}

    // start'em all
    actors foreach (a => a.start())
  }

  doAfter {
    // stop'em all
    actors foreach (a => a ! Stop)
  }

  "run with thousands of others Harvesters in parallel" in {
    // send a harvest message to each harvester
    val futures = hs map (h => h !! Harvest)

    val responses = Futures.awaitAll(2000, futures : _*)

    // check responses
    responses foreach (r => r must be_==(Some(Harvested("g", 0))))
  }
}

Before diving into the details, I will explain my intentions. The idea is to create max producers and harvesters, to pair them up and then to send a Harvest message to each Harvester. After that, all the actors get some time to run – given that Scala promised me that actors are highly concurrent, they will get 2 entire seconds. And finally, I want to check that each Harvester sent the correct response. So how did I do that?

The first step was to create 10,000 producers:

val ps = for (i <- 1 to max) yield new Producer("p" + i, "g")

Yes, a list comprehension. Given that I have a Python background, I knew what to look for. In Python, this would be:

[new Producer("p" + i, "g") for i in range(max)]

The same for creating the harvesters. And then, the following code:

(hs zip ps) foreach {case (h, p) => h.connectTo(p)}

Yes, it's getting interesting. As far as I understand Scala syntax (and hey, I've just started), this is identical to

def connect(h: Harvester, p: Producer) = {
  h.connectTo(p)
}
val tuples: List(Harvester, Producer) = hs.zip(ps)
tuples.foreach(connect)

When the setup was complete, it was time to send some messages:

val futures = hs map (h => h !! Harvest)

And voila, I introduced myself to futures – temporary objects that will eventually be filled with a result. Obtaining a future from an actor is done by using !! instead of !?. Given that !? runs synchronously, this would of course be totally unsuitable to test the concurrency of my actors. By using !!, I actually send the message asynchronously – the test will not block until I have start accessing the future values.

The usual way to obtain the value from a future is to call it: someFuture() is the correct way to start waiting for the actual result. However, someFuture() does not have a timeout, and I want to protect myself from actors that never send a reply.

The solution could be found in Futures.awaitAll. For example, Futures.awaitAll(100, someFuture1, someFuture2, ..., someFutureN) waits for at most 100ms. For each future that actually has a value, it returns Some(value); for all the other futures, it returns None.

Unfortunately, Futures.awaitAll expects the futures as varargs, and all I've got is a list. Some Googling later, I found the solution:

val responses = Futures.awaitAll(2000, futures : _*)

Yikes! Apparently, defining the type of futures as _* allows me to use the list as a varargs argument. It works, but it just goes to show that every language has its dark secrets. By the way, this construct is called an Ascription.

Lessons learned

Implementing the HarvesterSpec was an interesting experience. Maybe I wanted to much at once, but writing the spec required a lot of Google and searching StackOverflow.

• Lists – I have written my first lists, and applied some anonymous functions to them using foreach and map. I've also created some very large lists using list comprehensions.
• Futures - I have discovered how to work with futures, and was pleasantly surprised at their implementation.
• Inline actors – I have learned about inline actors; so far, they seem especially useful for mocking. I have yet to find a useful example of inline actors in actual code. But then again, I haven't really searched for such an example.
• Varargs – I have discovered how to pass lists to methods expecting varargs arguments. Again, I was surprised, but not very pleasantly.