Jay Fields's Blog, page 12
February 7, 2011
Clojure: Web Socket Introduction
Web Sockets may sound intimidating, but if you use Clojure and jQuery it's actually quite simple.
The first thing you need to do is grab a WebSocket capable server. Jetty is a decent option, but (Joe Walnes' recently released) webbit was the fastest choice for getting up and running.
You'll probably want to:git clone https://github.com/joewalnes/webbit.git
cd webbit
make jarAt that point you should have both build and lib directories in your webbit directory. You'll want to copy webbit/build/webbit.jar and webbit/lib/netty-3.2.3.Final.jar to a new directory where you're going to create your Web Socket application. You might as well create a blank server.clj and index.html as well. Now you should have a directory that looks something like this:-rw-r--r-- 1 jfields jfields 674 Feb 7 08:58 index.html
-rw-r--r-- 1 jfields jfields 786229 Feb 6 10:49 netty-3.2.3.Final.jar
-rw-r--r-- 1 jfields jfields 693 Feb 7 08:58 server.clj
-rw-r--r-- 1 jfields jfields 59616 Feb 6 10:49 webbit.jarNext we'll get webbit fired up using Clojure. The webbit README gives us a java example that is easily convertible. The following code starts a server and prints it's args when a web socket is opened, closed, or sent a message.(ns server
(:require [clojure.contrib.json :as json]
[clojure.string :as s])
(:import [webbit WebServer WebServers WebSocketHandler]
[webbit.handler StaticFileHandler]))
(doto (WebServers/createWebServer 8080)
(.add "/websocket"
(proxy [WebSocketHandler] []
(onOpen [c] (println "opened" c))
(onClose [c] (println "closed" c))
(onMessage [c j] (println c j))))
(.add (StaticFileHandler. "."))
(.start))
(println "server up")That's it. The server doesn't do much, but it's up and running. Start it up if you'd like.java -cp clojure.jar:webbit.jar:netty-3.2.3.Final.jar:clojure-contrib-1.2.0.jar clojure.main server.clj(note: I'm using Clojure 1.2)
If all went well, you should see "server up" printed to the console. We'll leave the server running for now and put together a simple client.
Again, the examples already on the internet give us 90% of what we need. We're going to use jquery-websocket, and the example at the bottom of the page is just about exactly what we need. The following code is a slightly modified version that should suit our purposes.<html>
<body>
<h1>WebSocket Demo</h1>
<input id="message" type="text"/>
<section id="content"></section>
<script src="http://www.google.com/jsapi">...
<script>google.load("jquery", "1.3")</script>
<script src="http://jquery-json.googlecode.com/fil...
</script>
<script src="http://jquery-websocket.googlecode.co...
</script>
<script>
var ws = $.websocket("ws://127.0.0.1:8080/websocket", {
events: {
upcased: function(e) { $("#content").html(e.message); }}});
$('#message').change(function(){
ws.send('message', {type: "downcase", message: $("#message").val()});});
</script>
</body>
</html>Our client isn't much, but it does connect to a web socket and it sends a message to the web socket when the text in the input is changed.
Assuming you still have the server running you should be able to load up your page.
Here's an updated version of server.clj.(ns server
(:require [clojure.contrib.json :as json]
[clojure.string :as s])
(:import [webbit WebServer WebServers WebSocketHandler]
[webbit.handler StaticFileHandler]))
(defn on-message [connection json-message]
(let [message (-> json-message json/read-json (get-in [:data :message]))]
(.send connection (json/json-str {:type "upcased" :message (s/upper-case message) }))))
(doto (WebServers/createWebServer 8080)
(.add "/websocket"
(proxy [WebSocketHandler] []
(onOpen [c] (println "opened" c))
(onClose [c] (println "closed" c))
(onMessage [c j] (on-message c j))))
(.add (StaticFileHandler. "."))
(.start))
(println "server up")The new version of server.clj takes advantage of clojure.contrib json support. The net result of this is that we can work with Clojure maps and basically ignore json throughout our application.
After making the above changes to server.clj we can restart our server, refresh our webpage, enter some text, tab out of the input, and then we should see our text on the webpage as upper-case.
And, we're done. We have working client-server interaction. We're ready to put this into production. It's that easy.
You might have noticed a few things that make the magic happen. On the server we sent a map that has :type "upcased". This type corresponds to the events that are defined in our client. The jquery-websocket takes care of routing our new message to the function associated with upcased. Extending on this idea, you can send messages from the server with different types and handle each one on the ui as a different event.
That's it. The app should be working, and you should have everything you need to begin expanding the capabilities of the application. If you run into any trouble, the documentation for webbit and jquery-websocket should get you through.
© Jay Fields - www.jayfields.com
The first thing you need to do is grab a WebSocket capable server. Jetty is a decent option, but (Joe Walnes' recently released) webbit was the fastest choice for getting up and running.
You'll probably want to:git clone https://github.com/joewalnes/webbit.git
cd webbit
make jarAt that point you should have both build and lib directories in your webbit directory. You'll want to copy webbit/build/webbit.jar and webbit/lib/netty-3.2.3.Final.jar to a new directory where you're going to create your Web Socket application. You might as well create a blank server.clj and index.html as well. Now you should have a directory that looks something like this:-rw-r--r-- 1 jfields jfields 674 Feb 7 08:58 index.html
-rw-r--r-- 1 jfields jfields 786229 Feb 6 10:49 netty-3.2.3.Final.jar
-rw-r--r-- 1 jfields jfields 693 Feb 7 08:58 server.clj
-rw-r--r-- 1 jfields jfields 59616 Feb 6 10:49 webbit.jarNext we'll get webbit fired up using Clojure. The webbit README gives us a java example that is easily convertible. The following code starts a server and prints it's args when a web socket is opened, closed, or sent a message.(ns server
(:require [clojure.contrib.json :as json]
[clojure.string :as s])
(:import [webbit WebServer WebServers WebSocketHandler]
[webbit.handler StaticFileHandler]))
(doto (WebServers/createWebServer 8080)
(.add "/websocket"
(proxy [WebSocketHandler] []
(onOpen [c] (println "opened" c))
(onClose [c] (println "closed" c))
(onMessage [c j] (println c j))))
(.add (StaticFileHandler. "."))
(.start))
(println "server up")That's it. The server doesn't do much, but it's up and running. Start it up if you'd like.java -cp clojure.jar:webbit.jar:netty-3.2.3.Final.jar:clojure-contrib-1.2.0.jar clojure.main server.clj(note: I'm using Clojure 1.2)
If all went well, you should see "server up" printed to the console. We'll leave the server running for now and put together a simple client.
Again, the examples already on the internet give us 90% of what we need. We're going to use jquery-websocket, and the example at the bottom of the page is just about exactly what we need. The following code is a slightly modified version that should suit our purposes.<html>
<body>
<h1>WebSocket Demo</h1>
<input id="message" type="text"/>
<section id="content"></section>
<script src="http://www.google.com/jsapi">...
<script>google.load("jquery", "1.3")</script>
<script src="http://jquery-json.googlecode.com/fil...
</script>
<script src="http://jquery-websocket.googlecode.co...
</script>
<script>
var ws = $.websocket("ws://127.0.0.1:8080/websocket", {
events: {
upcased: function(e) { $("#content").html(e.message); }}});
$('#message').change(function(){
ws.send('message', {type: "downcase", message: $("#message").val()});});
</script>
</body>
</html>Our client isn't much, but it does connect to a web socket and it sends a message to the web socket when the text in the input is changed.
Assuming you still have the server running you should be able to load up your page.
Is your page not loading? =(If your page loads, your server must be up and running. You should see something similar to the following line in your server console.opened #<NettyWebSocketConnection webbit.netty.NettyWebSocketConnection@8c5488>You might as well type something into the input and tab out (or whatever you prefer to do that fires the "change" event). I typed in "hello" and got the following output in my server console.#<NettyWebSocketConnection webbit.netty.NettyWebSocketConnection@8c5488> {"type":"message","data":{"type":"downcase","message":"hello"}}Okay, everything is working. Let's add a little behavior to our server. Upon receiving a message, our server is going to take the text, upcase it, and send it back to the client.
What URL did you use? I've been told http://localhost:8080/ doesn't work as well as http://127.0.0.1:8080/
What browser did you use? Everything works for me in Chrome (version 8.0.552.237)
Here's an updated version of server.clj.(ns server
(:require [clojure.contrib.json :as json]
[clojure.string :as s])
(:import [webbit WebServer WebServers WebSocketHandler]
[webbit.handler StaticFileHandler]))
(defn on-message [connection json-message]
(let [message (-> json-message json/read-json (get-in [:data :message]))]
(.send connection (json/json-str {:type "upcased" :message (s/upper-case message) }))))
(doto (WebServers/createWebServer 8080)
(.add "/websocket"
(proxy [WebSocketHandler] []
(onOpen [c] (println "opened" c))
(onClose [c] (println "closed" c))
(onMessage [c j] (on-message c j))))
(.add (StaticFileHandler. "."))
(.start))
(println "server up")The new version of server.clj takes advantage of clojure.contrib json support. The net result of this is that we can work with Clojure maps and basically ignore json throughout our application.
After making the above changes to server.clj we can restart our server, refresh our webpage, enter some text, tab out of the input, and then we should see our text on the webpage as upper-case.
And, we're done. We have working client-server interaction. We're ready to put this into production. It's that easy.
You might have noticed a few things that make the magic happen. On the server we sent a map that has :type "upcased". This type corresponds to the events that are defined in our client. The jquery-websocket takes care of routing our new message to the function associated with upcased. Extending on this idea, you can send messages from the server with different types and handle each one on the ui as a different event.
That's it. The app should be working, and you should have everything you need to begin expanding the capabilities of the application. If you run into any trouble, the documentation for webbit and jquery-websocket should get you through.
© Jay Fields - www.jayfields.com
February 4, 2011
Clojure: &env and &form
Inside the body of defmacro you can call &env and &form to get a bit of interesting information that may or may not be helpful.
Here's a few examples that demonstrate how &env and &form can be used.
&env
By default &env is nil.user=> (defmacro show-env [] (println &env))
#'user/show-env
user=> (show-env)
nilHowever, if any bindings exist, &env gives you the names of the bindings as the keys of a map.user=> (let [band "zeppelin" city "london"] (show-env))
{city #<LocalBinding clojure.lang.Compiler$LocalBinding@78ff9053>, band #<LocalBinding clojure.lang.Compiler$LocalBinding@525c7734>}Okay, now we're getting somewhere. What's a Compiler$LocalBinding? I'm not exactly sure, and I've never bothered to look into it. I've been told that the 'values' from &env may change in the future, so I wouldn't rely on them anyway. Since I can't rely on them, I haven't found the need to look into what they are.
Back to the keys. They sure look like symbols.user=> (defmacro show-env [] (println (keys &env)) (println (map class (keys &env))))
#'user/show-env
user=> (let [band "zeppelin" city "london"] (show-env))
(city band)
(clojure.lang.Symbol clojure.lang.Symbol)As the example shows, they are definitely symbols. However, these symbols don't have a namespace.user=> (defmacro show-env [] (println (keys &env)) (println (map namespace (keys &env))))
#'user/show-env
user=> (let [band "zeppelin" city "london"] (show-env))
(city band)
(nil nil)Since the symbols don't have a namespace there didn't seem to be much fun I could do with them; however, you can use the symbols in your macro to print the values, as the following example shows.user=> (defmacro show-env [] (println (keys &env)) `(println ~@(keys &env)))
#'user/show-env
user=> (let [band "zeppelin" city "london"] (show-env))
(city band)
london zeppelinPrinting the values of bindings can be a helpful trick while you are debugging.
&form
&form can be used to get the original macro invocation.user=> (defmacro show-form [] (println &form))
#'user/show-form
user=> (show-form)
(show-form)Okay, not very interesting so far. It gets a bit more interesting when your macro takes a few arguments.user=> (defmacro show-form [a b] (println &form))
#'user/show-form
user=> (show-form 50 100)
(show-form 50 100)
user=> (show-form a 100)
(show-form a 100)So, you can get the arguments. Notice you can grab both 50 and 100.user=> (defmacro show-form [a b] (println (next &form)))
#'user/show-form
user=> (show-form 50 100)
(50 100)
user=> (defmacro show-form [a b] (println (map class (next &form))))
#'user/show-form
user=> (show-form 50 100)
(java.lang.Integer java.lang.Integer)Interesting. So I have a few integers I can work with, if I wish. What about 'show-form'?user=> (defmacro show-form [a b] (println (map class &form)))
#'user/show-form
user=> (show-form 50 100)
(clojure.lang.Symbol java.lang.Integer java.lang.Integer)'show-form' is a symbol, as expected. Which brings us back to a previous example, shown again below.user=> (defmacro show-form [a b] (println (map class &form)))
#'user/show-form
user=> (show-form a 100)
(clojure.lang.Symbol clojure.lang.Symbol java.lang.Integer)Okay, 'a' is also a symbol, unsurprising, but perhaps it's interesting since 'a' doesn't exist anywhere except in our invocation. You can probably do some interesting things here, like allow people to specify enum values and append the enum yourself.user=> (ns user (:import [java.util.concurrent TimeUnit]))
java.util.concurrent.TimeUnit
user=> (defmacro time-units [& l] (->> (next &form) (map (partial str "TimeUnit/")) (map symbol) (cons 'list)))
#'user/time-units
user=> (time-units SECONDS MILLISECONDS)
(#< SECONDS> #< MILLISECONDS>)Would you want to use &form instead of just using the arguments (stored in l)? Probably not. This isn't an exercise in what you should do, but it does demonstrate what you could do.
So &form must be returning a list, right?user=> (defmacro show-form [a b] (println (map class &form)) (println (class &form)))
#'user/show-form
user=> (show-form 50 100)
(clojure.lang.Symbol java.lang.Integer java.lang.Integer)
clojure.lang.PersistentListA list. Correct.
And, I'm in a macro, so I can do anything I want with this list. Maybe I just want to print the arguments. Easy enough.user=> (defmacro show-form [a b] `(println ~@(next &form)))
#'user/show-form
user=> (show-form 50 100)
50 100Of course, there are a million things I could do. You get the idea.
One other interesting thing to note is that &form has metadata.user=> (show-form 50 100)
{:line 132}Perhaps you don't care about line numbers, but they definitely can come in handy when you are writing a testing framework.
I use &form in expectations and I believe LazyTest uses &env. I guess you never know what you're going to need...
© Jay Fields - www.jayfields.com
Here's a few examples that demonstrate how &env and &form can be used.
&env
By default &env is nil.user=> (defmacro show-env [] (println &env))
#'user/show-env
user=> (show-env)
nilHowever, if any bindings exist, &env gives you the names of the bindings as the keys of a map.user=> (let [band "zeppelin" city "london"] (show-env))
{city #<LocalBinding clojure.lang.Compiler$LocalBinding@78ff9053>, band #<LocalBinding clojure.lang.Compiler$LocalBinding@525c7734>}Okay, now we're getting somewhere. What's a Compiler$LocalBinding? I'm not exactly sure, and I've never bothered to look into it. I've been told that the 'values' from &env may change in the future, so I wouldn't rely on them anyway. Since I can't rely on them, I haven't found the need to look into what they are.
Back to the keys. They sure look like symbols.user=> (defmacro show-env [] (println (keys &env)) (println (map class (keys &env))))
#'user/show-env
user=> (let [band "zeppelin" city "london"] (show-env))
(city band)
(clojure.lang.Symbol clojure.lang.Symbol)As the example shows, they are definitely symbols. However, these symbols don't have a namespace.user=> (defmacro show-env [] (println (keys &env)) (println (map namespace (keys &env))))
#'user/show-env
user=> (let [band "zeppelin" city "london"] (show-env))
(city band)
(nil nil)Since the symbols don't have a namespace there didn't seem to be much fun I could do with them; however, you can use the symbols in your macro to print the values, as the following example shows.user=> (defmacro show-env [] (println (keys &env)) `(println ~@(keys &env)))
#'user/show-env
user=> (let [band "zeppelin" city "london"] (show-env))
(city band)
london zeppelinPrinting the values of bindings can be a helpful trick while you are debugging.
&form
&form can be used to get the original macro invocation.user=> (defmacro show-form [] (println &form))
#'user/show-form
user=> (show-form)
(show-form)Okay, not very interesting so far. It gets a bit more interesting when your macro takes a few arguments.user=> (defmacro show-form [a b] (println &form))
#'user/show-form
user=> (show-form 50 100)
(show-form 50 100)
user=> (show-form a 100)
(show-form a 100)So, you can get the arguments. Notice you can grab both 50 and 100.user=> (defmacro show-form [a b] (println (next &form)))
#'user/show-form
user=> (show-form 50 100)
(50 100)
user=> (defmacro show-form [a b] (println (map class (next &form))))
#'user/show-form
user=> (show-form 50 100)
(java.lang.Integer java.lang.Integer)Interesting. So I have a few integers I can work with, if I wish. What about 'show-form'?user=> (defmacro show-form [a b] (println (map class &form)))
#'user/show-form
user=> (show-form 50 100)
(clojure.lang.Symbol java.lang.Integer java.lang.Integer)'show-form' is a symbol, as expected. Which brings us back to a previous example, shown again below.user=> (defmacro show-form [a b] (println (map class &form)))
#'user/show-form
user=> (show-form a 100)
(clojure.lang.Symbol clojure.lang.Symbol java.lang.Integer)Okay, 'a' is also a symbol, unsurprising, but perhaps it's interesting since 'a' doesn't exist anywhere except in our invocation. You can probably do some interesting things here, like allow people to specify enum values and append the enum yourself.user=> (ns user (:import [java.util.concurrent TimeUnit]))
java.util.concurrent.TimeUnit
user=> (defmacro time-units [& l] (->> (next &form) (map (partial str "TimeUnit/")) (map symbol) (cons 'list)))
#'user/time-units
user=> (time-units SECONDS MILLISECONDS)
(#< SECONDS> #< MILLISECONDS>)Would you want to use &form instead of just using the arguments (stored in l)? Probably not. This isn't an exercise in what you should do, but it does demonstrate what you could do.
So &form must be returning a list, right?user=> (defmacro show-form [a b] (println (map class &form)) (println (class &form)))
#'user/show-form
user=> (show-form 50 100)
(clojure.lang.Symbol java.lang.Integer java.lang.Integer)
clojure.lang.PersistentListA list. Correct.
And, I'm in a macro, so I can do anything I want with this list. Maybe I just want to print the arguments. Easy enough.user=> (defmacro show-form [a b] `(println ~@(next &form)))
#'user/show-form
user=> (show-form 50 100)
50 100Of course, there are a million things I could do. You get the idea.
One other interesting thing to note is that &form has metadata.user=> (show-form 50 100)
{:line 132}Perhaps you don't care about line numbers, but they definitely can come in handy when you are writing a testing framework.
I use &form in expectations and I believe LazyTest uses &env. I guess you never know what you're going to need...
© Jay Fields - www.jayfields.com
January 19, 2011
Compatible Opinions on Software
I'm switching teams this month. I'll be working out of the DRW New York City office in the near future. I've been with my current team for almost 2 years, and I've learned plenty of valuable lessons. I'm not sure if the most valuable take-away centers around Compatible Opinions on Software; however, it's definitely the take-away that seems to remain in the fore-front of my memories.
The guys I work with in the DRW CT office are truly talented. Each individual on my team is someone I would consider to be a great software developer. As a result, we don't tend to have discussions on issues that where one answer is clearly right and another is clearly wrong. Instead, we often find ourselves in situations where one or more individuals have differing opinions and a "correct" answer cannot be found. Despite everyone being an excellent developer, several members of the team have incompatible opinions on software.
Here's the net result: Quality software. We aren't talking about a problem that caused projects to fail. On the contrary, our customer is very happy and the team is largely happy with the product we deliver. Unfortunately, as a teammate puts it, I seem to be immune to complacency. Even though things are good, I wont be satisfied until they are great. And, I feel that incompatible opinions on software is our largest production thief.
In general, it seems that anything can cause incompatible opinions. However, to make clear what I'm talking about, here is a short sample list of incompatible opinions I've run into:powerful language vs. powerful IDEmonolithic application vs many small componentswhat level of duplication is acceptable, and whyspecialization vs generalizationteam sizeFor the majority of my career I've believed - if I work on a team of excellent developers everything will simply fall into place. I now believe that being technically excellent is only the first requirement. I also believe that you can have two technically excellent people who have vastly different opinions on the most effective way to deliver software.
Two developers who prefer a powerful language may or may not be more productive than two developers who prefer a powerful IDE. However, both of the previous pairs will be more productive than one developer who prefers a powerful language teamed up with one developer who prefers a powerful IDE.
I'm not talking about people who love IntelliJ but are perfectly willing to learn emacs. If you're open to working in emacs then you have a compatible opinion with an emacs zealot. I'm talking about people who have tried the opposing point of view and still have a strong opinion that it's not the best way to deliver software. Putting those people on the same team only ensures a never-ending battle about why things aren't as good as they could be.
Of course, having differing views can be a good thing. Diversity ensures all kinds of benefits. I'm not suggesting that you rid yourself of all diversity. However, for the sake of productivity, I suggest that you do your best to avoid working with someone who has both an opposing view and is as inflexible as you are on the subject. The more central the subject is to the project, the more likely it is that productivity will be lost.
Let's get back to a concrete example. For me personally, I strongly prefer Clojure to Java. That's not to say that Java doesn't have it's place, it does. But, if both languages are valid choices, I'll always opt for Clojure. Given that opinion, I'm more effective if I'm working with someone else that prefers Clojure to Java. The language decision is important to me, very central to the project, and likely to be an ongoing issue.
Building off of the previous example, I don't personally care if we use IntelliJ or Emacs. As long as the IDE or editor choice is a reasonable one, I'm happy to use whatever someone else on the team prefers. I welcome diversity on compatible opinions. In fact, it's likely that diversity on compatible opinions will lead to greater productivity as better solutions are found.
So what can you do? The obvious answers: I've started making a list of opinions that I'm less flexible about. If I'm interviewing for a teammate, I'll ask questions to ensure we have compatible opinions on software. If I'm interviewing for a new job, I'll ask the interviewers about their opinions on software. To a certain extent I've always asked these questions, but I think I'll have better results if I identify which of my opinions I'm less flexible about and focus a bit more on ensuring that there aren't any conflicts.
If you already have a team in place and aren't looking to leave it's a bit harder to resolve incompatible opinions on software. Depending on the size of your team, you may be able to split the team up into smaller groups of people who have compatible opinions on software. Obviously, this will depend on having enough people to allow for a split, and having software that can be logically divided. If you can pull this off, you also get the nice benefit of seeing the results from doing things differently, without having to change your approach until it makes sense for you.
At this point I've worked on three different teams that achieved a level of productivity greater than that of any other teams I've been a part of. Of course, there are several factors that contributed to the success of all three teams. However, all three teams had compatible opinions on software, and avoided an entire class of problems that erode productivity. You'd have a hard time convincing me that those two things weren't somehow related.
© Jay Fields - www.jayfields.com
The guys I work with in the DRW CT office are truly talented. Each individual on my team is someone I would consider to be a great software developer. As a result, we don't tend to have discussions on issues that where one answer is clearly right and another is clearly wrong. Instead, we often find ourselves in situations where one or more individuals have differing opinions and a "correct" answer cannot be found. Despite everyone being an excellent developer, several members of the team have incompatible opinions on software.
Here's the net result: Quality software. We aren't talking about a problem that caused projects to fail. On the contrary, our customer is very happy and the team is largely happy with the product we deliver. Unfortunately, as a teammate puts it, I seem to be immune to complacency. Even though things are good, I wont be satisfied until they are great. And, I feel that incompatible opinions on software is our largest production thief.
In general, it seems that anything can cause incompatible opinions. However, to make clear what I'm talking about, here is a short sample list of incompatible opinions I've run into:powerful language vs. powerful IDEmonolithic application vs many small componentswhat level of duplication is acceptable, and whyspecialization vs generalizationteam sizeFor the majority of my career I've believed - if I work on a team of excellent developers everything will simply fall into place. I now believe that being technically excellent is only the first requirement. I also believe that you can have two technically excellent people who have vastly different opinions on the most effective way to deliver software.
Two developers who prefer a powerful language may or may not be more productive than two developers who prefer a powerful IDE. However, both of the previous pairs will be more productive than one developer who prefers a powerful language teamed up with one developer who prefers a powerful IDE.
I'm not talking about people who love IntelliJ but are perfectly willing to learn emacs. If you're open to working in emacs then you have a compatible opinion with an emacs zealot. I'm talking about people who have tried the opposing point of view and still have a strong opinion that it's not the best way to deliver software. Putting those people on the same team only ensures a never-ending battle about why things aren't as good as they could be.
Of course, having differing views can be a good thing. Diversity ensures all kinds of benefits. I'm not suggesting that you rid yourself of all diversity. However, for the sake of productivity, I suggest that you do your best to avoid working with someone who has both an opposing view and is as inflexible as you are on the subject. The more central the subject is to the project, the more likely it is that productivity will be lost.
Let's get back to a concrete example. For me personally, I strongly prefer Clojure to Java. That's not to say that Java doesn't have it's place, it does. But, if both languages are valid choices, I'll always opt for Clojure. Given that opinion, I'm more effective if I'm working with someone else that prefers Clojure to Java. The language decision is important to me, very central to the project, and likely to be an ongoing issue.
Building off of the previous example, I don't personally care if we use IntelliJ or Emacs. As long as the IDE or editor choice is a reasonable one, I'm happy to use whatever someone else on the team prefers. I welcome diversity on compatible opinions. In fact, it's likely that diversity on compatible opinions will lead to greater productivity as better solutions are found.
So what can you do? The obvious answers: I've started making a list of opinions that I'm less flexible about. If I'm interviewing for a teammate, I'll ask questions to ensure we have compatible opinions on software. If I'm interviewing for a new job, I'll ask the interviewers about their opinions on software. To a certain extent I've always asked these questions, but I think I'll have better results if I identify which of my opinions I'm less flexible about and focus a bit more on ensuring that there aren't any conflicts.
If you already have a team in place and aren't looking to leave it's a bit harder to resolve incompatible opinions on software. Depending on the size of your team, you may be able to split the team up into smaller groups of people who have compatible opinions on software. Obviously, this will depend on having enough people to allow for a split, and having software that can be logically divided. If you can pull this off, you also get the nice benefit of seeing the results from doing things differently, without having to change your approach until it makes sense for you.
At this point I've worked on three different teams that achieved a level of productivity greater than that of any other teams I've been a part of. Of course, there are several factors that contributed to the success of all three teams. However, all three teams had compatible opinions on software, and avoided an entire class of problems that erode productivity. You'd have a hard time convincing me that those two things weren't somehow related.
© Jay Fields - www.jayfields.com
January 12, 2011
Clojure: Using Java Inner Classes
Recently, I wanted to add some specific behavior to my Clojure application when any uncaught exception occurred. This is easy enough by calling the setDefaultUncaughtExceptionHandler (sDUEH); however, it's worth noting that the sDUEH takes a Thread.UncaughtExceptionHandler argument. Clojure has no problem giving you access to inner classes, but the syntax is slightly different: Outer$Inner in Clojure is the same as Outer.Inner in Java.
It's easy to use the REPL and determine if you have things correct.user=> Thread$UncaughtExceptionHandler
java.lang.Thread$UncaughtExceptionHandlerThis works without issue, because java.lang is (generally) available in Clojure.
If you want access to an inner class that is not in java.lang you'll want to import it directly. For example, the following REPL session shows how you can access the ThreadPoolExecutor.DiscardPolicy class.user=> ThreadPoolExecutor$DiscardPolicy
java.lang.Exception: Unable to resolve symbol: ThreadPoolExecutor$DiscardPolicy in this context (NO_SOURCE_FILE:0)
user=> (ns my-ns (:import [java.util.concurrent ThreadPoolExecutor$DiscardPolicy]))
java.util.concurrent.ThreadPoolExecutor$DiscardPolicy
my-ns=> ThreadPoolExecutor$DiscardPolicy
java.util.concurrent.ThreadPoolExecutor$DiscardPolicyReturning the entire class in the REPL should suffice as proof that everything is working fine; however, the code below is a brief example of using Thread$UncaughtExceptionHandler in a larger context.jfields:Documents jfields$ cat inner.clj
(def handler (proxy [Thread$UncaughtExceptionHandler] []
(uncaughtException [thread exception]
(println thread exception))))
(Thread/setDefaultUncaughtExceptionHandler handler)
(/ 12 0)
jfields:Documents jfields$ java -cp ../dev/clojure-current/clojure.jar clojure.main -i inner.clj
#<Thread Thread[main,5,main]> #<CompilerException java.lang.ArithmeticException: Divide by zero (inner.clj:0)>As you can see from the command-line output, the exception is handled by the proxy of Thread$UncaughtExceptionHandler.
© Jay Fields - www.jayfields.com
It's easy to use the REPL and determine if you have things correct.user=> Thread$UncaughtExceptionHandler
java.lang.Thread$UncaughtExceptionHandlerThis works without issue, because java.lang is (generally) available in Clojure.
If you want access to an inner class that is not in java.lang you'll want to import it directly. For example, the following REPL session shows how you can access the ThreadPoolExecutor.DiscardPolicy class.user=> ThreadPoolExecutor$DiscardPolicy
java.lang.Exception: Unable to resolve symbol: ThreadPoolExecutor$DiscardPolicy in this context (NO_SOURCE_FILE:0)
user=> (ns my-ns (:import [java.util.concurrent ThreadPoolExecutor$DiscardPolicy]))
java.util.concurrent.ThreadPoolExecutor$DiscardPolicy
my-ns=> ThreadPoolExecutor$DiscardPolicy
java.util.concurrent.ThreadPoolExecutor$DiscardPolicyReturning the entire class in the REPL should suffice as proof that everything is working fine; however, the code below is a brief example of using Thread$UncaughtExceptionHandler in a larger context.jfields:Documents jfields$ cat inner.clj
(def handler (proxy [Thread$UncaughtExceptionHandler] []
(uncaughtException [thread exception]
(println thread exception))))
(Thread/setDefaultUncaughtExceptionHandler handler)
(/ 12 0)
jfields:Documents jfields$ java -cp ../dev/clojure-current/clojure.jar clojure.main -i inner.clj
#<Thread Thread[main,5,main]> #<CompilerException java.lang.ArithmeticException: Divide by zero (inner.clj:0)>As you can see from the command-line output, the exception is handled by the proxy of Thread$UncaughtExceptionHandler.
© Jay Fields - www.jayfields.com
January 11, 2011
Clojure: fnil
The fnil function was added to Clojure in version 1.2. The fnil function is a great addition that allows you to write code that works for all cases where an argument isn't nil, and handle the case where it is nil.
From the documentation: The fnil function takes a function f, and returns a function that calls f, replacing a nil first argument to f with a supplied value.
A simple example is working with and nil.user=> ( nil 1)
java.lang.NullPointerException (NO_SOURCE_FILE:0)
user=> (def new (fnil 0))
#'user/new
user=> (new nil 1)
1As you can see, the function throws an exception if an argument is nil; however, we were easily able to create our own new function that handles the first argument being nil.
In isolation, it might be hard to see how this is valuable. However, once combined with high order functions it's easy to see the benefit.
Several months ago I wrote about composing functions and used the update-in function as my final example of what I thought was the best implementation. What I didn't address in the blog entry was how to handle the first update.
As you can see from the following code, the first update will fail if a default value isn't populated.user=> (def current-score {})
#'user/current-score
user=> (defn update-score [current {:keys [country player score]}]
(update-in current [country player] score))
#'user/update-score
user=> (update-score current-score {:player "Paul Casey" :country :England :score -1})
java.lang.NullPointerException (NO_SOURCE_FILE:0)At the time of the writing Clojure 1.2 was not production ready, and I used a definition of update-score that was much more verbose, but did handle nil.user=> (defn update-score [current {:keys [country player score]}]
(update-in current [country player] #( (or %1 0) score)))
#'user/update-score
user=> (update-score current-score {:player "Paul Casey" :country :England :score -1})
{:England {"Paul Casey" -1}}While the above code works perfectly well, it's obviously not nearly as nice to read as the example that doesn't need to concern itself with nil.
However, Clojure 1.2 is now production ready and the fnil function is available. As a result, you can now write the following version of the update-score function that is obviously preferable to the version that uses the or function.user=> (defn update-score [current {:keys [country player score]}]
(update-in current [country player] (fnil 0) score))
#'user/update-score
user=> (update-score current-score {:player "Paul Casey" :country :England :score -1})
{:England {"Paul Casey" -1}}I'll admit that fnil wasn't my favorite function when I first found it; however, it's become indispensable. Looking through my code I find (fnil 0) and (fnil - 0) a few times, and I definitely prefer those to the versions that use the or function.
© Jay Fields - www.jayfields.com
From the documentation: The fnil function takes a function f, and returns a function that calls f, replacing a nil first argument to f with a supplied value.
A simple example is working with and nil.user=> ( nil 1)
java.lang.NullPointerException (NO_SOURCE_FILE:0)
user=> (def new (fnil 0))
#'user/new
user=> (new nil 1)
1As you can see, the function throws an exception if an argument is nil; however, we were easily able to create our own new function that handles the first argument being nil.
In isolation, it might be hard to see how this is valuable. However, once combined with high order functions it's easy to see the benefit.
Several months ago I wrote about composing functions and used the update-in function as my final example of what I thought was the best implementation. What I didn't address in the blog entry was how to handle the first update.
As you can see from the following code, the first update will fail if a default value isn't populated.user=> (def current-score {})
#'user/current-score
user=> (defn update-score [current {:keys [country player score]}]
(update-in current [country player] score))
#'user/update-score
user=> (update-score current-score {:player "Paul Casey" :country :England :score -1})
java.lang.NullPointerException (NO_SOURCE_FILE:0)At the time of the writing Clojure 1.2 was not production ready, and I used a definition of update-score that was much more verbose, but did handle nil.user=> (defn update-score [current {:keys [country player score]}]
(update-in current [country player] #( (or %1 0) score)))
#'user/update-score
user=> (update-score current-score {:player "Paul Casey" :country :England :score -1})
{:England {"Paul Casey" -1}}While the above code works perfectly well, it's obviously not nearly as nice to read as the example that doesn't need to concern itself with nil.
However, Clojure 1.2 is now production ready and the fnil function is available. As a result, you can now write the following version of the update-score function that is obviously preferable to the version that uses the or function.user=> (defn update-score [current {:keys [country player score]}]
(update-in current [country player] (fnil 0) score))
#'user/update-score
user=> (update-score current-score {:player "Paul Casey" :country :England :score -1})
{:England {"Paul Casey" -1}}I'll admit that fnil wasn't my favorite function when I first found it; however, it's become indispensable. Looking through my code I find (fnil 0) and (fnil - 0) a few times, and I definitely prefer those to the versions that use the or function.
© Jay Fields - www.jayfields.com
January 6, 2011
Clojure: partial and comp
Clojure provides a few different options for creating functions inline: fn (or the #() reader macro), partial, and comp. When I first got started with Clojure I found I could do everything with fn and #(); and that's a good place to start. However, as I produced more Clojure code I found there were also opportunities to use both partial and comp to create more concise code.
The following examples are contrived. They will show how partial and comp can be used; however, they aren't great examples of when they should be used. As always, context is important, and you'll need to decide when (or if) you want to use either function.
The partial function takes a function and fewer than the normal arguments to the function, and returns a function that takes a variable number of additional args. When called, the returned function calls the function with the specified args and any additional args.
The partial function can often be used as an alternative to fn or #(). The following example shows how you can use either #() or partial to multiply a list of integers by .01 (convert pennies to dollars).user=> (map #(* 0.01 %1) [5000 100 50])
(50.0 1.0 0.5)
user=> (map (partial * 0.01) [5000 100 50])
(50.0 1.0 0.5)In a straightforward example, such as the one above, it's really up to you which you'd prefer. However, if you want to specify a predicate that takes a variable number of args then the case for partial starts to become a bit more noticeable.user=> (map #(apply str "price & tip: " %&) [5000 100 50] (repeat " ") [2000 40 10])
("price & tip: 5000 2000" "price & tip: 100 40" "price & tip: 50 10")
user=> (map (partial str "price & tip: ") [5000 100 50] (repeat " ") [2000 40 10])
("price & tip: 5000 2000" "price & tip: 100 40" "price & tip: 50 10")I haven't come across an example yet that made me think: The partial function is definitely the right choice here! However, I have passed around a few functions that had several arguments and found I prefer (partial f arg1) to #(f arg1 %1 %2 %3) and #(apply f arg1 %&).
The comp function takes a variable number of functions and returns a function that is the composition of those functions. The returned function takes a variable number of args, applies the rightmost of functions to the args, the next function (right-to-left) to the result, etc.
In a previous blog entry I used comp to return the values from a map given a list of keys. Below you can find the same example that shows the definition and usage.user=> (def select-values (comp vals select-keys))
#'user/select-values
user=> (select-values {:a 1 :b 2} [:a])
(1)As you can see from the example, comp creates a function that takes a sequence of keys, calls select-keys with that sequence, then calls vals with the result of calling select-keys. As the documentation specifies, the functions are called from right to left.
In general I find myself executing some functions directly with some data. In that case I generally use the -> macro. For example, if I already have a map and I want a list of the keys I'm probably going to write code similar to what's found below.user=> (-> {:a 1 :b 2} (select-keys [:a]) vals)
(1)However, there are times when you need a function that is the composition of a few other functions. For example, taking a list of numbers, converting them to strings, and then converting them to keywords.user=> (map (comp keyword str) [1 2])
(:1 :2)The same thing can be done with #(), as the example below shows.user=> (map #(keyword (str %1)) [1 2])
(:1 :2)While the code above works perfectly well, I definitely prefer the version that uses the comp function.
Like so many other functions in Clojure, you can get by without partial and comp. However, I find my code more readable and maintainable when I use tools specifically designed to handle my current problem.
© Jay Fields - www.jayfields.com
The following examples are contrived. They will show how partial and comp can be used; however, they aren't great examples of when they should be used. As always, context is important, and you'll need to decide when (or if) you want to use either function.
The partial function takes a function and fewer than the normal arguments to the function, and returns a function that takes a variable number of additional args. When called, the returned function calls the function with the specified args and any additional args.
The partial function can often be used as an alternative to fn or #(). The following example shows how you can use either #() or partial to multiply a list of integers by .01 (convert pennies to dollars).user=> (map #(* 0.01 %1) [5000 100 50])
(50.0 1.0 0.5)
user=> (map (partial * 0.01) [5000 100 50])
(50.0 1.0 0.5)In a straightforward example, such as the one above, it's really up to you which you'd prefer. However, if you want to specify a predicate that takes a variable number of args then the case for partial starts to become a bit more noticeable.user=> (map #(apply str "price & tip: " %&) [5000 100 50] (repeat " ") [2000 40 10])
("price & tip: 5000 2000" "price & tip: 100 40" "price & tip: 50 10")
user=> (map (partial str "price & tip: ") [5000 100 50] (repeat " ") [2000 40 10])
("price & tip: 5000 2000" "price & tip: 100 40" "price & tip: 50 10")I haven't come across an example yet that made me think: The partial function is definitely the right choice here! However, I have passed around a few functions that had several arguments and found I prefer (partial f arg1) to #(f arg1 %1 %2 %3) and #(apply f arg1 %&).
The comp function takes a variable number of functions and returns a function that is the composition of those functions. The returned function takes a variable number of args, applies the rightmost of functions to the args, the next function (right-to-left) to the result, etc.
In a previous blog entry I used comp to return the values from a map given a list of keys. Below you can find the same example that shows the definition and usage.user=> (def select-values (comp vals select-keys))
#'user/select-values
user=> (select-values {:a 1 :b 2} [:a])
(1)As you can see from the example, comp creates a function that takes a sequence of keys, calls select-keys with that sequence, then calls vals with the result of calling select-keys. As the documentation specifies, the functions are called from right to left.
In general I find myself executing some functions directly with some data. In that case I generally use the -> macro. For example, if I already have a map and I want a list of the keys I'm probably going to write code similar to what's found below.user=> (-> {:a 1 :b 2} (select-keys [:a]) vals)
(1)However, there are times when you need a function that is the composition of a few other functions. For example, taking a list of numbers, converting them to strings, and then converting them to keywords.user=> (map (comp keyword str) [1 2])
(:1 :2)The same thing can be done with #(), as the example below shows.user=> (map #(keyword (str %1)) [1 2])
(:1 :2)While the code above works perfectly well, I definitely prefer the version that uses the comp function.
Like so many other functions in Clojure, you can get by without partial and comp. However, I find my code more readable and maintainable when I use tools specifically designed to handle my current problem.
© Jay Fields - www.jayfields.com
January 5, 2011
Clojure: select-keys, select-values, and apply-values
Clojure provides the get and get-in functions for returning values from a map and the select-keys function for returning a new map of only the specified keys. Clojure doesn't provide a function that returns a list of values; however, it's very easy to create such a function (which I call select-values). Once you have the ability to select-values it becomes very easy to create a function that applies a function to the selected values (which I call apply-values).
The select-keys function returns a map containing only the entries of the specified keys. The following (pasted) REPL session shows a few different select-keys behaviors.user=> (select-keys {:a 1 :b 2} [:a])
{:a 1}
user=> (select-keys {:a 1 :b 2} [:a :b])
{:b 2, :a 1}
user=> (select-keys {:a 1 :b 2} [:a :b :c])
{:b 2, :a 1}
user=> (select-keys {:a 1 :b 2} [])
{}
user=> (select-keys {:a 1 :b 2} nil)
{}The select-keys function is helpful in many occassions; however, sometimes you only care about selecting the values of certain keys in a map. A simple solution is to call select-keys and then vals. Below you can find the results of applying this idea.user=> (def select-values (comp vals select-keys))
#'user/select-values
user=> (select-values {:a 1 :b 2} [:a])
(1)
user=> (select-values {:a 1 :b 2} [:a :b])
(2 1)
user=> (select-values {:a 1 :b 2} [:a :b :c])
(2 1)
user=> (select-values {:a 1 :b 2} [])
nil
user=> (select-values {:a 1 :b 2} nil)
nilThe select-values implementation from above may be sufficient for what you are doing, but there are two things worth noticing: in cases where you might be expecting an empty list you are seeing nil; and, the values are not in the same order that the keys were specified in. Given that (standard) maps are unsorted, you can't be sure of the ordering the values.
(side-note: If you are concerned with microseconds, it's also been reported that select-keys is a bit slow/garbage heavy.)
An alternative definition of select-values uses the reduce function and pulls the values by key and incrementally builds the (vector) result.user=> (defn select-values [map ks]
(reduce #(conj %1 (map %2)) [] ks))
#'user/select-values
user=> (select-values {:a 1 :b 2} [:a])
[1]
user=> (select-values {:a 1 :b 2} [:a :b])
[1 2]
user=> (select-values {:a 1 :b 2} [:a :b :c])
[1 2 nil]
user=> (select-values {:a 1 :b 2} [])
[]
user=> (select-values {:a 1 :b 2} nil)
[]The new select-values function returns the values in order and returns an empty vector in the cases where previous examples returned nil, but we have a new problem: Keys specified that don't exist in the map are now included in the vector as nil. This issue is easily addressed by adding a call to the remove function.
The implementation that includes removing nils can be found below.user=> (defn select-values [map ks]
(remove nil? (reduce #(conj %1 (map %2)) [] ks)))
#'user/select-values
user=> (select-values {:a 1 :b 2} [:a])
(1)
user=> (select-values {:a 1 :b 2} [:a :b])
(1 2)
user=> (select-values {:a 1 :b 2} [:a :b :c])
(1 2)
user=> (select-values {:a 1 :b 2} [])
()
user=> (select-values {:a 1 :b 2} nil)
()There is no "correct" implementation for select-values. If you don't care about ordering and nil is a reasonable return value: the first implementation is the correct choice due to it's concise definition. If you do care about ordering and performance: the second implementation might be the right choice. If you want something that follows the principle of least surprise: the third implementation is probably the right choice. You'll have to decide what's best for your context. In fact, here's a few more implementations that might be better based on your context.user=> (defn select-values [m ks]
(map #({:a 1 :b 2} %) ks))
#'user/select-values
user=> (select-values {:a 1 :b 2} [:a])
(1)
user=> (select-values {:a 1 :b 2} [:a :b :c])
(1 2 nil)
user=> (defn select-values [m ks]
(reduce #(if-let [v ({:a 1 :b 2} %2)] (conj %1 v) %1) [] ks))
#'user/select-values
user=> (select-values {:a 1 :b 2} [:a])
[1]
user=> (select-values {:a 1 :b 2} [:a :b :c])
[1 2]Pulling values from a map is helpful, but it's generally not the end goal. If you find yourself pulling values from a map, it's likely that you're going to want to apply a function to the extracted values. With that in mind, I generally define an apply-values function that returns the result of applying a function to the values returned from specified keys.
A good example of this is returning the total for a line item represented as a map. Given a map that specifies a line item costing $5 and having a quantity of 4, you can use (* price quantity) to determine the total price for the line item.
Using our previously defined select-values function we can do the work ourselves, as the example below shows.user=> (let [[price quantity] (select-values {:price 5 :quantity 4 :upc 1123} [:price :quantity])]
(* price quantity))
20The example above works perfectly well; however, applying a function to the values of a map seems like a fairly generic operation that can easily be extracted to it's own function (the apply-values function). The example below shows the definition and usage of my definition of apply-values.user=> (defn apply-values [map f & ks]
(apply f (select-values map ks)))
#'user/apply-values
user=> (apply-values {:price 5 :quantity 4 :upc 1123} * :price :quantity)
20I find select-keys, select-values, & apply-values to be helpful when writing Clojure applications. If you find you need these functions, feel free to use them in your own code. However, you'll probably want to check the comments - I'm sure someone with more Clojure experience than I have will provide superior implementations.
© Jay Fields - www.jayfields.com
The select-keys function returns a map containing only the entries of the specified keys. The following (pasted) REPL session shows a few different select-keys behaviors.user=> (select-keys {:a 1 :b 2} [:a])
{:a 1}
user=> (select-keys {:a 1 :b 2} [:a :b])
{:b 2, :a 1}
user=> (select-keys {:a 1 :b 2} [:a :b :c])
{:b 2, :a 1}
user=> (select-keys {:a 1 :b 2} [])
{}
user=> (select-keys {:a 1 :b 2} nil)
{}The select-keys function is helpful in many occassions; however, sometimes you only care about selecting the values of certain keys in a map. A simple solution is to call select-keys and then vals. Below you can find the results of applying this idea.user=> (def select-values (comp vals select-keys))
#'user/select-values
user=> (select-values {:a 1 :b 2} [:a])
(1)
user=> (select-values {:a 1 :b 2} [:a :b])
(2 1)
user=> (select-values {:a 1 :b 2} [:a :b :c])
(2 1)
user=> (select-values {:a 1 :b 2} [])
nil
user=> (select-values {:a 1 :b 2} nil)
nilThe select-values implementation from above may be sufficient for what you are doing, but there are two things worth noticing: in cases where you might be expecting an empty list you are seeing nil; and, the values are not in the same order that the keys were specified in. Given that (standard) maps are unsorted, you can't be sure of the ordering the values.
(side-note: If you are concerned with microseconds, it's also been reported that select-keys is a bit slow/garbage heavy.)
An alternative definition of select-values uses the reduce function and pulls the values by key and incrementally builds the (vector) result.user=> (defn select-values [map ks]
(reduce #(conj %1 (map %2)) [] ks))
#'user/select-values
user=> (select-values {:a 1 :b 2} [:a])
[1]
user=> (select-values {:a 1 :b 2} [:a :b])
[1 2]
user=> (select-values {:a 1 :b 2} [:a :b :c])
[1 2 nil]
user=> (select-values {:a 1 :b 2} [])
[]
user=> (select-values {:a 1 :b 2} nil)
[]The new select-values function returns the values in order and returns an empty vector in the cases where previous examples returned nil, but we have a new problem: Keys specified that don't exist in the map are now included in the vector as nil. This issue is easily addressed by adding a call to the remove function.
The implementation that includes removing nils can be found below.user=> (defn select-values [map ks]
(remove nil? (reduce #(conj %1 (map %2)) [] ks)))
#'user/select-values
user=> (select-values {:a 1 :b 2} [:a])
(1)
user=> (select-values {:a 1 :b 2} [:a :b])
(1 2)
user=> (select-values {:a 1 :b 2} [:a :b :c])
(1 2)
user=> (select-values {:a 1 :b 2} [])
()
user=> (select-values {:a 1 :b 2} nil)
()There is no "correct" implementation for select-values. If you don't care about ordering and nil is a reasonable return value: the first implementation is the correct choice due to it's concise definition. If you do care about ordering and performance: the second implementation might be the right choice. If you want something that follows the principle of least surprise: the third implementation is probably the right choice. You'll have to decide what's best for your context. In fact, here's a few more implementations that might be better based on your context.user=> (defn select-values [m ks]
(map #({:a 1 :b 2} %) ks))
#'user/select-values
user=> (select-values {:a 1 :b 2} [:a])
(1)
user=> (select-values {:a 1 :b 2} [:a :b :c])
(1 2 nil)
user=> (defn select-values [m ks]
(reduce #(if-let [v ({:a 1 :b 2} %2)] (conj %1 v) %1) [] ks))
#'user/select-values
user=> (select-values {:a 1 :b 2} [:a])
[1]
user=> (select-values {:a 1 :b 2} [:a :b :c])
[1 2]Pulling values from a map is helpful, but it's generally not the end goal. If you find yourself pulling values from a map, it's likely that you're going to want to apply a function to the extracted values. With that in mind, I generally define an apply-values function that returns the result of applying a function to the values returned from specified keys.
A good example of this is returning the total for a line item represented as a map. Given a map that specifies a line item costing $5 and having a quantity of 4, you can use (* price quantity) to determine the total price for the line item.
Using our previously defined select-values function we can do the work ourselves, as the example below shows.user=> (let [[price quantity] (select-values {:price 5 :quantity 4 :upc 1123} [:price :quantity])]
(* price quantity))
20The example above works perfectly well; however, applying a function to the values of a map seems like a fairly generic operation that can easily be extracted to it's own function (the apply-values function). The example below shows the definition and usage of my definition of apply-values.user=> (defn apply-values [map f & ks]
(apply f (select-values map ks)))
#'user/apply-values
user=> (apply-values {:price 5 :quantity 4 :upc 1123} * :price :quantity)
20I find select-keys, select-values, & apply-values to be helpful when writing Clojure applications. If you find you need these functions, feel free to use them in your own code. However, you'll probably want to check the comments - I'm sure someone with more Clojure experience than I have will provide superior implementations.
© Jay Fields - www.jayfields.com
December 6, 2010
Clojure: get, get-in, contains?, and some
Clojure provides a get function that returns the value mapped to a key in a set or map. The documentation shows the example: (get map key). While that's completely valid, I tend to use sets and maps as functions when the get is that simple.
For example, I'd use ({"FSU" 31 "UF" 7} "FSU") if I wanted the value of the key "FSU". It's much less likely that I'd use (get {"FSU" 31 "UF" 7} "FSU"), largely because the former example is less typing.
However, if I'm doing something more complicated I've found the get function to be helpful. Often, I like to use the combination of get and -> or ->>.
The following example takes some json-data, converts it to a clojure map, and pulls the value from the "FSU" key.
(-> json-data read-json (get "FSU"))
It's also worth noting, in our example we have to use get, since strings are not Clojure functions. If instead we chose to make our keys keywords, we could choose either of the following solutions. I don't believe there is a right or wrong solution; which you use will likely be a personal preference.
(-> json-data (read-json true) (get :FSU))
(-> json-data (read-json true) :FSU)
We can modify the example and assume nested json that results in the following clojure map: {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}}
Building off of a previous example, we could use a slightly modified version to get the score for FSU.
(-> json-data read-json (get "scores") (get "FSU"))
However, getting nested values is common enough that Clojure provides a function designed specifically to address that need: get-in
The get-in function returns the value in a nested associative structure when given a sequence of keys. Using get-in you can replace the last example with the following code.
(-> json-data read-json (get-in ["scores" "FSU"]))
The get-in function is very helpful when dealing with nested structures; however, there is one gotcha that I've run into. The following shows a REPL session and what get-in returns with various keys.user=> (get-in {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}} ["scores" "FSU"])
31
user=> (get-in {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}} ["scores"])
{"FSU" 31, "UF" 7}
user=> (get-in {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}} [])
{"timestamp" 1291578985220, "scores" {"FSU" 31, "UF" 7}}
user=> (get-in {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}} nil)
{"timestamp" 1291578985220, "scores" {"FSU" 31, "UF" 7}}Everything looks logical enough; however, if you are pulling your key sequence from somewhere else you could end up with unexpected results. The following example shows how a simple mistake could result in a bug.user=> (def score-key-seqs {"FSU" ["scores" "FSU"]})
#'user/score-key-seqs
user=> (get-in {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}} (score-key-seqs "FSU"))
31
user=> (get-in {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}} (score-key-seqs "UF"))
{"timestamp" 1291578985220, "scores" {"FSU" 31, "UF" 7}}If you're always expecting a number and you get a map instead, things might not work out well.
It's also worth noting that both get and get-in allow you to specify default values. You can check the documentation on clojure.org for more information on default values.
You don't always need to get a value, sometimes it's good enough to know that a key is in a map or set. In general I use the value returned from a map or set to determine if a key exists - the following snippet uses that pattern.(if (a-map :key)
(do-true-behaviors)
(do-false-behaviors))However, that pattern fails if the value of :key is nil. If it's possible that the value might be nil you might want to use Clojure's contains? function. The contains? function returns true if key is present in the given collection, otherwise returns false. The following code pasted from a REPL session demonstrates that contains? works perfectly well with nil.user=> (contains? {:foo nil} :foo)
trueThe contains? function works well with sets and maps; however, if you try to use it on a list you might get surprising results.user=> (contains? [1 3 4] 2)
trueFor numerically indexed collections like vectors and Java arrays, the contains? function tests if the numeric key is within the range of indexes. The Clojure documentation recommends looking at the some function if you're looking for an item in a list.
The some function returns the first logical true value of a predicate for any item in the list, else nil. The following REPL session shows how you can use a set as the predicate with some to determine if a value is found in a list.user=> (some #{2} [1 3 4])
nil
user=> (some #{1} [1 3 4])
1Clojure provides various functions for operating on maps and sets. At first glance some of them may look superfluous; however, as you spend more time working with sets and maps you'll start to appreciate the subtle differences and the value they provide.
© Jay Fields - www.jayfields.com
For example, I'd use ({"FSU" 31 "UF" 7} "FSU") if I wanted the value of the key "FSU". It's much less likely that I'd use (get {"FSU" 31 "UF" 7} "FSU"), largely because the former example is less typing.
However, if I'm doing something more complicated I've found the get function to be helpful. Often, I like to use the combination of get and -> or ->>.
The following example takes some json-data, converts it to a clojure map, and pulls the value from the "FSU" key.
(-> json-data read-json (get "FSU"))
It's also worth noting, in our example we have to use get, since strings are not Clojure functions. If instead we chose to make our keys keywords, we could choose either of the following solutions. I don't believe there is a right or wrong solution; which you use will likely be a personal preference.
(-> json-data (read-json true) (get :FSU))
(-> json-data (read-json true) :FSU)
We can modify the example and assume nested json that results in the following clojure map: {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}}
Building off of a previous example, we could use a slightly modified version to get the score for FSU.
(-> json-data read-json (get "scores") (get "FSU"))
However, getting nested values is common enough that Clojure provides a function designed specifically to address that need: get-in
The get-in function returns the value in a nested associative structure when given a sequence of keys. Using get-in you can replace the last example with the following code.
(-> json-data read-json (get-in ["scores" "FSU"]))
The get-in function is very helpful when dealing with nested structures; however, there is one gotcha that I've run into. The following shows a REPL session and what get-in returns with various keys.user=> (get-in {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}} ["scores" "FSU"])
31
user=> (get-in {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}} ["scores"])
{"FSU" 31, "UF" 7}
user=> (get-in {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}} [])
{"timestamp" 1291578985220, "scores" {"FSU" 31, "UF" 7}}
user=> (get-in {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}} nil)
{"timestamp" 1291578985220, "scores" {"FSU" 31, "UF" 7}}Everything looks logical enough; however, if you are pulling your key sequence from somewhere else you could end up with unexpected results. The following example shows how a simple mistake could result in a bug.user=> (def score-key-seqs {"FSU" ["scores" "FSU"]})
#'user/score-key-seqs
user=> (get-in {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}} (score-key-seqs "FSU"))
31
user=> (get-in {"timestamp" 1291578985220 "scores" {"FSU" 31 "UF" 7}} (score-key-seqs "UF"))
{"timestamp" 1291578985220, "scores" {"FSU" 31, "UF" 7}}If you're always expecting a number and you get a map instead, things might not work out well.
It's also worth noting that both get and get-in allow you to specify default values. You can check the documentation on clojure.org for more information on default values.
You don't always need to get a value, sometimes it's good enough to know that a key is in a map or set. In general I use the value returned from a map or set to determine if a key exists - the following snippet uses that pattern.(if (a-map :key)
(do-true-behaviors)
(do-false-behaviors))However, that pattern fails if the value of :key is nil. If it's possible that the value might be nil you might want to use Clojure's contains? function. The contains? function returns true if key is present in the given collection, otherwise returns false. The following code pasted from a REPL session demonstrates that contains? works perfectly well with nil.user=> (contains? {:foo nil} :foo)
trueThe contains? function works well with sets and maps; however, if you try to use it on a list you might get surprising results.user=> (contains? [1 3 4] 2)
trueFor numerically indexed collections like vectors and Java arrays, the contains? function tests if the numeric key is within the range of indexes. The Clojure documentation recommends looking at the some function if you're looking for an item in a list.
The some function returns the first logical true value of a predicate for any item in the list, else nil. The following REPL session shows how you can use a set as the predicate with some to determine if a value is found in a list.user=> (some #{2} [1 3 4])
nil
user=> (some #{1} [1 3 4])
1Clojure provides various functions for operating on maps and sets. At first glance some of them may look superfluous; however, as you spend more time working with sets and maps you'll start to appreciate the subtle differences and the value they provide.
© Jay Fields - www.jayfields.com
November 30, 2010
Taking a Second Look at Collective Code Ownership
It's common to hear proponents of Agile discussing the benefits of collective code ownership. The benefits can be undeniable. Sharing knowledge ensures at least one other perspective and drastically reduces Bus Risk. However, sharing comes at a cost: time. The ROI of sharing with a few people can be greatly different than the ROI of sharing with 5 or more people.
I do believe in the benefits of collective code ownership. Collective code ownership is a step towards taking an underachieving team and turning them into a good team. However, I'm becoming more and more convinced that it's not the way to take good team and make them great.
(context: I believe these ideas apply to teams larger than 3. Teams of 2-3 should likely be working with everyone on everything.)
If you pair program, you will incur a context switch every time you rotate pairs. Context switches always have a non-zero cost, and the more you work on, the larger the context switch is likely to be. Teams where everyone works on everything are very likely paying for expensive context switches on a regular basis. In fact, the words Context Switch are often used specifically to point out the cost.
Working on everything also ensures a non-trivial amount of time ramping up on whatever code you are about to start working on. Sometimes you work on something you know fairly well. Other times you're working on something you've never seen before. Working on something you've never seen before creates two choices (assuming pair-programming): go along for the ride, understanding little - or - slow your pair down significantly while they explain what's going on.
Let's say you choose to slow your pair down for the full explanation: was it worth it? If you're the only other person that knows the component, it's very likely that it was worth it. What if everyone else on the team already knows that component deeply? Well, if they all die, you can maintain the app, but I don't think that's going to be the largest issue on the team.
(the same ideas apply if you don't pair-program, except you don't have the "go along for the ride" option)
I can hear some of you right now: We rotate enough that the context switch is virtually free and because we rotate so much there's little ramp up time. You might be right. Your problem domain might be so simple that jumping on and off of parts of your system is virtually free. However, if you're domain is complex in anyway, I think you're underestimating the cost of context switches and ramp up time. Also, the devil is traditionally in the details, so you're "simple domain" probably isn't as simple as you think.
Let's assume your domain is that simple: it might be cheaper to rewrite the software than take your Bus Number from 3 to 4.
Another benefit of pair programming combined with collective code ownership is bringing up everyone's skill level to that of the most skilled team member. In my opinion, that's something you need to worry about on an underachieving team, not a good team. If you're on a good team, it's likely that you can learn just as much from any member of your team; therefore, you are not losing anything by sticking to working with just a few of them in specific areas. You really only run into an education problem if you're team has more learners than mentors - and, in that case, you're not ready to worry about going from good to great.
There's also opportunity cost of not sticking to certain areas. Focusing on a problem allows you to create better solutions. Specifically, it allows you to create a vision of what needs to be done, work towards that vision and constantly revise where necessary.
Mark Twain once wrote that his letters would be shorter if he had more time. The same is often true of software. The simplest solution is not always the most obvious. If you're jumping from problem to problem, you're more likely to create an inferior solution. You'll solve problems, but you'll be creating higher maintenance costs for the project in the long term.
Instead, I often find it very helpful to ponder a problem and create a more simple and, very often, a more concise solution. In my experience, the maintenance costs are also greatly reduced by the simplified, condensed solution.
I'd like to repeat for clarity: Collective code ownership has benefits. There's no doubt that it is better to have everyone work on everything than have everyone focused on individual parts of the codebase. However, it's worth considering the cost of complete sharing if you are trying to move from good to great.
© Jay Fields - www.jayfields.com
I do believe in the benefits of collective code ownership. Collective code ownership is a step towards taking an underachieving team and turning them into a good team. However, I'm becoming more and more convinced that it's not the way to take good team and make them great.
(context: I believe these ideas apply to teams larger than 3. Teams of 2-3 should likely be working with everyone on everything.)
If you pair program, you will incur a context switch every time you rotate pairs. Context switches always have a non-zero cost, and the more you work on, the larger the context switch is likely to be. Teams where everyone works on everything are very likely paying for expensive context switches on a regular basis. In fact, the words Context Switch are often used specifically to point out the cost.
Working on everything also ensures a non-trivial amount of time ramping up on whatever code you are about to start working on. Sometimes you work on something you know fairly well. Other times you're working on something you've never seen before. Working on something you've never seen before creates two choices (assuming pair-programming): go along for the ride, understanding little - or - slow your pair down significantly while they explain what's going on.
Let's say you choose to slow your pair down for the full explanation: was it worth it? If you're the only other person that knows the component, it's very likely that it was worth it. What if everyone else on the team already knows that component deeply? Well, if they all die, you can maintain the app, but I don't think that's going to be the largest issue on the team.
(the same ideas apply if you don't pair-program, except you don't have the "go along for the ride" option)
I can hear some of you right now: We rotate enough that the context switch is virtually free and because we rotate so much there's little ramp up time. You might be right. Your problem domain might be so simple that jumping on and off of parts of your system is virtually free. However, if you're domain is complex in anyway, I think you're underestimating the cost of context switches and ramp up time. Also, the devil is traditionally in the details, so you're "simple domain" probably isn't as simple as you think.
Let's assume your domain is that simple: it might be cheaper to rewrite the software than take your Bus Number from 3 to 4.
Another benefit of pair programming combined with collective code ownership is bringing up everyone's skill level to that of the most skilled team member. In my opinion, that's something you need to worry about on an underachieving team, not a good team. If you're on a good team, it's likely that you can learn just as much from any member of your team; therefore, you are not losing anything by sticking to working with just a few of them in specific areas. You really only run into an education problem if you're team has more learners than mentors - and, in that case, you're not ready to worry about going from good to great.
There's also opportunity cost of not sticking to certain areas. Focusing on a problem allows you to create better solutions. Specifically, it allows you to create a vision of what needs to be done, work towards that vision and constantly revise where necessary.
Mark Twain once wrote that his letters would be shorter if he had more time. The same is often true of software. The simplest solution is not always the most obvious. If you're jumping from problem to problem, you're more likely to create an inferior solution. You'll solve problems, but you'll be creating higher maintenance costs for the project in the long term.
Instead, I often find it very helpful to ponder a problem and create a more simple and, very often, a more concise solution. In my experience, the maintenance costs are also greatly reduced by the simplified, condensed solution.
I'd like to repeat for clarity: Collective code ownership has benefits. There's no doubt that it is better to have everyone work on everything than have everyone focused on individual parts of the codebase. However, it's worth considering the cost of complete sharing if you are trying to move from good to great.
© Jay Fields - www.jayfields.com
October 30, 2010
Experience Report: Feature Toggle over Feature Branch
We often use Feature Toggle on my current team (when gradual release isn't possible). My experience so far has been: gradual release is better than Feature Toggle, and Feature Toggle is better than Feature Branch.
I found Martin's bliki entry on Feature Toggle to be a great description, but the entry doesn't touch on the primary reasons why I prefer Feature Toggle to Feature Branch.
When using Feature Branch I have to constantly rebase to avoid massive merge issues, in general. That means rebasing during development and testing. Rebasing a branch while it's complete and being tested, has often lead to subtle, merge related bugs that go unnoticed because the feature is not under active development.
Additionally, (and likely more problematic) once I merge a feature branch I am committed (no pun intended). If a bug in the new feature, that requires rolling production back to the previous release, is found after the branch has been merged I find myself rolling back the feature commit or continuing with a un-releasable trunk. Rolling back the commit is painful because it is likely a large commit. If I continue on with an un-releasable trunk and another (unrelated to the new feature) bug is found in trunk I'm in trouble: I can't fix the new bug and release.
That's bad. I either lose significant time rolling back the release (terrible), or I roll the dice (terrifying). I was burned by this situation once already this year. It is not something I'm looking to suffer again in the near future.
Feature Toggle avoids this entirely by allowing me to run with the toggle available and turn it back on if things go wrong. When I feel comfortable that everything is okay (usually, a week in prod is good enough), I clean up the unnecessary toggles.
Of course, nothing is black & white. Sometimes a Feature Toggle increases the scope of the work to an unacceptable level, and Feature Branch is the correct decision. However, I always weigh the terrible/terrifying situation when I'm choosing which direction is optimal.
It's also worth noting, I roll the current day's changes into production every night. It's possible that your experience will be vastly different if your release schedules are multi-day or multi-week.
© Jay Fields - www.jayfields.com
I found Martin's bliki entry on Feature Toggle to be a great description, but the entry doesn't touch on the primary reasons why I prefer Feature Toggle to Feature Branch.
When using Feature Branch I have to constantly rebase to avoid massive merge issues, in general. That means rebasing during development and testing. Rebasing a branch while it's complete and being tested, has often lead to subtle, merge related bugs that go unnoticed because the feature is not under active development.
Additionally, (and likely more problematic) once I merge a feature branch I am committed (no pun intended). If a bug in the new feature, that requires rolling production back to the previous release, is found after the branch has been merged I find myself rolling back the feature commit or continuing with a un-releasable trunk. Rolling back the commit is painful because it is likely a large commit. If I continue on with an un-releasable trunk and another (unrelated to the new feature) bug is found in trunk I'm in trouble: I can't fix the new bug and release.
That's bad. I either lose significant time rolling back the release (terrible), or I roll the dice (terrifying). I was burned by this situation once already this year. It is not something I'm looking to suffer again in the near future.
Feature Toggle avoids this entirely by allowing me to run with the toggle available and turn it back on if things go wrong. When I feel comfortable that everything is okay (usually, a week in prod is good enough), I clean up the unnecessary toggles.
Of course, nothing is black & white. Sometimes a Feature Toggle increases the scope of the work to an unacceptable level, and Feature Branch is the correct decision. However, I always weigh the terrible/terrifying situation when I'm choosing which direction is optimal.
It's also worth noting, I roll the current day's changes into production every night. It's possible that your experience will be vastly different if your release schedules are multi-day or multi-week.
© Jay Fields - www.jayfields.com
