Weird Science Day 20: Crowd Sourcing with DARPA–The Shredder Challenge

The contest started on October 27 and ended on December 2, much to the surprise of Dan Kaufman, director of DARPA's Information Innovation Office. "Lots of experts were skeptical that a solution could be produced at all let alone within the short time frame," he said.

According to the announcement on December 2, over 9000 contestants participated. Here's  how the challenge went:

"Today's troops often confiscate the remnants of destroyed documents in war zones, but reconstructing them is a daunting task. DARPA's Shredder Challenge called upon computer scientists, puzzle enthusiasts and anyone else who likes solving complex problems to compete for up to $50,000 by piecing together a series of shredded documents."

Five separate puzzles comprised the challenge. Each one different in subject matter and shredding method. There was a range of difficulty: number 1 was relatively easy, number five tough as nails.

Below is an example of before and after.

Before

After

Q: So what was the prize? A: $50,000.

Q: Who won? A: A small team that called themselves "All Your Shreds Are Belong to U.S."

Q: How did All Your Shreds solve the puzzles? A: They used "custom-coded, computer-vision algorithms to suggest fragment pairings to human assemblers for verification."

Oh. They used an algorithm. Okay, I see. Hopefully the troops will remember to pack the algorithm with them when they go on their search-and-repaste missions.

Q: Who is All Your Shreds are Belong to U.S.? A: Three guys (Otavio Good, Luke Alonso, and Keith Walker) along with a few friends.

Q: What's up with the name? A:  Figure it out with this interesting item at Wikipedia.

I admit the title of the group had me baffled until I read the article. I get the joke now. I'm in on it. This one solidly are belong to me.

The DARPA challenge was not without drama. Apparently the UC-San Diego team was the fave to win. Its strategy entailed grouping similar pieces together using a cluster analysis algorithm (again with the algorithms) and then crowdsourcing the groups. Over 3000 people joined with them and for much of the puzzle's run, UCSD was in a good spot. But then they had trouble with the crowd. Some people are born spoilers and the UCSD puzzle effort came under attack when some meanies moved a few pieces around and locked them into incorrect positions.

UCSD leaders rushed in to contain the damage but their efforts were not enough. All Your Shreds swept passed and picked up a cool fifty thou. According to New Scientist, the leader of the UCSD team had received an email from "ucsdsabateur"  who admitted to recruiting spoilers to disrupt UCSD's efforts. The reason: "crowd sourcing is basically cheating." In my opinion, unless it was specifically prohibited in the challenge, it wasn't. I do agree though, the puzzle seems like it would be more fun if you have to do it alone instead of using 6000 friends. What a sense of accomplishment.

I feel the same about the algorithm. What's wrong with good ol' human pattern recognition skills? Sure, humans are too slow for DARPA's needs. They want a solution yesterday. Still I can see  Dell jumping all over this new thing. Could it be the new Sudoku?

Thanks for reading!

Sue Lange's latest ebook, Tritcheon Hash, is full of lapses of logic and weird science. "It's a wild, good read." Get your copy from Amazon or read a couple of free chapters at the publisher's website.