Sam Newman's Blog, page 2

Magpie Talkshow Episode 17 - Corey Latislaw

After far too long a hiatus, the Magpie Talkshow is back with it's 17th episode. This week we wrap up the last of the interviews I recorded at ��redev last year, and hear my chat with Android developer Corey Latislaw. Mobile development in general is somewhat outside my wheelhouse, so I was very keen to hear about the challenges in that space. At ��redev, Corey did a talk about Android as the world phone, and during our interview we touch on a number of aspects of developing solutions for the emerging world, and how as a mobile developer you can start targetting the next 5 billion users.

We also find time to talk about the merits of minesweeper for identifying algorithmic aptitude, Corey's work helping bring small-scale solar to off-grid communities, and why Android fragmentation might not be as bad as you might have previously heard.

You can find more about Corey on her blog, follow her on Twitter, or find out more about her company Off Grid Electric. Finally, you can find Corey's talk from ��redev below, which is a great way of finding out not only about Android One, but also some of the challenges developers face when developing for emerging markets.

ANDROID IS THE WORLD PHONE from Øredev Conference on Vimeo.

Magpie Talkshow Episode 16 - Neha Narula

Distributed systems can be pretty hard things to understand, so I'm always on the lookout for someone who manages to explain these things well. Neha Narula is one such person, and I was lucky enough to see one of her sessions at ��redev late last year.

After spending several years researching distributed systems at Google, Neha is now working at the MIT Media Lab looking into uses of the blockchain algorithm. During our interview, we talk about distributed databases, why distributed transactions can be a pretty good solution sometimes, and how much Postgres rocks!

You can follow Neha on Twitter as @neha, and find out more over at her blog.

Magpie Talkshow Episode 15 - Ben Stopford

Back in November, again at the ��redev conference, I sat down with an old colleague of mine, Ben Stopford. After starting out in physics (specialising in cosmology no less!) Ben shifted over to computing, where he has been ever since. After leaving ThoughtWorks, he spent much of his time in financial institutions working in the areas of high performance computing and data engineering.

Since then he has taken up a role at Confluent, the company behind Kafka, the open source distributed message broker. During our interview we talk about the challenges of handling volume at scale, why Kafka is different to other sorts of message brokers, and which Fortran is the best Fortran!

As always, links!

Follow Ben on Twitter: @benstopford
Ben's Elements Of Scale talk from ��redev
And also his The Future Of Data Technology, also from last year's excellent ��redev 2015

Magpie Talkshow Episode 14 - Sam Aaron

I've enjoyed all the interviews I've done for The Magpie Talkshow so far, but some do stick out more than others. Often the key distinction is the enthusiasm the interviewee has for a given topic - their passion really shines through when I listen back. Irinia's obvious love of Erlang was fantastic way back on Episode 4, but she might have a rival in , this week's guest on the show.

I've known Sam for a while. Back when we first met, he was using Overtone to create music using Clojure. Nowadays he works in research at Cambridge University, where amongst other things he works on SonicPi, a Ruby-based tool for making music and livecoding. During the episode, we explore his work on SonicPi, the challenges of live coding, how graphing calculators rock and why Aphex Twin should be using code to create his music.

Sam's main passion isn't necessarily music, more the idea that coding can and should be seen as a creative endeavor. As he puts it in the podcast, wouldn't it be dull if we only used English to describe business logic, rather than plays or poetry? Sam hopes that platforms like SonicPi can shine a light on the potential for code to be a creative tool.

SonicPi itself is now on the default distro for the Raseberry Pi, and is being used all over to help children code (which is a neat link back to Lynn Langit's work all the way back in episode 1). But it isn't just for kids. It's easy to use, comes with a great set of tutorials, and if you needed any encouragement to give it a go, then just listen to the episode!

When not working on SonicPi itself, Sam uses it in livecoding events - I was lucky enough to see him perform at an after-hours concert at ��redev after seeing his earlier talk. I'd highly recommend seeing Sam in person or at least watching one of the various videos of his work available to get a sense of how it all works - I have spliced in a sample of music from one of his recent presentations. The talk he gave at ��redev is probably a good place to start. I hope you enjoy the episode!

His talk at ��redev on livecoding and SonicPi
SonicPi
Examples of his livecoding:
A recent interview he did with the BBC

Magpie Talkshow Episode 13 - H��kan Jonsson

After an unplanned two week hiatus, the Magpie Talkshow is back. The break was due to a bunch of craziness going on which I'll hopefully let you know about soon, but I'm hoping from here on out we can get back to an episode a week.

Aside from ��redev being a generally great conference, it was also my most successful in managing to grab people for interviews. You've already heard me chat with Amy Phillips about continuous delivery, and in the coming weeks I'll be talking to Ben Stopford about Kafka, Neha Narula about databases, Corey Latislaw about Android, and next week you'll hear me talking to Sam Aaron about his work on Sonic Pi. This week though, I'm chatting to H��kan Jonsson about data science.

H��kan is a researcher working on Sony's Lifelog project, which aims to help you collect data about yourself to build insights. The sensitive nature of the data involved with Lifelog puts H��kan in an interesting position with regards to the privacy of the data collected, a subject we explore at some length, especially the role of ongoing informed consent.

We also try and tease apart the distinction between data science and data engineering, explore why terms & conditions are pretty useless in building trust with your users, and talk about some of the tools H��kan uses in his work.

I do have to put in my oft-used disclaimer here around the quality of the recording. We chatted in what I thought was a deserted restaurant at a nearby hotel only to find that not only was there piped music which is way more audible in the recording that I'd hoped but that there was also a lot of noise coming from the restaurant's kitchen. Hopefully it doesn't impact too much on your enjoyment of the episode though!

During the interview H��kan mentoned a tutorial on how to access your data from Lifelog which you can find over on Sony's Developer site. He also mentioned Animashree Anandkumars work on Tensors which you can find here. On the subject of tensors, since H��kan and I chatted Google have released their Tensorflow library which is worth looking at if the subject interests you. Finally, if you'd like to know more about H��kan's work with Lifelog, his presentation from ��redev is available now.

More coverage on BFFs

Late last year I wrote an article on an architectural pattern called "Backend For Frontends". This is a pattern that describes the use of a specific-purpose server-side API which can acts as an aggregating facade for user interfaces and other external interfaces. The article proved to be quite popular, and continues to generate debate in different parts of the interwebs from time to time.

The article itself was partly a result of spending some time at SoundCloud's offices, where my ThoughtWorks colleague Lukasz Plotnicki and I interviewed several engineers about their use of the pattern. After I published my own generic BFF piece, Lukasz also released "a great writeup" which goes into more detail in how the pattern emerged at SoundCloud, which is well worth a look. Now, Lukasz has also found time to sit down for an interview with the Software Engineering Daily podcast to talk more generally about SoundCloud's use of microservices.

By the way, if you're intertested in another take of microservice adoption at SoundCloud, I can recommend both Phil Cal��ado's "How We Ended Up With Microservices" piece from last year and SoundCloud's own Bora Tunca's talk from micoxchg just a couple of weeks ago. While in Berlin for microxchg I managed to sit down with Bora for a chat for The Magpie Talkshow, and I hope that episode will be out in April sometime.

I am planning to update the original BFF piece to cover the use of alternative middleware solutions like Falcor and Relay. In advance of that, if any of you have experience you could share of these sorts of tools I'd love to hear it!

Telstra, Human Error and Blame Culture

Australia's biggest Telco, Telstra, suffered a pretty serious outage yesterday. For several hours, millions of customers had serious disruption to both voice and data services. In the face of understandable customer anger, Telstra's Chief Operating Officer apologised for the 'embarrasing' outage, and explained what had happened.

"We took that node down, unfortunately the individual that was managing that issue did not follow the correct procedure, and he reconnected the customers to the malfunctioning node, rather than transferring them to the nine other redundant nodes that he should have transferred people to," Ms McKenzie told reporters on Tuesday afternoon.

"We apologise right across our customer base. This is an embarrassing human error.

- Sydney Morning Herald

I took to Twitter, to express my incredulity:

Such bullshit @telstra. A network where a single mistake by one person causes a huge outage is just plain broken. https://t.co/cgnvprQ2NY

— Sam Newman (@samnewman) February 9, 2016

and

Seeing @telstra's COO throwing one person under the bus for a major outage is just plain embarrassing. https://t.co/cgnvprQ2NY

— Sam Newman (@samnewman) February 9, 2016

This in turn led to a brief chat with Charis Chang, a journalist for news.com.au, and a short time later quite a few of my comments ended up on a piece on their website. It's not every day a tweet from me gets picked up by a national news website, so I thought it was worth following up with a blog post to try and explain why I reacted in the way I did.

First, we should deal with the first reaction that many of you probably had when reading this. 'One person caused all this?'. Yes, the mind boggles. Telstra seem to have a system which is not prepared for people to make mistakes. This is either a triumph of hope over reality, or just plain daft.

The implication is that a human being is able to make a change with no checks and balances that causes a huge, nationwide outage. For the nations largest telco, which is relied upon by many business across the country, 'embarrassing' doesn't quite do it justice. However, it isn't the only problem that is evident here.

We Don't Know Why This Happened, But We Know Who To Blame

Now let's see some platinum grade buck-passing in action:

"[The employee responsible] didn't follow procedures and clearly that's not a good thing but I wouldn't want to pre-empt the proper investigation and we'll figure out what the right response is when we've had a chance to dig into the detail."

- Australian Financial Review

Lovely. So one person made a mistake which caused the outage, which the COO feels happy in stating publically. But at the same time claims there hasn't been a proper investigation. Clearly blame first and ask questions later is modus operandi at Telstra.

For the nations biggest telco to have a serious outage that can be caused by an individual is crazy. To rush out a bit of misguided PR which confirms this is even more nuts.

Blame Culture

Hands up who thinks it is healthy for senior leadership to publically throw an employee under a bus. Sure, the manager in question wasn't named, but are you really telling me that people inside Telstra don't know who this person is? I wouldn't want to be them this morning.

Environments where senior management feel OK about singling out individuals are toxic. Blame cultures like this create a poisnous work environment. They lead to situations where mistakes cannot be admitted to, which in turn means that systematic issues go ignored. It hurts morale and hinders the creation of open collaborative workforces which can innovate and make companies more successful.

If I was working at Telstra, I'd be looking for another job right now.

It's The System

From what we have been told so far, an individual was placed in a situation where a simple mistake was allowed to have huge ramifications. Aside from the concern about how ready Telstra are to point blame, it speaks to a complete lack of understanding in Telstra's leadership as to how failures occur.

Singling out an individual to blame is perhaps common, but it is rarely if ever correct or healthy to do so. Most people who spend any time looking at the causes of failures understand that it is not the individuals that cause failures, it is the systems that they are part of.

As John Allspaw puts it, looking for a single source of failure is like looking for a single source of success. Success or failure is down to a number of factors, and can rarely if ever be put down to 'this person screwed up'. A system in which a single individual can make a simple mistake which leads to a serious outage like this is in and of itself seriously broken.

Homework For Telstra's Leadership Team

For examples of how to better look at failures in systems, and how to handle problems when they occur, I have some reading and viewing to recommend for Telstra's senior management. In no particular order:

Sydney Dekker - System Failure, Human Error: Who���s to Blame?
John Allspaw - Blameless Postmortems and A Just Culture
John Allspaw - Each necessary, but only jointly sufficient

Magpie Talkshow Episode 12 - Kristof Adriaenssens

This episode is a week later than planned, due to me being on holiday. Good for me, not so good for all of you I'm afraid! It'll be weekly from here on out I hope, and I'll get better at letting you know if they'll be any interruptions in service. At last count I have 12 unpublished interviews in the backlog, so I certainly have plenty of content! Anyway, on with the episode details.

I had the chance to visit SoundCloud's Berlin HQ last year, and sat down with Kristof Adriaenssens, an engineer in their Core Engineering team. Among other things he contributes to a project called Zipkin,a distributed tracing tool. We talk about Zipkin itself, his work on the Brave library, the general challenges of managing microservices at scale, and some of the solutions that they have come up with at SoundCloud to handle the complexities.

Magpie Talkshow Episode 11 - Amy Phillips

I got to record a number of great interviews at ��redev in Sweden late last year. My interview with Songkick's Amy Phillips was the third, but due to slightly confusing reasons on my side, is actually the first to get released.

Amy is a software engineer in test at Songkick, and for a while now has been focusing on helping the organisation move towards Continuous Delivery. During the interview we talk about many, many things, including the value of parent pester power, why university courses with sandwich (work placement) years are a good idea, the different types of testing, and of course how to adopt continuous delivery. We also briefly mention Jason Huggins' Tapsterbot project, which I completely forgot the name of during the interview!

You can follow Amy on Twitter via the awesome handle @itjustbroke, and check out her blog to see what she's been up to, including her involvement with the Weekend Testing Group. Finally, although the ��redev presentation isn't yet live, you can still see one of her earlier talks, "Testing in a Continuous Delivery World", filmed at BBC Future Media back in 2014.

Magpie Talkshow Episode 10 - Martin Fowler

Many of you have probably heard of my colleague Martin Fowler before. I'm lucky enough to get to work with him on occasion as part of my role on ThoughtWorks' Technology Advisory Board (or TAB), which is an internal body inside ThoughtWorks itself. One of the main responsibilities of the TAB is putting out the twice yearly Technology Radar, and I was in our New York office recently to work on the new version when Martin and I found time to catch up for a chat.

Recorded in a rather cozy room, we talked about many things, including how Martin got where he is today, how the tech radar is created, and even the future of computing books.

During the interview we reference quite a few things, so there are a bunch of links for further reading if you're so inclined:

Toby Clemson's Infodeck on Testing Microservices
Martin's early writing (late 90's)
Recent refactoring articles
General microserservices resources
Infodeck on the future of data
What makes a good Anti-pattern
On Feature branches
And on Taking responsibility for what we build

Finally you can follow Martin On Twitter http://twitter.com/martinfowler.

I hope you enjoy the episode!