The Architecture of Open Source Applications

The Architecture of Open Source Applications #1

by Amy Brown (Editor), Greg Wilson (Editor)

Architects look at thousands of buildings during their training, and study critiques of those buildings written by masters. In contrast, most software developers only ever get to know a handful of large programs well—usually programs they wrote themselves—and never study the great programs of history. As a result, they repeat one another's mistakes rather than building on one another's successes.

This book's goal is to change that. In it, the authors of twenty-five open source applications explain how their software is structured, and why. What are each program's major components? How do they interact? And what did their builders learn during their development? In answering these questions, the contributors to this book provide unique insights into how they think.

If you are a junior developer, and want to learn how your more experienced colleagues think, this book is the place to start. If you are an intermediate or senior developer, and want to see how your peers have solved hard design problems, this book can help you too.

Genres: Programming, Technology, Computer Science, Technical, Nonfiction, Architecture, Software

432 pages, Kindle Edition

First published June 8, 2011

Reviews

[Igor Tsinman · Rating 5 out of 5]

The Architecture of Open Source Applications это сборник эссе, которые написаны корифеями OSS, авторами очень известных open source программ. Вот несколько примеров: Asterisk, Bash, Berkeley DB, CMake, Eclipse, LLVM, Mercurial, Riak and Erlang/OTP, Sendmail и ещё дюжина не менее известных.

В аннотации написано, что довольно часто рядовой программер не разбирается во внутреннем устройстве хорошо известных программ/пакетов (в отличие от архитектора, который просто обязан это знать - ой ли?))). Поэтому, как правило, программеры наступают на одни и те же грабли раз за разом, вместо того, чтобы использовать успешные наработки известных коллег.

Цель "The Architecture of Open Source Applications" изменить эту порочную ситуацию/круг. Авторы известных open source программ описывают структуру своих программ и то как они работают. Из каких компонентов они состоят и как они взаимосвязаны. Размышляют о своих ошибках и делятся рецептами.

Всю книгу целиком я не предполагал читать (не тот случай), но те главы, которые прочёл, мне понравились!

Вот несколько новых вещей, которые я узнал:

* The Bourne-Again Shell - ежедневно используя bash, я не предполагал, какая часть имплементации является самой сложной (оказывается это undo redirections, job bookkeeping).
* Riak and Erlang/OTP - distributed nosql db Riak использует 120% мощи OTP библиотеки Erlang.
* Berkeley DB - я не знал, что один из со-автором проекта это женщина. Первая версия с транзакциями была использована в Netscape в 1996 году.
* CMake - я был приятно удивлен, прочитав, что CMake использует Lua как встроенный язык.
* Continuous Integration (CI) - в нескольких проектах я использовал Buildbot и CDash, так что было интересно прочесть как правильно их использовать и сравнить с собственным опытом)))
* NoSQL - я использовал только Key-Value Stores (Mnesia), теперь я понял как работают Key-Data Structure, Key-Document, BigTable Column Family Stores.
* Sendmail - я давно пользую Postfix, но было интересно почитать как начинался проект (part time job) и что из этого получилось (кому довелось хоть раз конфигурировать Sendmail, тот меня поймёт)))

Книга может быть полезной для новичков и для матёрых программеров.

С момента публикации, книгу можно читать онлайн, прямо с сайта книжки. Вот здесь. Постоянно добавляются новые главы (о них можно узнать на сайте книги). Для тех, кто ждёт русскую версию, на сайте есть сообщение, что перевод уже в "процессе".

[Vasil Kolev · Rating 4 out of 5]

The book is somewhat hit and miss - there are some extremely boring and (IMO) useless chapters, and some very good. I should definitely mention the chapters on sendmail, graphite, hadoop, llvm, python packaging and riak. Everyone should be able to find some interesting examples to use in his/her own work.

[Brian · Rating 3 out of 5]

(3.0) Think of each chapter as a separate book, some good some bad.

Each chapter is on a different open source project, written by one of the principals involved in that project. There was a lot of variability in what I took away from the chapters. Perhaps it has to do with which applications I could relate to best, but some felt like just an explanation of the features and APIs of an app, not as much about the structure, the big architectural decisions and the evolution that the architectures went through. Others did much better there I thought. I did like that they pretty much all had a lessons learned section and reflected back on what succeeded, what didn't, what they learned.

The chapters I appreciated the most were Berkeley DB (woot, Margo Seltzer!), Graphite, HDFS, NoSQL, Riak, Sendmail. The others I read but didn't take much from, and they brought down the overall rating.

[Babak Ghadiri · Rating 3 out of 5]

این کتاب ۲۵ فصل داره که در هر فصلش معماری(؟) یک نرم‌افزار متن باز توسط افرادی که تسلط خوبی بهش دارند (معمولا نویسندگانش) توصیف میشه. در کل خیلی از فصلهاش کم ارزشند. ولی چند تا از فصلهاش مثل
Graphite
LLVM
The NoSQL Ecosystem
Riak and Erlang/OTP
Battle For Wesnoth
به نظر من خوب بودند.
http://aosabook.org/en/index.html

[Eric · Rating 3 out of 5]

An interesting collection of essays by open source maintainers, varying from supplementary documentation for the project to historical retrospectives on the projects’ evolutions to educational pieces about a given concept or decision.

A bit old at this point (published in 2011), but still has a number of interesting lessons to teach. Essay quality is variable as one would expect, but this is unfortunately made more prominent by the decision to order them alphabetically by title, leading to a few (several) slogs right at the beginning. The middle of the book is quite meaty though, and overall I found there to be more hits than misses.

[Benoit Blanchon · Rating 2 out of 5]

The quality varies greatly from chapter to chapter.
A few chapters are really good, but the majority is uninstructive and poorly written.
Overall, I don't think it was worth reading.

[Sergey Machulskis · Rating 1 out of 5]

I expected something different and it doesn't worth the time spent.

Didn't read the entire book (reviews say that some essays are badly written) so I chose the following list:
* Berkeley DB. Well-written and contains good pieces of advice. But it's very sparse.
* HDFS. Nothing useful to anyone who knows what NameNode is.
* NoSQL Ecosystem. It's not about architecture. It's an outdated review of NoSQL solutions.
* Python packaging. Interesting from a historical perspective. What to say? Modules in programming languages is not a solved problem and in Python's case it's complicated by legacy.
* Riak and Erlang/OTP. Well, Erlang has means for client-server communications. They're called behaviours and you can implement a server using it (aka write your own middleware). That's it.
* Selenium WebDriver. The first page of google gives better insights.
* Sendmail. It's a history of long-dead problems with email. It might be interesting if you use sendmail, but I doubt it.

[Steve · Rating 4 out of 5]

A fascinating but very uneven collection of articles about how a bunch of open source systems are built ranging from file systems and databases to scientific visualization and data processing pipelines to games. Each article focuses on a different system and different aspects of each system, some delving into class structures and others on very high-level component design.

It's a long read - getting in time on my commute it took over a year, these are not light reading- but a fascinating look at the many different ways people build software.

[Xavier Shay · Rating 3 out of 5]

Each chapter is written about a different application by a different author, and the quality varies widely.

The following are worth reading (and available online at http://www.aosabook.org/en/index.html though I read the kindle version):
* Graphite
* LLVM
* Selenium Webdriver
* Python packaging
* SnowFlock

[Mikhail Filatov · Rating 3 out of 5]

It's hard to give a single rating to a book with 20+ articles from even more authors.
It's actually one of the main concerns from my side -the book has several "research/Phd thesis-level" projects - e.g. why the article on "NoSQL"? If proper editing/selection was applied it could be lean and mean 250 pages book vs. bloated 400+ pages.
Still, there are several gems - e.g. Berkley DB which makes it worth reading. So my advice - go through chapters and read only interesting (for you), don't try to read the book as a whole - as you can get bored and may miss some interesting stuff closer to the end.

[pluton · Rating 2 out of 5]

A collection of "architectural stories" about a number of different open-source projects. I call them stories because they are of very varying quality and number of details; some had lots of code, some almost none at all; several described the program from the user's perspective and not so much from the code architecture's. I expected something else, but not sure what exactly.
Another point from a practical perspective, if I want to buy the book, the best way on the website is to buy from lulu.com — but why do they sell epub and pdf formats separately?!

[Anton Antonov · Rating 2 out of 5]

A very rough read. As a newbie software architect in 2017, this book was my need for creativity and inspiration to learn how other Open Source applications are architected.

The book invites authors to describe their experience in Open Source Software (OSS) projects, but the architecture modeling is neither consistent nor reliable in terms of quality.

Most chapters are a waste of time if not for the design lessons.
I gained much more by reading Neal Ford and Mark Richards' books.

[Christian · Rating 3 out of 5]

a diverse collection of basically blog posts or READMEs, some better than others, also depending on your own knowledge and interests.
I had hoped there were more points about architecture itself, or why something was build this way, and not that way.
Nonetheless can be a good read, just check which projects are written about, and if you have interest in them.

[Damien · Rating 3 out of 5]

This is an interesting, but very unequal book. Each chapter being written by different authors, some are very well written and interesting, while other aren't really good.

[Keren Xu]

finished chapter 11 matplotlib https://aosabook.org/en/v2/matplotlib...

[Dominik · Rating 2 out of 5]

some chapters worth it, the most I just skimmed

[Kam Yung Soh · Rating 3 out of 5]

An interesting book that covers just what the title says: how some of today's Open Source Applications were designed. And understanding how applications are designed is good to know when it is your turn to design an application that will be seen by the world, warts and all.

The book covers over twenty applications and range from the old but still much used (Bash, Sendmail), single purpose (Audacity) to generic purpose (Mercurial) and to generic design (Python Packaging, NoSQL).

It is not possible for me to understand or be interested in every project in this book, but by concentrating on those that interested me, and generally skimming through the others, you'll get a idea of what is involved in the design of software that most of us would use, whether directly or indirectly.

Some projects are covered in very general terms (Hadoop Distributed File System), other expose some of the internal structures of the software (Telepathy), while others give a fascinating look into how the software has developed and changed over the years (Sendmail).

Probably the best way to get the most out of this book is to read those parts that interest you, but also skim through the other projects to see what ideas were rejected or implemented. You never know; the way one unknown (to you) project was done in the book may well guide you on how to do your own software projects better.

This book can also be found online.

[Alexey · Rating 3 out of 5]

Книга – сборник статей, написанных разработчиками опенсорсного ПО. Помимо книги, статьи, как это и положенно OS, находятся в открытом доступе на официальном сайте проекта – https://aosabook.org/ (Амазон об этом вежливо умалчивает, а знал бы – сэкономил бы 20-ку).

В целом, опыт чтения – так себе. Во-первых, все главы очень различаются по качеству. Какие-то рассказывают всю историю создания – например, целый трактат о Sendmail, а какие-то нудно перечисляют инженерные решения, многие из которых устарели (книга написана в 2011-м году, и, когда, например, читаешь главу о системе тестирования Selenium, то кажется, что описываемые события произошли миллион лет назад).

Когда я покупал книгу, я думал, что уровень анализа архитектуры будет на более высоком уровне, т.е. будет больше аналитики. А так, многие статьи скорее напоминают длинные комментарии к коду. Читать это дело не практикующему программисту – унылое занятие.

Помимо этого, проекты отличаются по масштабу: в одной главе про Audacity (OS программа для редактирования аудиофайлов) или Jitsi, а в другой – кусок экосистемы вроде Python Packaging.

Наиболее интересные статьи (их можно найти на сайте проекта): Jitsi, Bash, Selenium, Sendmail.

[Thomas Zeeman · Rating 4 out of 5]

As with any book bundling a lot of stories from different authors, the style and quality differs. A lot in some cases, although the quality tends to stay on the higher end.

The various projects cover a wide range of topics (i.e. games, data processing, media), programming languages and ages (some applications have been around for several decades, others only a few years. All of them still relevant.

It was interesting to read about the way various projects were designed and how things sometimes grew over time.

Two things I missed there were some more graphic ways of describing the architecture of an application and some reflection on the design choices. The later chapters sometimes did contain a few diagrams, but most chapters don't have any pictures at all.

Design choices were often explained, but if there was a section on reflection or lessons learned it was relatively brief; typically a page out of a 30-40 page essay. And from the ones I've read only one struck me as interesting.

[amisai · Rating 4 out of 5]

A revision of architecture of several open source applications. Written by apps' authors or mainteners, this book allows you to understand main architectural ideas of applications. Variety of applications (communications, compilers, graph generators, ...) is the best idea of this book. Sometimes a chapter writer is not focused, and doesn't explain his application architecture properly, but that also a way of learning (in this case, how not to describe an architecture).

Expecting following volume of this serie...

[Abdul Qavi · Rating 4 out of 5]

Very informative book, a must-read for every IT professional. The book can't be rated as a whole since it consists of over 2 dozen chapters, and each chapter is written by different individual. Those chapters that are written by developers themselves like sendmail by Eric Allman, and others like Eclipse and Jitsi are excellent chapters with complete background and architecture nicely explained, while others are fairly good. So overall rating goes 4/5.

[Joey · Rating 3 out of 5]

Very uneaven, as is typical with this sort of book. Good chapters included llvm, bdb, bash. Too many block diagrams, and if the premise is we're not exposed to enough software architecture, why do I feel I've seen far too many of those? Although bdb used them to good effect showing evolution over time.

Looking forward to the upcoming chapters on git and ghc.

[Matt · Rating 3 out of 5]

Like other compilations books about software that I've read, this book really works or fails depending on the author of each chapter. I enjoyed most of the chapters, loved a few of them, and groaned at a few others. I'm looking forward to reading the second volume soon.

[Manzur · Rating 4 out of 5]

Quite interesting and useful book about architecture of widely used software. As it's a collection of chapters from different authors, writing quality of each of them are different. Nevertheless, I liked the book, especially "Lessons learned" part of the chapters.

[Ben Sowell]

This book is interesting and well done, I just probably won't make it all the way through. I'll probably pick out a chapter here and there instead.

[Uwe Dauernheim · Rating 3 out of 5]

The selection of projects is not great (though I can understand why). Volume II is magnitude better collection.

[Michiel]

interesting!