Release It!: Design and Deploy Production-Ready Software (Pragmatic Programmers)

by Michael T. Nygard

Increase the size of your web tier Your web servers will be dealing with more requests. Be sure to increase the maximum number of connections your web tier can handle, either by adding servers or by increa...

Set the session timeout to one standard deviation past the average think time.

Curtail session retention Keep sessions in memory for as short a time as reasonable. This time will vary from one domain to another.

Remember that users don’t understand sessions Users understand automatic logout for security reasons. They won’t understand why their shopping cart disappears because they spent twenty minutes researching a product on another website. Things shouldn’t disappear just because the user goes away for a while.

Keep keys, not whole objects If you keep whole objects in the session, do it with soft references. Keep keys to persistent objects instead. You will get better capacity, and your users will have a better experience: they can co...

Omit needless characters Omit wasted characters in HTML. Generating it takes CPU cycles on the application server. It takes bandwidth on the application servers’ network cards, bandwidth on the network switches, bandwidth on the web servers, and bandwidth on the users’ connections. Downloading it takes time, particularly for users on slow connections. Eliminate whitespace to save your users’ time and your company’s money.

Remove whitespace Whitespace sneaks into generated pages near loops, conditionals, and includes. The directive gets replaced, maybe by a zero-length string, but the coder’s nice formatting remains. I’ve seen pages with 400KB of tabs, spaces, and newlines.

Replace spacer images with nonbreaking spaces or CSS Web design programs leave spacer images such as mouse droppings in a page. Tiny but noisome. When turning a page des...

Replace HTML tables with CSS layout Aside from being more “Web 2.0,” CSS style sheets don’t have to be downloaded on every page the way a table-based superstructure does. HTML files with styles and classes c...

Make the Reload button irrelevant Fast sites don’t provoke the user into hitting the Reload button. You want your site to serve pages so fast that the Reload button never comes into it. Reload requests hurt your site when it’s already suffering.

Minimize handcrafted SQL You might be tempted to roll your own SQL as a performance optimization. It’s always a good idea to see whether there are ways to do the same in the database itself—through hinting, indexing, or views.

See whether the DBA laughs at the queries If it doesn’t pass the laugh test, it shouldn’t go into production. Period.

Verify gains against real data Try your handcrafted SQL against production-sized data. Gains observed in a development database sometimes evaporate when they hit r...

Don’t mix transactions and reporting.

Create indexes; it’s not just the DBA’s responsibility You know your application’s intentions better than the DBA. You should know which columns will be used for lookups, which tables are read-mostly, and which are write-mostly. It’s your responsibility to come up with the first iteration of indexes.

Purge sludge Old data just slows down queries and inserts. Unless the user cares about it—such as an order history table—you should get it off the production servers.

Keep reports out of production Reports can, and should, be served elsewhere. Don’t jeopardize production operations by letting reports run expensive queries. Besides, reports are better se...

Expose yourself to latency as seldom as possible Integration point latency is like the house advantage in blackjack. The more often you play, the more often it works against you. Avoid chatty remote protocols. They take longer to execute, and they tie up those precious request-handling threads.

Serve small cookies Use cookies for identifiers, not entire objects. Keep session data on the server, where it can’t be altered by a malicious client.

Pool connections Connection pooling is basic. There’s no excuse not to do it.

Protect request-handling threads Do not allow callers to block forever. Make sure that any checkout call has a timeout and that the caller knows what to do when it doesn’t get a connection back.

Size the pools for maximum throughput Undersized resource pools lead to contention and increased latency. This defeats the purpose of pooling the connections in the first place. Monitor calls to the connection pools to see...

Limit cache sizes Unbounded caches consume memory that is better spent handling requests. Holding every object you’ve ever loaded in memory doesn’t do the users any good.

Build a flush mechanism Whether it’s based on the clock, the calendar, or an event on the network, every cache needs to be flushed sooner or later. A cache flush can be expensive, though, so consider limiting how often a cache flush can be triggered, or you just might end up with attacks of self-denial.

Don’t cache trivial objects Not every domain object and HTML fragment is worth caching. Seldom-used, tiny, or inexpensive objects aren’t worth caching: the cost of bookkeeping and r...

Compare access and change frequency Don’t cache things that are likely to change be...

Precompute content that changes infrequently Any content that you present many times before it changes could be precomputed to save time during request handling. Factor the cost of generating the content out of individual requests and into the deployment process.

Tune the garbage collector in production User access patterns make a huge difference in the optimal settings, so you can’t tune the garbage collector in development or QA.

Keep it up You will need to tune the garbage collector after each major application release. If you have an annual demand cycle, you will also need to tune it at different times during the year, as user traffic shifts between features.

Don’t pool ordinary objects The only objects worth pooling are external connections and threads. For everything els...

It’s best to disable core dumps on production applications.

How often will the monitoring device execute its synthetic transaction? What is the maximum acceptable response time for each step of the transaction? What response codes or text patterns indicate success? What response codes or text patterns indicate failure? How frequently should the synthetic transaction be executed? From how many locations? Where will the data be recorded? What formula will be used to compute the percentage availability? Based on time or number of samples?

Configuration files are often obscurely named, buried deep in the directory structure of the code base, or just plain inscrutable. When a property is named hostname, is that “my hostname,” “the name of the authorized caller,” or “the host I call during the autumnal solstice?” Property files suffer from hidden linkages and high complexity—two of the biggest factors leading to operator error.

Memory Minimum heap size, maximum heap size, generation sizes

Garbage collection Type, frequency, memory reclaimed, size of request

Worker threads, for each thread pool Number of threads, threads busy, threads busy more than five seconds, high-water mark (maximum concurrent threads in use), low-water mark, number of...

Database connection pools, for each pool Number of connections, connections in use, high-water mark, low-water mark, number of times a connec...

Traffic statistics, for each request channel Total requests processed, average response time, requests aborted, requests per second, time of...

Business transaction, for each type Number processed, number aborted, dollar value, transaction aging, conversion rate, completion rate

Users Demographics or classification, technographics, percentage of users who are registered, number of users, usage patterns, errors encountered

Integration points Current state, manual override applied, number of times used, average response time from r...

Circuit breakers Current state, manual override applied, number of failed calls, time of last successful ca...

Ask, “Does this architecture make IT respond better to users’ needs?”