Building Microservices: Designing Fine-Grained Systems

by Sam Newman

Rather than having each service manage handshaking with our identity provider, a more common approach is to use a gateway to act as a proxy, sitting between your services and the outside world (as shown in Figure 11-7).

Beyond allowing (or disallowing) access to specific resources or endpoints, though, we need to leave the rest to the microservice itself; it will need to make further decisions about what operations to allow.

One option for avoiding the confused deputy problem is to perform all required authorization as soon as the request is received in our system.

Therefore, we’d want to push the decision about whether or not the call should be authorized into the same microservice where the functionality being requested lives.

Generating a JWT token on a per-request basis seems to be the most common solution to this problem, as we show in Figure 11-9.

In such a situation, it made sense to deal with the more complex rights management authorization process in a different way—essentially using the JWT token for the initial “simple” authorization, and then doing a subsequent lookup on a data store to fetch the additional fields as required. This meant the bulk of the system could just work off the tokens.

Creating a culture that prioritizes creating an environment in which people can share information freely, without fear of retribution, is vital to encourage learning in the wake of an incident.

We ended up implementing three fixes to avoid this happening again: getting our time-outs right, implementing bulkheads to separate out different connection pools, and implementing a circuit breaker to avoid sending calls to an unhealthy system in the first place.

Time-outs are incredibly useful. Put time-outs on all out-of-process calls, and pick a default time-out for everything. Log when time-outs occur, look at what happens, and change them accordingly. Look at “normal” healthy response times for your downstream services, and use that to guide where you set the time-out threshold.

Don’t just think about the time-out for a single service call; also think about a time-out for the overall operation, and abort the operation if this overall time-out budget is exceeded.

In many ways, bulkheads are the most important of the patterns we’ve looked at so far.

They can also give you the ability to reject requests in certain conditions to ensure that resources don’t become even more saturated; this is known as load shedding.

With a circuit breaker, after a certain number of requests to the downstream resource have failed (due either to error or to a time-out), the circuit breaker is blown.

While the circuit breaker is blown, you have some options. One is to queue up the requests and retry them later on. For some use cases, this might be appropriate, especially if you’re carrying out some work as part of an asynchronous job. If this call is being made as part of a synchronous call chain, however, it is probably better to fail fast.

If operations are idempotent, we can repeat the call multiple times without adverse impact.

At its heart it tells us that in a distributed system, we have three things we can trade off against each other: consistency, availability, and partition tolerance.

Optimizing our system to solve problems we don’t have is a great way to waste time that could be better spent on other activities, and also to ensure that we have a system that is needlessly more complex.

If you’ve decided to implement a microservice by splitting responsibility for reads and writes across different processes and models, for example, this should be invisible to consumers of the microservice.

The main thing is that you’d likely need to configure your cache invalidation mechanism to not automatically evict stale data, and to keep data in the cache until it can be updated.

It’s worth noting that, although from the consumer point of view this caching is invisible (it’s an internal implementation concern), that doesn’t mean we’d have to implement this by caching in code in a microservice instance.

The main concern around write-behind caches is going to be the potential for data loss.

Treat caching primarily as a performance optimization. Cache in as few places as possible to make it easier to reason about the freshness of data.

If you are using the enabling team model, with specialists spending time with multiple teams, they can help ensure that the work done by each team is consistent.

The downside is that you can end up with a lot of duplication between dependencies, which in turn can lead to a large bloat in terms of page load size.

The BFF is tightly coupled to a specific user experience and will typically be maintained by the same team as the user interface, thereby making it easier to define and adapt the API as the UI requires, while also simplifying the process of lining up release of both the client and server components.

So you can see your organizational structure as being one of the main drivers for deciding which model makes the most sense (Conway’s law wins again).

The first, which is often cheaper but more fraught, is to extract a shared library of some sort.

The other option is to extract the shared functionality into a new microservice.

For an application that is only providing a web UI, I suspect a BFF will make sense only if and when you have a significant amount of aggregation required on the server side.

Fundamentally, achieving loosely coupled organizational structures requires that power and accountability are decentralized.

With full life-cycle ownership, a single team comes up with the design, makes the changes, deploys the microservice, manages it in production, and ultimately decommissions the microservice when it is no longer required.

With a more collective ownership model, you are likely going to need to reduce options in order to introduce a stronger degree of consistency across what teams do and how microservices are implemented.

In the new, modern, decentralized organization, architects are surveying the landscape, spotting trends, helping connect people, and acting as a sounding board to help other teams get stuff done.

The less stable or mature a service is, the harder it will be to allow people outside the core team to submit patches.

If a team has a lot of inbound pull requests, it could be a sign that you really have a microservice that is being shared by multiple teams.

So, in general, we want to make use of peer change reviews and avoid the need for external code reviews.

If you want to do code reviews and aren’t pair programming, do the review as quickly as possible after the change is submitted, and go through the feedback in as synchronous a way as possible, ideally face to face with the reviewer.

The core delivery services team had the job of providing advice, guidance, and tooling to the squads in the LOBs, helping these squads deliver more effectively.

Well, I would suggest that geographical boundaries between people involved with the development should be a strong consideration when looking to define both team boundaries and software boundaries.

Thus, software architects need to shift their thinking away from creating the perfect end product and focus instead on helping create a framework in which the right systems can emerge and continue to grow as we learn more.

As architects, we need to worry much less about what happens inside a zone and more about what happens between the zones.

“be worried about what happens between the boxes, and be liberal in what happens inside.”

A model I greatly favor is to have a small number of dedicated architects in this team (perhaps just one or two people in many cases), but having this team augmented over time with technologists from each delivery team—the technical leads of each team at a minimum.