Django Design Patterns and Best Practices: Industry-standard web development techniques and solutions using Python, 2nd Edition

by Arun Ravindran

while writing a characterization test, you will first write a failing test with a dummy output, say X, because you don't know what to expect. When the test harness fails with an error, such as Expected output X but got Y, you will change your test to expect Y. So, now the test will pass, and it becomes a record of the code's existing behavior.

python manage.py inspectdb > models.py

migrating legacy database into django models

Django deprectations: Deprectations tell you whether a feature or idiom will be discontinued from Django in the future. Since Django 1.11, they are quiet by default. Use python -Wd so that deprecation warnings do appear.

view = resolve('/')

Using resolve() is great way to test the routing, instead of calling the view directly

self.assertListEqual(self.l1, self.l2)

A more specific assertion results in a more descriptive errors

it is always better to use the most specific assert* method for your tests. Since the second argument is not a list, the error clearly tells you that a list was expected.

Here are some qualities of a good test case (which is a subjective term, of course) in the form of an easy-to-remember mnemonic fast, independent, repeatable, small, transparent (FIRST)

Do not (re)test the framework: Django is well tested. Don't check for URL lookup, template rendering, and other framework-related functionalities.

Test models most, templates least:

Avoid using the web test client in unit tests:

Avoid interacting with external systems: Mock them if possible.

a test fixture is a file that contains a set of data that can be imported into your database to bring it to a known state. Typically, they are YAML or JSON files previously exported from the same database when it had some data.

Legacy and uncommon way of handling testing data

many consider using fixtures as an anti-pattern. It is recommended that you use factories instead.

The django-extensions package is shipped with the fantastic Werkzeug debugger that provides exactly this feature. In the following screenshot of the same exception, note the fully interactive Python interpreter available at each level of the call stack:

Logging has several advantages over printing: it has a timestamp, a clearly marked level of urgency (for example, INFO, DEBUG), and you don't have to remove them from your code later.

Once you have configured the LOGGING variable in settings.py, adding a logger to your existing code is quite easy, as shown here: # views.py import logging logger = logging.getLogger(__name__)

We need to set TEMPLATE_DEBUG to True (in addition to DEBUG) in settings.py so that Django shows a better error page when there is an error in your templates.

if you want to dump all the variables, use the built-in debug tag like this (inside a conveniently clickable text area):

However, you might want to pause in the middle of a template to inspect the state (say, inside a loop). A debugger will be perfect for such cases. In fact, it is possible to use any one of the aforementioned Python debuggers for your templates using custom template tags.

Debugging templates with pudb!

class XSSDemoView(View): def get(self, request): # WARNING: This code is insecure and prone to XSS attacks # *** Do not use it!!! *** if 'q' in request.GET: return HttpResponse("Searched for: {}".format( request.GET['q'])) else: return HttpResponse("""<form method="get"> <input type="text" name="q" placeholder="Search" value=""> <button type="submit">Go</button> </form>""") The preceding code is a View class that shows a search form when accessed without any GET parameters. If the search form is submitted, it shows the Search string exactly as entered by the user in the form. Now, open this view in a dated browser (say, IE 8) and enter the following search term in the form and submit it: <script>alert("pwned")</script>

Basic example of cross-site-scripting

we can use a RegexValidator class in our search-term field to restrict the user to alphanumeric characters and allow punctuation symbols recognized by your search module. Restrict the acceptable range of the user input as strictly as possible.

For instance, the following fake 0x0 image can be embedded in a comment: <img src="http://superbook.com/post?message=I+am+a+Dufus" width="0" height="0" border="0">

Basic example of cross-site request forgery

The basic protection against CSRF is to use an HTTP POST (or PUT and DELETE, if supported) for any action that has side effects. Any GET (or HEAD) request must be used for information retrieval, for example, read-only.

name = request.GET['user'] sql = "SELECT email FROM users WHERE username = '{}';".format(name) At first glance, it might appear that only the email address corresponds to the username mentioned as the GET parameter will be returned. However, imagine if an attacker entered ' OR '1'='1' in the form field, then the SQL code would be as follows: SELECT email FROM users WHERE username = '' OR '1'='1';

Basic example of SQL injection

Clickjacking is a means of misleading a user to click on a hidden link or button in the browser when they were intending to click on something else. This is typically implemented using an invisible IFRAME that contains the target website over a dummy web page

It works by setting the X-Frame-Options header to SAMEORIGIN for every outgoing HttpResponse. Most modern browsers recognize the header, which means that this page should not be inside a frame in other domains.

Django setting enabled by default to prevent iframe embedding

os.system("ls -l {}".format(filename)) An attacker can enter the filename as manage.py; rm -rf * and delete all the files in your directory. In

Basic example of shell injection

use call() from the subprocess module to run command-line programs with its default shell=False parameter to handle arguments securely if shell interpolation is not necessary.

Don't trust data from a browser, API, or any outside sources

Don't keep SECRET_KEY in version control

Don't store passwords in plain text

Don't log any sensitive data

use SSL

Avoid using redirects to user-supplied URLs

Use the strictest possible regular expressions

Don't keep your Python code in web root

CSRF should be enabled and used

Limit the size and type of user-uploaded files

Have a backup and recovery plan

there must be no development performed directly on the production environment. In fact, there is no need to install development tools, such as a compiler or debugger, in production. The presence of any unneeded software increases the attack surface of your site and could pose a security risk.

even if an internal component fails, there is enough redundancy to prevent the entire system crashing. This concept of avoiding a single point of failure (SPOF) can be applied at every level, hardware or software.

For example, username availability might be looked up on Redis, while the primary database might be PostgreSQL.

As you can imagine, most real-world applications will be composed of multiple Microservices and each of them would require multiple containers. If you run them on multiple servers, how would you deploy these containers across them? How can you scale individual microservices up or down? Kubernetes is the most widely recommended solution for managing such container clusters.

Platform as a Service (PaaS) is defined as a cloud service where the solution stack is already provided and managed for you. Popular platforms for Django hosting include Heroku, PythonAnywhere, and Google App Engine.

A more appropriate term would be Function as a Service (FaaS),

Monitoring tools usually need a backend service (sometimes called agents) to collect the statistics and frontend service to display dashboards or generate reports. Popular data collection backends include StatsD and Monit. This data can be passed to frontend tools, such as Graphite. There are several hosted monitoring tools, such as New Relic and Status.io, which are easier to set up and use.

tests at Amazon in 2007 revealed that for every 100 ms increase in load time of amazon.com, the sales decreased by 1 percent.

The key to improving performance is finding where the bottlenecks are. Rather than relying on guesswork, it is always recommended that you measure and profile your application to identify these performance bottlenecks. As Lord Kelvin would say: "If you can't measure it, you can't improve it."

Cache infinitely with CachedStaticFilesStorage:

Use a static asset manager: