This talk has been previously presented at:

  • DjangoCon 2021
  • PyCon 2020 Online
  • PyGotham 2019

  • The slides below are from the DjangoCon 2021 iteration.


    Hi! I'm Katie, and this is "What is deployment, anyway?".

    This isn't the talk you're expecting.
    This isn't a talk about the "One Way" to do deployments.

    No talk can do that.

    This also isn't a talk about how to take your proof-of-concept VC-funded startup site and turning it into the next big dot com. If you want to talk about that
    Here's my work email. Let's chat.

    But for today, the scope of my discussions are going to be talking about the minimum number of things you need to consider in order to host your django website. This talk is more about taking your little cool, Django blog application that you've made on your local machine and putting it on the internet so you can share it with your friends.

    This talk is going to answer the question posed in the title:
    What is deployment, anyway?

    More specifically, what is django deployment, anyway?
    (because you know, this is a djangocon!)

    and we're going to focus on the "what" here. We're going to briefly mention some of the "where" and the "how", but this is going to focus on the "what"
    This talk and the code examples in this talk are specifically django 3.2 and Python 3.9, as these are the most current minor versions at time of recording.
    If you're joining me on YouTube from the year 2032 I'm sorry?
    This talk might be a little bit out of date.
    But hopefully you still learn something.
    So, let's start by looking at django, as it comes, out of the box. Just as is.
    If you haven't create a new django project in a while, it's nice to be reminded about how it all starts.
    So in my terminal, I want to install django so I type "pip install django" and pip goes and collects django and it's main dependencies.
    I then want to start a new project So for that I'll use the django-admin CLI that was just installed, and use it's
    startproject template function. and i'll call it myproject, and create it in the current
    folder.
    now I have no output from this command but if I check what files I now have in this folder I can see I have a manage.py file, and then a myproject folder with some different files
    in there.
    I can see an asgi.py, a settings.py, a urls.py, and a wsgi.py. .. Huh.
    so, clearing my terminal, I want to now start up django so I run python manage.py runserver which then runs, but I get a big red error message, I have unapplied by migrations
    but django has helpfully suggested that I run "python manage.py migrate" to apply
    them. Great! let's quit that process
    and run that migrate command. All the migrations are applied, and they all report OK great!
    so then let's see what's in my folder now Ah, I see there's a new file Looks like i have a dee-bee dot ess-qu-lite-three file. That's interesting. since I've applied my migrations, I should be able to run my application
    so if I runserver again I get no red warnings, and I'm told the development server is available so let's go there
    Oh hey, the installation worked successfully! Go me! I also know I should now have a django admin
    Buttt I can't login. I don't have an account yet.
    so let's go back to the terminal and do that
    Let's stop that process Clear our terminal
    And then run the createsuperuser command This command, will create a superuser for me, that will have access to the admin,
    I'll use my name For this I don't need an email I'll give it a super secure long password twice and now I have a superuser. I can then restart the process
    Then back in my login screen
    I can enter my username and passwrd
    And I have access to the admin! Hooray!
    Django has a great local development story we can get our app running locally really easily. and a lot of this boils down to
    runserver
    This command runs our application for us, and also does nice things like
    automatically restart itself when made an edit to our code.
    The django documentation describes what runserver does.
    Seriously, the django docs are great, and they are very good at explaining what
    django does. If you're newer to django, or any tech, some docs can be hard to parse
    and understand, but that's okay, I'm here.
    Let's look at the docs.

    https://docs.djangoproject.com/en/3.2/ref/django-admin/#runserver
    from the docs, the runserver command Starts a lightweight development web server on the local machine.
    But if you look down a few paragraphs, in all caps do πŸ‘ not πŸ‘ use πŸ‘ this πŸ‘ server πŸ‘ in πŸ‘ a production setting
    Why? it continues.
    It has not gone through security audits or performance tests and that's now it's gonna stay we're in the business of making web frameworks, not web servers.
    let me pull that last part out into it's own slide
    Django is an extremely stable, production ready, web framework.
    Django is very good at being a web framework.
    The fact that it provides a local web server is amazing for getting started
    but the fact that it calls out it's limitations is also very very good.
    So we're going to need to find another webserver.
    This part of the documentation also brings up some .. terms.
    what is "production" anyway? and also
    why is it called "production"?

    Historically, when computers were fed punch cards, you could say it was a
    "production" to process data. You had to physically move different things around.
    Much like a production line in manufacturing.
    You could also think of it nowdays as a stage production. It's the thing that paterons,
    attendees, or users see.
    Your production environment is the one you want people to go to. It's your "live"
    environment. It needs to be able to support a crowd.
    The size of your crowd is going to influence when you start thinking about things like
    load balancing and caching, but right now, it's just us, and our mate on the other side
    of the country who we want to show this website to.
    not only is runserver a development web server,
    runserver also serves the pretty, that being the images, css, javascript, etc that comes
    with our application.
    To be able to see the nice centrally formatted blue-on-grey login window, there needs
    to be css to make that happen.
    these comes with django itself. If you don't have this setup correctly
    your login screen looks very 1997
    I mean, it's still usable -- web accessibility FTW -- but you can tell that something isn't
    setup correctly.
    Collectively, all these styling files are known as "static"
    But why is it called static?
    They're called "static" because they are unchanging.
    They came with Django itself, and they don't change. But they have to be 'served' so
    that the browser can make use of them.
    For instance
    if we take a look at the source of the login screen, we can see that we have some css and javascript being served from /static/ path -- The django documentation describes how runserver achieves this.

    https://docs.djangoproject.com/en/3.2/howto/static-files/#serving-static-files-during-development
    In the documentation for serving static files during development, there's another bit in
    here This is not suitable for production use!
    Also noted that runserver will only serve static files when you have DEBUG set to True.
    Well, we can see these files are being served, so we should have DEBUG as true,
    right? How can we check?
    This is where we start looking at some of the code that was automatically created
    when we run startproject
    if we remember from our file listing we have a settings.py sitting under our myproject folder. so let's check what's in that file.
    this file is quite long, and was automatically created for us.
    but one of the first things we see is quickstart development settings. unsuitable for production and these warnings just keep going
    going down the page, see security warning after security warning these settings all work for a local development environment, but it's completely
    understandable that all these warnings and stuff make it seem just so... inaccessible
    to a beginning developer who just wants to share their cool blog with their friends.
    this lack of knowledge is because they are a beginner developer, not a server
    operations person, or a system administrator. it's a completely different engineering
    specialisation.
    this is the entire reason for this talk
    to demystify deployment.
    oh look, there's that debug setting. it is true by default. this makes sense. while we're here, we also saw that db.sqlite3 file appear after running our migrations.
    If we scroll down the settings file we can see that that string is defined here as part of our DATABASES definition.
    so what is this sqlite thing? To the Docs! o/

    https://docs.djangoproject.com/en/3.2/ref/databases/#sqlite-notes
    SQLite provides an excellent development alternative... for applications that are predominantly read-only or require
    a smaller installation footprint.
    --
    Again, that word development.
    django has a great local development story you saw just how quickly we could get a django site working on our local machine.
    but to get our site to production is complex
    as we've discovered together, you need to provide you own production ready
    webserver, database, and static host.
    And if you aren't familiar with the production grade offerings out there, you'll end up
    confused
    or worse, using runserver in production
    which... you should not do. Good work!
    Oh dear, I enabled audience participation. Yes? What's that I hear from the back of
    the room, Do you have a question? ... Flask isn't this hard.
    ... not exactly a question, but let's talk about that.
    flask is simpler in production.
    a lot of deployment tutorials you'll see that say "How to Deploy Python in Production!"
    often use Flask as the target.
    "Just copy your code and run! Easy!"
    Well yes, that's true. You can get a out-the-box Hello World application running quite
    easily on many different platforms using flask.
    the issue that django has is one of
    state.
    django is a stateful application. Full of state. That is, it contains information and data
    that we really want to keep a hold of.
    the absolute bare bones django app we just ran comes with the django admin. the
    django admin requires a database. therefore, django requires a database. therefore
    django requires state.
    a Flask app that prints helloworld doesn't need a database.
    Yes you can absolutely deploy flask with a database, and django without a database,
    but any application that has state is going to be complex to deploy.
    Let me share another gem from the Django documentation.

    https://docs.djangoproject.com/en/3.2/howto/static-files/#serving-static-files-during-development
    Earlier I shared this section on serving static files.
    This Deploying static files link goes to

    https://docs.djangoproject.com/en/3.2/howto/static-files/deployment/
    a page full of strategies for how to deploy static files.
    But it also has this line As with all deployment tasks, the devil’s in the details. Every production setup will be
    a bit different.
    _EVERY_ production setup will be a bit different.
    This line has been in the docs for over a decade now, but it still holds true.
    Every production setup will be a bit different.
    Most every deployment talk you'll see will include a line like "It depends" when telling
    you the ""one true way"" to do deployments
    Sidenote, there is a talk called "The one true way to do django deployments". It's quite
    good.
    But that's because it's true. It does depend.
    But I want to use a different line.
    I'm a sysadmin. I'm not your sysadmin.
    I'm not the one who will be making sure your django site stays online.
    So any recommendations I make in this talk are just that. Recommendations based
    on my experience.
    It's up to you and your team to understand what you're doing, because you're the
    ones that'll have to maintain it, and fix it when it's broken..
    That being said:
    to be able to deploy to production, we need to upgrade the components we are using
    on our local machine to something more production ready
    we need a production grade webserver
    a production grade database
    and production grade static
    Django has support for a number of the most common of these, which will *really*
    help us here. That means we can move out of the scope of Django specific things,
    and find solutions that work for the generic use case, as that has more options, and
    more support.
    For instance, Flask
    I've said before that Flask is simpler in production.
    This is because while you may not need a database or static for a flask app, you will
    need a webserver. and the reason you can get flask up and running super easily is that hosting a site on
    a webserver is simpler than it's ever been
    Let's take a look at the Flask docs!

    https://flask.palletsprojects.com/en/2.0.x/deploying/index.html
    Yes, I'm showing you the Flask _docs_ at a DjangoCon. Fight me in the hallway.
    The Flask documentation don't need me to do the highlighting here, because it's
    already in bold: Flask's built-in server is not suitable for production.
    They also offer a list of hosting options.
    All these links go out to the hosting vendor's documentation, and show step by step
    ways to "just copy your code and host it". This isn't a complete list, of course.
    The host you use is going to be based on a few factors: what platform you already
    know, what platform your team already knows how to support, or just what platform
    you're interested in learning. It's your choice.
    There are also going to be different options for the same platform.
    Some of them provide a complete and ready to go python environment for you, and
    you just have to supply your code, others you have to configure and supply more.
    A lot of them cover basic load balancing for you, by "autoscaling"
    They'll provide HTTPS by default by handling SSL certification for you
    You can of course do this yourself, but if it's just you, your friend, and your blog, you
    don't have to.
    Hosting options are always evolving, and the best practice changes over time. Django
    doesn't list them because they are so in flux. This is why you need to understand
    where django let's go of your hand and says "Off you go, have fun storming the
    castle".
    Many hosting providers will have Django tutorials much like they do for Flask. I should
    know, I wrote a few of them. That'll get you to the how, as of now.We'll continue here with the "what" though.
    These Flask docs go onto mention this WSGI thing.

    Wait, we saw that earlier... waaay back in the beginning in the Django runserver docs

    https://docs.djangoproject.com/en/3.2/ref/django-admin/#runserver
    Sitting in there, there was the line "This server uses the WSGI application object specified by the WSGI_APPLICATION setting."

    Wait, that was in our settings file!
    Just above databases, we saw WSGI_APPLICATION = 'myproject.wsgi.application' wait, we saw a myproject.wsgi earlier!
    In our file listing we have a myproject/wsgi.py file. so let's open that up.
    This is the entire file.
    most of it is comments, but we can see that It exposes the WSGI callable as a module-level variable named ``application``.
    Ooh, a docs link! Let's check that

    https://docs.djangoproject.com/en/3.2/howto/deployment/wsgi/
    These docs say that Django’s primary deployment platform is WSGI, the Python standard for web servers
    and applications.
    Well what's what then?
    TO THE PYTHON DOCS

    https://www.python.org/dev/peps/pep-0333/
    This is Python Enhancement Proposal 333 (or PEP 3333) for the Python Web Server
    Gateway Interface
    The abstract is pretty self explanatory: "This document specifies a proposed standard interface between web servers and
    Python web applications or frameworks, to promote web application portability across
    a variety of web servers."
    it's this portability that we're taking advantage of. All those potential hosting options from earlier, they either directly provide, or allow
    you to easily provide, a WSGI server to host your web application. This is why hosting
    is so easy now days
    You could create a virtual machine and do this all manually, continuing your firewall
    and routing, but there are places that'll say "Give us your code, here's the URL for
    your site have fun". They may things so much easier, because this is a problem all
    web developers have. They wanna host a web. And so, they provide.
    But it's not just WSGI now.

    https://asgi.readthedocs.io
    There is also ASGI an Asynchronous server gateway interface
    You may have noticed an ASGI.py file in the project template from earlier.
    For right now, unless your blog does something fancy with ASGI, you don't have to
    worry about this. Support for ASGI is still increasing, but unless you're doing async
    stuff, just stay with WSGI.
    So we've established you to have a WSGI server on your platform host of choice.
    There are many, but the one I use is gunicorn. It's been listed on the Flask and
    Django pages we've seen so far.

    https://docs.gunicorn.org/en/latest/run.html
    I use guniron
    It is functionally very similar to runserver's main function -- it serves websites -- but
    importantly, it is a webserver that has been security reviewed and is suitable for
    production.
    You install it as a python package, then you can run the gunicorn command,
    specifying your WSGI app
    So we can do just that
    so in the terminal pip install gunicorn which pip does
    then we run gunicorn myproject.wsgi:application
    and run our web server
    and look, it's running just at the same place it was before for runserver
    Rocketship! But...
    our admin is 90's.
    And we know from our preivous pokering that's because of static issues.
    Indeed, looking back at our terminal, all our assets aren't found.
    this is because runserver handled static for us, which gunicorn doesn't do.
    so we can use gunicorn for our webserver, but it doesn'thandle the static, that's still
    another component.
    that's okay. We want to choose the best available options out there.
    as for hosting gunicorn, *How* you deploy is going to depend on your host of choice,
    check their documenation.
    This is a "what" talk, not a "how" talk. "How" changes over time. "what" is pretty
    consistent.
    But since you've now got our webserver, we now need to create and connect our
    stateful services. Our database, and our static.
    Your host will probably have these offerings. Starting with the database.
    Unless you have experience in another database that django supports, or a team that
    knows how to use another database I suggest using postgres.
    Django supports many databases, but PostgreSQL has the most support.
    So says me and the docs

    https://docs.djangoproject.com/en/3.2/ref/contrib/postgres/
    From the reference for the postgres contrib module, in a high level note right at the
    top of the page. "Django is, and will continue to be, a database-agnostic web framework. We would
    encourage those writing reusable applications for the Django community to write
    database-agnostic code where practical. However, we recognize that real world
    projects written using Django need not be database-agnostic."
    "Django provides support for a number of data types which will only work with
    PostgreSQL.
    There is no fundamental reason why (for example) a contrib.mysql module does not
    exist,
    except that PostgreSQL has the richest feature set of the supported databases so its
    users have the most to gain."
    A lot of people put a lot of work into django, and supporting postgres in particular. If
    you'd like to get more support for your preferred database, go for it! We'd love to have
    more support for these things.But if you want an answer right now, use Postgres.
    But if you want an answer right now, use Postgres.
    Our original settings.py file, it's engine is the sqlite3 version. So what we need to do is
    replace that with postgres
    When we setup our postgres server on our host of choice, we'll be provided with
    information about to connect to it.
    This will include our host and port that the server can be accessed from, and there will
    be a database on that server that is ours.
    But we probably want to make sure that this data is secure, so we'll need to specify a
    username and password
    but i really don't ever want see these appear in plain text in your settings.py.
    what you should be doing is having these values stored securely else. There are
    choices for this.
    your host may allow you to set encrypted environment variables, you can then
    configure your database settings to pull these values from your environment.
    these are a lot of values to set.
    one thing I like to do is instead define an environment file.

    https://django-environ.readthedocs.io
    I use a package called Django-environ
    It's a package that allows you to use the Twleve Factor Methodoloy to configure our
    application with environment variables.
    What this code snippet does, it that it'll load your environment, then read in the file.
    So instead of using os.environ.get everywhere, we can just use "env".
    But Katie, what is this "twelve factor methodology"?

    https://www.12factor.net/
    the twelve factors are a methodology to building web apps that make them easier to
    deploy, scale, and maintain.
    it was developed by people who made Heroku, and talks about the lessons they
    learnt there.
    This isn't a talk about 12 factor apps, but I will say that the method we're using to
    deploy our application is already covering a bunch of them, or we easily can.
    Dependencies: well, we if we throw our dependencies into a requirements file, then
    we're good there.
    Config: literally what we're doing now.
    Backing services: this is what our databases and static are. They're attaching to our
    process.
    And portbinding, that's our 8000 port right there.
    If you're using a hosted service, you're going to get processes, concurrency,
    disposability and logging for free.
    These are the concepts those providers work on.
    Instead of giving you an entire machine, they'll give you the ability to use part of one.These factors are what help them scale.
    Of the ones we haven't covered,
    you can think about using github to store your codebase,
    using your hosts build process to automatically update when you push code there,
    and replicate that setup if you want a testing environment.
    And then your admin process, that's your migrate command.
    There are other great talks that step into this methodology more, both in general and
    for django.
    But should get back to implementing django-envoron
    These factors are what help them scale.
    Of the ones we haven't covered,
    you can think about using github to store your codebase,
    using your hosts build process to automatically update when you push code there,
    and replicate that setup if you want a testing environment.
    And then your admin process, that's your migrate command.
    There are other great talks that step into this methodology more, both in general and
    for django.
    But should get back to implementing django-envoron
    Adding a little bit of code to the top of my file I can import and setup django-environ,
    then set all my configs in either my environment variables or environment file and my
    settings.py will pick them up.
    checkout the django-environ documentation for some more ways you can use
    package. the database url feature is quite neat.
    so, we need postgres for feature coverage, and django-environ for configuration
    portability.
    Now we have a database, we need to ensure we're actually applying the migrations to
    it.
    That's the admin process we mentioned serlier. Running manage.py migrate
    Later you're going to have to consider when you'll run these migrations, and if you'll
    automatically make them, or do any customisations.
    Markus Holtermann is a subject matter expert on this, check out some of his talks and
    blog posts.
    Back to the "what".
    so for our database, we've now postgres with a side of django-environ
    now for static.
    again, your host of choice is going to have a backing service solution for this. It'll be
    an "object store" of some sort. These are some of the oldest parts of cloud computing,
    and are exceptionally robust now.
    There's also a helper package for django for integrating these options into your
    application

    https://django-storages.readthedocs.io/en/latest/
    Django-storages is a collection of custom storage backends for django.
    These hook in pretty deep into django, and allow you to do all sorts of things with
    whatever static host you want to use.
    The documentation will guide you through the config you need for each type.
    And you'll be adding a few things to the end of settings.py. Somewhere around here where your STATIC_URL setting is.
    Now we have a place to put ourstatic, how do we put in in there? How do we collect
    out static? We know we're missing that step as our admin looked web 1.0.
    But you see, that's another of those "HOW" words. The command you need is called... collectstatic.
    But the HOW for this is a completely separate talk in and of itself.
    A good example of this talk is Jacob Kaplan-Moss' talk "Assets in Django". He goes
    through different strategies for static, including a deeper dive into what collectstatic
    does.
    so here we have it.
    you use gunicorn as a webserver on your host of choice
    you use a hosted postgresql database and django-environ for settings management,
    applying said migrations with migrate
    and hosted static with django-storages, and applying with collectstatic.
    This is my recommendation of what deployment is in 2021
    people in 2032, that disclaimer is also for you!
    Things may have changed! Giving this kind of talk is going to be an artefact of it's
    time!
    Please check the current django documentation for up to date recommendations! Try
    creating a django project from the template and see what comments are in the
    settings file! I've shown you how to do this!
    There are a lot of "how" and "when" things that I just didn't cover today
    There's a LOT to this space, and it's absolutely ok if you don't understand or even
    know about all of them.
    I don't.
    Each one of these are it's own talk, or at worst, it's own **conference**.
    The specialisation of operations engineering is complex and opinionated and ever
    evolving
    But what we're done in the last little bit is step through django gives you,
    helped you parse some of it's base documentation, understood some the concepts
    and made some suggestions,
    and given you helpers to get you deploying.
    Thank you so much for watching!
    I'm on glasnt on all the platforms, whatever those are in 2032, and if you want to chat,
    here's my email too.
    If you or your CTO wants to talk about serverless scalability and django hosting,
    Here's my work email.
    Have a brilliant rest of your day!