:title: Django Python Tutorial
:description: How to build a Django project from scratch on dotCloud, storing data in a PostgreSQL database.
:keywords: dotCloud, tutorial, documentation, python, Django, PostgreSQL
Django
======
.. include:: ../../dotcloud2note.inc
According to the `Django project homepage
Hello World. The current day and time is {{current_time|date:"DATETIME_FORMAT"}}.
Before we can use it, we need to add the view to our ``urls.py`` file so that we can link our url to our view. This is what it looks like when we are done. If someone goes to the root of the domain it will show the result of ``hello_view``. ``hellodjango/urls.py``: .. code-block:: python from helloapp.views import hello_view urlpatterns = patterns('', url(r'^$', view=hello_view, name='hello_page'), ) wsgi.py ------- The ``wsgi.py`` file will bridge between the ``python`` service and our Django app. We need three things here: * append our hellodjango directory to our python sys path. * inject the ``DJANGO_SETTINGS_MODULE`` variable into the environment, pointing to our project settings module; * setup the ``application`` callable, since that is what the dotCloud service will be looking for. ``wsgi.py``: .. code-block:: python import os import sys sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__),'hellodjango'))) os.environ['DJANGO_SETTINGS_MODULE'] = 'hellodjango.settings' import django.core.handlers.wsgi application = django.core.handlers.wsgi.WSGIHandler() .. note:: Django 1.4 now comes with a ``wsgi.py`` file under your project. This is a different one that needs to be at the root of your project, next to your ``dotcloud.yml`` file. We can now create our application with the :doc:`flavor ` of our choice and push our application, by running the following commands: .. code-block:: bash dotcloud create djangotutorial dotcloud push (you can of course use any application name you like). A Python service will be created, the code will be deployed, and the URL of the service will be shown at the end of the build. If you go to this URL, you will see the plain and boring Django page, typical of the "just started" project. Add PostgreSQL Database ----------------------- It's time to add a database to our project! The first step is to tell dotCloud that we want a PostgreSQL server to be added to our application. Just edit the Build File again. ``dotcloud.yml``: .. code-block:: yaml www: type: python db: type: postgresql Note that we called our database ``db``, but it could have been anything else, really. If you ``dotcloud push`` again, you will see that the database service will be created (dotCloud will notice that we added a section to the Build File). Creating your Database ---------------------- When you create a new database service, you will be given a raw database server with full root access. It will be up to you to create the database and users that you want for your application. Normally you would need to add the database service to your dotcloud.yml file, push the service. Let the service start up and then login to create your database. Here is what that looks like, if you had a Postgresql service. .. code-block:: bash # connect to dotcloud mysql server instance dotcloud run db -- psql You can then type the following to create a database, user, and grant the user access to the newly created database:: create database happydb; create user db_username with password 'strong_password'; grant all privileges on database happydb to db_username; \q Here is what that looks like, if you had a MySQL service. .. code-block:: bash # connect to dotcloud mysql server instance dotcloud run db -- mysql You can then type the following to create a database, user, and grant the user access to the newly created database:: create database happydb; create user 'db_username' identified by 'strong_password'; grant all on happydb.* to 'db_username'@'%'; flush privileges; exit; That is the long way, now I'll show you the short way. You can create a script that will run during the ``postinstall`` process, and see if the database has been created or not, and if not, it will create it. All other times, it does nothing. The script will parse the information from your ``settings.py`` DATABASES values. This script only works with MySQL and PostgreSQL, but can be easily changed to support other databases. Here is what ``createdb.py`` looks like: .. code-block:: python import os import time import MySQLdb import psycopg2 import _mysql_exceptions from wsgi import * def create_dbs(): deadline = time.time() + 60 while time.time() < deadline: try: print("create_dbs: let's go.") django_settings = __import__(os.environ['DJANGO_SETTINGS_MODULE'], fromlist='DATABASES') print("create_dbs: got settings.") databases = django_settings.DATABASES for name, db in databases.iteritems(): host = db['HOST'] user = db['USER'] password = db['PASSWORD'] port = db['PORT'] db_name = db['NAME'] db_type = db['ENGINE'] # see if it is mysql if db_type.endswith('mysql'): print 'creating database %s on %s' % (db_name, host) db = MySQLdb.connect(user=user, passwd=password, host=host, port=port) cur = db.cursor() print("Check if database is already there.") cur.execute("""SELECT SCHEMA_NAME FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME = %s""", (db_name,)) results = cur.fetchone() if not results: print("Database %s doesn't exist, lets create it." % db_name) sql = """CREATE DATABASE IF NOT EXISTS %s """ % (db_name,) print("> %s" % sql) cur.execute(sql) print(".....") else: print("database already exists, moving on to next step.") exit(0) # see if it is postgresql elif db_type.endswith('postgresql_psycopg2'): print 'creating database %s on %s' % (db_name, host) con = psycopg2.connect(host=host, user=user, password=password, port=port, database='postgres') con.set_isolation_level(0) cur = con.cursor() try: cur.execute('CREATE DATABASE %s' % db_name) except psycopg2.ProgrammingError as detail: print detail print 'moving right along...' exit(0) else: print("ERROR: {0} is not supported by this script, you will need to create your database by hand.".format(db_type)) exit(1) except psycopg2.OperationalError: print "Could not connect to database. Waiting a little bit." time.sleep(10) except _mysql_exceptions.OperationalError: print "Could not connect to database. Waiting a little bit." time.sleep(10) print 'Could not connect to database after 1 minutes. Something is wrong.' exit(1) if __name__ == '__main__': import sys print("create_dbs start") create_dbs() print("create_dbs all done") Add Database Credentials to settings.py --------------------------------------- Now, we need to edit ``settings.py`` to specify the host, port, user, and password to connect to our database. When you deploy your application, these parameters are stored in the dotCloud Environment File. This allows you to repeat the deployment of your application (e.g. for staging purposes) without having to manually copy-paste the parameters into your settings each time. If you don't want to use the Environment File, you can retrieve the same information with ``dotcloud info db``. The Environment File is a JSON file holding a lot of information about our stack. It contains (among other things) our database connection parameters. We will load this file, and use those parameters in Django's settings. See :doc:`dotCloud environment guide ` for more details about the Environment File. ``hellodjango/settings.py``: .. code-block:: python # Django settings for hellodjango project. import json with open('/home/dotcloud/environment.json') as f: env = json.load(f) DEBUG = True TEMPLATE_DEBUG = DEBUG # … DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'happydb', 'USER': env['DOTCLOUD_DB_SQL_LOGIN'], 'PASSWORD': env['DOTCLOUD_DB_SQL_PASSWORD'], 'HOST': env['DOTCLOUD_DB_SQL_HOST'], 'PORT': int(env['DOTCLOUD_DB_SQL_PORT']), } } # … .. note:: We decided to use the name ``happydb`` here. You can call it what ever you want. If you use the ``createdb.py`` script, it will look for this database name, and if it isn't created, it will create it for you. If you don't use the ``createdb.py`` script you will need to create the database yourself. Django Admin Site ----------------- We will now activate the Django administration application. Nothing is specific to dotCloud here: we just uncomment the relevant lines of code in ``settings.py`` and ``urls.py``. ``hellodjango/settings.py``: .. code-block:: python # … INSTALLED_APPS = ( 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', # Uncomment the next line to enable the admin: 'django.contrib.admin', # Uncomment the next line to enable admin documentation: # 'django.contrib.admindocs', 'helloapp', ) # … ``hellodjango/urls.py``: .. code-block:: python from django.conf.urls.defaults import patterns, include, url # import the view from helloapp from helloapp.views import hello_view # Uncomment the next two lines to enable the admin: from django.contrib import admin admin.autodiscover() urlpatterns = patterns('', url(r'^$', view=hello_view, name='hello_page'), url(r'^admin/', include(admin.site.urls)), ) If we push our application now, we can go to the ``/admin`` URL, but since we didn't call ``syncdb`` yet, the database structure doesn't exist, and Django will refuse to do anything useful for us. Automatically Call syncdb ------------------------- To make sure that the database structure is properly created, we want to call ``manage.py syncdb`` automatically each time we push our code. On the first push, this will create the Django tables; later, it will create new tables that might be required by new models you will define. To make that happen, we create a ``postinstall`` script. It is called automatically at the end of each push operation. ``postinstall``: .. code-block:: sh #!/bin/sh python createdb.py python hellodjango/manage.py syncdb --noinput A few remarks: * this is a shell script (hence the ``#!/bin/sh`` shebang at the beginning), but you can also use a Python script if you like; * The ``createdb.py`` script is first so that it creates the database before we try to ``syncdb`` * by default, ``syncdb`` will interactively prompt you to create a Django superuser in the database, but we cannot interact with the terminal during the push process, so we disable this thanks to ``--noinput``. * make sure that you have set the correct execute permissions on all files that you plan to run with your ``postinstall`` script including the ``postinstall`` script. Database Migrations ------------------- This project is too small, and doesn't require database migrations, but if you are using a database migration tool like ``Django South`` you can add the following to your ``postinstall`` script to run your migrations at the correct time. For more information about ``Django south`` visit this website: http://south.aeracode.org .. code-block:: sh python hellodjango/manage.py migrate If you push the code at this point, hitting the ``/admin`` URL will display the login form, but we don't have a valid user yet, and the login form won't have the usual Django CSS since we didn't take care about the static assets yet. Create Django Superuser ----------------------- Since the ``syncdb`` command was run non-interactively, it did not prompt us to create a superuser, and therefore, we don't have a user to login. To create an admin user automatically, we will write a simple Python script that will use Django's environment, load the authentication models, create a ``User`` object, set a password, and give him superuser privileges. The user login will be ``admin``, and its password will be ``password``. Note that if the user already exists, it won't be touched. However, if it does not exist, it will be re-created. If you don't like this ``admin`` user, you should not delete it (it would be re-added each time you push your code) but just remove its privileges and reset its password, for instance. ``mkadmin.py``: .. code-block:: python #!/usr/bin/env python from wsgi import * from django.contrib.auth.models import User u, created = User.objects.get_or_create(username='admin') if created: u.set_password('password') u.is_superuser = True u.is_staff = True u.save() ``postinstall``: .. code-block:: sh #!/bin/sh python createdb.py python hellodjango/manage.py syncdb --noinput python mkadmin.py At this point, if we push the code, we will be able to login, but we still lack the CSS that will make the admin site look nicer. Handle Static and Media Assets ------------------------------ We still lack the CSS required to make our admin interface look nice. We need to do three things here. First, we will edit ``settings.py`` to specify ``STATIC_ROOT``, ``STATIC_URL``, ``MEDIA_ROOT``, and ``MEDIA_URL``. ``MEDIA_ROOT`` will point to ``/home/dotcloud/data``. By convention, the ``data`` directory will persist across pushes. This is important: You don't want to store media (user uploaded files...) in ``current`` or ``code``, because those directories are wiped out at each push. We decided to point ``STATIC_ROOT`` to ``/home/dotcloud/volatile``, since the static files are “generated” at each push. We could have put them in ``current`` but to avoid conflict and confusion we chose a separate directory. Your application will need a place to store your static files, so we are going to put them in a directory called ``static`` at the same level as ``hellodjango``. In order for Django to know that there is static media there, we need to tell it, by adding that directory to ``STATICFILES_DIRS`` in ``settings.py``. ``hellodjango/settings.py``: .. code-block:: python # … # Absolute filesystem path to the directory that will hold user-uploaded files. # Example: "/home/media/media.lawrence.com/media/" MEDIA_ROOT = '/home/dotcloud/data/media/' # URL that handles the media served from MEDIA_ROOT. Make sure to use a # trailing slash. # Examples: "http://media.lawrence.com/media/", "http://example.com/media/" MEDIA_URL = '/media/' # Absolute path to the directory static files should be collected to. # Don't put anything in this directory yourself; store your static files # in apps' "static/" subdirectories and in STATICFILES_DIRS. # Example: "/home/media/media.lawrence.com/static/" STATIC_ROOT = '/home/dotcloud/volatile/static/' # URL prefix for static files. # Example: "http://media.lawrence.com/static/" STATIC_URL = '/static/' # Additional locations of static files STATICFILES_DIRS = ( os.path.join(PROJECT_ROOT, 'static/'), ) # … The next step is to instruct Nginx to map ``/media`` to that directory in ``/home/dotcloud/data`` and ``/static`` to that directory in ``/home/dotcloud/volatile``. This is done through a Nginx configuration snippet. You can do many interesting things with custom Nginx configuration files; the :doc:`dotCloud nginx guide ` gives some details about that. ``nginx.conf``: .. code-block:: nginx location /media/ { root /home/dotcloud/data ; } location /static/ { root /home/dotcloud/volatile ; } .. warning:: The python service no longer sets up the ``/static/`` directory for us. You must define your static location in nginx.conf as of August 2012. If you want to store your static media in a different directory that is fine, you just need to make the appropriate changes in the django ``settings.py``. The last step is to add the ``collectstatic`` management command to our ``postinstall`` script. Before calling it, we create the required directories, just in case. ``postinstall``: .. code-block:: sh #!/bin/sh python createdb.py python hellodjango/manage.py syncdb --noinput python mkadmin.py mkdir -p /home/dotcloud/data/media /home/dotcloud/volatile/static python hellodjango/manage.py collectstatic --noinput After pushing this last round of modifications, the CSS for the admin site (and other static assets) will be found correctly, and we have a very basic (but functional) Django project to build on! .. note:: If you are using ``collectstatic`` make sure you have it enabled in your ``INSTALLED_APPS`` in ``settings.py``. Django Logging -------------- Setting up your application to use Django logging is pretty easy, you just need to make some minor changes to your ``settings.py``. This change below will put your Django logs in ``/var/log/supervisor/helloapp.log``. .. code-block:: python # A sample logging configuration. The only tangible logging # performed by this configuration is to send an email to # the site admins on every HTTP 500 error. # See http://docs.djangoproject.com/en/dev/topics/logging for # more details on how to customize your logging configuration. LOGGING = { 'version': 1, 'disable_existing_loggers': True, 'formatters': { 'verbose': { 'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s' }, 'simple': { 'format': '%(levelname)s %(message)s' }, }, 'handlers': { 'null': { 'level':'DEBUG', 'class':'django.utils.log.NullHandler', }, 'console': { 'level': 'DEBUG', 'class': 'logging.StreamHandler', 'formatter': 'verbose' }, 'log_file': { 'level': 'DEBUG', 'class': 'logging.handlers.RotatingFileHandler', 'formatter': 'verbose', 'filename': '/var/log/supervisor/helloapp.log', 'maxBytes': 1024*1024*25, # 25 MB 'backupCount': 5, }, 'mail_admins': { 'level': 'ERROR', 'class': 'django.utils.log.AdminEmailHandler' } }, 'loggers': { 'django': { 'handlers': ['console', 'log_file', 'mail_admins'], 'level': 'INFO', 'propagate': True, }, 'django.request': { 'handlers': ['console', 'log_file', 'mail_admins'], 'level': 'ERROR', 'propagate': False, }, 'django.db.backends': { 'handlers': ['console', 'log_file', 'mail_admins'], 'level': 'INFO', 'propagate': False, }, # Catch All Logger -- Captures any other logging '': { 'handlers': ['console', 'log_file', 'mail_admins'], 'level': 'INFO', 'propagate': True, } } } Logs ---- If you need to look at the logs to see how it is going you can do it two ways. The first way will tail your logs for you to your console. .. code-block:: sh # look at logs of your service, it will tail them to your console. ctrl-c to stop. dotcloud logs www Or login via ssh and look at your logs. .. code-block:: sh # Open up a shell dotcloud run www Here are the ones you most likely care about. .. code-block:: sh # nginx access and error logs. /var/log/nginx/