We removed our free Sandbox April 25th.
You can read more on our blog.
Sometimes, you need to run a program for a longer time than a single HTTP request. Use cases include:
- CPU-intensive jobs (e.g. video transcoding);
- running some code at a specified time, or on regular intervals;
- background activity (e.g. crawling 3rd party service to update you database);
- run a specific web server, like Node.js or Tornado;
- and much more!
DotCloud provides “worker services” dedicated to those tasks. There is a different service for each language: Ruby worker, PHP & PHP Worker, Perl Worker, Python worker, Node.Js... The only difference between them is the set of pre-installed packages, and dependencies handling: python-worker supports requirements.txt, while —for instance— ruby-worker supports Gemfile.
All worker services rely on Supervisor to start and monitor your processes. Supervisor will start defined programs automatically, and will restart them automatically if they crash or exit. If you just need to run a program at a specified interval, you can also use a crontab and ignore Supervisor.
You can also use a “non-worker” service to run some background jobs. More specifically, all services feature crontab, allowing you to run Periodic Tasks. So if you want to run a daily Python script just need to update, e.g., a stock ticker in your database, you don’t have to dedicate a python-worker to this task: you can setup the crontab in the same python service that you use for your web application. However, you should be aware that when you scale your application, the cron tasks will be scheduled in all scaled instances – which is probably not what you need! So in many cases, it will still be better to use a separate service.
Similarly, a lot of (non-worker) services already run Supervisor, so you can run additional background jobs in those services. Then again, remember that those background jobs will run in multiple instances if you scale your application. Moreover, if you add background jobs to your web service, it will get less resources to serve pages, and your performance will take a significant hit.
Technically, if you really want to know – there is almost no difference between worker and non-worker services. For instance, the python-worker service is basically the python service without Nginx (HTTP server) and uWSGI (Python web workers) running. Both can optionally run background processes using Supervisor.
Let’s assume that you are building a DotCloud application called “ramen”. For the sake of simplicity, we will also assume that everything related to this application is located in a directory called “ramen-on-dotcloud”.
Let’s setup our environment:
$ dotcloud create ramen Created application "ramen" using the flavor "sandbox" (Use for development, free and unlimited apps. DO NOT use for production.) $ mkdir ramen-on-dotcloud
A DotCloud application is described by a Build Files, which is a simple YAML file named “dotcloud.yml” located in our “ramen-on-dotcloud” directory. To add a new service to our app, we just add the following lines to “ramen-on-dotcloud/dotcloud.yml”:
worker: type: perl-worker
With your DotCloud build file in hands you are ready to define your daemons.
The perl-worker service supports three different branches of Perl (5.12.x, 5.14.x, 5.16.x), it will default to Perl 5.12.x unless you specify otherwise. The current versions of each Perl branch are listed in the table below. Pick the branch that works best for you.
* Perl 5.12.x is the default
To configure your service to use a version of Perl other than 5.12.x, you will set the perl_version config option in your dotcloud.yml
For example, this is what you would have if you wanted Perl 5.14.x:
www: type: perl-worker approot: helloperl config: perl_version: v5.14.x
We use PerlBrew to manage the different Perl versions. This causes problems when you want to run cron jobs because cron wants to use the system version of Perl, instead of the one you selected.
In order to get your cron jobs to use the correct version of Perl, you will need to use the following fully qualified path:
We realize this isn’t ideal, and we are working on a better solution, but in the meantime this should get you working.
Here is an example crontab file:
MAILTOemail@example.com' # cron job that runs every minute. # m h dom mon dow command * * * * * /opt/perl5/perls/current/bin/perl ~/current/cron_command.py
For more information about running cron jobs on dotCloud, checkout the periodic tasks guide.