We removed our free Sandbox April 25th.
You can read more on our blog.
Sometimes, you need to run a program for a longer time than a single HTTP request. Use cases include:
- CPU-intensive jobs (e.g. video transcoding);
- running some code at a specified time, or on regular intervals;
- background activity (e.g. crawling 3rd party service to update you database);
- run a specific web server, like Node.js or Tornado;
- and much more!
DotCloud provides “worker services” dedicated to those tasks. There is a different service for each language: Ruby worker, PHP & PHP Worker, Perl Worker, Python worker, Node.Js... The only difference between them is the set of pre-installed packages, and dependencies handling: python-worker supports requirements.txt, while —for instance— ruby-worker supports Gemfile.
All worker services rely on Supervisor to start and monitor your processes. Supervisor will start defined programs automatically, and will restart them automatically if they crash or exit. If you just need to run a program at a specified interval, you can also use a crontab and ignore Supervisor.
You can also use a “non-worker” service to run some background jobs. More specifically, all services feature crontab, allowing you to run Periodic Tasks. So if you want to run a daily Python script just need to update, e.g., a stock ticker in your database, you don’t have to dedicate a python-worker to this task: you can setup the crontab in the same python service that you use for your web application. However, you should be aware that when you scale your application, the cron tasks will be scheduled in all scaled instances – which is probably not what you need! So in many cases, it will still be better to use a separate service.
Similarly, a lot of (non-worker) services already run Supervisor, so you can run additional background jobs in those services. Then again, remember that those background jobs will run in multiple instances if you scale your application. Moreover, if you add background jobs to your web service, it will get less resources to serve pages, and your performance will take a significant hit.
Technically, if you really want to know – there is almost no difference between worker and non-worker services. For instance, the python-worker service is basically the python service without Nginx (HTTP server) and uWSGI (Python web workers) running. Both can optionally run background processes using Supervisor.
Let’s assume that you are building a DotCloud application called “ramen”. For the sake of simplicity, we will also assume that everything related to this application is located in a directory called “ramen-on-dotcloud”.
Let’s setup our environment:
$ dotcloud create ramen Created application "ramen" using the flavor "sandbox" (Use for development, free and unlimited apps. DO NOT use for production.) $ mkdir ramen-on-dotcloud
A DotCloud application is described by a Build Files, which is a simple YAML file named “dotcloud.yml” located in our “ramen-on-dotcloud” directory. To add a new service to our app, we just add the following lines to “ramen-on-dotcloud/dotcloud.yml”:
worker: type: ruby-worker
With your DotCloud build file in hands you are ready to define your daemons.
As in the Ruby service you can specify your external dependencies in a Gemfile. For example:
source :rubygems gem 'redis'
Unsupported gems and how to exclude them¶
Some Ruby gems are not suited to environments like dotCloud, and should not be installed. Those are gems like:
- linecache19 (which is only used for debugging purposes, and should not be present on a production environment);
Actually, if you try to install them anyway, you will generally get a “permission denied” without much dire consequences.
To exclude those gems without juggling with multiple Gemfiles, you can put them in groups, and then tell bundler to exclude them when deploying on dotCloud.
You will need to edit your Build File. Assuming that your Ruby service is named “www”, modify its section like the following example:
www: type: ruby exclude_bundler_groups: - development
Then, in your Gemfile, regroup the unwanted gems in a group:
group :development do gem "ruby-debug19" gem "my-funky-gem" end
If you want to learn more about it, feel free to read:
Running cron jobs¶
We use the Ruby Version Manager (RVM) to manage the different versions of Ruby that you can use on the ruby service. This causes some problems when you want to run cron jobs. In order to use the correct version of Ruby, you need to first tell cron what Ruby version to use. There are a few different ways of doing this, but we will recommend the most straightforward way here.
To use an RVM Ruby from cron, we need to invoke our own bash session, like so:
MAILTOemail@example.com' # cron job that runs every minute. # m h dom mon dow command * * * * * /bin/bash -l -c '~/current/cron_command.rb'
/bin/bash -l -c Explained¶
We are wrapping our command in a bash session. The -c option tells bash a command follows. The -l option tells bash to load the entire environment. This option is crucial. Leave out the -l and you won’t load the correct version of Ruby.
The wrapper is necessary because cron runs a very limited environment. RVM relies on environment variables to find Ruby, gems and gem executables.
bash -l brings all this into cron’s environment, so RVM just works. You don’t have to worry about PATH, gem executables, etc. They all work just like they do in your interactive bash sessions.
Important note: in most cases, you’ll want to use single quotes for the command you pass to bash.
For more information about running cron jobs on dotCloud, checkout the periodic tasks guide.
Use a specific version of Ruby¶
If you don’t like the default version of Ruby used by your ruby or ruby-worker service, you can switch to another one. Here is how.
Edit your dotcloud.yml file to include a “ruby-version” parameter in the config section:
hello: type: ruby config: ruby-version: X
X can be one of the following:
- ree – for Ruby Enterprise Edition (1.8 series);
- 1.9.2 – for “plain” Ruby 1.9.2 (the default);
- 1.9.3 – for “plain” Ruby 1.9.3.
If you ask for a non-existent version, the deployment will fail. So please don’t try it!
However, if you specify a non-existent parameter (for example, if you use ruby_version instead of ruby-version), the parameter will be silently ignored, for back-and-forth compatibility reasons.
Like all the “config” parameters, the ruby-version cannot be changed on a live service. You have to destroy the service and push it again with a new configuration in order to change ruby-version.