Tornado Web Server

Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed. By using non-blocking network I/O, Tornado can scale to tens of thousands of open connections, making it ideal for long polling, WebSockets, and other applications that require a long-lived connection to each user.

Hello, world

Here is a simple “Hello, world” example web app for Tornado:

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")

def make_app():
    return tornado.web.Application([
        (r"/", MainHandler),
    ])

if __name__ == "__main__":
    app = make_app()
    app.listen(8888)
    tornado.ioloop.IOLoop.current().start()

This example does not use any of Tornado’s asynchronous features; for that see this simple chat room.

Installation

Automatic installation:

pip install tornado

Tornado is listed in PyPI and can be installed with pip or easy_install. Note that the source distribution includes demo applications that are not present when Tornado is installed in this way, so you may wish to download a copy of the source tarball as well.

Manual installation: Download tornado-4.3.tar.gz:

tar xvzf tornado-4.3.tar.gz
cd tornado-4.3
python setup.py build
sudo python setup.py install

The Tornado source code is hosted on GitHub.

Prerequisites: Tornado 4.3 runs on Python 2.6, 2.7, and 3.2+ (support for Python 2.6 and 3.2 is deprecated and will be removed in the next release). For Python 2, version 2.7.9 or newer is strongly recommended for the improved SSL support. In addition to the requirements which will be installed automatically by pip or setup.py install, the following optional packages may be useful:

  • unittest2 is needed to run Tornado’s test suite on Python 2.6 (it is unnecessary on more recent versions of Python)
  • concurrent.futures is the recommended thread pool for use with Tornado and enables the use of ThreadedResolver. It is needed only on Python 2; Python 3 includes this package in the standard library.
  • pycurl is used by the optional tornado.curl_httpclient. Libcurl version 7.19.3.1 or higher is required; version 7.21.1 or higher is recommended.
  • Twisted may be used with the classes in tornado.platform.twisted.
  • pycares is an alternative non-blocking DNS resolver that can be used when threads are not appropriate.
  • Monotime adds support for a monotonic clock, which improves reliability in environments where clock adjustments are frequent. No longer needed in Python 3.3.

Platforms: Tornado should run on any Unix-like platform, although for the best performance and scalability only Linux (with epoll) and BSD (with kqueue) are recommended for production deployment (even though Mac OS X is derived from BSD and supports kqueue, its networking performance is generally poor so it is recommended only for development use). Tornado will also run on Windows, although this configuration is not officially supported and is recommended only for development use.

Documentation

This documentation is also available in PDF and Epub formats.

User’s guide

Introduction

Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed. By using non-blocking network I/O, Tornado can scale to tens of thousands of open connections, making it ideal for long polling, WebSockets, and other applications that require a long-lived connection to each user.

Tornado can be roughly divided into four major components:

  • A web framework (including RequestHandler which is subclassed to create web applications, and various supporting classes).
  • Client- and server-side implementions of HTTP (HTTPServer and AsyncHTTPClient).
  • An asynchronous networking library (IOLoop and IOStream), which serve as the building blocks for the HTTP components and can also be used to implement other protocols.
  • A coroutine library (tornado.gen) which allows asynchronous code to be written in a more straightforward way than chaining callbacks.

The Tornado web framework and HTTP server together offer a full-stack alternative to WSGI. While it is possible to use the Tornado web framework in a WSGI container (WSGIAdapter), or use the Tornado HTTP server as a container for other WSGI frameworks (WSGIContainer), each of these combinations has limitations and to take full advantage of Tornado you will need to use the Tornado’s web framework and HTTP server together.

Asynchronous and non-Blocking I/O

Real-time web features require a long-lived mostly-idle connection per user. In a traditional synchronous web server, this implies devoting one thread to each user, which can be very expensive.

To minimize the cost of concurrent connections, Tornado uses a single-threaded event loop. This means that all application code should aim to be asynchronous and non-blocking because only one operation can be active at a time.

The terms asynchronous and non-blocking are closely related and are often used interchangeably, but they are not quite the same thing.

Blocking

A function blocks when it waits for something to happen before returning. A function may block for many reasons: network I/O, disk I/O, mutexes, etc. In fact, every function blocks, at least a little bit, while it is running and using the CPU (for an extreme example that demonstrates why CPU blocking must be taken as seriously as other kinds of blocking, consider password hashing functions like bcrypt, which by design use hundreds of milliseconds of CPU time, far more than a typical network or disk access).

A function can be blocking in some respects and non-blocking in others. For example, tornado.httpclient in the default configuration blocks on DNS resolution but not on other network access (to mitigate this use ThreadedResolver or a tornado.curl_httpclient with a properly-configured build of libcurl). In the context of Tornado we generally talk about blocking in the context of network I/O, although all kinds of blocking are to be minimized.

Asynchronous

An asynchronous function returns before it is finished, and generally causes some work to happen in the background before triggering some future action in the application (as opposed to normal synchronous functions, which do everything they are going to do before returning). There are many styles of asynchronous interfaces:

  • Callback argument
  • Return a placeholder (Future, Promise, Deferred)
  • Deliver to a queue
  • Callback registry (e.g. POSIX signals)

Regardless of which type of interface is used, asynchronous functions by definition interact differently with their callers; there is no free way to make a synchronous function asynchronous in a way that is transparent to its callers (systems like gevent use lightweight threads to offer performance comparable to asynchronous systems, but they do not actually make things asynchronous).

Examples

Here is a sample synchronous function:

from tornado.httpclient import HTTPClient

def synchronous_fetch(url):
    http_client = HTTPClient()
    response = http_client.fetch(url)
    return response.body

And here is the same function rewritten to be asynchronous with a callback argument:

from tornado.httpclient import AsyncHTTPClient

def asynchronous_fetch(url, callback):
    http_client = AsyncHTTPClient()
    def handle_response(response):
        callback(response.body)
    http_client.fetch(url, callback=handle_response)

And again with a Future instead of a callback:

from tornado.concurrent import Future

def async_fetch_future(url):
    http_client = AsyncHTTPClient()
    my_future = Future()
    fetch_future = http_client.fetch(url)
    fetch_future.add_done_callback(
        lambda f: my_future.set_result(f.result()))
    return my_future

The raw Future version is more complex, but Futures are nonetheless recommended practice in Tornado because they have two major advantages. Error handling is more consistent since the Future.result method can simply raise an exception (as opposed to the ad-hoc error handling common in callback-oriented interfaces), and Futures lend themselves well to use with coroutines. Coroutines will be discussed in depth in the next section of this guide. Here is the coroutine version of our sample function, which is very similar to the original synchronous version:

from tornado import gen

@gen.coroutine
def fetch_coroutine(url):
    http_client = AsyncHTTPClient()
    response = yield http_client.fetch(url)
    raise gen.Return(response.body)

The statement raise gen.Return(response.body) is an artifact of Python 2 (and 3.2), in which generators aren’t allowed to return values. To overcome this, Tornado coroutines raise a special kind of exception called a Return. The coroutine catches this exception and treats it like a returned value. In Python 3.3 and later, a return response.body achieves the same result.

Coroutines

Coroutines are the recommended way to write asynchronous code in Tornado. Coroutines use the Python yield keyword to suspend and resume execution instead of a chain of callbacks (cooperative lightweight threads as seen in frameworks like gevent are sometimes called coroutines as well, but in Tornado all coroutines use explicit context switches and are called as asynchronous functions).

Coroutines are almost as simple as synchronous code, but without the expense of a thread. They also make concurrency easier to reason about by reducing the number of places where a context switch can happen.

Example:

from tornado import gen

@gen.coroutine
def fetch_coroutine(url):
    http_client = AsyncHTTPClient()
    response = yield http_client.fetch(url)
    # In Python versions prior to 3.3, returning a value from
    # a generator is not allowed and you must use
    #   raise gen.Return(response.body)
    # instead.
    return response.body
Python 3.5: async and await

Python 3.5 introduces the async and await keywords (functions using these keywords are also called “native coroutines”). Starting in Tornado 4.3, you can use them in place of yield-based coroutines. Simply use async def foo() in place of a function definition with the @gen.coroutine decorator, and await in place of yield. The rest of this document still uses the yield style for compatibility with older versions of Python, but async and await will run faster when they are available:

async def fetch_coroutine(url):
    http_client = AsyncHTTPClient()
    response = await http_client.fetch(url)
    return response.body

The await keyword is less versatile than the yield keyword. For example, in a yield-based coroutine you can yield a list of Futures, while in a native coroutine you must wrap the list in tornado.gen.multi. You can also use tornado.gen.convert_yielded to convert anything that would work with yield into a form that will work with await.

While native coroutines are not visibly tied to a particular framework (i.e. they do not use a decorator like tornado.gen.coroutine or asyncio.coroutine), not all coroutines are compatible with each other. There is a coroutine runner which is selected by the first coroutine to be called, and then shared by all coroutines which are called directly with await. The Tornado coroutine runner is designed to be versatile and accept awaitable objects from any framework; other coroutine runners may be more limited (for example, the asyncio coroutine runner does not accept coroutines from other frameworks). For this reason, it is recommended to use the Tornado coroutine runner for any application which combines multiple frameworks. To call a coroutine using the Tornado runner from within a coroutine that is already using the asyncio runner, use the tornado.platform.asyncio.to_asyncio_future adapter.

How it works

A function containing yield is a generator. All generators are asynchronous; when called they return a generator object instead of running to completion. The @gen.coroutine decorator communicates with the generator via the yield expressions, and with the coroutine’s caller by returning a Future.

Here is a simplified version of the coroutine decorator’s inner loop:

# Simplified inner loop of tornado.gen.Runner
def run(self):
    # send(x) makes the current yield return x.
    # It returns when the next yield is reached
    future = self.gen.send(self.next)
    def callback(f):
        self.next = f.result()
        self.run()
    future.add_done_callback(callback)

The decorator receives a Future from the generator, waits (without blocking) for that Future to complete, then “unwraps” the Future and sends the result back into the generator as the result of the yield expression. Most asynchronous code never touches the Future class directly except to immediately pass the Future returned by an asynchronous function to a yield expression.

How to call a coroutine

Coroutines do not raise exceptions in the normal way: any exception they raise will be trapped in the Future until it is yielded. This means it is important to call coroutines in the right way, or you may have errors that go unnoticed:

@gen.coroutine
def divide(x, y):
    return x / y

def bad_call():
    # This should raise a ZeroDivisionError, but it won't because
    # the coroutine is called incorrectly.
    divide(1, 0)

In nearly all cases, any function that calls a coroutine must be a coroutine itself, and use the yield keyword in the call. When you are overriding a method defined in a superclass, consult the documentation to see if coroutines are allowed (the documentation should say that the method “may be a coroutine” or “may return a Future”):

@gen.coroutine
def good_call():
    # yield will unwrap the Future returned by divide() and raise
    # the exception.
    yield divide(1, 0)

Sometimes you may want to “fire and forget” a coroutine without waiting for its result. In this case it is recommended to use IOLoop.spawn_callback, which makes the IOLoop responsible for the call. If it fails, the IOLoop will log a stack trace:

# The IOLoop will catch the exception and print a stack trace in
# the logs. Note that this doesn't look like a normal call, since
# we pass the function object to be called by the IOLoop.
IOLoop.current().spawn_callback(divide, 1, 0)

Finally, at the top level of a program, if the `.IOLoop` is not yet running, you can start the IOLoop, run the coroutine, and then stop the IOLoop with the IOLoop.run_sync method. This is often used to start the main function of a batch-oriented program:

# run_sync() doesn't take arguments, so we must wrap the
# call in a lambda.
IOLoop.current().run_sync(lambda: divide(1, 0))
Coroutine patterns
Interaction with callbacks

To interact with asynchronous code that uses callbacks instead of Future, wrap the call in a Task. This will add the callback argument for you and return a Future which you can yield:

@gen.coroutine
def call_task():
    # Note that there are no parens on some_function.
    # This will be translated by Task into
    #   some_function(other_args, callback=callback)
    yield gen.Task(some_function, other_args)
Calling blocking functions

The simplest way to call a blocking function from a coroutine is to use a ThreadPoolExecutor, which returns Futures that are compatible with coroutines:

thread_pool = ThreadPoolExecutor(4)

@gen.coroutine
def call_blocking():
    yield thread_pool.submit(blocking_func, args)
Parallelism

The coroutine decorator recognizes lists and dicts whose values are Futures, and waits for all of those Futures in parallel:

@gen.coroutine
def parallel_fetch(url1, url2):
    resp1, resp2 = yield [http_client.fetch(url1),
                          http_client.fetch(url2)]

@gen.coroutine
def parallel_fetch_many(urls):
    responses = yield [http_client.fetch(url) for url in urls]
    # responses is a list of HTTPResponses in the same order

@gen.coroutine
def parallel_fetch_dict(urls):
    responses = yield {url: http_client.fetch(url)
                        for url in urls}
    # responses is a dict {url: HTTPResponse}
Interleaving

Sometimes it is useful to save a Future instead of yielding it immediately, so you can start another operation before waiting:

@gen.coroutine
def get(self):
    fetch_future = self.fetch_next_chunk()
    while True:
        chunk = yield fetch_future
        if chunk is None: break
        self.write(chunk)
        fetch_future = self.fetch_next_chunk()
        yield self.flush()
Looping

Looping is tricky with coroutines since there is no way in Python to yield on every iteration of a for or while loop and capture the result of the yield. Instead, you’ll need to separate the loop condition from accessing the results, as in this example from Motor:

import motor
db = motor.MotorClient().test

@gen.coroutine
def loop_example(collection):
    cursor = db.collection.find()
    while (yield cursor.fetch_next):
        doc = cursor.next_object()
Running in the background

PeriodicCallback is not normally used with coroutines. Instead, a coroutine can contain a while True: loop and use tornado.gen.sleep:

@gen.coroutine
def minute_loop():
    while True:
        yield do_something()
        yield gen.sleep(60)

# Coroutines that loop forever are generally started with
# spawn_callback().
IOLoop.current().spawn_callback(minute_loop)

Sometimes a more complicated loop may be desirable. For example, the previous loop runs every 60+N seconds, where N is the running time of do_something(). To run exactly every 60 seconds, use the interleaving pattern from above:

@gen.coroutine
def minute_loop2():
    while True:
        nxt = gen.sleep(60)   # Start the clock.
        yield do_something()  # Run while the clock is ticking.
        yield nxt             # Wait for the timer to run out.

Queue example - a concurrent web spider

Tornado’s tornado.queues module implements an asynchronous producer / consumer pattern for coroutines, analogous to the pattern implemented for threads by the Python standard library’s queue module.

A coroutine that yields Queue.get pauses until there is an item in the queue. If the queue has a maximum size set, a coroutine that yields Queue.put pauses until there is room for another item.

A Queue maintains a count of unfinished tasks, which begins at zero. put increments the count; task_done decrements it.

In the web-spider example here, the queue begins containing only base_url. When a worker fetches a page it parses the links and puts new ones in the queue, then calls task_done to decrement the counter once. Eventually, a worker fetches a page whose URLs have all been seen before, and there is also no work left in the queue. Thus that worker’s call to task_done decrements the counter to zero. The main coroutine, which is waiting for join, is unpaused and finishes.

import time
from datetime import timedelta

try:
    from HTMLParser import HTMLParser
    from urlparse import urljoin, urldefrag
except ImportError:
    from html.parser import HTMLParser
    from urllib.parse import urljoin, urldefrag

from tornado import httpclient, gen, ioloop, queues

base_url = 'http://www.tornadoweb.org/en/stable/'
concurrency = 10


@gen.coroutine
def get_links_from_url(url):
    """Download the page at `url` and parse it for links.

    Returned links have had the fragment after `#` removed, and have been made
    absolute so, e.g. the URL 'gen.html#tornado.gen.coroutine' becomes
    'http://www.tornadoweb.org/en/stable/gen.html'.
    """
    try:
        response = yield httpclient.AsyncHTTPClient().fetch(url)
        print('fetched %s' % url)

        html = response.body if isinstance(response.body, str) \
            else response.body.decode()
        urls = [urljoin(url, remove_fragment(new_url))
                for new_url in get_links(html)]
    except Exception as e:
        print('Exception: %s %s' % (e, url))
        raise gen.Return([])

    raise gen.Return(urls)


def remove_fragment(url):
    pure_url, frag = urldefrag(url)
    return pure_url


def get_links(html):
    class URLSeeker(HTMLParser):
        def __init__(self):
            HTMLParser.__init__(self)
            self.urls = []

        def handle_starttag(self, tag, attrs):
            href = dict(attrs).get('href')
            if href and tag == 'a':
                self.urls.append(href)

    url_seeker = URLSeeker()
    url_seeker.feed(html)
    return url_seeker.urls


@gen.coroutine
def main():
    q = queues.Queue()
    start = time.time()
    fetching, fetched = set(), set()

    @gen.coroutine
    def fetch_url():
        current_url = yield q.get()
        try:
            if current_url in fetching:
                return

            print('fetching %s' % current_url)
            fetching.add(current_url)
            urls = yield get_links_from_url(current_url)
            fetched.add(current_url)

            for new_url in urls:
                # Only follow links beneath the base URL
                if new_url.startswith(base_url):
                    yield q.put(new_url)

        finally:
            q.task_done()

    @gen.coroutine
    def worker():
        while True:
            yield fetch_url()

    q.put(base_url)

    # Start workers, then wait for the work queue to be empty.
    for _ in range(concurrency):
        worker()
    yield q.join(timeout=timedelta(seconds=300))
    assert fetching == fetched
    print('Done in %d seconds, fetched %s URLs.' % (
        time.time() - start, len(fetched)))


if __name__ == '__main__':
    import logging
    logging.basicConfig()
    io_loop = ioloop.IOLoop.current()
    io_loop.run_sync(main)

Structure of a Tornado web application

A Tornado web application generally consists of one or more RequestHandler subclasses, an Application object which routes incoming requests to handlers, and a main() function to start the server.

A minimal “hello world” example looks something like this:

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")

def make_app():
    return tornado.web.Application([
        (r"/", MainHandler),
    ])

if __name__ == "__main__":
    app = make_app()
    app.listen(8888)
    tornado.ioloop.IOLoop.current().start()
The Application object

The Application object is responsible for global configuration, including the routing table that maps requests to handlers.

The routing table is a list of URLSpec objects (or tuples), each of which contains (at least) a regular expression and a handler class. Order matters; the first matching rule is used. If the regular expression contains capturing groups, these groups are the path arguments and will be passed to the handler’s HTTP method. If a dictionary is passed as the third element of the URLSpec, it supplies the initialization arguments which will be passed to RequestHandler.initialize. Finally, the URLSpec may have a name, which will allow it to be used with RequestHandler.reverse_url.

For example, in this fragment the root URL / is mapped to MainHandler and URLs of the form /story/ followed by a number are mapped to StoryHandler. That number is passed (as a string) to StoryHandler.get.

class MainHandler(RequestHandler):
    def get(self):
        self.write('<a href="%s">link to story 1</a>' %
                   self.reverse_url("story", "1"))

class StoryHandler(RequestHandler):
    def initialize(self, db):
        self.db = db

    def get(self, story_id):
        self.write("this is story %s" % story_id)

app = Application([
    url(r"/", MainHandler),
    url(r"/story/([0-9]+)", StoryHandler, dict(db=db), name="story")
    ])

The Application constructor takes many keyword arguments that can be used to customize the behavior of the application and enable optional features; see Application.settings for the complete list.

Subclassing RequestHandler

Most of the work of a Tornado web application is done in subclasses of RequestHandler. The main entry point for a handler subclass is a method named after the HTTP method being handled: get(), post(), etc. Each handler may define one or more of these methods to handle different HTTP actions. As described above, these methods will be called with arguments corresponding to the capturing groups of the routing rule that matched.

Within a handler, call methods such as RequestHandler.render or RequestHandler.write to produce a response. render() loads a Template by name and renders it with the given arguments. write() is used for non-template-based output; it accepts strings, bytes, and dictionaries (dicts will be encoded as JSON).

Many methods in RequestHandler are designed to be overridden in subclasses and be used throughout the application. It is common to define a BaseHandler class that overrides methods such as write_error and get_current_user and then subclass your own BaseHandler instead of RequestHandler for all your specific handlers.

Handling request input

The request handler can access the object representing the current request with self.request. See the class definition for HTTPServerRequest for a complete list of attributes.

Request data in the formats used by HTML forms will be parsed for you and is made available in methods like get_query_argument and get_body_argument.

class MyFormHandler(tornado.web.RequestHandler):
    def get(self):
        self.write('<html><body><form action="/myform" method="POST">'
                   '<input type="text" name="message">'
                   '<input type="submit" value="Submit">'
                   '</form></body></html>')

    def post(self):
        self.set_header("Content-Type", "text/plain")
        self.write("You wrote " + self.get_body_argument("message"))

Since the HTML form encoding is ambiguous as to whether an argument is a single value or a list with one element, RequestHandler has distinct methods to allow the application to indicate whether or not it expects a list. For lists, use get_query_arguments and get_body_arguments instead of their singular counterparts.

Files uploaded via a form are available in self.request.files, which maps names (the name of the HTML <input type="file"> element) to a list of files. Each file is a dictionary of the form {"filename":..., "content_type":..., "body":...}. The files object is only present if the files were uploaded with a form wrapper (i.e. a multipart/form-data Content-Type); if this format was not used the raw uploaded data is available in self.request.body. By default uploaded files are fully buffered in memory; if you need to handle files that are too large to comfortably keep in memory see the stream_request_body class decorator.

Due to the quirks of the HTML form encoding (e.g. the ambiguity around singular versus plural arguments), Tornado does not attempt to unify form arguments with other types of input. In particular, we do not parse JSON request bodies. Applications that wish to use JSON instead of form-encoding may override prepare to parse their requests:

def prepare(self):
    if self.request.headers["Content-Type"].startswith("application/json"):
        self.json_args = json.loads(self.request.body)
    else:
        self.json_args = None
Overriding RequestHandler methods

In addition to get()/post()/etc, certain other methods in RequestHandler are designed to be overridden by subclasses when necessary. On every request, the following sequence of calls takes place:

  1. A new RequestHandler object is created on each request
  2. initialize() is called with the initialization arguments from the Application configuration. initialize should typically just save the arguments passed into member variables; it may not produce any output or call methods like send_error.
  3. prepare() is called. This is most useful in a base class shared by all of your handler subclasses, as prepare is called no matter which HTTP method is used. prepare may produce output; if it calls finish (or redirect, etc), processing stops here.
  4. One of the HTTP methods is called: get(), post(), put(), etc. If the URL regular expression contains capturing groups, they are passed as arguments to this method.
  5. When the request is finished, on_finish() is called. For synchronous handlers this is immediately after get() (etc) return; for asynchronous handlers it is after the call to finish().

All methods designed to be overridden are noted as such in the RequestHandler documentation. Some of the most commonly overridden methods include:

Error Handling

If a handler raises an exception, Tornado will call RequestHandler.write_error to generate an error page. tornado.web.HTTPError can be used to generate a specified status code; all other exceptions return a 500 status.

The default error page includes a stack trace in debug mode and a one-line description of the error (e.g. “500: Internal Server Error”) otherwise. To produce a custom error page, override RequestHandler.write_error (probably in a base class shared by all your handlers). This method may produce output normally via methods such as write and render. If the error was caused by an exception, an exc_info triple will be passed as a keyword argument (note that this exception is not guaranteed to be the current exception in sys.exc_info, so write_error must use e.g. traceback.format_exception instead of traceback.format_exc).

It is also possible to generate an error page from regular handler methods instead of write_error by calling set_status, writing a response, and returning. The special exception tornado.web.Finish may be raised to terminate the handler without calling write_error in situations where simply returning is not convenient.

For 404 errors, use the default_handler_class Application setting. This handler should override prepare instead of a more specific method like get() so it works with any HTTP method. It should produce its error page as described above: either by raising a HTTPError(404) and overriding write_error, or calling self.set_status(404) and producing the response directly in prepare().

Redirection

There are two main ways you can redirect requests in Tornado: RequestHandler.redirect and with the RedirectHandler.

You can use self.redirect() within a RequestHandler method to redirect users elsewhere. There is also an optional parameter permanent which you can use to indicate that the redirection is considered permanent. The default value of permanent is False, which generates a 302 Found HTTP response code and is appropriate for things like redirecting users after successful POST requests. If permanent is true, the 301 Moved Permanently HTTP response code is used, which is useful for e.g. redirecting to a canonical URL for a page in an SEO-friendly manner.

RedirectHandler lets you configure redirects directly in your Application routing table. For example, to configure a single static redirect:

app = tornado.web.Application([
    url(r"/app", tornado.web.RedirectHandler,
        dict(url="http://itunes.apple.com/my-app-id")),
    ])

RedirectHandler also supports regular expression substitutions. The following rule redirects all requests beginning with /pictures/ to the prefix /photos/ instead:

app = tornado.web.Application([
    url(r"/photos/(.*)", MyPhotoHandler),
    url(r"/pictures/(.*)", tornado.web.RedirectHandler,
        dict(url=r"/photos/\1")),
    ])

Unlike RequestHandler.redirect, RedirectHandler uses permanent redirects by default. This is because the routing table does not change at runtime and is presumed to be permanent, while redirects found in handlers are likely to be the result of other logic that may change. To send a temporary redirect with a RedirectHandler, add permanent=False to the RedirectHandler initialization arguments.

Asynchronous handlers

Tornado handlers are synchronous by default: when the get()/post() method returns, the request is considered finished and the response is sent. Since all other requests are blocked while one handler is running, any long-running handler should be made asynchronous so it can call its slow operations in a non-blocking way. This topic is covered in more detail in Asynchronous and non-Blocking I/O; this section is about the particulars of asynchronous techniques in RequestHandler subclasses.

The simplest way to make a handler asynchronous is to use the coroutine decorator. This allows you to perform non-blocking I/O with the yield keyword, and no response will be sent until the coroutine has returned. See Coroutines for more details.

In some cases, coroutines may be less convenient than a callback-oriented style, in which case the tornado.web.asynchronous decorator can be used instead. When this decorator is used the response is not automatically sent; instead the request will be kept open until some callback calls RequestHandler.finish. It is up to the application to ensure that this method is called, or else the user’s browser will simply hang.

Here is an example that makes a call to the FriendFeed API using Tornado’s built-in AsyncHTTPClient:

class MainHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        http = tornado.httpclient.AsyncHTTPClient()
        http.fetch("http://friendfeed-api.com/v2/feed/bret",
                   callback=self.on_response)

    def on_response(self, response):
        if response.error: raise tornado.web.HTTPError(500)
        json = tornado.escape.json_decode(response.body)
        self.write("Fetched " + str(len(json["entries"])) + " entries "
                   "from the FriendFeed API")
        self.finish()

When get() returns, the request has not finished. When the HTTP client eventually calls on_response(), the request is still open, and the response is finally flushed to the client with the call to self.finish().

For comparison, here is the same example using a coroutine:

class MainHandler(tornado.web.RequestHandler):
    @tornado.gen.coroutine
    def get(self):
        http = tornado.httpclient.AsyncHTTPClient()
        response = yield http.fetch("http://friendfeed-api.com/v2/feed/bret")
        json = tornado.escape.json_decode(response.body)
        self.write("Fetched " + str(len(json["entries"])) + " entries "
                   "from the FriendFeed API")

For a more advanced asynchronous example, take a look at the chat example application, which implements an AJAX chat room using long polling. Users of long polling may want to override on_connection_close() to clean up after the client closes the connection (but see that method’s docstring for caveats).

Templates and UI

Tornado includes a simple, fast, and flexible templating language. This section describes that language as well as related issues such as internationalization.

Tornado can also be used with any other Python template language, although there is no provision for integrating these systems into RequestHandler.render. Simply render the template to a string and pass it to RequestHandler.write

Configuring templates

By default, Tornado looks for template files in the same directory as the .py files that refer to them. To put your template files in a different directory, use the template_path Application setting (or override RequestHandler.get_template_path if you have different template paths for different handlers).

To load templates from a non-filesystem location, subclass tornado.template.BaseLoader and pass an instance as the template_loader application setting.

Compiled templates are cached by default; to turn off this caching and reload templates so changes to the underlying files are always visible, use the application settings compiled_template_cache=False or debug=True.

Template syntax

A Tornado template is just HTML (or any other text-based format) with Python control sequences and expressions embedded within the markup:

<html>
   <head>
      <title>{{ title }}</title>
   </head>
   <body>
     <ul>
       {% for item in items %}
         <li>{{ escape(item) }}</li>
       {% end %}
     </ul>
   </body>
 </html>

If you saved this template as “template.html” and put it in the same directory as your Python file, you could render this template with:

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        items = ["Item 1", "Item 2", "Item 3"]
        self.render("template.html", title="My title", items=items)

Tornado templates support control statements and expressions. Control statements are surrounded by {% and %}, e.g., {% if len(items) > 2 %}. Expressions are surrounded by {{ and }}, e.g., {{ items[0] }}.

Control statements more or less map exactly to Python statements. We support if, for, while, and try, all of which are terminated with {% end %}. We also support template inheritance using the extends and block statements, which are described in detail in the documentation for the tornado.template.

Expressions can be any Python expression, including function calls. Template code is executed in a namespace that includes the following objects and functions (Note that this list applies to templates rendered using RequestHandler.render and render_string. If you’re using the tornado.template module directly outside of a RequestHandler many of these entries are not present).

When you are building a real application, you are going to want to use all of the features of Tornado templates, especially template inheritance. Read all about those features in the tornado.template section (some features, including UIModules are implemented in the tornado.web module)

Under the hood, Tornado templates are translated directly to Python. The expressions you include in your template are copied verbatim into a Python function representing your template. We don’t try to prevent anything in the template language; we created it explicitly to provide the flexibility that other, stricter templating systems prevent. Consequently, if you write random stuff inside of your template expressions, you will get random Python errors when you execute the template.

All template output is escaped by default, using the tornado.escape.xhtml_escape function. This behavior can be changed globally by passing autoescape=None to the Application or tornado.template.Loader constructors, for a template file with the {% autoescape None %} directive, or for a single expression by replacing {{ ... }} with {% raw ...%}. Additionally, in each of these places the name of an alternative escaping function may be used instead of None.

Note that while Tornado’s automatic escaping is helpful in avoiding XSS vulnerabilities, it is not sufficient in all cases. Expressions that appear in certain locations, such as in Javascript or CSS, may need additional escaping. Additionally, either care must be taken to always use double quotes and xhtml_escape in HTML attributes that may contain untrusted content, or a separate escaping function must be used for attributes (see e.g. http://wonko.com/post/html-escaping)

Internationalization

The locale of the current user (whether they are logged in or not) is always available as self.locale in the request handler and as locale in templates. The name of the locale (e.g., en_US) is available as locale.name, and you can translate strings with the Locale.translate method. Templates also have the global function call _() available for string translation. The translate function has two forms:

_("Translate this string")

which translates the string directly based on the current locale, and:

_("A person liked this", "%(num)d people liked this",
  len(people)) % {"num": len(people)}

which translates a string that can be singular or plural based on the value of the third argument. In the example above, a translation of the first string will be returned if len(people) is 1, or a translation of the second string will be returned otherwise.

The most common pattern for translations is to use Python named placeholders for variables (the %(num)d in the example above) since placeholders can move around on translation.

Here is a properly internationalized template:

<html>
   <head>
      <title>FriendFeed - {{ _("Sign in") }}</title>
   </head>
   <body>
     <form action="{{ request.path }}" method="post">
       <div>{{ _("Username") }} <input type="text" name="username"/></div>
       <div>{{ _("Password") }} <input type="password" name="password"/></div>
       <div><input type="submit" value="{{ _("Sign in") }}"/></div>
       {% module xsrf_form_html() %}
     </form>
   </body>
 </html>

By default, we detect the user’s locale using the Accept-Language header sent by the user’s browser. We choose en_US if we can’t find an appropriate Accept-Language value. If you let user’s set their locale as a preference, you can override this default locale selection by overriding RequestHandler.get_user_locale:

class BaseHandler(tornado.web.RequestHandler):
    def get_current_user(self):
        user_id = self.get_secure_cookie("user")
        if not user_id: return None
        return self.backend.get_user_by_id(user_id)

    def get_user_locale(self):
        if "locale" not in self.current_user.prefs:
            # Use the Accept-Language header
            return None
        return self.current_user.prefs["locale"]

If get_user_locale returns None, we fall back on the Accept-Language header.

The tornado.locale module supports loading translations in two formats: the .mo format used by gettext and related tools, and a simple .csv format. An application will generally call either tornado.locale.load_translations or tornado.locale.load_gettext_translations once at startup; see those methods for more details on the supported formats..

You can get the list of supported locales in your application with tornado.locale.get_supported_locales(). The user’s locale is chosen to be the closest match based on the supported locales. For example, if the user’s locale is es_GT, and the es locale is supported, self.locale will be es for that request. We fall back on en_US if no close match can be found.

UI modules

Tornado supports UI modules to make it easy to support standard, reusable UI widgets across your application. UI modules are like special function calls to render components of your page, and they can come packaged with their own CSS and JavaScript.

For example, if you are implementing a blog, and you want to have blog entries appear on both the blog home page and on each blog entry page, you can make an Entry module to render them on both pages. First, create a Python module for your UI modules, e.g., uimodules.py:

class Entry(tornado.web.UIModule):
    def render(self, entry, show_comments=False):
        return self.render_string(
            "module-entry.html", entry=entry, show_comments=show_comments)

Tell Tornado to use uimodules.py using the ui_modules setting in your application:

from . import uimodules

class HomeHandler(tornado.web.RequestHandler):
    def get(self):
        entries = self.db.query("SELECT * FROM entries ORDER BY date DESC")
        self.render("home.html", entries=entries)

class EntryHandler(tornado.web.RequestHandler):
    def get(self, entry_id):
        entry = self.db.get("SELECT * FROM entries WHERE id = %s", entry_id)
        if not entry: raise tornado.web.HTTPError(404)
        self.render("entry.html", entry=entry)

settings = {
    "ui_modules": uimodules,
}
application = tornado.web.Application([
    (r"/", HomeHandler),
    (r"/entry/([0-9]+)", EntryHandler),
], **settings)

Within a template, you can call a module with the {% module %} statement. For example, you could call the Entry module from both home.html:

{% for entry in entries %}
  {% module Entry(entry) %}
{% end %}

and entry.html:

{% module Entry(entry, show_comments=True) %}

Modules can include custom CSS and JavaScript functions by overriding the embedded_css, embedded_javascript, javascript_files, or css_files methods:

class Entry(tornado.web.UIModule):
    def embedded_css(self):
        return ".entry { margin-bottom: 1em; }"

    def render(self, entry, show_comments=False):
        return self.render_string(
            "module-entry.html", show_comments=show_comments)

Module CSS and JavaScript will be included once no matter how many times a module is used on a page. CSS is always included in the <head> of the page, and JavaScript is always included just before the </body> tag at the end of the page.

When additional Python code is not required, a template file itself may be used as a module. For example, the preceding example could be rewritten to put the following in module-entry.html:

{{ set_resources(embedded_css=".entry { margin-bottom: 1em; }") }}
<!-- more template html... -->

This revised template module would be invoked with:

{% module Template("module-entry.html", show_comments=True) %}

The set_resources function is only available in templates invoked via {% module Template(...) %}. Unlike the {% include ... %} directive, template modules have a distinct namespace from their containing template - they can only see the global template namespace and their own keyword arguments.

Authentication and security

Cookies and secure cookies

You can set cookies in the user’s browser with the set_cookie method:

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        if not self.get_cookie("mycookie"):
            self.set_cookie("mycookie", "myvalue")
            self.write("Your cookie was not set yet!")
        else:
            self.write("Your cookie was set!")

Cookies are not secure and can easily be modified by clients. If you need to set cookies to, e.g., identify the currently logged in user, you need to sign your cookies to prevent forgery. Tornado supports signed cookies with the set_secure_cookie and get_secure_cookie methods. To use these methods, you need to specify a secret key named cookie_secret when you create your application. You can pass in application settings as keyword arguments to your application:

application = tornado.web.Application([
    (r"/", MainHandler),
], cookie_secret="__TODO:_GENERATE_YOUR_OWN_RANDOM_VALUE_HERE__")

Signed cookies contain the encoded value of the cookie in addition to a timestamp and an HMAC signature. If the cookie is old or if the signature doesn’t match, get_secure_cookie will return None just as if the cookie isn’t set. The secure version of the example above:

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        if not self.get_secure_cookie("mycookie"):
            self.set_secure_cookie("mycookie", "myvalue")
            self.write("Your cookie was not set yet!")
        else:
            self.write("Your cookie was set!")

Tornado’s secure cookies guarantee integrity but not confidentiality. That is, the cookie cannot be modified but its contents can be seen by the user. The cookie_secret is a symmetric key and must be kept secret – anyone who obtains the value of this key could produce their own signed cookies.

By default, Tornado’s secure cookies expire after 30 days. To change this, use the expires_days keyword argument to set_secure_cookie and the max_age_days argument to get_secure_cookie. These two values are passed separately so that you may e.g. have a cookie that is valid for 30 days for most purposes, but for certain sensitive actions (such as changing billing information) you use a smaller max_age_days when reading the cookie.

Tornado also supports multiple signing keys to enable signing key rotation. cookie_secret then must be a dict with integer key versions as keys and the corresponding secrets as values. The currently used signing key must then be set as key_version application setting but all other keys in the dict are allowed for cookie signature validation, if the correct key version is set in the cookie. To implement cookie updates, the current signing key version can be queried via get_secure_cookie_key_version.

User authentication

The currently authenticated user is available in every request handler as self.current_user, and in every template as current_user. By default, current_user is None.

To implement user authentication in your application, you need to override the get_current_user() method in your request handlers to determine the current user based on, e.g., the value of a cookie. Here is an example that lets users log into the application simply by specifying a nickname, which is then saved in a cookie:

class BaseHandler(tornado.web.RequestHandler):
    def get_current_user(self):
        return self.get_secure_cookie("user")

class MainHandler(BaseHandler):
    def get(self):
        if not self.current_user:
            self.redirect("/login")
            return
        name = tornado.escape.xhtml_escape(self.current_user)
        self.write("Hello, " + name)

class LoginHandler(BaseHandler):
    def get(self):
        self.write('<html><body><form action="/login" method="post">'
                   'Name: <input type="text" name="name">'
                   '<input type="submit" value="Sign in">'
                   '</form></body></html>')

    def post(self):
        self.set_secure_cookie("user", self.get_argument("name"))
        self.redirect("/")

application = tornado.web.Application([
    (r"/", MainHandler),
    (r"/login", LoginHandler),
], cookie_secret="__TODO:_GENERATE_YOUR_OWN_RANDOM_VALUE_HERE__")

You can require that the user be logged in using the Python decorator tornado.web.authenticated. If a request goes to a method with this decorator, and the user is not logged in, they will be redirected to login_url (another application setting). The example above could be rewritten:

class MainHandler(BaseHandler):
    @tornado.web.authenticated
    def get(self):
        name = tornado.escape.xhtml_escape(self.current_user)
        self.write("Hello, " + name)

settings = {
    "cookie_secret": "__TODO:_GENERATE_YOUR_OWN_RANDOM_VALUE_HERE__",
    "login_url": "/login",
}
application = tornado.web.Application([
    (r"/", MainHandler),
    (r"/login", LoginHandler),
], **settings)

If you decorate post() methods with the authenticated decorator, and the user is not logged in, the server will send a 403 response. The @authenticated decorator is simply shorthand for if not self.current_user: self.redirect() and may not be appropriate for non-browser-based login schemes.

Check out the Tornado Blog example application for a complete example that uses authentication (and stores user data in a MySQL database).

Third party authentication

The tornado.auth module implements the authentication and authorization protocols for a number of the most popular sites on the web, including Google/Gmail, Facebook, Twitter, and FriendFeed. The module includes methods to log users in via these sites and, where applicable, methods to authorize access to the service so you can, e.g., download a user’s address book or publish a Twitter message on their behalf.

Here is an example handler that uses Google for authentication, saving the Google credentials in a cookie for later access:

class GoogleOAuth2LoginHandler(tornado.web.RequestHandler,
                               tornado.auth.GoogleOAuth2Mixin):
    @tornado.gen.coroutine
    def get(self):
        if self.get_argument('code', False):
            user = yield self.get_authenticated_user(
                redirect_uri='http://your.site.com/auth/google',
                code=self.get_argument('code'))
            # Save the user with e.g. set_secure_cookie
        else:
            yield self.authorize_redirect(
                redirect_uri='http://your.site.com/auth/google',
                client_id=self.settings['google_oauth']['key'],
                scope=['profile', 'email'],
                response_type='code',
                extra_params={'approval_prompt': 'auto'})

See the tornado.auth module documentation for more details.

Cross-site request forgery protection

Cross-site request forgery, or XSRF, is a common problem for personalized web applications. See the Wikipedia article for more information on how XSRF works.

The generally accepted solution to prevent XSRF is to cookie every user with an unpredictable value and include that value as an additional argument with every form submission on your site. If the cookie and the value in the form submission do not match, then the request is likely forged.

Tornado comes with built-in XSRF protection. To include it in your site, include the application setting xsrf_cookies:

settings = {
    "cookie_secret": "__TODO:_GENERATE_YOUR_OWN_RANDOM_VALUE_HERE__",
    "login_url": "/login",
    "xsrf_cookies": True,
}
application = tornado.web.Application([
    (r"/", MainHandler),
    (r"/login", LoginHandler),
], **settings)

If xsrf_cookies is set, the Tornado web application will set the _xsrf cookie for all users and reject all POST, PUT, and DELETE requests that do not contain a correct _xsrf value. If you turn this setting on, you need to instrument all forms that submit via POST to contain this field. You can do this with the special UIModule xsrf_form_html(), available in all templates:

<form action="/new_message" method="post">
  {% module xsrf_form_html() %}
  <input type="text" name="message"/>
  <input type="submit" value="Post"/>
</form>

If you submit AJAX POST requests, you will also need to instrument your JavaScript to include the _xsrf value with each request. This is the jQuery function we use at FriendFeed for AJAX POST requests that automatically adds the _xsrf value to all requests:

function getCookie(name) {
    var r = document.cookie.match("\\b" + name + "=([^;]*)\\b");
    return r ? r[1] : undefined;
}

jQuery.postJSON = function(url, args, callback) {
    args._xsrf = getCookie("_xsrf");
    $.ajax({url: url, data: $.param(args), dataType: "text", type: "POST",
        success: function(response) {
        callback(eval("(" + response + ")"));
    }});
};

For PUT and DELETE requests (as well as POST requests that do not use form-encoded arguments), the XSRF token may also be passed via an HTTP header named X-XSRFToken. The XSRF cookie is normally set when xsrf_form_html is used, but in a pure-Javascript application that does not use any regular forms you may need to access self.xsrf_token manually (just reading the property is enough to set the cookie as a side effect).

If you need to customize XSRF behavior on a per-handler basis, you can override RequestHandler.check_xsrf_cookie(). For example, if you have an API whose authentication does not use cookies, you may want to disable XSRF protection by making check_xsrf_cookie() do nothing. However, if you support both cookie and non-cookie-based authentication, it is important that XSRF protection be used whenever the current request is authenticated with a cookie.

Running and deploying

Since Tornado supplies its own HTTPServer, running and deploying it is a little different from other Python web frameworks. Instead of configuring a WSGI container to find your application, you write a main() function that starts the server:

def main():
    app = make_app()
    app.listen(8888)
    IOLoop.current().start()

if __name__ == '__main__':
    main()

Configure your operating system or process manager to run this program to start the server. Please note that it may be necessary to increase the number of open files per process (to avoid “Too many open files”-Error). To raise this limit (setting it to 50000 for example) you can use the ulimit command, modify /etc/security/limits.conf or setting minfds in your supervisord config.

Processes and ports

Due to the Python GIL (Global Interpreter Lock), it is necessary to run multiple Python processes to take full advantage of multi-CPU machines. Typically it is best to run one process per CPU.

Tornado includes a built-in multi-process mode to start several processes at once. This requires a slight alteration to the standard main function:

def main():
    app = make_app()
    server = tornado.httpserver.HTTPServer(app)
    server.bind(8888)
    server.start(0)  # forks one process per cpu
    IOLoop.current().start()

This is the easiest way to start multiple processes and have them all share the same port, although it has some limitations. First, each child process will have its own IOLoop, so it is important that nothing touch the global IOLoop instance (even indirectly) before the fork. Second, it is difficult to do zero-downtime updates in this model. Finally, since all the processes share the same port it is more difficult to monitor them individually.

For more sophisticated deployments, it is recommended to start the processes independently, and have each one listen on a different port. The “process groups” feature of supervisord is one good way to arrange this. When each process uses a different port, an external load balancer such as HAProxy or nginx is usually needed to present a single address to outside visitors.

Running behind a load balancer

When running behind a load balancer like nginx, it is recommended to pass xheaders=True to the HTTPServer constructor. This will tell Tornado to use headers like X-Real-IP to get the user’s IP address instead of attributing all traffic to the balancer’s IP address.

This is a barebones nginx config file that is structurally similar to the one we use at FriendFeed. It assumes nginx and the Tornado servers are running on the same machine, and the four Tornado servers are running on ports 8000 - 8003:

user nginx;
worker_processes 1;

error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;

events {
    worker_connections 1024;
    use epoll;
}

http {
    # Enumerate all the Tornado servers here
    upstream frontends {
        server 127.0.0.1:8000;
        server 127.0.0.1:8001;
        server 127.0.0.1:8002;
        server 127.0.0.1:8003;
    }

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    access_log /var/log/nginx/access.log;

    keepalive_timeout 65;
    proxy_read_timeout 200;
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    gzip on;
    gzip_min_length 1000;
    gzip_proxied any;
    gzip_types text/plain text/html text/css text/xml
               application/x-javascript application/xml
               application/atom+xml text/javascript;

    # Only retry if there was a communication error, not a timeout
    # on the Tornado server (to avoid propagating "queries of death"
    # to all frontends)
    proxy_next_upstream error;

    server {
        listen 80;

        # Allow file uploads
        client_max_body_size 50M;

        location ^~ /static/ {
            root /var/www;
            if ($query_string) {
                expires max;
            }
        }
        location = /favicon.ico {
            rewrite (.*) /static/favicon.ico;
        }
        location = /robots.txt {
            rewrite (.*) /static/robots.txt;
        }

        location / {
            proxy_pass_header Server;
            proxy_set_header Host $http_host;
            proxy_redirect off;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Scheme $scheme;
            proxy_pass http://frontends;
        }
    }
}
Static files and aggressive file caching

You can serve static files from Tornado by specifying the static_path setting in your application:

settings = {
    "static_path": os.path.join(os.path.dirname(__file__), "static"),
    "cookie_secret": "__TODO:_GENERATE_YOUR_OWN_RANDOM_VALUE_HERE__",
    "login_url": "/login",
    "xsrf_cookies": True,
}
application = tornado.web.Application([
    (r"/", MainHandler),
    (r"/login", LoginHandler),
    (r"/(apple-touch-icon\.png)", tornado.web.StaticFileHandler,
     dict(path=settings['static_path'])),
], **settings)

This setting will automatically make all requests that start with /static/ serve from that static directory, e.g., http://localhost:8888/static/foo.png will serve the file foo.png from the specified static directory. We also automatically serve /robots.txt and /favicon.ico from the static directory (even though they don’t start with the /static/ prefix).

In the above settings, we have explicitly configured Tornado to serve apple-touch-icon.png from the root with the StaticFileHandler, though it is physically in the static file directory. (The capturing group in that regular expression is necessary to tell StaticFileHandler the requested filename; recall that capturing groups are passed to handlers as method arguments.) You could do the same thing to serve e.g. sitemap.xml from the site root. Of course, you can also avoid faking a root apple-touch-icon.png by using the appropriate <link /> tag in your HTML.

To improve performance, it is generally a good idea for browsers to cache static resources aggressively so browsers won’t send unnecessary If-Modified-Since or Etag requests that might block the rendering of the page. Tornado supports this out of the box with static content versioning.

To use this feature, use the static_url method in your templates rather than typing the URL of the static file directly in your HTML:

<html>
   <head>
      <title>FriendFeed - {{ _("Home") }}</title>
   </head>
   <body>
     <div><img src="{{ static_url("images/logo.png") }}"/></div>
   </body>
 </html>

The static_url() function will translate that relative path to a URI that looks like /static/images/logo.png?v=aae54. The v argument is a hash of the content in logo.png, and its presence makes the Tornado server send cache headers to the user’s browser that will make the browser cache the content indefinitely.

Since the v argument is based on the content of the file, if you update a file and restart your server, it will start sending a new v value, so the user’s browser will automatically fetch the new file. If the file’s contents don’t change, the browser will continue to use a locally cached copy without ever checking for updates on the server, significantly improving rendering performance.

In production, you probably want to serve static files from a more optimized static file server like nginx. You can configure most any web server to recognize the version tags used by static_url() and set caching headers accordingly. Here is the relevant portion of the nginx configuration we use at FriendFeed:

location /static/ {
    root /var/friendfeed/static;
    if ($query_string) {
        expires max;
    }
 }
Debug mode and automatic reloading

If you pass debug=True to the Application constructor, the app will be run in debug/development mode. In this mode, several features intended for convenience while developing will be enabled (each of which is also available as an individual flag; if both are specified the individual flag takes precedence):

  • autoreload=True: The app will watch for changes to its source files and reload itself when anything changes. This reduces the need to manually restart the server during development. However, certain failures (such as syntax errors at import time) can still take the server down in a way that debug mode cannot currently recover from.
  • compiled_template_cache=False: Templates will not be cached.
  • static_hash_cache=False: Static file hashes (used by the static_url function) will not be cached
  • serve_traceback=True: When an exception in a RequestHandler is not caught, an error page including a stack trace will be generated.

Autoreload mode is not compatible with the multi-process mode of HTTPServer. You must not give HTTPServer.start an argument other than 1 (or call tornado.process.fork_processes) if you are using autoreload mode.

The automatic reloading feature of debug mode is available as a standalone module in tornado.autoreload. The two can be used in combination to provide extra robustness against syntax errors: set autoreload=True within the app to detect changes while it is running, and start it with python -m tornado.autoreload myserver.py to catch any syntax errors or other errors at startup.

Reloading loses any Python interpreter command-line arguments (e.g. -u) because it re-executes Python using sys.executable and sys.argv. Additionally, modifying these variables will cause reloading to behave incorrectly.

On some platforms (including Windows and Mac OSX prior to 10.6), the process cannot be updated “in-place”, so when a code change is detected the old server exits and a new one starts. This has been known to confuse some IDEs.

WSGI and Google App Engine

Tornado is normally intended to be run on its own, without a WSGI container. However, in some environments (such as Google App Engine), only WSGI is allowed and applications cannot run their own servers. In this case Tornado supports a limited mode of operation that does not support asynchronous operation but allows a subset of Tornado’s functionality in a WSGI-only environment. The features that are not allowed in WSGI mode include coroutines, the @asynchronous decorator, AsyncHTTPClient, the auth module, and WebSockets.

You can convert a Tornado Application to a WSGI application with tornado.wsgi.WSGIAdapter. In this example, configure your WSGI container to find the application object:

import tornado.web
import tornado.wsgi

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")

tornado_app = tornado.web.Application([
    (r"/", MainHandler),
])
application = tornado.wsgi.WSGIAdapter(tornado_app)

See the appengine example application for a full-featured AppEngine app built on Tornado.

Web framework

tornado.webRequestHandler and Application classes

tornado.web provides a simple web framework with asynchronous features that allow it to scale to large numbers of open connections, making it ideal for long polling.

Here is a simple “Hello, world” example app:

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")

if __name__ == "__main__":
    application = tornado.web.Application([
        (r"/", MainHandler),
    ])
    application.listen(8888)
    tornado.ioloop.IOLoop.current().start()

See the User’s guide for additional information.

Thread-safety notes

In general, methods on RequestHandler and elsewhere in Tornado are not thread-safe. In particular, methods such as write(), finish(), and flush() must only be called from the main thread. If you use multiple threads it is important to use IOLoop.add_callback to transfer control back to the main thread before finishing the request.

Request handlers
class tornado.web.RequestHandler(application, request, **kwargs)[source]

Base class for HTTP request handlers.

Subclasses must define at least one of the methods defined in the “Entry points” section below.

Entry points
RequestHandler.initialize()[source]

Hook for subclass initialization.

A dictionary passed as the third argument of a url spec will be supplied as keyword arguments to initialize().

Example:

class ProfileHandler(RequestHandler):
    def initialize(self, database):
        self.database = database

    def get(self, username):
        ...

app = Application([
    (r'/user/(.*)', ProfileHandler, dict(database=database)),
    ])
RequestHandler.prepare()[source]

Called at the beginning of a request before get/post/etc.

Override this method to perform common initialization regardless of the request method.

Asynchronous support: Decorate this method with gen.coroutine or return_future to make it asynchronous (the asynchronous decorator cannot be used on prepare). If this method returns a Future execution will not proceed until the Future is done.

New in version 3.1: Asynchronous support.

RequestHandler.on_finish()[source]

Called after the end of a request.

Override this method to perform cleanup, logging, etc. This method is a counterpart to prepare. on_finish may not produce any output, as it is called after the response has been sent to the client.

Implement any of the following methods (collectively known as the HTTP verb methods) to handle the corresponding HTTP method. These methods can be made asynchronous with one of the following decorators: gen.coroutine, return_future, or asynchronous.

To support a method not on this list, override the class variable SUPPORTED_METHODS:

class WebDAVHandler(RequestHandler):
    SUPPORTED_METHODS = RequestHandler.SUPPORTED_METHODS + ('PROPFIND',)

    def propfind(self):
        pass
RequestHandler.get(*args, **kwargs)[source]
RequestHandler.head(*args, **kwargs)[source]
RequestHandler.post(*args, **kwargs)[source]
RequestHandler.delete(*args, **kwargs)[source]
RequestHandler.patch(*args, **kwargs)[source]
RequestHandler.put(*args, **kwargs)[source]
RequestHandler.options(*args, **kwargs)[source]
Input
RequestHandler.get_argument(name, default=[], strip=True)[source]

Returns the value of the argument with the given name.

If default is not provided, the argument is considered to be required, and we raise a MissingArgumentError if it is missing.

If the argument appears in the url more than once, we return the last value.

The returned value is always unicode.

RequestHandler.get_arguments(name, strip=True)[source]

Returns a list of the arguments with the given name.

If the argument is not present, returns an empty list.

The returned values are always unicode.

RequestHandler.get_query_argument(name, default=[], strip=True)[source]

Returns the value of the argument with the given name from the request query string.

If default is not provided, the argument is considered to be required, and we raise a MissingArgumentError if it is missing.

If the argument appears in the url more than once, we return the last value.

The returned value is always unicode.

New in version 3.2.

RequestHandler.get_query_arguments(name, strip=True)[source]

Returns a list of the query arguments with the given name.

If the argument is not present, returns an empty list.

The returned values are always unicode.

New in version 3.2.

RequestHandler.get_body_argument(name, default=[], strip=True)[source]

Returns the value of the argument with the given name from the request body.

If default is not provided, the argument is considered to be required, and we raise a MissingArgumentError if it is missing.

If the argument appears in the url more than once, we return the last value.

The returned value is always unicode.

New in version 3.2.

RequestHandler.get_body_arguments(name, strip=True)[source]

Returns a list of the body arguments with the given name.

If the argument is not present, returns an empty list.

The returned values are always unicode.

New in version 3.2.

RequestHandler.decode_argument(value, name=None)[source]

Decodes an argument from the request.

The argument has been percent-decoded and is now a byte string. By default, this method decodes the argument as utf-8 and returns a unicode string, but this may be overridden in subclasses.

This method is used as a filter for both get_argument() and for values extracted from the url and passed to get()/post()/etc.

The name of the argument is provided if known, but may be None (e.g. for unnamed groups in the url regex).

RequestHandler.request

The tornado.httputil.HTTPServerRequest object containing additional request parameters including e.g. headers and body data.

RequestHandler.path_args
RequestHandler.path_kwargs

The path_args and path_kwargs attributes contain the positional and keyword arguments that are passed to the HTTP verb methods. These attributes are set before those methods are called, so the values are available during prepare.

Output
RequestHandler.set_status(status_code, reason=None)[source]

Sets the status code for our response.

Parameters:
  • status_code (int) – Response status code. If reason is None, it must be present in httplib.responses.
  • reason (string) – Human-readable reason phrase describing the status code. If None, it will be filled in from httplib.responses.
RequestHandler.set_header(name, value)[source]

Sets the given response header name and value.

If a datetime is given, we automatically format it according to the HTTP specification. If the value is not a string, we convert it to a string. All header values are then encoded as UTF-8.

RequestHandler.add_header(name, value)[source]

Adds the given response header and value.

Unlike set_header, add_header may be called multiple times to return multiple values for the same header.

RequestHandler.clear_header(name)[source]

Clears an outgoing header, undoing a previous set_header call.

Note that this method does not apply to multi-valued headers set by add_header.

RequestHandler.set_default_headers()[source]

Override this to set HTTP headers at the beginning of the request.

For example, this is the place to set a custom Server header. Note that setting such headers in the normal flow of request processing may not do what you want, since headers may be reset during error handling.

RequestHandler.write(chunk)[source]

Writes the given chunk to the output buffer.

To write the output to the network, use the flush() method below.

If the given chunk is a dictionary, we write it as JSON and set the Content-Type of the response to be application/json. (if you want to send JSON as a different Content-Type, call set_header after calling write()).

Note that lists are not converted to JSON because of a potential cross-site security vulnerability. All JSON output should be wrapped in a dictionary. More details at http://haacked.com/archive/2009/06/25/json-hijacking.aspx/ and https://github.com/facebook/tornado/issues/1009

RequestHandler.flush(include_footers=False, callback=None)[source]

Flushes the current output buffer to the network.

The callback argument, if given, can be used for flow control: it will be run when all flushed data has been written to the socket. Note that only one flush callback can be outstanding at a time; if another flush occurs before the previous flush’s callback has been run, the previous callback will be discarded.

Changed in version 4.0: Now returns a Future if no callback is given.

RequestHandler.finish(chunk=None)[source]

Finishes this response, ending the HTTP request.

RequestHandler.render(template_name, **kwargs)[source]

Renders the template with the given arguments as the response.

RequestHandler.render_string(template_name, **kwargs)[source]

Generate the given template with the given arguments.

We return the generated byte string (in utf8). To generate and write a template as a response, use render() above.

RequestHandler.get_template_namespace()[source]

Returns a dictionary to be used as the default template namespace.

May be overridden by subclasses to add or modify values.

The results of this method will be combined with additional defaults in the tornado.template module and keyword arguments to render or render_string.

RequestHandler.redirect(url, permanent=False, status=None)[source]

Sends a redirect to the given (optionally relative) URL.

If the status argument is specified, that value is used as the HTTP status code; otherwise either 301 (permanent) or 302 (temporary) is chosen based on the permanent argument. The default is 302 (temporary).

RequestHandler.send_error(status_code=500, **kwargs)[source]

Sends the given HTTP error code to the browser.

If flush() has already been called, it is not possible to send an error, so this method will simply terminate the response. If output has been written but not yet flushed, it will be discarded and replaced with the error page.

Override write_error() to customize the error page that is returned. Additional keyword arguments are passed through to write_error.

RequestHandler.write_error(status_code, **kwargs)[source]

Override to implement custom error pages.

write_error may call write, render, set_header, etc to produce output as usual.

If this error was caused by an uncaught exception (including HTTPError), an exc_info triple will be available as kwargs["exc_info"]. Note that this exception may not be the “current” exception for purposes of methods like sys.exc_info() or traceback.format_exc.

RequestHandler.clear()[source]

Resets all headers and content for this response.

RequestHandler.data_received(chunk)[source]

Implement this method to handle streamed request data.

Requires the stream_request_body decorator.

Cookies
RequestHandler.cookies

An alias for self.request.cookies.

Gets the value of the cookie with the given name, else default.

Sets the given cookie name/value with the given options.

Additional keyword arguments are set on the Cookie.Morsel directly. See http://docs.python.org/library/cookie.html#morsel-objects for available attributes.

Deletes the cookie with the given name.

Due to limitations of the cookie protocol, you must pass the same path and domain to clear a cookie as were used when that cookie was set (but there is no way to find out on the server side which values were used for a given cookie).

RequestHandler.clear_all_cookies(path='/', domain=None)[source]

Deletes all the cookies the user sent with this request.

See clear_cookie for more information on the path and domain parameters.

Changed in version 3.2: Added the path and domain parameters.

Returns the given signed cookie if it validates, or None.

The decoded cookie value is returned as a byte string (unlike get_cookie).

Changed in version 3.2.1: Added the min_version argument. Introduced cookie version 2; both versions 1 and 2 are accepted by default.

Returns the signing key version of the secure cookie.

The version is returned as int.

Signs and timestamps a cookie so it cannot be forged.

You must specify the cookie_secret setting in your Application to use this method. It should be a long, random sequence of bytes to be used as the HMAC secret for the signature.

To read a cookie set with this method, use get_secure_cookie().

Note that the expires_days parameter sets the lifetime of the cookie in the browser, but is independent of the max_age_days parameter to get_secure_cookie.

Secure cookies may contain arbitrary byte values, not just unicode strings (unlike regular cookies)

Changed in version 3.2.1: Added the version argument. Introduced cookie version 2 and made it the default.

RequestHandler.create_signed_value(name, value, version=None)[source]

Signs and timestamps a string so it cannot be forged.

Normally used via set_secure_cookie, but provided as a separate method for non-cookie uses. To decode a value not stored as a cookie use the optional value argument to get_secure_cookie.

Changed in version 3.2.1: Added the version argument. Introduced cookie version 2 and made it the default.

tornado.web.MIN_SUPPORTED_SIGNED_VALUE_VERSION = 1

The oldest signed value version supported by this version of Tornado.

Signed values older than this version cannot be decoded.

New in version 3.2.1.

tornado.web.MAX_SUPPORTED_SIGNED_VALUE_VERSION = 2

The newest signed value version supported by this version of Tornado.

Signed values newer than this version cannot be decoded.

New in version 3.2.1.

tornado.web.DEFAULT_SIGNED_VALUE_VERSION = 2

The signed value version produced by RequestHandler.create_signed_value.

May be overridden by passing a version keyword argument.

New in version 3.2.1.

tornado.web.DEFAULT_SIGNED_VALUE_MIN_VERSION = 1

The oldest signed value accepted by RequestHandler.get_secure_cookie.

May be overridden by passing a min_version keyword argument.

New in version 3.2.1.

Other
RequestHandler.application

The Application object serving this request

RequestHandler.check_etag_header()[source]

Checks the Etag header against requests’s If-None-Match.

Returns True if the request’s Etag matches and a 304 should be returned. For example:

self.set_etag_header()
if self.check_etag_header():
    self.set_status(304)
    return

This method is called automatically when the request is finished, but may be called earlier for applications that override compute_etag and want to do an early check for If-None-Match before completing the request. The Etag header should be set (perhaps with set_etag_header) before calling this method.

Verifies that the _xsrf cookie matches the _xsrf argument.

To prevent cross-site request forgery, we set an _xsrf cookie and include the same value as a non-cookie field with all POST requests. If the two do not match, we reject the form submission as a potential forgery.

The _xsrf value may be set as either a form field named _xsrf or in a custom HTTP header named X-XSRFToken or X-CSRFToken (the latter is accepted for compatibility with Django).

See http://en.wikipedia.org/wiki/Cross-site_request_forgery

Prior to release 1.1.1, this check was ignored if the HTTP header X-Requested-With: XMLHTTPRequest was present. This exception has been shown to be insecure and has been removed. For more information please see http://www.djangoproject.com/weblog/2011/feb/08/security/ http://weblog.rubyonrails.org/2011/2/8/csrf-protection-bypass-in-ruby-on-rails

Changed in version 3.2.2: Added support for cookie version 2. Both versions 1 and 2 are supported.

RequestHandler.compute_etag()[source]

Computes the etag header to be used for this request.

By default uses a hash of the content written so far.

May be overridden to provide custom etag implementations, or may return None to disable tornado’s default etag support.

RequestHandler.create_template_loader(template_path)[source]

Returns a new template loader for the given path.

May be overridden by subclasses. By default returns a directory-based loader on the given path, using the autoescape and template_whitespace application settings. If a template_loader application setting is supplied, uses that instead.

RequestHandler.current_user

The authenticated user for this request.

This is set in one of two ways:

  • A subclass may override get_current_user(), which will be called automatically the first time self.current_user is accessed. get_current_user() will only be called once per request, and is cached for future access:

    def get_current_user(self):
        user_cookie = self.get_secure_cookie("user")
            if user_cookie:
                return json.loads(user_cookie)
        return None
    
  • It may be set as a normal variable, typically from an overridden prepare():

    @gen.coroutine
    def prepare(self):
        user_id_cookie = self.get_secure_cookie("user_id")
        if user_id_cookie:
            self.current_user = yield load_user(user_id_cookie)
    

Note that prepare() may be a coroutine while get_current_user() may not, so the latter form is necessary if loading the user requires asynchronous operations.

The user object may any type of the application’s choosing.

RequestHandler.get_browser_locale(default='en_US')[source]

Determines the user’s locale from Accept-Language header.

See http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.4

RequestHandler.get_current_user()[source]

Override to determine the current user from, e.g., a cookie.

This method may not be a coroutine.

RequestHandler.get_login_url()[source]

Override to customize the login URL based on the request.

By default, we use the login_url application setting.

RequestHandler.get_status()[source]

Returns the status code for our response.

RequestHandler.get_template_path()[source]

Override to customize template path for each handler.

By default, we use the template_path application setting. Return None to load templates relative to the calling file.

RequestHandler.get_user_locale()[source]

Override to determine the locale from the authenticated user.

If None is returned, we fall back to get_browser_locale().

This method should return a tornado.locale.Locale object, most likely obtained via a call like tornado.locale.get("en")

RequestHandler.locale

The locale for the current session.

Determined by either get_user_locale, which you can override to set the locale based on, e.g., a user preference stored in a database, or get_browser_locale, which uses the Accept-Language header.

RequestHandler.log_exception(typ, value, tb)[source]

Override to customize logging of uncaught exceptions.

By default logs instances of HTTPError as warnings without stack traces (on the tornado.general logger), and all other exceptions as errors with stack traces (on the tornado.application logger).

New in version 3.1.

RequestHandler.on_connection_close()[source]

Called in async handlers if the client closed the connection.

Override this to clean up resources associated with long-lived connections. Note that this method is called only if the connection was closed during asynchronous processing; if you need to do cleanup after every request override on_finish instead.

Proxies may keep a connection open for a time (perhaps indefinitely) after the client has gone away, so this method may not be called promptly after the end user closes their connection.

RequestHandler.require_setting(name, feature='this feature')[source]

Raises an exception if the given app setting is not defined.

RequestHandler.reverse_url(name, *args)[source]

Alias for Application.reverse_url.

RequestHandler.set_etag_header()[source]

Sets the response’s Etag header using self.compute_etag().

Note: no header will be set if compute_etag() returns None.

This method is called automatically when the request is finished.

RequestHandler.settings

An alias for self.application.settings.

RequestHandler.static_url(path, include_host=None, **kwargs)[source]

Returns a static URL for the given relative static file path.

This method requires you set the static_path setting in your application (which specifies the root directory of your static files).

This method returns a versioned url (by default appending ?v=<signature>), which allows the static files to be cached indefinitely. This can be disabled by passing include_version=False (in the default implementation; other static file implementations are not required to support this, but they may support other options).

By default this method returns URLs relative to the current host, but if include_host is true the URL returned will be absolute. If this handler has an include_host attribute, that value will be used as the default for all static_url calls that do not pass include_host as a keyword argument.

RequestHandler.xsrf_form_html()[source]

An HTML <input/> element to be included with all POST forms.

It defines the _xsrf input value, which we check on all POST requests to prevent cross-site request forgery. If you have set the xsrf_cookies application setting, you must include this HTML within all of your HTML forms.

In a template, this method should be called with {% module xsrf_form_html() %}

See check_xsrf_cookie() above for more information.

RequestHandler.xsrf_token

The XSRF-prevention token for the current user/session.

To prevent cross-site request forgery, we set an ‘_xsrf’ cookie and include the same ‘_xsrf’ value as an argument with all POST requests. If the two do not match, we reject the form submission as a potential forgery.

See http://en.wikipedia.org/wiki/Cross-site_request_forgery

Changed in version 3.2.2: The xsrf token will now be have a random mask applied in every request, which makes it safe to include the token in pages that are compressed. See http://breachattack.com for more information on the issue fixed by this change. Old (version 1) cookies will be converted to version 2 when this method is called unless the xsrf_cookie_version Application setting is set to 1.

Changed in version 4.3: The xsrf_cookie_kwargs Application setting may be used to supply additional cookie options (which will be passed directly to set_cookie). For example, xsrf_cookie_kwargs=dict(httponly=True, secure=True) will set the secure and httponly flags on the _xsrf cookie.

Application configuration
class tornado.web.Application(handlers=None, default_host='', transforms=None, **settings)[source]

A collection of request handlers that make up a web application.

Instances of this class are callable and can be passed directly to HTTPServer to serve the application:

application = web.Application([
    (r"/", MainPageHandler),
])
http_server = httpserver.HTTPServer(application)
http_server.listen(8080)
ioloop.IOLoop.current().start()

The constructor for this class takes in a list of URLSpec objects or (regexp, request_class) tuples. When we receive requests, we iterate over the list in order and instantiate an instance of the first request class whose regexp matches the request path. The request class can be specified as either a class object or a (fully-qualified) name.

Each tuple can contain additional elements, which correspond to the arguments to the URLSpec constructor. (Prior to Tornado 3.2, only tuples of two or three elements were allowed).

A dictionary may be passed as the third element of the tuple, which will be used as keyword arguments to the handler’s constructor and initialize method. This pattern is used for the StaticFileHandler in this example (note that a StaticFileHandler can be installed automatically with the static_path setting described below):

application = web.Application([
    (r"/static/(.*)", web.StaticFileHandler, {"path": "/var/www"}),
])

We support virtual hosts with the add_handlers method, which takes in a host regular expression as the first argument:

application.add_handlers(r"www\.myhost\.com", [
    (r"/article/([0-9]+)", ArticleHandler),
])

You can serve static files by sending the static_path setting as a keyword argument. We will serve those files from the /static/ URI (this is configurable with the static_url_prefix setting), and we will serve /favicon.ico and /robots.txt from the same directory. A custom subclass of StaticFileHandler can be specified with the static_handler_class setting.

settings

Additional keyword arguments passed to the constructor are saved in the settings dictionary, and are often referred to in documentation as “application settings”. Settings are used to customize various aspects of Tornado (although in some cases richer customization is possible by overriding methods in a subclass of RequestHandler). Some applications also like to use the settings dictionary as a way to make application-specific settings available to handlers without using global variables. Settings used in Tornado are described below.

General settings:

  • autoreload: If True, the server process will restart when any source files change, as described in Debug mode and automatic reloading. This option is new in Tornado 3.2; previously this functionality was controlled by the debug setting.
  • debug: Shorthand for several debug mode settings, described in Debug mode and automatic reloading. Setting debug=True is equivalent to autoreload=True, compiled_template_cache=False, static_hash_cache=False, serve_traceback=True.
  • default_handler_class and default_handler_args: This handler will be used if no other match is found; use this to implement custom 404 pages (new in Tornado 3.2).
  • compress_response: If True, responses in textual formats will be compressed automatically. New in Tornado 4.0.
  • gzip: Deprecated alias for compress_response since Tornado 4.0.
  • log_function: This function will be called at the end of every request to log the result (with one argument, the RequestHandler object). The default implementation writes to the logging module’s root logger. May also be customized by overriding Application.log_request.
  • serve_traceback: If true, the default error page will include the traceback of the error. This option is new in Tornado 3.2; previously this functionality was controlled by the debug setting.
  • ui_modules and ui_methods: May be set to a mapping of UIModule or UI methods to be made available to templates. May be set to a module, dictionary, or a list of modules and/or dicts. See UI modules for more details.

Authentication and security settings:

  • cookie_secret: Used by RequestHandler.get_secure_cookie and set_secure_cookie to sign cookies.
  • key_version: Used by requestHandler set_secure_cookie to sign cookies with a specific key when cookie_secret is a key dictionary.
  • login_url: The authenticated decorator will redirect to this url if the user is not logged in. Can be further customized by overriding RequestHandler.get_login_url
  • xsrf_cookies: If true, Cross-site request forgery protection will be enabled.
  • xsrf_cookie_version: Controls the version of new XSRF cookies produced by this server. Should generally be left at the default (which will always be the highest supported version), but may be set to a lower value temporarily during version transitions. New in Tornado 3.2.2, which introduced XSRF cookie version 2.
  • xsrf_cookie_kwargs: May be set to a dictionary of additional arguments to be passed to RequestHandler.set_cookie for the XSRF cookie.
  • twitter_consumer_key, twitter_consumer_secret, friendfeed_consumer_key, friendfeed_consumer_secret, google_consumer_key, google_consumer_secret, facebook_api_key, facebook_secret: Used in the tornado.auth module to authenticate to various APIs.

Template settings:

  • autoescape: Controls automatic escaping for templates. May be set to None to disable escaping, or to the name of a function that all output should be passed through. Defaults to "xhtml_escape". Can be changed on a per-template basis with the {% autoescape %} directive.
  • compiled_template_cache: Default is True; if False templates will be recompiled on every request. This option is new in Tornado 3.2; previously this functionality was controlled by the debug setting.
  • template_path: Directory containing template files. Can be further customized by overriding RequestHandler.get_template_path
  • template_loader: Assign to an instance of tornado.template.BaseLoader to customize template loading. If this setting is used the template_path and autoescape settings are ignored. Can be further customized by overriding RequestHandler.create_template_loader.
  • template_whitespace: Controls handling of whitespace in templates; see tornado.template.filter_whitespace for allowed values. New in Tornado 4.3.

Static file settings:

  • static_hash_cache: Default is True; if False static urls will be recomputed on every request. This option is new in Tornado 3.2; previously this functionality was controlled by the debug setting.
  • static_path: Directory from which static files will be served.
  • static_url_prefix: Url prefix for static files, defaults to "/static/".
  • static_handler_class, static_handler_args: May be set to use a different handler for static files instead of the default tornado.web.StaticFileHandler. static_handler_args, if set, should be a dictionary of keyword arguments to be passed to the handler’s initialize method.
listen(port, address='', **kwargs)[source]

Starts an HTTP server for this application on the given port.

This is a convenience alias for creating an HTTPServer object and calling its listen method. Keyword arguments not supported by HTTPServer.listen are passed to the HTTPServer constructor. For advanced uses (e.g. multi-process mode), do not use this method; create an HTTPServer and call its TCPServer.bind/TCPServer.start methods directly.

Note that after calling this method you still need to call IOLoop.current().start() to start the server.

Returns the HTTPServer object.

Changed in version 4.3: Now returns the HTTPServer object.

add_handlers(host_pattern, host_handlers)[source]

Appends the given handlers to our handler list.

Host patterns are processed sequentially in the order they were added. All matching patterns will be considered.

reverse_url(name, *args)[source]

Returns a URL path for handler named name

The handler must be added to the application as a named URLSpec.

Args will be substituted for capturing groups in the URLSpec regex. They will be converted to strings if necessary, encoded as utf8, and url-escaped.

log_request(handler)[source]

Writes a completed HTTP request to the logs.

By default writes to the python root logger. To change this behavior either subclass Application and override this method, or pass a function in the application settings dictionary as log_function.

class tornado.web.URLSpec(pattern, handler, kwargs=None, name=None)[source]

Specifies mappings between URLs and handlers.

Parameters:

  • pattern: Regular expression to be matched. Any groups in the regex will be passed in to the handler’s get/post/etc methods as arguments.
  • handler: RequestHandler subclass to be invoked.
  • kwargs (optional): A dictionary of additional arguments to be passed to the handler’s constructor.
  • name (optional): A name for this handler. Used by Application.reverse_url.

The URLSpec class is also available under the name tornado.web.url.

Decorators
tornado.web.asynchronous(method)[source]

Wrap request handler methods with this if they are asynchronous.

This decorator is for callback-style asynchronous methods; for coroutines, use the @gen.coroutine decorator without @asynchronous. (It is legal for legacy reasons to use the two decorators together provided @asynchronous is first, but @asynchronous will be ignored in this case)

This decorator should only be applied to the HTTP verb methods; its behavior is undefined for any other method. This decorator does not make a method asynchronous; it tells the framework that the method is asynchronous. For this decorator to be useful the method must (at least sometimes) do something asynchronous.

If this decorator is given, the response is not finished when the method returns. It is up to the request handler to call self.finish() to finish the HTTP request. Without this decorator, the request is automatically finished when the get() or post() method returns. Example:

class MyRequestHandler(RequestHandler):
    @asynchronous
    def get(self):
       http = httpclient.AsyncHTTPClient()
       http.fetch("http://friendfeed.com/", self._on_download)

    def _on_download(self, response):
       self.write("Downloaded!")
       self.finish()

Changed in version 3.1: The ability to use @gen.coroutine without @asynchronous.

Changed in version 4.3: Returning anything but None or a yieldable object from a method decorated with @asynchronous is an error. Such return values were previously ignored silently.

tornado.web.authenticated(method)[source]

Decorate methods with this to require that the user be logged in.

If the user is not logged in, they will be redirected to the configured login url.

If you configure a login url with a query parameter, Tornado will assume you know what you’re doing and use it as-is. If not, it will add a next parameter so the login page knows where to send you once you’re logged in.

tornado.web.addslash(method)[source]

Use this decorator to add a missing trailing slash to the request path.

For example, a request to /foo would redirect to /foo/ with this decorator. Your request handler mapping should use a regular expression like r'/foo/?' in conjunction with using the decorator.

tornado.web.removeslash(method)[source]

Use this decorator to remove trailing slashes from the request path.

For example, a request to /foo/ would redirect to /foo with this decorator. Your request handler mapping should use a regular expression like r'/foo/*' in conjunction with using the decorator.

tornado.web.stream_request_body(cls)[source]

Apply to RequestHandler subclasses to enable streaming body support.

This decorator implies the following changes:

  • HTTPServerRequest.body is undefined, and body arguments will not be included in RequestHandler.get_argument.
  • RequestHandler.prepare is called when the request headers have been read instead of after the entire body has been read.
  • The subclass must define a method data_received(self, data):, which will be called zero or more times as data is available. Note that if the request has an empty body, data_received may not be called.
  • prepare and data_received may return Futures (such as via @gen.coroutine, in which case the next method will not be called until those futures have completed.
  • The regular HTTP method (post, put, etc) will be called after the entire body has been read.

There is a subtle interaction between data_received and asynchronous prepare: The first call to data_received may occur at any point after the call to prepare has returned or yielded.

Everything else
exception tornado.web.HTTPError(status_code=500, log_message=None, *args, **kwargs)[source]

An exception that will turn into an HTTP error response.

Raising an HTTPError is a convenient alternative to calling RequestHandler.send_error since it automatically ends the current function.

To customize the response sent with an HTTPError, override RequestHandler.write_error.

Parameters:
  • status_code (int) – HTTP status code. Must be listed in httplib.responses unless the reason keyword argument is given.
  • log_message (string) – Message to be written to the log for this error (will not be shown to the user unless the Application is in debug mode). May contain %s-style placeholders, which will be filled in with remaining positional parameters.
  • reason (string) – Keyword-only argument. The HTTP “reason” phrase to pass in the status line along with status_code. Normally determined automatically from status_code, but can be used to use a non-standard numeric code.
exception tornado.web.Finish[source]

An exception that ends the request without producing an error response.

When Finish is raised in a RequestHandler, the request will end (calling RequestHandler.finish if it hasn’t already been called), but the error-handling methods (including RequestHandler.write_error) will not be called.

If Finish() was created with no arguments, the pending response will be sent as-is. If Finish() was given an argument, that argument will be passed to RequestHandler.finish().

This can be a more convenient way to implement custom error pages than overriding write_error (especially in library code):

if self.current_user is None:
    self.set_status(401)
    self.set_header('WWW-Authenticate', 'Basic realm="something"')
    raise Finish()

Changed in version 4.3: Arguments passed to Finish() will be passed on to RequestHandler.finish.

exception tornado.web.MissingArgumentError(arg_name)[source]

Exception raised by RequestHandler.get_argument.

This is a subclass of HTTPError, so if it is uncaught a 400 response code will be used instead of 500 (and a stack trace will not be logged).

New in version 3.1.

class tornado.web.UIModule(handler)[source]

A re-usable, modular UI unit on a page.

UI modules often execute additional queries, and they can include additional CSS and JavaScript that will be included in the output page, which is automatically inserted on page render.

Subclasses of UIModule must override the render method.

render(*args, **kwargs)[source]

Override in subclasses to return this module’s output.

embedded_javascript()[source]

Override to return a JavaScript string to be embedded in the page.

javascript_files()[source]

Override to return a list of JavaScript files needed by this module.

If the return values are relative paths, they will be passed to RequestHandler.static_url; otherwise they will be used as-is.

embedded_css()[source]

Override to return a CSS string that will be embedded in the page.

css_files()[source]

Override to returns a list of CSS files required by this module.

If the return values are relative paths, they will be passed to RequestHandler.static_url; otherwise they will be used as-is.

html_head()[source]

Override to return an HTML string that will be put in the <head/> element.

html_body()[source]

Override to return an HTML string that will be put at the end of the <body/> element.

render_string(path, **kwargs)[source]

Renders a template and returns it as a string.

class tornado.web.ErrorHandler(application, request, **kwargs)[source]

Generates an error response with status_code for all requests.

class tornado.web.FallbackHandler(application, request, **kwargs)[source]

A RequestHandler that wraps another HTTP server callback.

The fallback is a callable object that accepts an HTTPServerRequest, such as an Application or tornado.wsgi.WSGIContainer. This is most useful to use both Tornado RequestHandlers and WSGI in the same server. Typical usage:

wsgi_app = tornado.wsgi.WSGIContainer(
    django.core.handlers.wsgi.WSGIHandler())
application = tornado.web.Application([
    (r"/foo", FooHandler),
    (r".*", FallbackHandler, dict(fallback=wsgi_app),
])
class tornado.web.RedirectHandler(application, request, **kwargs)[source]

Redirects the client to the given URL for all GET requests.

You should provide the keyword argument url to the handler, e.g.:

application = web.Application([
    (r"/oldpath", web.RedirectHandler, {"url": "/newpath"}),
])
class tornado.web.StaticFileHandler(application, request, **kwargs)[source]

A simple handler that can serve static content from a directory.

A StaticFileHandler is configured automatically if you pass the static_path keyword argument to Application. This handler can be customized with the static_url_prefix, static_handler_class, and static_handler_args settings.

To map an additional path to this handler for a static data directory you would add a line to your application like:

application = web.Application([
    (r"/content/(.*)", web.StaticFileHandler, {"path": "/var/www"}),
])

The handler constructor requires a path argument, which specifies the local root directory of the content to be served.

Note that a capture group in the regex is required to parse the value for the path argument to the get() method (different than the constructor argument above); see URLSpec for details.

To serve a file like index.html automatically when a directory is requested, set static_handler_args=dict(default_filename="index.html") in your application settings, or add default_filename as an initializer argument for your StaticFileHandler.

To maximize the effectiveness of browser caching, this class supports versioned urls (by default using the argument ?v=). If a version is given, we instruct the browser to cache this file indefinitely. make_static_url (also available as RequestHandler.static_url) can be used to construct a versioned url.

This handler is intended primarily for use in development and light-duty file serving; for heavy traffic it will be more efficient to use a dedicated static file server (such as nginx or Apache). We support the HTTP Accept-Ranges mechanism to return partial content (because some browsers require this functionality to be present to seek in HTML5 audio or video).

Subclassing notes

This class is designed to be extensible by subclassing, but because of the way static urls are generated with class methods rather than instance methods, the inheritance patterns are somewhat unusual. Be sure to use the @classmethod decorator when overriding a class method. Instance methods may use the attributes self.path self.absolute_path, and self.modified.

Subclasses should only override methods discussed in this section; overriding other methods is error-prone. Overriding StaticFileHandler.get is particularly problematic due to the tight coupling with compute_etag and other methods.

To change the way static urls are generated (e.g. to match the behavior of another server or CDN), override make_static_url, parse_url_path, get_cache_time, and/or get_version.

To replace all interaction with the filesystem (e.g. to serve static content from a database), override get_content, get_content_size, get_modified_time, get_absolute_path, and validate_absolute_path.

Changed in version 3.1: Many of the methods for subclasses were added in Tornado 3.1.

compute_etag()[source]

Sets the Etag header based on static url version.

This allows efficient If-None-Match checks against cached versions, and sends the correct Etag for a partial response (i.e. the same Etag as the full file).

New in version 3.1.

set_headers()[source]

Sets the content and caching headers on the response.

New in version 3.1.

should_return_304()[source]

Returns True if the headers indicate that we should return 304.

New in version 3.1.

classmethod get_absolute_path(root, path)[source]

Returns the absolute location of path relative to root.

root is the path configured for this StaticFileHandler (in most cases the static_path Application setting).

This class method may be overridden in subclasses. By default it returns a filesystem path, but other strings may be used as long as they are unique and understood by the subclass’s overridden get_content.

New in version 3.1.

validate_absolute_path(root, absolute_path)[source]

Validate and return the absolute path.

root is the configured path for the StaticFileHandler, and path is the result of get_absolute_path

This is an instance method called during request processing, so it may raise HTTPError or use methods like RequestHandler.redirect (return None after redirecting to halt further processing). This is where 404 errors for missing files are generated.

This method may modify the path before returning it, but note that any such modifications will not be understood by make_static_url.

In instance methods, this method’s result is available as self.absolute_path.

New in version 3.1.

classmethod get_content(abspath, start=None, end=None)[source]

Retrieve the content of the requested resource which is located at the given absolute path.

This class method may be overridden by subclasses. Note that its signature is different from other overridable class methods (no settings argument); this is deliberate to ensure that abspath is able to stand on its own as a cache key.

This method should either return a byte string or an iterator of byte strings. The latter is preferred for large files as it helps reduce memory fragmentation.

New in version 3.1.

classmethod get_content_version(abspath)[source]

Returns a version string for the resource at the given path.

This class method may be overridden by subclasses. The default implementation is a hash of the file’s contents.

New in version 3.1.

get_content_size()[source]

Retrieve the total size of the resource at the given path.

This method may be overridden by subclasses.

New in version 3.1.

Changed in version 4.0: This method is now always called, instead of only when partial results are requested.

get_modified_time()[source]

Returns the time that self.absolute_path was last modified.

May be overridden in subclasses. Should return a datetime object or None.

New in version 3.1.

get_content_type()[source]

Returns the Content-Type header to be used for this request.

New in version 3.1.

set_extra_headers(path)[source]

For subclass to add extra headers to the response

get_cache_time(path, modified, mime_type)[source]

Override to customize cache control behavior.

Return a positive number of seconds to make the result cacheable for that amount of time or 0 to mark resource as cacheable for an unspecified amount of time (subject to browser heuristics).

By default returns cache expiry of 10 years for resources requested with v argument.

classmethod make_static_url(settings, path, include_version=True)[source]

Constructs a versioned url for the given path.

This method may be overridden in subclasses (but note that it is a class method rather than an instance method). Subclasses are only required to implement the signature make_static_url(cls, settings, path); other keyword arguments may be passed through static_url but are not standard.

settings is the Application.settings dictionary. path is the static path being requested. The url returned should be relative to the current host.

include_version determines whether the generated URL should include the query string containing the version hash of the file corresponding to the given path.

parse_url_path(url_path)[source]

Converts a static URL path into a filesystem path.

url_path is the path component of the URL with static_url_prefix removed. The return value should be filesystem path relative to static_path.

This is the inverse of make_static_url.

classmethod get_version(settings, path)[source]

Generate the version string to be used in static URLs.

settings is the Application.settings dictionary and path is the relative location of the requested asset on the filesystem. The returned value should be a string, or None if no version could be determined.

Changed in version 3.1: This method was previously recommended for subclasses to override; get_content_version is now preferred as it allows the base class to handle caching of the result.

tornado.template — Flexible output generation

A simple template system that compiles templates to Python code.

Basic usage looks like:

t = template.Template("<html>{{ myvalue }}</html>")
print t.generate(myvalue="XXX")

Loader is a class that loads templates from a root directory and caches the compiled templates:

loader = template.Loader("/home/btaylor")
print loader.load("test.html").generate(myvalue="XXX")

We compile all templates to raw Python. Error-reporting is currently... uh, interesting. Syntax for the templates:

### base.html
<html>
  <head>
    <title>{% block title %}Default title{% end %}</title>
  </head>
  <body>
    <ul>
      {% for student in students %}
        {% block student %}
          <li>{{ escape(student.name) }}</li>
        {% end %}
      {% end %}
    </ul>
  </body>
</html>

### bold.html
{% extends "base.html" %}

{% block title %}A bolder title{% end %}

{% block student %}
  <li><span style="bold">{{ escape(student.name) }}</span></li>
{% end %}

Unlike most other template systems, we do not put any restrictions on the expressions you can include in your statements. if and for blocks get translated exactly into Python, so you can do complex expressions like:

{% for student in [p for p in people if p.student and p.age > 23] %}
  <li>{{ escape(student.name) }}</li>
{% end %}

Translating directly to Python means you can apply functions to expressions easily, like the escape() function in the examples above. You can pass functions in to your template just like any other variable (In a RequestHandler, override RequestHandler.get_template_namespace):

### Python code
def add(x, y):
   return x + y
template.execute(add=add)

### The template
{{ add(1, 2) }}

We provide the functions escape(), url_escape(), json_encode(), and squeeze() to all templates by default.

Typical applications do not create Template or Loader instances by hand, but instead use the render and render_string methods of tornado.web.RequestHandler, which load templates automatically based on the template_path Application setting.

Variable names beginning with _tt_ are reserved by the template system and should not be used by application code.

Syntax Reference

Template expressions are surrounded by double curly braces: {{ ... }}. The contents may be any python expression, which will be escaped according to the current autoescape setting and inserted into the output. Other template directives use {% %}. These tags may be escaped as {{! and {%! if you need to include a literal {{ or {% in the output.

To comment out a section so that it is omitted from the output, surround it with {# ... #}.

{% apply *function* %}...{% end %}

Applies a function to the output of all template code between apply and end:

{% apply linkify %}{{name}} said: {{message}}{% end %}

Note that as an implementation detail apply blocks are implemented as nested functions and thus may interact strangely with variables set via {% set %}, or the use of {% break %} or {% continue %} within loops.

{% autoescape *function* %}

Sets the autoescape mode for the current file. This does not affect other files, even those referenced by {% include %}. Note that autoescaping can also be configured globally, at the Application or Loader.:

{% autoescape xhtml_escape %}
{% autoescape None %}
{% block *name* %}...{% end %}

Indicates a named, replaceable block for use with {% extends %}. Blocks in the parent template will be replaced with the contents of the same-named block in a child template.:

<!-- base.html -->
<title>{% block title %}Default title{% end %}</title>

<!-- mypage.html -->
{% extends "base.html" %}
{% block title %}My page title{% end %}
{% comment ... %}
A comment which will be removed from the template output. Note that there is no {% end %} tag; the comment goes from the word comment to the closing %} tag.
{% extends *filename* %}
Inherit from another template. Templates that use extends should contain one or more block tags to replace content from the parent template. Anything in the child template not contained in a block tag will be ignored. For an example, see the {% block %} tag.
{% for *var* in *expr* %}...{% end %}
Same as the python for statement. {% break %} and {% continue %} may be used inside the loop.
{% from *x* import *y* %}
Same as the python import statement.
{% if *condition* %}...{% elif *condition* %}...{% else %}...{% end %}
Conditional statement - outputs the first section whose condition is true. (The elif and else sections are optional)
{% import *module* %}
Same as the python import statement.
{% include *filename* %}
Includes another template file. The included file can see all the local variables as if it were copied directly to the point of the include directive (the {% autoescape %} directive is an exception). Alternately, {% module Template(filename, **kwargs) %} may be used to include another template with an isolated namespace.
{% module *expr* %}

Renders a UIModule. The output of the UIModule is not escaped:

{% module Template("foo.html", arg=42) %}

UIModules are a feature of the tornado.web.RequestHandler class (and specifically its render method) and will not work when the template system is used on its own in other contexts.

{% raw *expr* %}
Outputs the result of the given expression without autoescaping.
{% set *x* = *y* %}
Sets a local variable.
{% try %}...{% except %}...{% else %}...{% finally %}...{% end %}
Same as the python try statement.
{% while *condition* %}... {% end %}
Same as the python while statement. {% break %} and {% continue %} may be used inside the loop.
{% whitespace *mode* %}
Sets the whitespace mode for the remainder of the current file (or until the next {% whitespace %} directive). See filter_whitespace for available options. New in Tornado 4.3.
Class reference
class tornado.template.Template(template_string, name="<string>", loader=None, compress_whitespace=None, autoescape="xhtml_escape", whitespace=None)[source]

A compiled template.

We compile into Python from the given template_string. You can generate the template from variables with generate().

Construct a Template.

Parameters:
  • template_string (str) – the contents of the template file.
  • name (str) – the filename from which the template was loaded (used for error message).
  • loader (tornado.template.BaseLoader) – the BaseLoader responsible for this template, used to resolve {% include %} and {% extend %} directives.
  • compress_whitespace (bool) – Deprecated since Tornado 4.3. Equivalent to whitespace="single" if true and whitespace="all" if false.
  • autoescape (str) – The name of a function in the template namespace, or None to disable escaping by default.
  • whitespace (str) – A string specifying treatment of whitespace; see filter_whitespace for options.

Changed in version 4.3: Added whitespace parameter; deprecated compress_whitespace.

generate(**kwargs)[source]

Generate this template with the given arguments.

class tornado.template.BaseLoader(autoescape='xhtml_escape', namespace=None, whitespace=None)[source]

Base class for template loaders.

You must use a template loader to use template constructs like {% extends %} and {% include %}. The loader caches all templates after they are loaded the first time.

Construct a template loader.

Parameters:
  • autoescape (str) – The name of a function in the template namespace, such as “xhtml_escape”, or None to disable autoescaping by default.
  • namespace (dict) – A dictionary to be added to the default template namespace, or None.
  • whitespace (str) – A string specifying default behavior for whitespace in templates; see filter_whitespace for options. Default is “single” for files ending in ”.html” and ”.js” and “all” for other files.

Changed in version 4.3: Added whitespace parameter.

reset()[source]

Resets the cache of compiled templates.

resolve_path(name, parent_path=None)[source]

Converts a possibly-relative path to absolute (used internally).

load(name, parent_path=None)[source]

Loads a template.

class tornado.template.Loader(root_directory, **kwargs)[source]

A template loader that loads from a single root directory.

class tornado.template.DictLoader(dict, **kwargs)[source]

A template loader that loads from a dictionary.

exception tornado.template.ParseError(message, filename, lineno)[source]

Raised for template syntax errors.

ParseError instances have filename and lineno attributes indicating the position of the error.

Changed in version 4.3: Added filename and lineno attributes.

tornado.template.filter_whitespace(mode, text)[source]

Transform whitespace in text according to mode.

Available modes are:

  • all: Return all whitespace unmodified.
  • single: Collapse consecutive whitespace with a single whitespace character, preserving newlines.
  • oneline: Collapse all runs of whitespace into a single space character, removing all newlines in the process.

New in version 4.3.

tornado.escape — Escaping and string manipulation

Escaping/unescaping methods for HTML, JSON, URLs, and others.

Also includes a few other miscellaneous string manipulation functions that have crept in over time.

Escaping functions
tornado.escape.xhtml_escape(value)[source]

Escapes a string so it is valid within HTML or XML.

Escapes the characters <, >, ", ', and &. When used in attribute values the escaped strings must be enclosed in quotes.

Changed in version 3.2: Added the single quote to the list of escaped characters.

tornado.escape.xhtml_unescape(value)[source]

Un-escapes an XML-escaped string.

tornado.escape.url_escape(value, plus=True)[source]

Returns a URL-encoded version of the given value.

If plus is true (the default), spaces will be represented as “+” instead of “%20”. This is appropriate for query strings but not for the path component of a URL. Note that this default is the reverse of Python’s urllib module.

New in version 3.1: The plus argument

tornado.escape.url_unescape(value, encoding='utf-8', plus=True)[source]

Decodes the given value from a URL.

The argument may be either a byte or unicode string.

If encoding is None, the result will be a byte string. Otherwise, the result is a unicode string in the specified encoding.

If plus is true (the default), plus signs will be interpreted as spaces (literal plus signs must be represented as “%2B”). This is appropriate for query strings and form-encoded values but not for the path component of a URL. Note that this default is the reverse of Python’s urllib module.

New in version 3.1: The plus argument

tornado.escape.json_encode(value)[source]

JSON-encodes the given Python object.

tornado.escape.json_decode(value)[source]

Returns Python objects for the given JSON string.

Byte/unicode conversions

These functions are used extensively within Tornado itself, but should not be directly needed by most applications. Note that much of the complexity of these functions comes from the fact that Tornado supports both Python 2 and Python 3.

tornado.escape.utf8(value)[source]

Converts a string argument to a byte string.

If the argument is already a byte string or None, it is returned unchanged. Otherwise it must be a unicode string and is encoded as utf8.

tornado.escape.to_unicode(value)[source]

Converts a string argument to a unicode string.

If the argument is already a unicode string or None, it is returned unchanged. Otherwise it must be a byte string and is decoded as utf8.

tornado.escape.native_str()

Converts a byte or unicode string into type str. Equivalent to utf8 on Python 2 and to_unicode on Python 3.

tornado.escape.to_basestring(value)[source]

Converts a string argument to a subclass of basestring.

In python2, byte and unicode strings are mostly interchangeable, so functions that deal with a user-supplied argument in combination with ascii string constants can use either and should return the type the user supplied. In python3, the two types are not interchangeable, so this method is needed to convert byte strings to unicode.

tornado.escape.recursive_unicode(obj)[source]

Walks a simple data structure, converting byte strings to unicode.

Supports lists, tuples, and dictionaries.

Miscellaneous functions
tornado.escape.linkify(text, shorten=False, extra_params='', require_protocol=False, permitted_protocols=['http', 'https'])[source]

Converts plain text into HTML with links.

For example: linkify("Hello http://tornadoweb.org!") would return Hello <a href="http://tornadoweb.org">http://tornadoweb.org</a>!

Parameters:

  • shorten: Long urls will be shortened for display.

  • extra_params: Extra text to include in the link tag, or a callable

    taking the link as an argument and returning the extra text e.g. linkify(text, extra_params='rel="nofollow" class="external"'), or:

    def extra_params_cb(url):
        if url.startswith("http://example.com"):
            return 'class="internal"'
        else:
            return 'class="external" rel="nofollow"'
    linkify(text, extra_params=extra_params_cb)
    
  • require_protocol: Only linkify urls which include a protocol. If

    this is False, urls such as www.facebook.com will also be linkified.

  • permitted_protocols: List (or set) of protocols which should be

    linkified, e.g. linkify(text, permitted_protocols=["http", "ftp", "mailto"]). It is very unsafe to include protocols such as javascript.

tornado.escape.squeeze(value)[source]

Replace all sequences of whitespace chars with a single space.

tornado.locale — Internationalization support

Translation methods for generating localized strings.

To load a locale and generate a translated string:

user_locale = tornado.locale.get("es_LA")
print user_locale.translate("Sign out")

tornado.locale.get() returns the closest matching locale, not necessarily the specific locale you requested. You can support pluralization with additional arguments to translate(), e.g.:

people = [...]
message = user_locale.translate(
    "%(list)s is online", "%(list)s are online", len(people))
print message % {"list": user_locale.list(people)}

The first string is chosen if len(people) == 1, otherwise the second string is chosen.

Applications should call one of load_translations (which uses a simple CSV format) or load_gettext_translations (which uses the .mo format supported by gettext and related tools). If neither method is called, the Locale.translate method will simply return the original string.

tornado.locale.get(*locale_codes)[source]

Returns the closest match for the given locale codes.

We iterate over all given locale codes in order. If we have a tight or a loose match for the code (e.g., “en” for “en_US”), we return the locale. Otherwise we move to the next code in the list.

By default we return en_US if no translations are found for any of the specified locales. You can change the default locale with set_default_locale().

tornado.locale.set_default_locale(code)[source]

Sets the default locale.

The default locale is assumed to be the language used for all strings in the system. The translations loaded from disk are mappings from the default locale to the destination locale. Consequently, you don’t need to create a translation file for the default locale.

tornado.locale.load_translations(directory, encoding=None)[source]

Loads translations from CSV files in a directory.

Translations are strings with optional Python-style named placeholders (e.g., My name is %(name)s) and their associated translations.

The directory should have translation files of the form LOCALE.csv, e.g. es_GT.csv. The CSV files should have two or three columns: string, translation, and an optional plural indicator. Plural indicators should be one of “plural” or “singular”. A given string can have both singular and plural forms. For example %(name)s liked this may have a different verb conjugation depending on whether %(name)s is one name or a list of names. There should be two rows in the CSV file for that string, one with plural indicator “singular”, and one “plural”. For strings with no verbs that would change on translation, simply use “unknown” or the empty string (or don’t include the column at all).

The file is read using the csv module in the default “excel” dialect. In this format there should not be spaces after the commas.

If no encoding parameter is given, the encoding will be detected automatically (among UTF-8 and UTF-16) if the file contains a byte-order marker (BOM), defaulting to UTF-8 if no BOM is present.

Example translation es_LA.csv:

"I love you","Te amo"
"%(name)s liked this","A %(name)s les gustó esto","plural"
"%(name)s liked this","A %(name)s le gustó esto","singular"

Changed in version 4.3: Added encoding parameter. Added support for BOM-based encoding detection, UTF-16, and UTF-8-with-BOM.

tornado.locale.load_gettext_translations(directory, domain)[source]

Loads translations from gettext‘s locale tree

Locale tree is similar to system’s /usr/share/locale, like:

{directory}/{lang}/LC_MESSAGES/{domain}.mo

Three steps are required to have you app translated:

  1. Generate POT translation file:

    xgettext --language=Python --keyword=_:1,2 -d mydomain file1.py file2.html etc
    
  2. Merge against existing POT file:

    msgmerge old.po mydomain.po > new.po
    
  3. Compile:

    msgfmt mydomain.po -o {directory}/pt_BR/LC_MESSAGES/mydomain.mo
    
tornado.locale.get_supported_locales()[source]

Returns a list of all the supported locale codes.

class tornado.locale.Locale(code, translations)[source]

Object representing a locale.

After calling one of load_translations or load_gettext_translations, call get or get_closest to get a Locale object.

classmethod get_closest(*locale_codes)[source]

Returns the closest match for the given locale code.

classmethod get(code)[source]

Returns the Locale for the given locale code.

If it is not supported, we raise an exception.

translate(message, plural_message=None, count=None)[source]

Returns the translation for the given message for this locale.

If plural_message is given, you must also provide count. We return plural_message when count != 1, and we return the singular form for the given message when count == 1.

format_date(date, gmt_offset=0, relative=True, shorter=False, full_format=False)[source]

Formats the given date (which should be GMT).

By default, we return a relative time (e.g., “2 minutes ago”). You can return an absolute date string with relative=False.

You can force a full format date (“July 10, 1980”) with full_format=True.

This method is primarily intended for dates in the past. For dates in the future, we fall back to full format.

format_day(date, gmt_offset=0, dow=True)[source]

Formats the given date as a day of week.

Example: “Monday, January 22”. You can remove the day of week with dow=False.

list(parts)[source]

Returns a comma-separated list for the given list of parts.

The format is, e.g., “A, B and C”, “A and B” or just “A” for lists of size 1.

friendly_number(value)[source]

Returns a comma-separated number for the given integer.

class tornado.locale.CSVLocale(code, translations)[source]

Locale implementation using tornado’s CSV translation format.

class tornado.locale.GettextLocale(code, translations)[source]

Locale implementation using the gettext module.

pgettext(context, message, plural_message=None, count=None)[source]

Allows to set context for translation, accepts plural forms.

Usage example:

pgettext("law", "right")
pgettext("good", "right")

Plural message example:

pgettext("organization", "club", "clubs", len(clubs))
pgettext("stick", "club", "clubs", len(clubs))

To generate POT file with context, add following options to step 1 of load_gettext_translations sequence:

xgettext [basic options] --keyword=pgettext:1c,2 --keyword=pgettext:1c,2,3

New in version 4.2.

tornado.websocket — Bidirectional communication to the browser

Implementation of the WebSocket protocol.

WebSockets allow for bidirectional communication between the browser and server.

WebSockets are supported in the current versions of all major browsers, although older versions that do not support WebSockets are still in use (refer to http://caniuse.com/websockets for details).

This module implements the final version of the WebSocket protocol as defined in RFC 6455. Certain browser versions (notably Safari 5.x) implemented an earlier draft of the protocol (known as “draft 76”) and are not compatible with this module.

Changed in version 4.0: Removed support for the draft 76 protocol version.

class tornado.websocket.WebSocketHandler(application, request, **kwargs)[source]

Subclass this class to create a basic WebSocket handler.

Override on_message to handle incoming messages, and use write_message to send messages to the client. You can also override open and on_close to handle opened and closed connections.

See http://dev.w3.org/html5/websockets/ for details on the JavaScript interface. The protocol is specified at http://tools.ietf.org/html/rfc6455.

Here is an example WebSocket handler that echos back all received messages back to the client:

class EchoWebSocket(tornado.websocket.WebSocketHandler):
    def open(self):
        print("WebSocket opened")

    def on_message(self, message):
        self.write_message(u"You said: " + message)

    def on_close(self):
        print("WebSocket closed")

WebSockets are not standard HTTP connections. The “handshake” is HTTP, but after the handshake, the protocol is message-based. Consequently, most of the Tornado HTTP facilities are not available in handlers of this type. The only communication methods available to you are write_message(), ping(), and close(). Likewise, your request handler class should implement open() method rather than get() or post().

If you map the handler above to /websocket in your application, you can invoke it in JavaScript with:

var ws = new WebSocket("ws://localhost:8888/websocket");
ws.onopen = function() {
   ws.send("Hello, world");
};
ws.onmessage = function (evt) {
   alert(evt.data);
};

This script pops up an alert box that says “You said: Hello, world”.

Web browsers allow any site to open a websocket connection to any other, instead of using the same-origin policy that governs other network access from javascript. This can be surprising and is a potential security hole, so since Tornado 4.0 WebSocketHandler requires applications that wish to receive cross-origin websockets to opt in by overriding the check_origin method (see that method’s docs for details). Failure to do so is the most likely cause of 403 errors when making a websocket connection.

When using a secure websocket connection (wss://) with a self-signed certificate, the connection from a browser may fail because it wants to show the “accept this certificate” dialog but has nowhere to show it. You must first visit a regular HTML page using the same certificate to accept it before the websocket connection will succeed.

Event handlers
WebSocketHandler.open(*args, **kwargs)[source]

Invoked when a new WebSocket is opened.

The arguments to open are extracted from the tornado.web.URLSpec regular expression, just like the arguments to tornado.web.RequestHandler.get.

WebSocketHandler.on_message(message)[source]

Handle incoming messages on the WebSocket

This method must be overridden.

WebSocketHandler.on_close()[source]

Invoked when the WebSocket is closed.

If the connection was closed cleanly and a status code or reason phrase was supplied, these values will be available as the attributes self.close_code and self.close_reason.

Changed in version 4.0: Added close_code and close_reason attributes.

WebSocketHandler.select_subprotocol(subprotocols)[source]

Invoked when a new WebSocket requests specific subprotocols.

subprotocols is a list of strings identifying the subprotocols proposed by the client. This method may be overridden to return one of those strings to select it, or None to not select a subprotocol. Failure to select a subprotocol does not automatically abort the connection, although clients may close the connection if none of their proposed subprotocols was selected.

Output
WebSocketHandler.write_message(message, binary=False)[source]

Sends the given message to the client of this Web Socket.

The message may be either a string or a dict (which will be encoded as json). If the binary argument is false, the message will be sent as utf8; in binary mode any byte string is allowed.

If the connection is already closed, raises WebSocketClosedError.

Changed in version 3.2: WebSocketClosedError was added (previously a closed connection would raise an AttributeError)

Changed in version 4.3: Returns a Future which can be used for flow control.

WebSocketHandler.close(code=None, reason=None)[source]

Closes this Web Socket.

Once the close handshake is successful the socket will be closed.

code may be a numeric status code, taken from the values defined in RFC 6455 section 7.4.1. reason may be a textual message about why the connection is closing. These values are made available to the client, but are not otherwise interpreted by the websocket protocol.

Changed in version 4.0: Added the code and reason arguments.

Configuration
WebSocketHandler.check_origin(origin)[source]

Override to enable support for allowing alternate origins.

The origin argument is the value of the Origin HTTP header, the url responsible for initiating this request. This method is not called for clients that do not send this header; such requests are always allowed (because all browsers that implement WebSockets support this header, and non-browser clients do not have the same cross-site security concerns).

Should return True to accept the request or False to reject it. By default, rejects all requests with an origin on a host other than this one.

This is a security protection against cross site scripting attacks on browsers, since WebSockets are allowed to bypass the usual same-origin policies and don’t use CORS headers.

To accept all cross-origin traffic (which was the default prior to Tornado 4.0), simply override this method to always return true:

def check_origin(self, origin):
    return True

To allow connections from any subdomain of your site, you might do something like:

def check_origin(self, origin):
    parsed_origin = urllib.parse.urlparse(origin)
    return parsed_origin.netloc.endswith(".mydomain.com")

New in version 4.0.

WebSocketHandler.get_compression_options()[source]

Override to return compression options for the connection.

If this method returns None (the default), compression will be disabled. If it returns a dict (even an empty one), it will be enabled. The contents of the dict may be used to control the memory and CPU usage of the compression, but no such options are currently implemented.

New in version 4.1.

WebSocketHandler.set_nodelay(value)[source]

Set the no-delay flag for this stream.

By default, small messages may be delayed and/or combined to minimize the number of packets sent. This can sometimes cause 200-500ms delays due to the interaction between Nagle’s algorithm and TCP delayed ACKs. To reduce this delay (at the expense of possibly increasing bandwidth usage), call self.set_nodelay(True) once the websocket connection is established.

See BaseIOStream.set_nodelay for additional details.

New in version 3.1.

Other
WebSocketHandler.ping(data)[source]

Send ping frame to the remote end.

WebSocketHandler.on_pong(data)[source]

Invoked when the response to a ping frame is received.

exception tornado.websocket.WebSocketClosedError[source]

Raised by operations on a closed connection.

New in version 3.2.

Client-side support
tornado.websocket.websocket_connect(url, io_loop=None, callback=None, connect_timeout=None, on_message_callback=None, compression_options=None)[source]

Client-side websocket support.

Takes a url and returns a Future whose result is a WebSocketClientConnection.

compression_options is interpreted in the same way as the return value of WebSocketHandler.get_compression_options.

The connection supports two styles of operation. In the coroutine style, the application typically calls read_message in a loop:

conn = yield websocket_connect(url)
while True:
    msg = yield conn.read_message()
    if msg is None: break
    # Do something with msg

In the callback style, pass an on_message_callback to websocket_connect. In both styles, a message of None indicates that the connection has been closed.

Changed in version 3.2: Also accepts HTTPRequest objects in place of urls.

Changed in version 4.1: Added compression_options and on_message_callback. The io_loop argument is deprecated.

class tornado.websocket.WebSocketClientConnection(io_loop, request, on_message_callback=None, compression_options=None)[source]

WebSocket client connection.

This class should not be instantiated directly; use the websocket_connect function instead.

close(code=None, reason=None)[source]

Closes the websocket connection.

code and reason are documented under WebSocketHandler.close.

New in version 3.2.

Changed in version 4.0: Added the code and reason arguments.

write_message(message, binary=False)[source]

Sends a message to the WebSocket server.

read_message(callback=None)[source]

Reads a message from the WebSocket server.

If on_message_callback was specified at WebSocket initialization, this function will never return messages

Returns a future whose result is the message, or None if the connection is closed. If a callback argument is given it will be called with the future when it is ready.

HTTP servers and clients

tornado.httpserver — Non-blocking HTTP server

A non-blocking, single-threaded HTTP server.

Typical applications have little direct interaction with the HTTPServer class except to start a server at the beginning of the process (and even that is often done indirectly via tornado.web.Application.listen).

Changed in version 4.0: The HTTPRequest class that used to live in this module has been moved to tornado.httputil.HTTPServerRequest. The old name remains as an alias.

HTTP Server
class tornado.httpserver.HTTPServer(*args, **kwargs)[source]

A non-blocking, single-threaded HTTP server.

A server is defined by a subclass of HTTPServerConnectionDelegate, or, for backwards compatibility, a callback that takes an HTTPServerRequest as an argument. The delegate is usually a tornado.web.Application.

HTTPServer supports keep-alive connections by default (automatically for HTTP/1.1, or for HTTP/1.0 when the client requests Connection: keep-alive).

If xheaders is True, we support the X-Real-Ip/X-Forwarded-For and X-Scheme/X-Forwarded-Proto headers, which override the remote IP and URI scheme/protocol for all requests. These headers are useful when running Tornado behind a reverse proxy or load balancer. The protocol argument can also be set to https if Tornado is run behind an SSL-decoding proxy that does not set one of the supported xheaders.

To make this server serve SSL traffic, send the ssl_options keyword argument with an ssl.SSLContext object. For compatibility with older versions of Python ssl_options may also be a dictionary of keyword arguments for the ssl.wrap_socket method.:

ssl_ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
ssl_ctx.load_cert_chain(os.path.join(data_dir, "mydomain.crt"),
                        os.path.join(data_dir, "mydomain.key"))
HTTPServer(applicaton, ssl_options=ssl_ctx)

HTTPServer initialization follows one of three patterns (the initialization methods are defined on tornado.tcpserver.TCPServer):

  1. listen: simple single-process:

    server = HTTPServer(app)
    server.listen(8888)
    IOLoop.current().start()
    

    In many cases, tornado.web.Application.listen can be used to avoid the need to explicitly create the HTTPServer.

  2. bind/start: simple multi-process:

    server = HTTPServer(app)
    server.bind(8888)
    server.start(0)  # Forks multiple sub-processes
    IOLoop.current().start()
    

    When using this interface, an IOLoop must not be passed to the HTTPServer constructor. start will always start the server on the default singleton IOLoop.

  3. add_sockets: advanced multi-process:

    sockets = tornado.netutil.bind_sockets(8888)
    tornado.process.fork_processes(0)
    server = HTTPServer(app)
    server.add_sockets(sockets)
    IOLoop.current().start()
    

    The add_sockets interface is more complicated, but it can be used with tornado.process.fork_processes to give you more flexibility in when the fork happens. add_sockets can also be used in single-process servers if you want to create your listening sockets in some way other than tornado.netutil.bind_sockets.

Changed in version 4.0: Added decompress_request, chunk_size, max_header_size, idle_connection_timeout, body_timeout, max_body_size arguments. Added support for HTTPServerConnectionDelegate instances as request_callback.

Changed in version 4.1: HTTPServerConnectionDelegate.start_request is now called with two arguments (server_conn, request_conn) (in accordance with the documentation) instead of one (request_conn).

Changed in version 4.2: HTTPServer is now a subclass of tornado.util.Configurable.

tornado.httpclient — Asynchronous HTTP client

Blocking and non-blocking HTTP client interfaces.

This module defines a common interface shared by two implementations, simple_httpclient and curl_httpclient. Applications may either instantiate their chosen implementation class directly or use the AsyncHTTPClient class from this module, which selects an implementation that can be overridden with the AsyncHTTPClient.configure method.

The default implementation is simple_httpclient, and this is expected to be suitable for most users’ needs. However, some applications may wish to switch to curl_httpclient for reasons such as the following:

  • curl_httpclient has some features not found in simple_httpclient, including support for HTTP proxies and the ability to use a specified network interface.
  • curl_httpclient is more likely to be compatible with sites that are not-quite-compliant with the HTTP spec, or sites that use little-exercised features of HTTP.
  • curl_httpclient is faster.
  • curl_httpclient was the default prior to Tornado 2.0.

Note that if you are using curl_httpclient, it is highly recommended that you use a recent version of libcurl and pycurl. Currently the minimum supported version of libcurl is 7.21.1, and the minimum version of pycurl is 7.18.2. It is highly recommended that your libcurl installation is built with asynchronous DNS resolver (threaded or c-ares), otherwise you may encounter various problems with request timeouts (for more information, see http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTCONNECTTIMEOUTMS and comments in curl_httpclient.py).

To select curl_httpclient, call AsyncHTTPClient.configure at startup:

AsyncHTTPClient.configure("tornado.curl_httpclient.CurlAsyncHTTPClient")
HTTP client interfaces
class tornado.httpclient.HTTPClient(async_client_class=None, **kwargs)[source]

A blocking HTTP client.

This interface is provided for convenience and testing; most applications that are running an IOLoop will want to use AsyncHTTPClient instead. Typical usage looks like this:

http_client = httpclient.HTTPClient()
try:
    response = http_client.fetch("http://www.google.com/")
    print response.body
except httpclient.HTTPError as e:
    # HTTPError is raised for non-200 responses; the response
    # can be found in e.response.
    print("Error: " + str(e))
except Exception as e:
    # Other errors are possible, such as IOError.
    print("Error: " + str(e))
http_client.close()
close()[source]

Closes the HTTPClient, freeing any resources used.

fetch(request, **kwargs)[source]

Executes a request, returning an HTTPResponse.

The request may be either a string URL or an HTTPRequest object. If it is a string, we construct an HTTPRequest using any additional kwargs: HTTPRequest(request, **kwargs)

If an error occurs during the fetch, we raise an HTTPError unless the raise_error keyword argument is set to False.

class tornado.httpclient.AsyncHTTPClient[source]

An non-blocking HTTP client.

Example usage:

def handle_request(response):
    if response.error:
        print "Error:", response.error
    else:
        print response.body

http_client = AsyncHTTPClient()
http_client.fetch("http://www.google.com/", handle_request)

The constructor for this class is magic in several respects: It actually creates an instance of an implementation-specific subclass, and instances are reused as a kind of pseudo-singleton (one per IOLoop). The keyword argument force_instance=True can be used to suppress this singleton behavior. Unless force_instance=True is used, no arguments other than io_loop should be passed to the AsyncHTTPClient constructor. The implementation subclass as well as arguments to its constructor can be set with the static method configure()

All AsyncHTTPClient implementations support a defaults keyword argument, which can be used to set default values for HTTPRequest attributes. For example:

AsyncHTTPClient.configure(
    None, defaults=dict(user_agent="MyUserAgent"))
# or with force_instance:
client = AsyncHTTPClient(force_instance=True,
    defaults=dict(user_agent="MyUserAgent"))

Changed in version 4.1: The io_loop argument is deprecated.

close()[source]

Destroys this HTTP client, freeing any file descriptors used.

This method is not needed in normal use due to the way that AsyncHTTPClient objects are transparently reused. close() is generally only necessary when either the IOLoop is also being closed, or the force_instance=True argument was used when creating the AsyncHTTPClient.

No other methods may be called on the AsyncHTTPClient after close().

fetch(request, callback=None, raise_error=True, **kwargs)[source]

Executes a request, asynchronously returning an HTTPResponse.

The request may be either a string URL or an HTTPRequest object. If it is a string, we construct an HTTPRequest using any additional kwargs: HTTPRequest(request, **kwargs)

This method returns a Future whose result is an HTTPResponse. By default, the Future will raise an HTTPError if the request returned a non-200 response code. Instead, if raise_error is set to False, the response will always be returned regardless of the response code.

If a callback is given, it will be invoked with the HTTPResponse. In the callback interface, HTTPError is not automatically raised. Instead, you must check the response’s error attribute or call its rethrow method.

classmethod configure(impl, **kwargs)[source]

Configures the AsyncHTTPClient subclass to use.

AsyncHTTPClient() actually creates an instance of a subclass. This method may be called with either a class object or the fully-qualified name of such a class (or None to use the default, SimpleAsyncHTTPClient)

If additional keyword arguments are given, they will be passed to the constructor of each subclass instance created. The keyword argument max_clients determines the maximum number of simultaneous fetch() operations that can execute in parallel on each IOLoop. Additional arguments may be supported depending on the implementation class in use.

Example:

AsyncHTTPClient.configure("tornado.curl_httpclient.CurlAsyncHTTPClient")
Request objects
class tornado.httpclient.HTTPRequest(url, method='GET', headers=None, body=None, auth_username=None, auth_password=None, auth_mode=None, connect_timeout=None, request_timeout=None, if_modified_since=None, follow_redirects=None, max_redirects=None, user_agent=None, use_gzip=None, network_interface=None, streaming_callback=None, header_callback=None, prepare_curl_callback=None, proxy_host=None, proxy_port=None, proxy_username=None, proxy_password=None, allow_nonstandard_methods=None, validate_cert=None, ca_certs=None, allow_ipv6=None, client_key=None, client_cert=None, body_producer=None, expect_100_continue=False, decompress_response=None, ssl_options=None)[source]

HTTP client request object.

All parameters except url are optional.

Parameters:
  • url (string) – URL to fetch
  • method (string) – HTTP method, e.g. “GET” or “POST”
  • headers (HTTPHeaders or dict) – Additional HTTP headers to pass on the request
  • body – HTTP request body as a string (byte or unicode; if unicode the utf-8 encoding will be used)
  • body_producer – Callable used for lazy/asynchronous request bodies. It is called with one argument, a write function, and should return a Future. It should call the write function with new data as it becomes available. The write function returns a Future which can be used for flow control. Only one of body and body_producer may be specified. body_producer is not supported on curl_httpclient. When using body_producer it is recommended to pass a Content-Length in the headers as otherwise chunked encoding will be used, and many servers do not support chunked encoding on requests. New in Tornado 4.0
  • auth_username (string) – Username for HTTP authentication
  • auth_password (string) – Password for HTTP authentication
  • auth_mode (string) – Authentication mode; default is “basic”. Allowed values are implementation-defined; curl_httpclient supports “basic” and “digest”; simple_httpclient only supports “basic”
  • connect_timeout (float) – Timeout for initial connection in seconds
  • request_timeout (float) – Timeout for entire request in seconds
  • if_modified_since (datetime or float) – Timestamp for If-Modified-Since header
  • follow_redirects (bool) – Should redirects be followed automatically or return the 3xx response?
  • max_redirects (int) – Limit for follow_redirects
  • user_agent (string) – String to send as User-Agent header
  • decompress_response (bool) – Request a compressed response from the server and decompress it after downloading. Default is True. New in Tornado 4.0.
  • use_gzip (bool) – Deprecated alias for decompress_response since Tornado 4.0.
  • network_interface (string) – Network interface to use for request. curl_httpclient only; see note below.
  • streaming_callback (callable) – If set, streaming_callback will be run with each chunk of data as it is received, and HTTPResponse.body and HTTPResponse.buffer will be empty in the final response.
  • header_callback (callable) – If set, header_callback will be run with each header line as it is received (including the first line, e.g. HTTP/1.0 200 OK\r\n, and a final line containing only \r\n. All lines include the trailing newline characters). HTTPResponse.headers will be empty in the final response. This is most useful in conjunction with streaming_callback, because it’s the only way to get access to header data while the request is in progress.
  • prepare_curl_callback (callable) – If set, will be called with a pycurl.Curl object to allow the application to make additional setopt calls.
  • proxy_host (string) – HTTP proxy hostname. To use proxies, proxy_host and proxy_port must be set; proxy_username and proxy_pass are optional. Proxies are currently only supported with curl_httpclient.
  • proxy_port (int) – HTTP proxy port
  • proxy_username (string) – HTTP proxy username
  • proxy_password (string) – HTTP proxy password
  • allow_nonstandard_methods (bool) – Allow unknown values for method argument?
  • validate_cert (bool) – For HTTPS requests, validate the server’s certificate?
  • ca_certs (string) – filename of CA certificates in PEM format, or None to use defaults. See note below when used with curl_httpclient.
  • client_key (string) – Filename for client SSL key, if any. See note below when used with curl_httpclient.
  • client_cert (string) – Filename for client SSL certificate, if any. See note below when used with curl_httpclient.
  • ssl_options (ssl.SSLContext) – ssl.SSLContext object for use in simple_httpclient (unsupported by curl_httpclient). Overrides validate_cert, ca_certs, client_key, and client_cert.
  • allow_ipv6 (bool) – Use IPv6 when available? Default is true.
  • expect_100_continue (bool) – If true, send the Expect: 100-continue header and wait for a continue response before sending the request body. Only supported with simple_httpclient.

Note

When using curl_httpclient certain options may be inherited by subsequent fetches because pycurl does not allow them to be cleanly reset. This applies to the ca_certs, client_key, client_cert, and network_interface arguments. If you use these options, you should pass them on every request (you don’t have to always use the same values, but it’s not possible to mix requests that specify these options with ones that use the defaults).

New in version 3.1: The auth_mode argument.

New in version 4.0: The body_producer and expect_100_continue arguments.

New in version 4.2: The ssl_options argument.

Response objects
class tornado.httpclient.HTTPResponse(request, code, headers=None, buffer=None, effective_url=None, error=None, request_time=None, time_info=None, reason=None)[source]

HTTP Response object.

Attributes:

  • request: HTTPRequest object
  • code: numeric HTTP status code, e.g. 200 or 404
  • reason: human-readable reason phrase describing the status code
  • headers: tornado.httputil.HTTPHeaders object
  • effective_url: final location of the resource after following any redirects
  • buffer: cStringIO object for response body
  • body: response body as string (created on demand from self.buffer)
  • error: Exception object, if any
  • request_time: seconds from request start to finish
  • time_info: dictionary of diagnostic timing information from the request. Available data are subject to change, but currently uses timings available from http://curl.haxx.se/libcurl/c/curl_easy_getinfo.html, plus queue, which is the delay (if any) introduced by waiting for a slot under AsyncHTTPClient‘s max_clients setting.
rethrow()[source]

If there was an error on the request, raise an HTTPError.

Exceptions
exception tornado.httpclient.HTTPError(code, message=None, response=None)[source]

Exception thrown for an unsuccessful HTTP request.

Attributes:

  • code - HTTP error integer error code, e.g. 404. Error code 599 is used when no HTTP response was received, e.g. for a timeout.
  • response - HTTPResponse object, if any.

Note that if follow_redirects is False, redirects become HTTPErrors, and you can look at error.response.headers['Location'] to see the destination of the redirect.

Command-line interface

This module provides a simple command-line interface to fetch a url using Tornado’s HTTP client. Example usage:

# Fetch the url and print its body
python -m tornado.httpclient http://www.google.com

# Just print the headers
python -m tornado.httpclient --print_headers --print_body=false http://www.google.com
Implementations
class tornado.simple_httpclient.SimpleAsyncHTTPClient[source]

Non-blocking HTTP client with no external dependencies.

This class implements an HTTP 1.1 client on top of Tornado’s IOStreams. Some features found in the curl-based AsyncHTTPClient are not yet supported. In particular, proxies are not supported, connections are not reused, and callers cannot select the network interface to be used.

initialize(io_loop, max_clients=10, hostname_mapping=None, max_buffer_size=104857600, resolver=None, defaults=None, max_header_size=None, max_body_size=None)[source]

Creates a AsyncHTTPClient.

Only a single AsyncHTTPClient instance exists per IOLoop in order to provide limitations on the number of pending connections. force_instance=True may be used to suppress this behavior.

Note that because of this implicit reuse, unless force_instance is used, only the first call to the constructor actually uses its arguments. It is recommended to use the configure method instead of the constructor to ensure that arguments take effect.

max_clients is the number of concurrent requests that can be in progress; when this limit is reached additional requests will be queued. Note that time spent waiting in this queue still counts against the request_timeout.

hostname_mapping is a dictionary mapping hostnames to IP addresses. It can be used to make local DNS changes when modifying system-wide settings like /etc/hosts is not possible or desirable (e.g. in unittests).

max_buffer_size (default 100MB) is the number of bytes that can be read into memory at once. max_body_size (defaults to max_buffer_size) is the largest response body that the client will accept. Without a streaming_callback, the smaller of these two limits applies; with a streaming_callback only max_body_size does.

Changed in version 4.2: Added the max_body_size argument.

class tornado.curl_httpclient.CurlAsyncHTTPClient(io_loop, max_clients=10, defaults=None)

libcurl-based HTTP client.

tornado.httputil — Manipulate HTTP headers and URLs

HTTP utility code shared by clients and servers.

This module also defines the HTTPServerRequest class which is exposed via tornado.web.RequestHandler.request.

class tornado.httputil.HTTPHeaders(*args, **kwargs)[source]

A dictionary that maintains Http-Header-Case for all keys.

Supports multiple values per key via a pair of new methods, add() and get_list(). The regular dictionary interface returns a single value per key, with multiple values joined by a comma.

>>> h = HTTPHeaders({"content-type": "text/html"})
>>> list(h.keys())
['Content-Type']
>>> h["Content-Type"]
'text/html'
>>> h.add("Set-Cookie", "A=B")
>>> h.add("Set-Cookie", "C=D")
>>> h["set-cookie"]
'A=B,C=D'
>>> h.get_list("set-cookie")
['A=B', 'C=D']
>>> for (k,v) in sorted(h.get_all()):
...    print('%s: %s' % (k,v))
...
Content-Type: text/html
Set-Cookie: A=B
Set-Cookie: C=D
add(name, value)[source]

Adds a new value for the given key.

get_list(name)[source]

Returns all values for the given header as a list.

get_all()[source]

Returns an iterable of all (name, value) pairs.

If a header has multiple values, multiple pairs will be returned with the same name.

parse_line(line)[source]

Updates the dictionary with a single header line.

>>> h = HTTPHeaders()
>>> h.parse_line("Content-Type: text/html")
>>> h.get('content-type')
'text/html'
classmethod parse(headers)[source]

Returns a dictionary from HTTP header text.

>>> h = HTTPHeaders.parse("Content-Type: text/html\r\nContent-Length: 42\r\n")
>>> sorted(h.items())
[('Content-Length', '42'), ('Content-Type', 'text/html')]
class tornado.httputil.HTTPServerRequest(method=None, uri=None, version='HTTP/1.0', headers=None, body=None, host=None, files=None, connection=None, start_line=None)[source]

A single HTTP request.

All attributes are type str unless otherwise noted.

method

HTTP request method, e.g. “GET” or “POST”

uri

The requested uri.

path

The path portion of uri

query

The query portion of uri

version

HTTP version specified in request, e.g. “HTTP/1.1”

headers

HTTPHeaders dictionary-like object for request headers. Acts like a case-insensitive dictionary with additional methods for repeated headers.

body

Request body, if present, as a byte string.

remote_ip

Client’s IP address as a string. If HTTPServer.xheaders is set, will pass along the real IP address provided by a load balancer in the X-Real-Ip or X-Forwarded-For header.

Changed in version 3.1: The list format of X-Forwarded-For is now supported.

protocol

The protocol used, either “http” or “https”. If HTTPServer.xheaders is set, will pass along the protocol used by a load balancer if reported via an X-Scheme header.

host

The requested hostname, usually taken from the Host header.

arguments

GET/POST arguments are available in the arguments property, which maps arguments names to lists of values (to support multiple values for individual names). Names are of type str, while arguments are byte strings. Note that this is different from RequestHandler.get_argument, which returns argument values as unicode strings.

query_arguments

Same format as arguments, but contains only arguments extracted from the query string.

New in version 3.2.

body_arguments

Same format as arguments, but contains only arguments extracted from the request body.

New in version 3.2.

files

File uploads are available in the files property, which maps file names to lists of HTTPFile.

connection

An HTTP request is attached to a single HTTP connection, which can be accessed through the “connection” attribute. Since connections are typically kept open in HTTP/1.1, multiple requests can be handled sequentially on a single connection.

Changed in version 4.0: Moved from tornado.httpserver.HTTPRequest.

supports_http_1_1()[source]

Returns True if this request supports HTTP/1.1 semantics.

Deprecated since version 4.0: Applications are less likely to need this information with the introduction of HTTPConnection. If you still need it, access the version attribute directly.

cookies

A dictionary of Cookie.Morsel objects.

write(chunk, callback=None)[source]

Writes the given chunk to the response stream.

Deprecated since version 4.0: Use request.connection and the HTTPConnection methods to write the response.

finish()[source]

Finishes this HTTP request on the open connection.

Deprecated since version 4.0: Use request.connection and the HTTPConnection methods to write the response.

full_url()[source]

Reconstructs the full URL for this request.

request_time()[source]

Returns the amount of time it took for this request to execute.

get_ssl_certificate(binary_form=False)[source]

Returns the client’s SSL certificate, if any.

To use client certificates, the HTTPServer’s ssl.SSLContext.verify_mode field must be set, e.g.:

ssl_ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
ssl_ctx.load_cert_chain("foo.crt", "foo.key")
ssl_ctx.load_verify_locations("cacerts.pem")
ssl_ctx.verify_mode = ssl.CERT_REQUIRED
server = HTTPServer(app, ssl_options=ssl_ctx)

By default, the return value is a dictionary (or None, if no client certificate is present). If binary_form is true, a DER-encoded form of the certificate is returned instead. See SSLSocket.getpeercert() in the standard library for more details. http://docs.python.org/library/ssl.html#sslsocket-objects

exception tornado.httputil.HTTPInputError[source]

Exception class for malformed HTTP requests or responses from remote sources.

New in version 4.0.

exception tornado.httputil.HTTPOutputError[source]

Exception class for errors in HTTP output.

New in version 4.0.

class tornado.httputil.HTTPServerConnectionDelegate[source]

Implement this interface to handle requests from HTTPServer.

New in version 4.0.

start_request(server_conn, request_conn)[source]

This method is called by the server when a new request has started.

Parameters:
  • server_conn – is an opaque object representing the long-lived (e.g. tcp-level) connection.
  • request_conn – is a HTTPConnection object for a single request/response exchange.

This method should return a HTTPMessageDelegate.

on_close(server_conn)[source]

This method is called when a connection has been closed.

Parameters:server_conn – is a server connection that has previously been passed to start_request.
class tornado.httputil.HTTPMessageDelegate[source]

Implement this interface to handle an HTTP request or response.

New in version 4.0.

headers_received(start_line, headers)[source]

Called when the HTTP headers have been received and parsed.

Parameters:

Some HTTPConnection methods can only be called during headers_received.

May return a Future; if it does the body will not be read until it is done.

data_received(chunk)[source]

Called when a chunk of data has been received.

May return a Future for flow control.

finish()[source]

Called after the last chunk of data has been received.

on_connection_close()[source]

Called if the connection is closed without finishing the request.

If headers_received is called, either finish or on_connection_close will be called, but not both.

class tornado.httputil.HTTPConnection[source]

Applications use this interface to write their responses.

New in version 4.0.

write_headers(start_line, headers, chunk=None, callback=None)[source]

Write an HTTP header block.

Parameters:
  • start_line – a RequestStartLine or ResponseStartLine.
  • headers – a HTTPHeaders instance.
  • chunk – the first (optional) chunk of data. This is an optimization so that small responses can be written in the same call as their headers.
  • callback – a callback to be run when the write is complete.

The version field of start_line is ignored.

Returns a Future if no callback is given.

write(chunk, callback=None)[source]

Writes a chunk of body data.

The callback will be run when the write is complete. If no callback is given, returns a Future.

finish()[source]

Indicates that the last body data has been written.

tornado.httputil.url_concat(url, args)[source]

Concatenate url and arguments regardless of whether url has existing query parameters.

args may be either a dictionary or a list of key-value pairs (the latter allows for multiple values with the same key.

>>> url_concat("http://example.com/foo", dict(c="d"))
'http://example.com/foo?c=d'
>>> url_concat("http://example.com/foo?a=b", dict(c="d"))
'http://example.com/foo?a=b&c=d'
>>> url_concat("http://example.com/foo?a=b", [("c", "d"), ("c", "d2")])
'http://example.com/foo?a=b&c=d&c=d2'
class tornado.httputil.HTTPFile[source]

Represents a file uploaded via a form.

For backwards compatibility, its instance attributes are also accessible as dictionary keys.

  • filename
  • body
  • content_type
tornado.httputil.parse_body_arguments(content_type, body, arguments, files, headers=None)[source]

Parses a form request body.

Supports application/x-www-form-urlencoded and multipart/form-data. The content_type parameter should be a string and body should be a byte string. The arguments and files parameters are dictionaries that will be updated with the parsed contents.

tornado.httputil.parse_multipart_form_data(boundary, data, arguments, files)[source]

Parses a multipart/form-data body.

The boundary and data parameters are both byte strings. The dictionaries given in the arguments and files parameters will be updated with the contents of the body.

tornado.httputil.format_timestamp(ts)[source]

Formats a timestamp in the format used by HTTP.

The argument may be a numeric timestamp as returned by time.time, a time tuple as returned by time.gmtime, or a datetime.datetime object.

>>> format_timestamp(1359312200)
'Sun, 27 Jan 2013 18:43:20 GMT'
class tornado.httputil.RequestStartLine

RequestStartLine(method, path, version)

method

Alias for field number 0

path

Alias for field number 1

version

Alias for field number 2

tornado.httputil.parse_request_start_line(line)[source]

Returns a (method, path, version) tuple for an HTTP 1.x request line.

The response is a collections.namedtuple.

>>> parse_request_start_line("GET /foo HTTP/1.1")
RequestStartLine(method='GET', path='/foo', version='HTTP/1.1')
class tornado.httputil.ResponseStartLine

ResponseStartLine(version, code, reason)

code

Alias for field number 1

reason

Alias for field number 2

version

Alias for field number 0

tornado.httputil.parse_response_start_line(line)[source]

Returns a (version, code, reason) tuple for an HTTP 1.x response line.

The response is a collections.namedtuple.

>>> parse_response_start_line("HTTP/1.1 200 OK")
ResponseStartLine(version='HTTP/1.1', code=200, reason='OK')
tornado.httputil.split_host_and_port(netloc)[source]

Returns (host, port) tuple from netloc.

Returned port will be None if not present.

New in version 4.1.

tornado.http1connection – HTTP/1.x client/server implementation

Client and server implementations of HTTP/1.x.

New in version 4.0.

class tornado.http1connection.HTTP1ConnectionParameters(no_keep_alive=False, chunk_size=None, max_header_size=None, header_timeout=None, max_body_size=None, body_timeout=None, decompress=False)[source]

Parameters for HTTP1Connection and HTTP1ServerConnection.

Parameters:
  • no_keep_alive (bool) – If true, always close the connection after one request.
  • chunk_size (int) – how much data to read into memory at once
  • max_header_size (int) – maximum amount of data for HTTP headers
  • header_timeout (float) – how long to wait for all headers (seconds)
  • max_body_size (int) – maximum amount of data for body
  • body_timeout (float) – how long to wait while reading body (seconds)
  • decompress (bool) – if true, decode incoming Content-Encoding: gzip
class tornado.http1connection.HTTP1Connection(stream, is_client, params=None, context=None)[source]

Implements the HTTP/1.x protocol.

This class can be on its own for clients, or via HTTP1ServerConnection for servers.

Parameters:
  • stream – an IOStream
  • is_client (bool) – client or server
  • params – a HTTP1ConnectionParameters instance or None
  • context – an opaque application-defined object that can be accessed as connection.context.
read_response(delegate)[source]

Read a single HTTP response.

Typical client-mode usage is to write a request using write_headers, write, and finish, and then call read_response.

Parameters:delegate – a HTTPMessageDelegate

Returns a Future that resolves to None after the full response has been read.

set_close_callback(callback)[source]

Sets a callback that will be run when the connection is closed.

Deprecated since version 4.0: Use HTTPMessageDelegate.on_connection_close instead.

detach()[source]

Take control of the underlying stream.

Returns the underlying IOStream object and stops all further HTTP processing. May only be called during HTTPMessageDelegate.headers_received. Intended for implementing protocols like websockets that tunnel over an HTTP handshake.

set_body_timeout(timeout)[source]

Sets the body timeout for a single request.

Overrides the value from HTTP1ConnectionParameters.

set_max_body_size(max_body_size)[source]

Sets the body size limit for a single request.

Overrides the value from HTTP1ConnectionParameters.

write_headers(start_line, headers, chunk=None, callback=None)[source]

Implements HTTPConnection.write_headers.

write(chunk, callback=None)[source]

Implements HTTPConnection.write.

For backwards compatibility is is allowed but deprecated to skip write_headers and instead call write() with a pre-encoded header block.

finish()[source]

Implements HTTPConnection.finish.

class tornado.http1connection.HTTP1ServerConnection(stream, params=None, context=None)[source]

An HTTP/1.x server.

Parameters:
close()[source]

Closes the connection.

Returns a Future that resolves after the serving loop has exited.

start_serving(delegate)[source]

Starts serving requests on this connection.

Parameters:delegate – a HTTPServerConnectionDelegate

Asynchronous networking

tornado.ioloop — Main event loop

An I/O event loop for non-blocking sockets.

Typical applications will use a single IOLoop object, in the IOLoop.instance singleton. The IOLoop.start method should usually be called at the end of the main() function. Atypical applications may use more than one IOLoop, such as one IOLoop per thread, or per unittest case.

In addition to I/O events, the IOLoop can also schedule time-based events. IOLoop.add_timeout is a non-blocking alternative to time.sleep.

IOLoop objects
class tornado.ioloop.IOLoop[source]

A level-triggered I/O loop.

We use epoll (Linux) or kqueue (BSD and Mac OS X) if they are available, or else we fall back on select(). If you are implementing a system that needs to handle thousands of simultaneous connections, you should use a system that supports either epoll or kqueue.

Example usage for a simple TCP server:

import errno
import functools
import tornado.ioloop
import socket

def connection_ready(sock, fd, events):
    while True:
        try:
            connection, address = sock.accept()
        except socket.error as e:
            if e.args[0] not in (errno.EWOULDBLOCK, errno.EAGAIN):
                raise
            return
        connection.setblocking(0)
        handle_connection(connection, address)

if __name__ == '__main__':
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    sock.setblocking(0)
    sock.bind(("", port))
    sock.listen(128)

    io_loop = tornado.ioloop.IOLoop.current()
    callback = functools.partial(connection_ready, sock)
    io_loop.add_handler(sock.fileno(), callback, io_loop.READ)
    io_loop.start()

By default, a newly-constructed IOLoop becomes the thread’s current IOLoop, unless there already is a current IOLoop. This behavior can be controlled with the make_current argument to the IOLoop constructor: if make_current=True, the new IOLoop will always try to become current and it raises an error if there is already a current instance. If make_current=False, the new IOLoop will not try to become current.

Changed in version 4.2: Added the make_current keyword argument to the IOLoop constructor.

Running an IOLoop
static IOLoop.current(instance=True)[source]

Returns the current thread’s IOLoop.

If an IOLoop is currently running or has been marked as current by make_current, returns that instance. If there is no current IOLoop, returns IOLoop.instance() (i.e. the main thread’s IOLoop, creating one if necessary) if instance is true.

In general you should use IOLoop.current as the default when constructing an asynchronous object, and use IOLoop.instance when you mean to communicate to the main thread from a different one.

Changed in version 4.1: Added instance argument to control the fallback to IOLoop.instance().

IOLoop.make_current()[source]

Makes this the IOLoop for the current thread.

An IOLoop automatically becomes current for its thread when it is started, but it is sometimes useful to call make_current explicitly before starting the IOLoop, so that code run at startup time can find the right instance.

Changed in version 4.1: An IOLoop created while there is no current IOLoop will automatically become current.

static IOLoop.instance()[source]

Returns a global IOLoop instance.

Most applications have a single, global IOLoop running on the main thread. Use this method to get this instance from another thread. In most other cases, it is better to use current() to get the current thread’s IOLoop.

static IOLoop.initialized()[source]

Returns true if the singleton instance has been created.

IOLoop.install()[source]

Installs this IOLoop object as the singleton instance.

This is normally not necessary as instance() will create an IOLoop on demand, but you may want to call install to use a custom subclass of IOLoop.

static IOLoop.clear_instance()[source]

Clear the global IOLoop instance.

New in version 4.0.

IOLoop.start()[source]

Starts the I/O loop.

The loop will run until one of the callbacks calls stop(), which will make the loop stop after the current event iteration completes.

IOLoop.stop()[source]

Stop the I/O loop.

If the event loop is not currently running, the next call to start() will return immediately.

To use asynchronous methods from otherwise-synchronous code (such as unit tests), you can start and stop the event loop like this:

ioloop = IOLoop()
async_method(ioloop=ioloop, callback=ioloop.stop)
ioloop.start()

ioloop.start() will return after async_method has run its callback, whether that callback was invoked before or after ioloop.start.

Note that even after stop has been called, the IOLoop is not completely stopped until IOLoop.start has also returned. Some work that was scheduled before the call to stop may still be run before the IOLoop shuts down.

IOLoop.run_sync(func, timeout=None)[source]

Starts the IOLoop, runs the given function, and stops the loop.

The function must return either a yieldable object or None. If the function returns a yieldable object, the IOLoop will run until the yieldable is resolved (and run_sync() will return the yieldable’s result). If it raises an exception, the IOLoop will stop and the exception will be re-raised to the caller.

The keyword-only argument timeout may be used to set a maximum duration for the function. If the timeout expires, a TimeoutError is raised.

This method is useful in conjunction with tornado.gen.coroutine to allow asynchronous calls in a main() function:

@gen.coroutine
def main():
    # do stuff...

if __name__ == '__main__':
    IOLoop.current().run_sync(main)

Changed in version 4.3: Returning a non-None, non-yieldable value is now an error.

IOLoop.close(all_fds=False)[source]

Closes the IOLoop, freeing any resources used.

If all_fds is true, all file descriptors registered on the IOLoop will be closed (not just the ones created by the IOLoop itself).

Many applications will only use a single IOLoop that runs for the entire lifetime of the process. In that case closing the IOLoop is not necessary since everything will be cleaned up when the process exits. IOLoop.close is provided mainly for scenarios such as unit tests, which create and destroy a large number of IOLoops.

An IOLoop must be completely stopped before it can be closed. This means that IOLoop.stop() must be called and IOLoop.start() must be allowed to return before attempting to call IOLoop.close(). Therefore the call to close will usually appear just after the call to start rather than near the call to stop.

Changed in version 3.1: If the IOLoop implementation supports non-integer objects for “file descriptors”, those objects will have their close method when all_fds is true.

I/O events
IOLoop.add_handler(fd, handler, events)[source]

Registers the given handler to receive the given events for fd.

The fd argument may either be an integer file descriptor or a file-like object with a fileno() method (and optionally a close() method, which may be called when the IOLoop is shut down).

The events argument is a bitwise or of the constants IOLoop.READ, IOLoop.WRITE, and IOLoop.ERROR.

When an event occurs, handler(fd, events) will be run.

Changed in version 4.0: Added the ability to pass file-like objects in addition to raw file descriptors.

IOLoop.update_handler(fd, events)[source]

Changes the events we listen for fd.

Changed in version 4.0: Added the ability to pass file-like objects in addition to raw file descriptors.

IOLoop.remove_handler(fd)[source]

Stop listening for events on fd.

Changed in version 4.0: Added the ability to pass file-like objects in addition to raw file descriptors.

Callbacks and timeouts
IOLoop.add_callback(callback, *args, **kwargs)[source]

Calls the given callback on the next I/O loop iteration.

It is safe to call this method from any thread at any time, except from a signal handler. Note that this is the only method in IOLoop that makes this thread-safety guarantee; all other interaction with the IOLoop must be done from that IOLoop‘s thread. add_callback() may be used to transfer control from other threads to the IOLoop‘s thread.

To add a callback from a signal handler, see add_callback_from_signal.

IOLoop.add_callback_from_signal(callback, *args, **kwargs)[source]

Calls the given callback on the next I/O loop iteration.

Safe for use from a Python signal handler; should not be used otherwise.

Callbacks added with this method will be run without any stack_context, to avoid picking up the context of the function that was interrupted by the signal.

IOLoop.add_future(future, callback)[source]

Schedules a callback on the IOLoop when the given Future is finished.

The callback is invoked with one argument, the Future.

IOLoop.add_timeout(deadline, callback, *args, **kwargs)[source]

Runs the callback at the time deadline from the I/O loop.

Returns an opaque handle that may be passed to remove_timeout to cancel.

deadline may be a number denoting a time (on the same scale as IOLoop.time, normally time.time), or a datetime.timedelta object for a deadline relative to the current time. Since Tornado 4.0, call_later is a more convenient alternative for the relative case since it does not require a timedelta object.

Note that it is not safe to call add_timeout from other threads. Instead, you must use add_callback to transfer control to the IOLoop‘s thread, and then call add_timeout from there.

Subclasses of IOLoop must implement either add_timeout or call_at; the default implementations of each will call the other. call_at is usually easier to implement, but subclasses that wish to maintain compatibility with Tornado versions prior to 4.0 must use add_timeout instead.

Changed in version 4.0: Now passes through *args and **kwargs to the callback.

IOLoop.call_at(when, callback, *args, **kwargs)[source]

Runs the callback at the absolute time designated by when.

when must be a number using the same reference point as IOLoop.time.

Returns an opaque handle that may be passed to remove_timeout to cancel. Note that unlike the asyncio method of the same name, the returned object does not have a cancel() method.

See add_timeout for comments on thread-safety and subclassing.

New in version 4.0.

IOLoop.call_later(delay, callback, *args, **kwargs)[source]

Runs the callback after delay seconds have passed.

Returns an opaque handle that may be passed to remove_timeout to cancel. Note that unlike the asyncio method of the same name, the returned object does not have a cancel() method.

See add_timeout for comments on thread-safety and subclassing.

New in version 4.0.

IOLoop.remove_timeout(timeout)[source]

Cancels a pending timeout.

The argument is a handle as returned by add_timeout. It is safe to call remove_timeout even if the callback has already been run.

IOLoop.spawn_callback(callback, *args, **kwargs)[source]

Calls the given callback on the next IOLoop iteration.

Unlike all other callback-related methods on IOLoop, spawn_callback does not associate the callback with its caller’s stack_context, so it is suitable for fire-and-forget callbacks that should not interfere with the caller.

New in version 4.0.

IOLoop.time()[source]

Returns the current time according to the IOLoop‘s clock.

The return value is a floating-point number relative to an unspecified time in the past.

By default, the IOLoop‘s time function is time.time. However, it may be configured to use e.g. time.monotonic instead. Calls to add_timeout that pass a number instead of a datetime.timedelta should use this function to compute the appropriate time, so they can work no matter what time function is chosen.

class tornado.ioloop.PeriodicCallback(callback, callback_time, io_loop=None)[source]

Schedules the given callback to be called periodically.

The callback is called every callback_time milliseconds. Note that the timeout is given in milliseconds, while most other time-related functions in Tornado use seconds.

If the callback runs for longer than callback_time milliseconds, subsequent invocations will be skipped to get back on schedule.

start must be called after the PeriodicCallback is created.

Changed in version 4.1: The io_loop argument is deprecated.

start()[source]

Starts the timer.

stop()[source]

Stops the timer.

is_running()[source]

Return True if this PeriodicCallback has been started.

New in version 4.1.

Debugging and error handling
IOLoop.handle_callback_exception(callback)[source]

This method is called whenever a callback run by the IOLoop throws an exception.

By default simply logs the exception as an error. Subclasses may override this method to customize reporting of exceptions.

The exception itself is not passed explicitly, but is available in sys.exc_info.

IOLoop.set_blocking_signal_threshold(seconds, action)[source]

Sends a signal if the IOLoop is blocked for more than s seconds.

Pass seconds=None to disable. Requires Python 2.6 on a unixy platform.

The action parameter is a Python signal handler. Read the documentation for the signal module for more information. If action is None, the process will be killed if it is blocked for too long.

IOLoop.set_blocking_log_threshold(seconds)[source]

Logs a stack trace if the IOLoop is blocked for more than s seconds.

Equivalent to set_blocking_signal_threshold(seconds, self.log_stack)

IOLoop.log_stack(signal, frame)[source]

Signal handler to log the stack trace of the current thread.

For use with set_blocking_signal_threshold.

Methods for subclasses
IOLoop.initialize(make_current=None)[source]
IOLoop.close_fd(fd)[source]

Utility method to close an fd.

If fd is a file-like object, we close it directly; otherwise we use os.close.

This method is provided for use by IOLoop subclasses (in implementations of IOLoop.close(all_fds=True) and should not generally be used by application code.

New in version 4.0.

IOLoop.split_fd(fd)[source]

Returns an (fd, obj) pair from an fd parameter.

We accept both raw file descriptors and file-like objects as input to add_handler and related methods. When a file-like object is passed, we must retain the object itself so we can close it correctly when the IOLoop shuts down, but the poller interfaces favor file descriptors (they will accept file-like objects and call fileno() for you, but they always return the descriptor itself).

This method is provided for use by IOLoop subclasses and should not generally be used by application code.

New in version 4.0.

tornado.iostream — Convenient wrappers for non-blocking sockets

Utility classes to write to and read from non-blocking files and sockets.

Contents:

  • BaseIOStream: Generic interface for reading and writing.
  • IOStream: Implementation of BaseIOStream using non-blocking sockets.
  • SSLIOStream: SSL-aware version of IOStream.
  • PipeIOStream: Pipe-based IOStream implementation.
Base class
class tornado.iostream.BaseIOStream(io_loop=None, max_buffer_size=None, read_chunk_size=None, max_write_buffer_size=None)[source]

A utility class to write to and read from a non-blocking file or socket.

We support a non-blocking write() and a family of read_*() methods. All of the methods take an optional callback argument and return a Future only if no callback is given. When the operation completes, the callback will be run or the Future will resolve with the data read (or None for write()). All outstanding Futures will resolve with a StreamClosedError when the stream is closed; users of the callback interface will be notified via BaseIOStream.set_close_callback instead.

When a stream is closed due to an error, the IOStream’s error attribute contains the exception object.

Subclasses must implement fileno, close_fd, write_to_fd, read_from_fd, and optionally get_fd_error.

BaseIOStream constructor.

Parameters:
  • io_loop – The IOLoop to use; defaults to IOLoop.current. Deprecated since Tornado 4.1.
  • max_buffer_size – Maximum amount of incoming data to buffer; defaults to 100MB.
  • read_chunk_size – Amount of data to read at one time from the underlying transport; defaults to 64KB.
  • max_write_buffer_size – Amount of outgoing data to buffer; defaults to unlimited.

Changed in version 4.0: Add the max_write_buffer_size parameter. Changed default read_chunk_size to 64KB.

Main interface
BaseIOStream.write(data, callback=None)[source]

Asynchronously write the given data to this stream.

If callback is given, we call it when all of the buffered write data has been successfully written to the stream. If there was previously buffered write data and an old write callback, that callback is simply overwritten with this new callback.

If no callback is given, this method returns a Future that resolves (with a result of None) when the write has been completed. If write is called again before that Future has resolved, the previous future will be orphaned and will never resolve.

Changed in version 4.0: Now returns a Future if no callback is given.

BaseIOStream.read_bytes(num_bytes, callback=None, streaming_callback=None, partial=False)[source]

Asynchronously read a number of bytes.

If a streaming_callback is given, it will be called with chunks of data as they become available, and the final result will be empty. Otherwise, the result is all the data that was read. If a callback is given, it will be run with the data as an argument; if not, this method returns a Future.

If partial is true, the callback is run as soon as we have any bytes to return (but never more than num_bytes)

Changed in version 4.0: Added the partial argument. The callback argument is now optional and a Future will be returned if it is omitted.

BaseIOStream.read_until(delimiter, callback=None, max_bytes=None)[source]

Asynchronously read until we have found the given delimiter.

The result includes all the data read including the delimiter. If a callback is given, it will be run with the data as an argument; if not, this method returns a Future.

If max_bytes is not None, the connection will be closed if more than max_bytes bytes have been read and the delimiter is not found.

Changed in version 4.0: Added the max_bytes argument. The callback argument is now optional and a Future will be returned if it is omitted.

BaseIOStream.read_until_regex(regex, callback=None, max_bytes=None)[source]

Asynchronously read until we have matched the given regex.

The result includes the data that matches the regex and anything that came before it. If a callback is given, it will be run with the data as an argument; if not, this method returns a Future.

If max_bytes is not None, the connection will be closed if more than max_bytes bytes have been read and the regex is not satisfied.

Changed in version 4.0: Added the max_bytes argument. The callback argument is now optional and a Future will be returned if it is omitted.

BaseIOStream.read_until_close(callback=None, streaming_callback=None)[source]

Asynchronously reads all data from the socket until it is closed.

If a streaming_callback is given, it will be called with chunks of data as they become available, and the final result will be empty. Otherwise, the result is all the data that was read. If a callback is given, it will be run with the data as an argument; if not, this method returns a Future.

Note that if a streaming_callback is used, data will be read from the socket as quickly as it becomes available; there is no way to apply backpressure or cancel the reads. If flow control or cancellation are desired, use a loop with read_bytes(partial=True) instead.

Changed in version 4.0: The callback argument is now optional and a Future will be returned if it is omitted.

BaseIOStream.close(exc_info=False)[source]

Close this stream.

If exc_info is true, set the error attribute to the current exception from sys.exc_info (or if exc_info is a tuple, use that instead of sys.exc_info).

BaseIOStream.set_close_callback(callback)[source]

Call the given callback when the stream is closed.

This is not necessary for applications that use the Future interface; all outstanding Futures will resolve with a StreamClosedError when the stream is closed.

BaseIOStream.closed()[source]

Returns true if the stream has been closed.

BaseIOStream.reading()[source]

Returns true if we are currently reading from the stream.

BaseIOStream.writing()[source]

Returns true if we are currently writing to the stream.

BaseIOStream.set_nodelay(value)[source]

Sets the no-delay flag for this stream.

By default, data written to TCP streams may be held for a time to make the most efficient use of bandwidth (according to Nagle’s algorithm). The no-delay flag requests that data be written as soon as possible, even if doing so would consume additional bandwidth.

This flag is currently defined only for TCP-based IOStreams.

New in version 3.1.

Methods for subclasses
BaseIOStream.fileno()[source]

Returns the file descriptor for this stream.

BaseIOStream.close_fd()[source]

Closes the file underlying this stream.

close_fd is called by BaseIOStream and should not be called elsewhere; other users should call close instead.

BaseIOStream.write_to_fd(data)[source]

Attempts to write data to the underlying file.

Returns the number of bytes written.

BaseIOStream.read_from_fd()[source]

Attempts to read from the underlying file.

Returns None if there was nothing to read (the socket returned EWOULDBLOCK or equivalent), otherwise returns the data. When possible, should return no more than self.read_chunk_size bytes at a time.

BaseIOStream.get_fd_error()[source]

Returns information about any error on the underlying file.

This method is called after the IOLoop has signaled an error on the file descriptor, and should return an Exception (such as socket.error with additional information, or None if no such information is available.

Implementations
class tornado.iostream.IOStream(socket, *args, **kwargs)[source]

Socket-based IOStream implementation.

This class supports the read and write methods from BaseIOStream plus a connect method.

The socket parameter may either be connected or unconnected. For server operations the socket is the result of calling socket.accept. For client operations the socket is created with socket.socket, and may either be connected before passing it to the IOStream or connected with IOStream.connect.

A very simple (and broken) HTTP client using this class:

import tornado.ioloop
import tornado.iostream
import socket

def send_request():
    stream.write(b"GET / HTTP/1.0\r\nHost: friendfeed.com\r\n\r\n")
    stream.read_until(b"\r\n\r\n", on_headers)

def on_headers(data):
    headers = {}
    for line in data.split(b"\r\n"):
       parts = line.split(b":")
       if len(parts) == 2:
           headers[parts[0].strip()] = parts[1].strip()
    stream.read_bytes(int(headers[b"Content-Length"]), on_body)

def on_body(data):
    print(data)
    stream.close()
    tornado.ioloop.IOLoop.current().stop()

if __name__ == '__main__':
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
    stream = tornado.iostream.IOStream(s)
    stream.connect(("friendfeed.com", 80), send_request)
    tornado.ioloop.IOLoop.current().start()
connect(address, callback=None, server_hostname=None)[source]

Connects the socket to a remote address without blocking.

May only be called if the socket passed to the constructor was not previously connected. The address parameter is in the same format as for socket.connect for the type of socket passed to the IOStream constructor, e.g. an (ip, port) tuple. Hostnames are accepted here, but will be resolved synchronously and block the IOLoop. If you have a hostname instead of an IP address, the TCPClient class is recommended instead of calling this method directly. TCPClient will do asynchronous DNS resolution and handle both IPv4 and IPv6.

If callback is specified, it will be called with no arguments when the connection is completed; if not this method returns a Future (whose result after a successful connection will be the stream itself).

In SSL mode, the server_hostname parameter will be used for certificate validation (unless disabled in the ssl_options) and SNI (if supported; requires Python 2.7.9+).

Note that it is safe to call IOStream.write while the connection is pending, in which case the data will be written as soon as the connection is ready. Calling IOStream read methods before the socket is connected works on some platforms but is non-portable.

Changed in version 4.0: If no callback is given, returns a Future.

Changed in version 4.2: SSL certificates are validated by default; pass ssl_options=dict(cert_reqs=ssl.CERT_NONE) or a suitably-configured ssl.SSLContext to the SSLIOStream constructor to disable.

start_tls(server_side, ssl_options=None, server_hostname=None)[source]

Convert this IOStream to an SSLIOStream.

This enables protocols that begin in clear-text mode and switch to SSL after some initial negotiation (such as the STARTTLS extension to SMTP and IMAP).

This method cannot be used if there are outstanding reads or writes on the stream, or if there is any data in the IOStream’s buffer (data in the operating system’s socket buffer is allowed). This means it must generally be used immediately after reading or writing the last clear-text data. It can also be used immediately after connecting, before any reads or writes.

The ssl_options argument may be either an ssl.SSLContext object or a dictionary of keyword arguments for the ssl.wrap_socket function. The server_hostname argument will be used for certificate validation unless disabled in the ssl_options.

This method returns a Future whose result is the new SSLIOStream. After this method has been called, any other operation on the original stream is undefined.

If a close callback is defined on this stream, it will be transferred to the new stream.

New in version 4.0.

Changed in version 4.2: SSL certificates are validated by default; pass ssl_options=dict(cert_reqs=ssl.CERT_NONE) or a suitably-configured ssl.SSLContext to disable.

class tornado.iostream.SSLIOStream(*args, **kwargs)[source]

A utility class to write to and read from a non-blocking SSL socket.

If the socket passed to the constructor is already connected, it should be wrapped with:

ssl.wrap_socket(sock, do_handshake_on_connect=False, **kwargs)

before constructing the SSLIOStream. Unconnected sockets will be wrapped when IOStream.connect is finished.

The ssl_options keyword argument may either be an ssl.SSLContext object or a dictionary of keywords arguments for ssl.wrap_socket

wait_for_handshake(callback=None)[source]

Wait for the initial SSL handshake to complete.

If a callback is given, it will be called with no arguments once the handshake is complete; otherwise this method returns a Future which will resolve to the stream itself after the handshake is complete.

Once the handshake is complete, information such as the peer’s certificate and NPN/ALPN selections may be accessed on self.socket.

This method is intended for use on server-side streams or after using IOStream.start_tls; it should not be used with IOStream.connect (which already waits for the handshake to complete). It may only be called once per stream.

New in version 4.2.

class tornado.iostream.PipeIOStream(fd, *args, **kwargs)[source]

Pipe-based IOStream implementation.

The constructor takes an integer file descriptor (such as one returned by os.pipe) rather than an open file object. Pipes are generally one-way, so a PipeIOStream can be used for reading or writing but not both.

Exceptions
exception tornado.iostream.StreamBufferFullError[source]

Exception raised by IOStream methods when the buffer is full.

exception tornado.iostream.StreamClosedError(real_error=None)[source]

Exception raised by IOStream methods when the stream is closed.

Note that the close callback is scheduled to run after other callbacks on the stream (to allow for buffered data to be processed), so you may see this error before you see the close callback.

The real_error attribute contains the underlying error that caused the stream to close (if any).

Changed in version 4.3: Added the real_error attribute.

exception tornado.iostream.UnsatisfiableReadError[source]

Exception raised when a read cannot be satisfied.

Raised by read_until and read_until_regex with a max_bytes argument.

tornado.netutil — Miscellaneous network utilities

Miscellaneous network utility code.

tornado.netutil.bind_sockets(port, address=None, family=<AddressFamily.AF_UNSPEC: 0>, backlog=128, flags=None, reuse_port=False)[source]

Creates listening sockets bound to the given port and address.

Returns a list of socket objects (multiple sockets are returned if the given address maps to multiple IP addresses, which is most common for mixed IPv4 and IPv6 use).

Address may be either an IP address or hostname. If it’s a hostname, the server will listen on all IP addresses associated with the name. Address may be an empty string or None to listen on all available interfaces. Family may be set to either socket.AF_INET or socket.AF_INET6 to restrict to IPv4 or IPv6 addresses, otherwise both will be used if available.

The backlog argument has the same meaning as for socket.listen().

flags is a bitmask of AI_* flags to getaddrinfo, like socket.AI_PASSIVE | socket.AI_NUMERICHOST.

resuse_port option sets SO_REUSEPORT option for every socket in the list. If your platform doesn’t support this option ValueError will be raised.

tornado.netutil.bind_unix_socket(file, mode=384, backlog=128)[source]

Creates a listening unix socket.

If a socket with the given name already exists, it will be deleted. If any other file with that name exists, an exception will be raised.

Returns a socket object (not a list of socket objects like bind_sockets)

tornado.netutil.add_accept_handler(sock, callback, io_loop=None)[source]

Adds an IOLoop event handler to accept new connections on sock.

When a connection is accepted, callback(connection, address) will be run (connection is a socket object, and address is the address of the other end of the connection). Note that this signature is different from the callback(fd, events) signature used for IOLoop handlers.

Changed in version 4.1: The io_loop argument is deprecated.

tornado.netutil.is_valid_ip(ip)[source]

Returns true if the given string is a well-formed IP address.

Supports IPv4 and IPv6.

class tornado.netutil.Resolver[source]

Configurable asynchronous DNS resolver interface.

By default, a blocking implementation is used (which simply calls socket.getaddrinfo). An alternative implementation can be chosen with the Resolver.configure class method:

Resolver.configure('tornado.netutil.ThreadedResolver')

The implementations of this interface included with Tornado are

resolve(host, port, family=<AddressFamily.AF_UNSPEC: 0>, callback=None)[source]

Resolves an address.

The host argument is a string which may be a hostname or a literal IP address.

Returns a Future whose result is a list of (family, address) pairs, where address is a tuple suitable to pass to socket.connect (i.e. a (host, port) pair for IPv4; additional fields may be present for IPv6). If a callback is passed, it will be run with the result as an argument when it is complete.

close()[source]

Closes the Resolver, freeing any resources used.

New in version 3.1.

class tornado.netutil.ExecutorResolver[source]

Resolver implementation using a concurrent.futures.Executor.

Use this instead of ThreadedResolver when you require additional control over the executor being used.

The executor will be shut down when the resolver is closed unless close_resolver=False; use this if you want to reuse the same executor elsewhere.

Changed in version 4.1: The io_loop argument is deprecated.

class tornado.netutil.BlockingResolver[source]

Default Resolver implementation, using socket.getaddrinfo.

The IOLoop will be blocked during the resolution, although the callback will not be run until the next IOLoop iteration.

class tornado.netutil.ThreadedResolver[source]

Multithreaded non-blocking Resolver implementation.

Requires the concurrent.futures package to be installed (available in the standard library since Python 3.2, installable with pip install futures in older versions).

The thread pool size can be configured with:

Resolver.configure('tornado.netutil.ThreadedResolver',
                   num_threads=10)

Changed in version 3.1: All ThreadedResolvers share a single thread pool, whose size is set by the first one to be created.

class tornado.netutil.OverrideResolver[source]

Wraps a resolver with a mapping of overrides.

This can be used to make local DNS changes (e.g. for testing) without modifying system-wide settings.

The mapping can contain either host strings or host-port pairs.

tornado.netutil.ssl_options_to_context(ssl_options)[source]

Try to convert an ssl_options dictionary to an SSLContext object.

The ssl_options dictionary contains keywords to be passed to ssl.wrap_socket. In Python 2.7.9+, ssl.SSLContext objects can be used instead. This function converts the dict form to its SSLContext equivalent, and may be used when a component which accepts both forms needs to upgrade to the SSLContext version to use features like SNI or NPN.

tornado.netutil.ssl_wrap_socket(socket, ssl_options, server_hostname=None, **kwargs)[source]

Returns an ssl.SSLSocket wrapping the given socket.

ssl_options may be either an ssl.SSLContext object or a dictionary (as accepted by ssl_options_to_context). Additional keyword arguments are passed to wrap_socket (either the SSLContext method or the ssl module function as appropriate).

tornado.tcpclientIOStream connection factory

A non-blocking TCP connection factory.

class tornado.tcpclient.TCPClient(resolver=None, io_loop=None)[source]

A non-blocking TCP connection factory.

Changed in version 4.1: The io_loop argument is deprecated.

connect(host, port, af=<AddressFamily.AF_UNSPEC: 0>, ssl_options=None, max_buffer_size=None)[source]

Connect to the given host and port.

Asynchronously returns an IOStream (or SSLIOStream if ssl_options is not None).

tornado.tcpserver — Basic IOStream-based TCP server

A non-blocking, single-threaded TCP server.

class tornado.tcpserver.TCPServer(io_loop=None, ssl_options=None, max_buffer_size=None, read_chunk_size=None)[source]

A non-blocking, single-threaded TCP server.

To use TCPServer, define a subclass which overrides the handle_stream method.

To make this server serve SSL traffic, send the ssl_options keyword argument with an ssl.SSLContext object. For compatibility with older versions of Python ssl_options may also be a dictionary of keyword arguments for the ssl.wrap_socket method.:

ssl_ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
ssl_ctx.load_cert_chain(os.path.join(data_dir, "mydomain.crt"),
                        os.path.join(data_dir, "mydomain.key"))
TCPServer(ssl_options=ssl_ctx)

TCPServer initialization follows one of three patterns:

  1. listen: simple single-process:

    server = TCPServer()
    server.listen(8888)
    IOLoop.current().start()
    
  2. bind/start: simple multi-process:

    server = TCPServer()
    server.bind(8888)
    server.start(0)  # Forks multiple sub-processes
    IOLoop.current().start()
    

    When using this interface, an IOLoop must not be passed to the TCPServer constructor. start will always start the server on the default singleton IOLoop.

  3. add_sockets: advanced multi-process:

    sockets = bind_sockets(8888)
    tornado.process.fork_processes(0)
    server = TCPServer()
    server.add_sockets(sockets)
    IOLoop.current().start()
    

    The add_sockets interface is more complicated, but it can be used with tornado.process.fork_processes to give you more flexibility in when the fork happens. add_sockets can also be used in single-process servers if you want to create your listening sockets in some way other than bind_sockets.

New in version 3.1: The max_buffer_size argument.

listen(port, address='')[source]

Starts accepting connections on the given port.

This method may be called more than once to listen on multiple ports. listen takes effect immediately; it is not necessary to call TCPServer.start afterwards. It is, however, necessary to start the IOLoop.

add_sockets(sockets)[source]

Makes this server start accepting connections on the given sockets.

The sockets parameter is a list of socket objects such as those returned by bind_sockets. add_sockets is typically used in combination with that method and tornado.process.fork_processes to provide greater control over the initialization of a multi-process server.

add_socket(socket)[source]

Singular version of add_sockets. Takes a single socket object.

bind(port, address=None, family=<AddressFamily.AF_UNSPEC: 0>, backlog=128)[source]

Binds this server to the given port on the given address.

To start the server, call start. If you want to run this server in a single process, you can call listen as a shortcut to the sequence of bind and start calls.

Address may be either an IP address or hostname. If it’s a hostname, the server will listen on all IP addresses associated with the name. Address may be an empty string or None to listen on all available interfaces. Family may be set to either socket.AF_INET or socket.AF_INET6 to restrict to IPv4 or IPv6 addresses, otherwise both will be used if available.

The backlog argument has the same meaning as for socket.listen.

This method may be called multiple times prior to start to listen on multiple ports or interfaces.

start(num_processes=1)[source]

Starts this server in the IOLoop.

By default, we run the server in this process and do not fork any additional child process.

If num_processes is None or <= 0, we detect the number of cores available on this machine and fork that number of child processes. If num_processes is given and > 1, we fork that specific number of sub-processes.

Since we use processes and not threads, there is no shared memory between any server code.

Note that multiple processes are not compatible with the autoreload module (or the autoreload=True option to tornado.web.Application which defaults to True when debug=True). When using multiple processes, no IOLoops can be created or referenced until after the call to TCPServer.start(n).

stop()[source]

Stops listening for new connections.

Requests currently in progress may still continue after the server is stopped.

handle_stream(stream, address)[source]

Override to handle a new IOStream from an incoming connection.

This method may be a coroutine; if so any exceptions it raises asynchronously will be logged. Accepting of incoming connections will not be blocked by this coroutine.

If this TCPServer is configured for SSL, handle_stream may be called before the SSL handshake has completed. Use SSLIOStream.wait_for_handshake if you need to verify the client’s certificate or use NPN/ALPN.

Changed in version 4.2: Added the option for this method to be a coroutine.

Coroutines and concurrency

tornado.gen — Simplify asynchronous code

tornado.gen is a generator-based interface to make it easier to work in an asynchronous environment. Code using the gen module is technically asynchronous, but it is written as a single generator instead of a collection of separate functions.

For example, the following asynchronous handler:

class AsyncHandler(RequestHandler):
    @asynchronous
    def get(self):
        http_client = AsyncHTTPClient()
        http_client.fetch("http://example.com",
                          callback=self.on_fetch)

    def on_fetch(self, response):
        do_something_with_response(response)
        self.render("template.html")

could be written with gen as:

class GenAsyncHandler(RequestHandler):
    @gen.coroutine
    def get(self):
        http_client = AsyncHTTPClient()
        response = yield http_client.fetch("http://example.com")
        do_something_with_response(response)
        self.render("template.html")

Most asynchronous functions in Tornado return a Future; yielding this object returns its result.

You can also yield a list or dict of Futures, which will be started at the same time and run in parallel; a list or dict of results will be returned when they are all finished:

@gen.coroutine
def get(self):
    http_client = AsyncHTTPClient()
    response1, response2 = yield [http_client.fetch(url1),
                                  http_client.fetch(url2)]
    response_dict = yield dict(response3=http_client.fetch(url3),
                               response4=http_client.fetch(url4))
    response3 = response_dict['response3']
    response4 = response_dict['response4']

If the singledispatch library is available (standard in Python 3.4, available via the singledispatch package on older versions), additional types of objects may be yielded. Tornado includes support for asyncio.Future and Twisted’s Deferred class when tornado.platform.asyncio and tornado.platform.twisted are imported. See the convert_yielded function to extend this mechanism.

Changed in version 3.2: Dict support added.

Changed in version 4.1: Support added for yielding asyncio Futures and Twisted Deferreds via singledispatch.

Decorators
tornado.gen.coroutine(func, replace_callback=True)[source]

Decorator for asynchronous generators.

Any generator that yields objects from this module must be wrapped in either this decorator or engine.

Coroutines may “return” by raising the special exception Return(value). In Python 3.3+, it is also possible for the function to simply use the return value statement (prior to Python 3.3 generators were not allowed to also return values). In all versions of Python a coroutine that simply wishes to exit early may use the return statement without a value.

Functions with this decorator return a Future. Additionally, they may be called with a callback keyword argument, which will be invoked with the future’s result when it resolves. If the coroutine fails, the callback will not be run and an exception will be raised into the surrounding StackContext. The callback argument is not visible inside the decorated function; it is handled by the decorator itself.

From the caller’s perspective, @gen.coroutine is similar to the combination of @return_future and @gen.engine.

Warning

When exceptions occur inside a coroutine, the exception information will be stored in the Future object. You must examine the result of the Future object, or the exception may go unnoticed by your code. This means yielding the function if called from another coroutine, using something like IOLoop.run_sync for top-level calls, or passing the Future to IOLoop.add_future.

tornado.gen.engine(func)[source]

Callback-oriented decorator for asynchronous generators.

This is an older interface; for new code that does not need to be compatible with versions of Tornado older than 3.0 the coroutine decorator is recommended instead.

This decorator is similar to coroutine, except it does not return a Future and the callback argument is not treated specially.

In most cases, functions decorated with engine should take a callback argument and invoke it with their result when they are finished. One notable exception is the RequestHandler HTTP verb methods, which use self.finish() in place of a callback argument.

Utility functions
exception tornado.gen.Return(value=None)[source]

Special exception to return a value from a coroutine.

If this exception is raised, its value argument is used as the result of the coroutine:

@gen.coroutine
def fetch_json(url):
    response = yield AsyncHTTPClient().fetch(url)
    raise gen.Return(json_decode(response.body))

In Python 3.3, this exception is no longer necessary: the return statement can be used directly to return a value (previously yield and return with a value could not be combined in the same function).

By analogy with the return statement, the value argument is optional, but it is never necessary to raise gen.Return(). The return statement can be used with no arguments instead.

tornado.gen.with_timeout(timeout, future, io_loop=None, quiet_exceptions=())[source]

Wraps a Future in a timeout.

Raises TimeoutError if the input future does not complete before timeout, which may be specified in any form allowed by IOLoop.add_timeout (i.e. a datetime.timedelta or an absolute time relative to IOLoop.time)

If the wrapped Future fails after it has timed out, the exception will be logged unless it is of a type contained in quiet_exceptions (which may be an exception type or a sequence of types).

Currently only supports Futures, not other YieldPoint classes.

New in version 4.0.

Changed in version 4.1: Added the quiet_exceptions argument and the logging of unhandled exceptions.

exception tornado.gen.TimeoutError[source]

Exception raised by with_timeout.

tornado.gen.sleep(duration)[source]

Return a Future that resolves after the given number of seconds.

When used with yield in a coroutine, this is a non-blocking analogue to time.sleep (which should not be used in coroutines because it is blocking):

yield gen.sleep(0.5)

Note that calling this function on its own does nothing; you must wait on the Future it returns (usually by yielding it).

New in version 4.1.

tornado.gen.moment

A special object which may be yielded to allow the IOLoop to run for one iteration.

This is not needed in normal use but it can be helpful in long-running coroutines that are likely to yield Futures that are ready instantly.

Usage: yield gen.moment

New in version 4.0.

class tornado.gen.WaitIterator(*args, **kwargs)[source]

Provides an iterator to yield the results of futures as they finish.

Yielding a set of futures like this:

results = yield [future1, future2]

pauses the coroutine until both future1 and future2 return, and then restarts the coroutine with the results of both futures. If either future is an exception, the expression will raise that exception and all the results will be lost.

If you need to get the result of each future as soon as possible, or if you need the result of some futures even if others produce errors, you can use WaitIterator:

wait_iterator = gen.WaitIterator(future1, future2)
while not wait_iterator.done():
    try:
        result = yield wait_iterator.next()
    except Exception as e:
        print("Error {} from {}".format(e, wait_iterator.current_future))
    else:
        print("Result {} received from {} at {}".format(
            result, wait_iterator.current_future,
            wait_iterator.current_index))

Because results are returned as soon as they are available the output from the iterator will not be in the same order as the input arguments. If you need to know which future produced the current result, you can use the attributes WaitIterator.current_future, or WaitIterator.current_index to get the index of the future from the input list. (if keyword arguments were used in the construction of the WaitIterator, current_index will use the corresponding keyword).

On Python 3.5, WaitIterator implements the async iterator protocol, so it can be used with the async for statement (note that in this version the entire iteration is aborted if any value raises an exception, while the previous example can continue past individual errors):

async for result in gen.WaitIterator(future1, future2):
    print("Result {} received from {} at {}".format(
        result, wait_iterator.current_future,
        wait_iterator.current_index))

New in version 4.1.

Changed in version 4.3: Added async for support in Python 3.5.

done()[source]

Returns True if this iterator has no more results.

next()[source]

Returns a Future that will yield the next available result.

Note that this Future will not be the same object as any of the inputs.

tornado.gen.multi(children, quiet_exceptions=())[source]

Runs multiple asynchronous operations in parallel.

children may either be a list or a dict whose values are yieldable objects. multi() returns a new yieldable object that resolves to a parallel structure containing their results. If children is a list, the result is a list of results in the same order; if it is a dict, the result is a dict with the same keys.

That is, results = yield multi(list_of_futures) is equivalent to:

results = []
for future in list_of_futures:
    results.append(yield future)

If any children raise exceptions, multi() will raise the first one. All others will be logged, unless they are of types contained in the quiet_exceptions argument.

If any of the inputs are YieldPoints, the returned yieldable object is a YieldPoint. Otherwise, returns a Future. This means that the result of multi can be used in a native coroutine if and only if all of its children can be.

In a yield-based coroutine, it is not normally necessary to call this function directly, since the coroutine runner will do it automatically when a list or dict is yielded. However, it is necessary in await-based coroutines, or to pass the quiet_exceptions argument.

This function is available under the names multi() and Multi() for historical reasons.

Changed in version 4.2: If multiple yieldables fail, any exceptions after the first (which is raised) will be logged. Added the quiet_exceptions argument to suppress this logging for selected exception types.

Changed in version 4.3: Replaced the class Multi and the function multi_future with a unified function multi. Added support for yieldables other than YieldPoint and Future.

tornado.gen.multi_future(children, quiet_exceptions=())[source]

Wait for multiple asynchronous futures in parallel.

This function is similar to multi, but does not support YieldPoints.

New in version 4.0.

Changed in version 4.2: If multiple Futures fail, any exceptions after the first (which is raised) will be logged. Added the quiet_exceptions argument to suppress this logging for selected exception types.

Deprecated since version 4.3: Use multi instead.

tornado.gen.Task(func, *args, **kwargs)[source]

Adapts a callback-based asynchronous function for use in coroutines.

Takes a function (and optional additional arguments) and runs it with those arguments plus a callback keyword argument. The argument passed to the callback is returned as the result of the yield expression.

Changed in version 4.0: gen.Task is now a function that returns a Future, instead of a subclass of YieldPoint. It still behaves the same way when yielded.

class tornado.gen.Arguments

The result of a Task or Wait whose callback had more than one argument (or keyword arguments).

The Arguments object is a collections.namedtuple and can be used either as a tuple (args, kwargs) or an object with attributes args and kwargs.

tornado.gen.convert_yielded(yielded)[source]

Convert a yielded object into a Future.

The default implementation accepts lists, dictionaries, and Futures.

If the singledispatch library is available, this function may be extended to support additional types. For example:

@convert_yielded.register(asyncio.Future)
def _(asyncio_future):
    return tornado.platform.asyncio.to_tornado_future(asyncio_future)

New in version 4.1.

tornado.gen.maybe_future(x)[source]

Converts x into a Future.

If x is already a Future, it is simply returned; otherwise it is wrapped in a new Future. This is suitable for use as result = yield gen.maybe_future(f()) when you don’t know whether f() returns a Future or not.

Deprecated since version 4.3: This function only handles Futures, not other yieldable objects. Instead of maybe_future, check for the non-future result types you expect (often just None), and yield anything unknown.

Legacy interface

Before support for Futures was introduced in Tornado 3.0, coroutines used subclasses of YieldPoint in their yield expressions. These classes are still supported but should generally not be used except for compatibility with older interfaces. None of these classes are compatible with native (await-based) coroutines.

class tornado.gen.YieldPoint[source]

Base class for objects that may be yielded from the generator.

Deprecated since version 4.0: Use Futures instead.

start(runner)[source]

Called by the runner after the generator has yielded.

No other methods will be called on this object before start.

is_ready()[source]

Called by the runner to determine whether to resume the generator.

Returns a boolean; may be called more than once.

get_result()[source]

Returns the value to use as the result of the yield expression.

This method will only be called once, and only after is_ready has returned true.

class tornado.gen.Callback(key)[source]

Returns a callable object that will allow a matching Wait to proceed.

The key may be any value suitable for use as a dictionary key, and is used to match Callbacks to their corresponding Waits. The key must be unique among outstanding callbacks within a single run of the generator function, but may be reused across different runs of the same function (so constants generally work fine).

The callback may be called with zero or one arguments; if an argument is given it will be returned by Wait.

Deprecated since version 4.0: Use Futures instead.

class tornado.gen.Wait(key)[source]

Returns the argument passed to the result of a previous Callback.

Deprecated since version 4.0: Use Futures instead.

class tornado.gen.WaitAll(keys)[source]

Returns the results of multiple previous Callbacks.

The argument is a sequence of Callback keys, and the result is a list of results in the same order.

WaitAll is equivalent to yielding a list of Wait objects.

Deprecated since version 4.0: Use Futures instead.

class tornado.gen.MultiYieldPoint(children, quiet_exceptions=())[source]

Runs multiple asynchronous operations in parallel.

This class is similar to multi, but it always creates a stack context even when no children require it. It is not compatible with native coroutines.

Changed in version 4.2: If multiple YieldPoints fail, any exceptions after the first (which is raised) will be logged. Added the quiet_exceptions argument to suppress this logging for selected exception types.

Changed in version 4.3: Renamed from Multi to MultiYieldPoint. The name Multi remains as an alias for the equivalent multi function.

Deprecated since version 4.3: Use multi instead.

tornado.concurrent — Work with threads and futures

Utilities for working with threads and Futures.

Futures are a pattern for concurrent programming introduced in Python 3.2 in the concurrent.futures package. This package defines a mostly-compatible Future class designed for use from coroutines, as well as some utility functions for interacting with the concurrent.futures package.

class tornado.concurrent.Future[source]

Placeholder for an asynchronous result.

A Future encapsulates the result of an asynchronous operation. In synchronous applications Futures are used to wait for the result from a thread or process pool; in Tornado they are normally used with IOLoop.add_future or by yielding them in a gen.coroutine.

tornado.concurrent.Future is similar to concurrent.futures.Future, but not thread-safe (and therefore faster for use with single-threaded event loops).

In addition to exception and set_exception, methods exc_info and set_exc_info are supported to capture tracebacks in Python 2. The traceback is automatically available in Python 3, but in the Python 2 futures backport this information is discarded. This functionality was previously available in a separate class TracebackFuture, which is now a deprecated alias for this class.

Changed in version 4.0: tornado.concurrent.Future is always a thread-unsafe Future with support for the exc_info methods. Previously it would be an alias for the thread-safe concurrent.futures.Future if that package was available and fall back to the thread-unsafe implementation if it was not.

Changed in version 4.1: If a Future contains an error but that error is never observed (by calling result(), exception(), or exc_info()), a stack trace will be logged when the Future is garbage collected. This normally indicates an error in the application, but in cases where it results in undesired logging it may be necessary to suppress the logging by ensuring that the exception is observed: f.add_done_callback(lambda f: f.exception()).

Consumer methods
Future.result(timeout=None)[source]

If the operation succeeded, return its result. If it failed, re-raise its exception.

This method takes a timeout argument for compatibility with concurrent.futures.Future but it is an error to call it before the Future is done, so the timeout is never used.

Future.exception(timeout=None)[source]

If the operation raised an exception, return the Exception object. Otherwise returns None.

This method takes a timeout argument for compatibility with concurrent.futures.Future but it is an error to call it before the Future is done, so the timeout is never used.

Future.exc_info()[source]

Returns a tuple in the same format as sys.exc_info or None.

New in version 4.0.

Future.add_done_callback(fn)[source]

Attaches the given callback to the Future.

It will be invoked with the Future as its argument when the Future has finished running and its result is available. In Tornado consider using IOLoop.add_future instead of calling add_done_callback directly.

Future.done()[source]

Returns True if the future has finished running.

Future.running()[source]

Returns True if this operation is currently running.

Future.cancel()[source]

Cancel the operation, if possible.

Tornado Futures do not support cancellation, so this method always returns False.

Future.cancelled()[source]

Returns True if the operation has been cancelled.

Tornado Futures do not support cancellation, so this method always returns False.

Producer methods
Future.set_result(result)[source]

Sets the result of a Future.

It is undefined to call any of the set methods more than once on the same object.

Future.set_exception(exception)[source]

Sets the exception of a Future.

Future.set_exc_info(exc_info)[source]

Sets the exception information of a Future.

Preserves tracebacks on Python 2.

New in version 4.0.

tornado.concurrent.run_on_executor(*args, **kwargs)[source]

Decorator to run a synchronous method asynchronously on an executor.

The decorated method may be called with a callback keyword argument and returns a future.

The IOLoop and executor to be used are determined by the io_loop and executor attributes of self. To use different attributes, pass keyword arguments to the decorator:

@run_on_executor(executor='_thread_pool')
def foo(self):
    pass

Changed in version 4.2: Added keyword arguments to use alternative attributes.

tornado.concurrent.return_future(f)[source]

Decorator to make a function that returns via callback return a Future.

The wrapped function should take a callback keyword argument and invoke it with one argument when it has finished. To signal failure, the function can simply raise an exception (which will be captured by the StackContext and passed along to the Future).

From the caller’s perspective, the callback argument is optional. If one is given, it will be invoked when the function is complete with Future.result() as an argument. If the function fails, the callback will not be run and an exception will be raised into the surrounding StackContext.

If no callback is given, the caller should use the Future to wait for the function to complete (perhaps by yielding it in a gen.engine function, or passing it to IOLoop.add_future).

Usage:

@return_future
def future_func(arg1, arg2, callback):
    # Do stuff (possibly asynchronous)
    callback(result)

@gen.engine
def caller(callback):
    yield future_func(arg1, arg2)
    callback()

Note that @return_future and @gen.engine can be applied to the same function, provided @return_future appears first. However, consider using @gen.coroutine instead of this combination.

tornado.concurrent.chain_future(a, b)[source]

Chain two futures together so that when one completes, so does the other.

The result (success or failure) of a will be copied to b, unless b has already been completed or cancelled by the time a finishes.

tornado.locks – Synchronization primitives

New in version 4.2.

Coordinate coroutines with synchronization primitives analogous to those the standard library provides to threads.

(Note that these primitives are not actually thread-safe and cannot be used in place of those from the standard library–they are meant to coordinate Tornado coroutines in a single-threaded app, not to protect shared objects in a multithreaded app.)

Condition
class tornado.locks.Condition[source]

A condition allows one or more coroutines to wait until notified.

Like a standard threading.Condition, but does not need an underlying lock that is acquired and released.

With a Condition, coroutines can wait to be notified by other coroutines:

from tornado import gen
from tornado.ioloop import IOLoop
from tornado.locks import Condition

condition = Condition()

@gen.coroutine
def waiter():
    print("I'll wait right here")
    yield condition.wait()  # Yield a Future.
    print("I'm done waiting")

@gen.coroutine
def notifier():
    print("About to notify")
    condition.notify()
    print("Done notifying")

@gen.coroutine
def runner():
    # Yield two Futures; wait for waiter() and notifier() to finish.
    yield [waiter(), notifier()]

IOLoop.current().run_sync(runner)
I'll wait right here
About to notify
Done notifying
I'm done waiting

wait takes an optional timeout argument, which is either an absolute timestamp:

io_loop = IOLoop.current()

# Wait up to 1 second for a notification.
yield condition.wait(timeout=io_loop.time() + 1)

...or a datetime.timedelta for a timeout relative to the current time:

# Wait up to 1 second.
yield condition.wait(timeout=datetime.timedelta(seconds=1))

The method raises tornado.gen.TimeoutError if there’s no notification before the deadline.

wait(timeout=None)[source]

Wait for notify.

Returns a Future that resolves True if the condition is notified, or False after a timeout.

notify(n=1)[source]

Wake n waiters.

notify_all()[source]

Wake all waiters.

Event
class tornado.locks.Event[source]

An event blocks coroutines until its internal flag is set to True.

Similar to threading.Event.

A coroutine can wait for an event to be set. Once it is set, calls to yield event.wait() will not block unless the event has been cleared:

from tornado import gen
from tornado.ioloop import IOLoop
from tornado.locks import Event

event = Event()

@gen.coroutine
def waiter():
    print("Waiting for event")
    yield event.wait()
    print("Not waiting this time")
    yield event.wait()
    print("Done")

@gen.coroutine
def setter():
    print("About to set the event")
    event.set()

@gen.coroutine
def runner():
    yield [waiter(), setter()]

IOLoop.current().run_sync(runner)
Waiting for event
About to set the event
Not waiting this time
Done
is_set()[source]

Return True if the internal flag is true.

set()[source]

Set the internal flag to True. All waiters are awakened.

Calling wait once the flag is set will not block.

clear()[source]

Reset the internal flag to False.

Calls to wait will block until set is called.

wait(timeout=None)[source]

Block until the internal flag is true.

Returns a Future, which raises tornado.gen.TimeoutError after a timeout.

Semaphore
class tornado.locks.Semaphore(value=1)[source]

A lock that can be acquired a fixed number of times before blocking.

A Semaphore manages a counter representing the number of release calls minus the number of acquire calls, plus an initial value. The acquire method blocks if necessary until it can return without making the counter negative.

Semaphores limit access to a shared resource. To allow access for two workers at a time:

from tornado import gen
from tornado.ioloop import IOLoop
from tornado.locks import Semaphore

sem = Semaphore(2)

@gen.coroutine
def worker(worker_id):
    yield sem.acquire()
    try:
        print("Worker %d is working" % worker_id)
        yield use_some_resource()
    finally:
        print("Worker %d is done" % worker_id)
        sem.release()

@gen.coroutine
def runner():
    # Join all workers.
    yield [worker(i) for i in range(3)]

IOLoop.current().run_sync(runner)
Worker 0 is working
Worker 1 is working
Worker 0 is done
Worker 2 is working
Worker 1 is done
Worker 2 is done

Workers 0 and 1 are allowed to run concurrently, but worker 2 waits until the semaphore has been released once, by worker 0.

acquire is a context manager, so worker could be written as:

@gen.coroutine
def worker(worker_id):
    with (yield sem.acquire()):
        print("Worker %d is working" % worker_id)
        yield use_some_resource()

    # Now the semaphore has been released.
    print("Worker %d is done" % worker_id)

In Python 3.5, the semaphore itself can be used as an async context manager:

async def worker(worker_id):
    async with sem:
        print("Worker %d is working" % worker_id)
        await use_some_resource()

    # Now the semaphore has been released.
    print("Worker %d is done" % worker_id)

Changed in version 4.3: Added async with support in Python 3.5.

release()[source]

Increment the counter and wake one waiter.

acquire(timeout=None)[source]

Decrement the counter. Returns a Future.

Block if the counter is zero and wait for a release. The Future raises TimeoutError after the deadline.

BoundedSemaphore
class tornado.locks.BoundedSemaphore(value=1)[source]

A semaphore that prevents release() being called too many times.

If release would increment the semaphore’s value past the initial value, it raises ValueError. Semaphores are mostly used to guard resources with limited capacity, so a semaphore released too many times is a sign of a bug.

release()[source]

Increment the counter and wake one waiter.

acquire(timeout=None)

Decrement the counter. Returns a Future.

Block if the counter is zero and wait for a release. The Future raises TimeoutError after the deadline.

Lock
class tornado.locks.Lock[source]

A lock for coroutines.

A Lock begins unlocked, and acquire locks it immediately. While it is locked, a coroutine that yields acquire waits until another coroutine calls release.

Releasing an unlocked lock raises RuntimeError.

acquire supports the context manager protocol in all Python versions:

>>> from tornado import gen, locks
>>> lock = locks.Lock()
>>>
>>> @gen.coroutine
... def f():
...    with (yield lock.acquire()):
...        # Do something holding the lock.
...        pass
...
...    # Now the lock is released.

In Python 3.5, Lock also supports the async context manager protocol. Note that in this case there is no acquire, because async with includes both the yield and the acquire (just as it does with threading.Lock):

>>> async def f():  
...    async with lock:
...        # Do something holding the lock.
...        pass
...
...    # Now the lock is released.

Changed in version 3.5: Added async with support in Python 3.5.

acquire(timeout=None)[source]

Attempt to lock. Returns a Future.

Returns a Future, which raises tornado.gen.TimeoutError after a timeout.

release()[source]

Unlock.

The first coroutine in line waiting for acquire gets the lock.

If not locked, raise a RuntimeError.

tornado.queues – Queues for coroutines

New in version 4.2.

Classes
Queue
class tornado.queues.Queue(maxsize=0)[source]

Coordinate producer and consumer coroutines.

If maxsize is 0 (the default) the queue size is unbounded.

from tornado import gen
from tornado.ioloop import IOLoop
from tornado.queues import Queue

q = Queue(maxsize=2)

@gen.coroutine
def consumer():
    while True:
        item = yield q.get()
        try:
            print('Doing work on %s' % item)
            yield gen.sleep(0.01)
        finally:
            q.task_done()

@gen.coroutine
def producer():
    for item in range(5):
        yield q.put(item)
        print('Put %s' % item)

@gen.coroutine
def main():
    # Start consumer without waiting (since it never finishes).
    IOLoop.current().spawn_callback(consumer)
    yield producer()     # Wait for producer to put all tasks.
    yield q.join()       # Wait for consumer to finish all tasks.
    print('Done')

IOLoop.current().run_sync(main)
Put 0
Put 1
Doing work on 0
Put 2
Doing work on 1
Put 3
Doing work on 2
Put 4
Doing work on 3
Doing work on 4
Done

In Python 3.5, Queue implements the async iterator protocol, so consumer() could be rewritten as:

async def consumer():
    async for item in q:
        try:
            print('Doing work on %s' % item)
            yield gen.sleep(0.01)
        finally:
            q.task_done()

Changed in version 4.3: Added async for support in Python 3.5.

maxsize

Number of items allowed in the queue.

qsize()[source]

Number of items in the queue.

put(item, timeout=None)[source]

Put an item into the queue, perhaps waiting until there is room.

Returns a Future, which raises tornado.gen.TimeoutError after a timeout.

put_nowait(item)[source]

Put an item into the queue without blocking.

If no free slot is immediately available, raise QueueFull.

get(timeout=None)[source]

Remove and return an item from the queue.

Returns a Future which resolves once an item is available, or raises tornado.gen.TimeoutError after a timeout.

get_nowait()[source]

Remove and return an item from the queue without blocking.

Return an item if one is immediately available, else raise QueueEmpty.

task_done()[source]

Indicate that a formerly enqueued task is complete.

Used by queue consumers. For each get used to fetch a task, a subsequent call to task_done tells the queue that the processing on the task is complete.

If a join is blocking, it resumes when all items have been processed; that is, when every put is matched by a task_done.

Raises ValueError if called more times than put.

join(timeout=None)[source]

Block until all items in the queue are processed.

Returns a Future, which raises tornado.gen.TimeoutError after a timeout.

PriorityQueue
class tornado.queues.PriorityQueue(maxsize=0)[source]

A Queue that retrieves entries in priority order, lowest first.

Entries are typically tuples like (priority number, data).

from tornado.queues import PriorityQueue

q = PriorityQueue()
q.put((1, 'medium-priority item'))
q.put((0, 'high-priority item'))
q.put((10, 'low-priority item'))

print(q.get_nowait())
print(q.get_nowait())
print(q.get_nowait())
(0, 'high-priority item')
(1, 'medium-priority item')
(10, 'low-priority item')
LifoQueue
class tornado.queues.LifoQueue(maxsize=0)[source]

A Queue that retrieves the most recently put items first.

from tornado.queues import LifoQueue

q = LifoQueue()
q.put(3)
q.put(2)
q.put(1)

print(q.get_nowait())
print(q.get_nowait())
print(q.get_nowait())
1
2
3
Exceptions
QueueEmpty
exception tornado.queues.QueueEmpty[source]

Raised by Queue.get_nowait when the queue has no items.

QueueFull
exception tornado.queues.QueueFull[source]

Raised by Queue.put_nowait when a queue is at its maximum size.

tornado.process — Utilities for multiple processes

Utilities for working with multiple processes, including both forking the server into multiple processes and managing subprocesses.

exception tornado.process.CalledProcessError[source]

An alias for subprocess.CalledProcessError.

tornado.process.cpu_count()[source]

Returns the number of processors on this machine.

tornado.process.fork_processes(num_processes, max_restarts=100)[source]

Starts multiple worker processes.

If num_processes is None or <= 0, we detect the number of cores available on this machine and fork that number of child processes. If num_processes is given and > 0, we fork that specific number of sub-processes.

Since we use processes and not threads, there is no shared memory between any server code.

Note that multiple processes are not compatible with the autoreload module (or the autoreload=True option to tornado.web.Application which defaults to True when debug=True). When using multiple processes, no IOLoops can be created or referenced until after the call to fork_processes.

In each child process, fork_processes returns its task id, a number between 0 and num_processes. Processes that exit abnormally (due to a signal or non-zero exit status) are restarted with the same id (up to max_restarts times). In the parent process, fork_processes returns None if all child processes have exited normally, but will otherwise only exit by throwing an exception.

tornado.process.task_id()[source]

Returns the current task id, if any.

Returns None if this process was not created by fork_processes.

class tornado.process.Subprocess(*args, **kwargs)[source]

Wraps subprocess.Popen with IOStream support.

The constructor is the same as subprocess.Popen with the following additions:

  • stdin, stdout, and stderr may have the value tornado.process.Subprocess.STREAM, which will make the corresponding attribute of the resulting Subprocess a PipeIOStream.
  • A new keyword argument io_loop may be used to pass in an IOLoop.

Changed in version 4.1: The io_loop argument is deprecated.

set_exit_callback(callback)[source]

Runs callback when this process exits.

The callback takes one argument, the return code of the process.

This method uses a SIGCHLD handler, which is a global setting and may conflict if you have other libraries trying to handle the same signal. If you are using more than one IOLoop it may be necessary to call Subprocess.initialize first to designate one IOLoop to run the signal handlers.

In many cases a close callback on the stdout or stderr streams can be used as an alternative to an exit callback if the signal handler is causing a problem.

wait_for_exit(raise_error=True)[source]

Returns a Future which resolves when the process exits.

Usage:

ret = yield proc.wait_for_exit()

This is a coroutine-friendly alternative to set_exit_callback (and a replacement for the blocking subprocess.Popen.wait).

By default, raises subprocess.CalledProcessError if the process has a non-zero exit status. Use wait_for_exit(raise_error=False) to suppress this behavior and return the exit status without raising.

New in version 4.2.

classmethod initialize(io_loop=None)[source]

Initializes the SIGCHLD handler.

The signal handler is run on an IOLoop to avoid locking issues. Note that the IOLoop used for signal handling need not be the same one used by individual Subprocess objects (as long as the IOLoops are each running in separate threads).

Changed in version 4.1: The io_loop argument is deprecated.

classmethod uninitialize()[source]

Removes the SIGCHLD handler.

Integration with other services

tornado.auth — Third-party login with OpenID and OAuth

This module contains implementations of various third-party authentication schemes.

All the classes in this file are class mixins designed to be used with the tornado.web.RequestHandler class. They are used in two ways:

  • On a login handler, use methods such as authenticate_redirect(), authorize_redirect(), and get_authenticated_user() to establish the user’s identity and store authentication tokens to your database and/or cookies.
  • In non-login handlers, use methods such as facebook_request() or twitter_request() to use the authentication tokens to make requests to the respective services.

They all take slightly different arguments due to the fact all these services implement authentication and authorization slightly differently. See the individual service classes below for complete documentation.

Example usage for Google OAuth:

class GoogleOAuth2LoginHandler(tornado.web.RequestHandler,
                               tornado.auth.GoogleOAuth2Mixin):
    @tornado.gen.coroutine
    def get(self):
        if self.get_argument('code', False):
            user = yield self.get_authenticated_user(
                redirect_uri='http://your.site.com/auth/google',
                code=self.get_argument('code'))
            # Save the user with e.g. set_secure_cookie
        else:
            yield self.authorize_redirect(
                redirect_uri='http://your.site.com/auth/google',
                client_id=self.settings['google_oauth']['key'],
                scope=['profile', 'email'],
                response_type='code',
                extra_params={'approval_prompt': 'auto'})

Changed in version 4.0: All of the callback interfaces in this module are now guaranteed to run their callback with an argument of None on error. Previously some functions would do this while others would simply terminate the request on their own. This change also ensures that errors are more consistently reported through the Future interfaces.

Common protocols

These classes implement the OpenID and OAuth standards. They will generally need to be subclassed to use them with any particular site. The degree of customization required will vary, but in most cases overridding the class attributes (which are named beginning with underscores for historical reasons) should be sufficient.

class tornado.auth.OpenIdMixin[source]

Abstract implementation of OpenID and Attribute Exchange.

Class attributes:

  • _OPENID_ENDPOINT: the identity provider’s URI.
authenticate_redirect(callback_uri=None, ax_attrs=['name', 'email', 'language', 'username'], callback=None)[source]

Redirects to the authentication URL for this service.

After authentication, the service will redirect back to the given callback URI with additional parameters including openid.mode.

We request the given attributes for the authenticated user by default (name, email, language, and username). If you don’t need all those attributes for your app, you can request fewer with the ax_attrs keyword argument.

Changed in version 3.1: Returns a Future and takes an optional callback. These are not strictly necessary as this method is synchronous, but they are supplied for consistency with OAuthMixin.authorize_redirect.

get_authenticated_user(callback, http_client=None)[source]

Fetches the authenticated user data upon redirect.

This method should be called by the handler that receives the redirect from the authenticate_redirect() method (which is often the same as the one that calls it; in that case you would call get_authenticated_user if the openid.mode parameter is present and authenticate_redirect if it is not).

The result of this method will generally be used to set a cookie.

get_auth_http_client()[source]

Returns the AsyncHTTPClient instance to be used for auth requests.

May be overridden by subclasses to use an HTTP client other than the default.

class tornado.auth.OAuthMixin[source]

Abstract implementation of OAuth 1.0 and 1.0a.

See TwitterMixin below for an example implementation.

Class attributes:

  • _OAUTH_AUTHORIZE_URL: The service’s OAuth authorization url.
  • _OAUTH_ACCESS_TOKEN_URL: The service’s OAuth access token url.
  • _OAUTH_VERSION: May be either “1.0” or “1.0a”.
  • _OAUTH_NO_CALLBACKS: Set this to True if the service requires advance registration of callbacks.

Subclasses must also override the _oauth_get_user_future and _oauth_consumer_token methods.

authorize_redirect(callback_uri=None, extra_params=None, http_client=None, callback=None)[source]

Redirects the user to obtain OAuth authorization for this service.

The callback_uri may be omitted if you have previously registered a callback URI with the third-party service. For some services (including Friendfeed), you must use a previously-registered callback URI and cannot specify a callback via this method.

This method sets a cookie called _oauth_request_token which is subsequently used (and cleared) in get_authenticated_user for security purposes.

Note that this method is asynchronous, although it calls RequestHandler.finish for you so it may not be necessary to pass a callback or use the Future it returns. However, if this method is called from a function decorated with gen.coroutine, you must call it with yield to keep the response from being closed prematurely.

Changed in version 3.1: Now returns a Future and takes an optional callback, for compatibility with gen.coroutine.

get_authenticated_user(callback, http_client=None)[source]

Gets the OAuth authorized user and access token.

This method should be called from the handler for your OAuth callback URL to complete the registration process. We run the callback with the authenticated user dictionary. This dictionary will contain an access_key which can be used to make authorized requests to this service on behalf of the user. The dictionary will also contain other fields such as name, depending on the service used.

_oauth_consumer_token()[source]

Subclasses must override this to return their OAuth consumer keys.

The return value should be a dict with keys key and secret.

_oauth_get_user_future(access_token, callback)[source]

Subclasses must override this to get basic information about the user.

Should return a Future whose result is a dictionary containing information about the user, which may have been retrieved by using access_token to make a request to the service.

The access token will be added to the returned dictionary to make the result of get_authenticated_user.

For backwards compatibility, the callback-based _oauth_get_user method is also supported.

get_auth_http_client()[source]

Returns the AsyncHTTPClient instance to be used for auth requests.

May be overridden by subclasses to use an HTTP client other than the default.

class tornado.auth.OAuth2Mixin[source]

Abstract implementation of OAuth 2.0.

See FacebookGraphMixin or GoogleOAuth2Mixin below for example implementations.

Class attributes:

  • _OAUTH_AUTHORIZE_URL: The service’s authorization url.
  • _OAUTH_ACCESS_TOKEN_URL: The service’s access token url.
authorize_redirect(redirect_uri=None, client_id=None, client_secret=None, extra_params=None, callback=None, scope=None, response_type='code')[source]

Redirects the user to obtain OAuth authorization for this service.

Some providers require that you register a redirect URL with your application instead of passing one via this method. You should call this method to log the user in, and then call get_authenticated_user in the handler for your redirect URL to complete the authorization process.

Changed in version 3.1: Returns a Future and takes an optional callback. These are not strictly necessary as this method is synchronous, but they are supplied for consistency with OAuthMixin.authorize_redirect.

oauth2_request(url, callback, access_token=None, post_args=None, **args)[source]

Fetches the given URL auth an OAuth2 access token.

If the request is a POST, post_args should be provided. Query string arguments should be given as keyword arguments.

Example usage:

..testcode:

class MainHandler(tornado.web.RequestHandler,
                  tornado.auth.FacebookGraphMixin):
    @tornado.web.authenticated
    @tornado.gen.coroutine
    def get(self):
        new_entry = yield self.oauth2_request(
            "https://graph.facebook.com/me/feed",
            post_args={"message": "I am posting from my Tornado application!"},
            access_token=self.current_user["access_token"])

        if not new_entry:
            # Call failed; perhaps missing permission?
            yield self.authorize_redirect()
            return
        self.finish("Posted a message!")

New in version 4.3.

get_auth_http_client()[source]

Returns the AsyncHTTPClient instance to be used for auth requests.

May be overridden by subclasses to use an HTTP client other than the default.

New in version 4.3.

Google
class tornado.auth.GoogleOAuth2Mixin[source]

Google authentication using OAuth2.

In order to use, register your application with Google and copy the relevant parameters to your application settings.

  • Go to the Google Dev Console at http://console.developers.google.com
  • Select a project, or create a new one.
  • In the sidebar on the left, select APIs & Auth.
  • In the list of APIs, find the Google+ API service and set it to ON.
  • In the sidebar on the left, select Credentials.
  • In the OAuth section of the page, select Create New Client ID.
  • Set the Redirect URI to point to your auth handler
  • Copy the “Client secret” and “Client ID” to the application settings as {“google_oauth”: {“key”: CLIENT_ID, “secret”: CLIENT_SECRET}}

New in version 3.2.

get_authenticated_user(redirect_uri, code, callback)[source]

Handles the login for the Google user, returning an access token.

The result is a dictionary containing an access_token field ([among others](https://developers.google.com/identity/protocols/OAuth2WebServer#handlingtheresponse)). Unlike other get_authenticated_user methods in this package, this method does not return any additional information about the user. The returned access token can be used with OAuth2Mixin.oauth2_request to request additional information (perhaps from https://www.googleapis.com/oauth2/v2/userinfo)

Example usage:

class GoogleOAuth2LoginHandler(tornado.web.RequestHandler,
                               tornado.auth.GoogleOAuth2Mixin):
    @tornado.gen.coroutine
    def get(self):
        if self.get_argument('code', False):
            access = yield self.get_authenticated_user(
                redirect_uri='http://your.site.com/auth/google',
                code=self.get_argument('code'))
            user = yield self.oauth2_request(
                "https://www.googleapis.com/oauth2/v1/userinfo",
                access_token=access["access_token"])
            # Save the user and access token with
            # e.g. set_secure_cookie.
        else:
            yield self.authorize_redirect(
                redirect_uri='http://your.site.com/auth/google',
                client_id=self.settings['google_oauth']['key'],
                scope=['profile', 'email'],
                response_type='code',
                extra_params={'approval_prompt': 'auto'})
Facebook
class tornado.auth.FacebookGraphMixin[source]

Facebook authentication using the new Graph API and OAuth2.

get_authenticated_user(redirect_uri, client_id, client_secret, code, callback, extra_fields=None)[source]

Handles the login for the Facebook user, returning a user object.

Example usage:

class FacebookGraphLoginHandler(tornado.web.RequestHandler,
                                tornado.auth.FacebookGraphMixin):
  @tornado.gen.coroutine
  def get(self):
      if self.get_argument("code", False):
          user = yield self.get_authenticated_user(
              redirect_uri='/auth/facebookgraph/',
              client_id=self.settings["facebook_api_key"],
              client_secret=self.settings["facebook_secret"],
              code=self.get_argument("code"))
          # Save the user with e.g. set_secure_cookie
      else:
          yield self.authorize_redirect(
              redirect_uri='/auth/facebookgraph/',
              client_id=self.settings["facebook_api_key"],
              extra_params={"scope": "read_stream,offline_access"})
facebook_request(path, callback, access_token=None, post_args=None, **args)[source]

Fetches the given relative API path, e.g., “/btaylor/picture”

If the request is a POST, post_args should be provided. Query string arguments should be given as keyword arguments.

An introduction to the Facebook Graph API can be found at http://developers.facebook.com/docs/api

Many methods require an OAuth access token which you can obtain through authorize_redirect and get_authenticated_user. The user returned through that process includes an access_token attribute that can be used to make authenticated requests via this method.

Example usage:

..testcode:

class MainHandler(tornado.web.RequestHandler,
                  tornado.auth.FacebookGraphMixin):
    @tornado.web.authenticated
    @tornado.gen.coroutine
    def get(self):
        new_entry = yield self.facebook_request(
            "/me/feed",
            post_args={"message": "I am posting from my Tornado application!"},
            access_token=self.current_user["access_token"])

        if not new_entry:
            # Call failed; perhaps missing permission?
            yield self.authorize_redirect()
            return
        self.finish("Posted a message!")

The given path is relative to self._FACEBOOK_BASE_URL, by default “https://graph.facebook.com”.

This method is a wrapper around OAuth2Mixin.oauth2_request; the only difference is that this method takes a relative path, while oauth2_request takes a complete url.

Changed in version 3.1: Added the ability to override self._FACEBOOK_BASE_URL.

Twitter
class tornado.auth.TwitterMixin[source]

Twitter OAuth authentication.

To authenticate with Twitter, register your application with Twitter at http://twitter.com/apps. Then copy your Consumer Key and Consumer Secret to the application settings twitter_consumer_key and twitter_consumer_secret. Use this mixin on the handler for the URL you registered as your application’s callback URL.

When your application is set up, you can use this mixin like this to authenticate the user with Twitter and get access to their stream:

class TwitterLoginHandler(tornado.web.RequestHandler,
                          tornado.auth.TwitterMixin):
    @tornado.gen.coroutine
    def get(self):
        if self.get_argument("oauth_token", None):
            user = yield self.get_authenticated_user()
            # Save the user using e.g. set_secure_cookie()
        else:
            yield self.authorize_redirect()

The user object returned by get_authenticated_user includes the attributes username, name, access_token, and all of the custom Twitter user attributes described at https://dev.twitter.com/docs/api/1.1/get/users/show

authenticate_redirect(callback_uri=None, callback=None)[source]

Just like authorize_redirect, but auto-redirects if authorized.

This is generally the right interface to use if you are using Twitter for single-sign on.

Changed in version 3.1: Now returns a Future and takes an optional callback, for compatibility with gen.coroutine.

twitter_request(path, callback=None, access_token=None, post_args=None, **args)[source]

Fetches the given API path, e.g., statuses/user_timeline/btaylor

The path should not include the format or API version number. (we automatically use JSON format and API version 1).

If the request is a POST, post_args should be provided. Query string arguments should be given as keyword arguments.

All the Twitter methods are documented at http://dev.twitter.com/

Many methods require an OAuth access token which you can obtain through authorize_redirect and get_authenticated_user. The user returned through that process includes an ‘access_token’ attribute that can be used to make authenticated requests via this method. Example usage:

class MainHandler(tornado.web.RequestHandler,
                  tornado.auth.TwitterMixin):
    @tornado.web.authenticated
    @tornado.gen.coroutine
    def get(self):
        new_entry = yield self.twitter_request(
            "/statuses/update",
            post_args={"status": "Testing Tornado Web Server"},
            access_token=self.current_user["access_token"])
        if not new_entry:
            # Call failed; perhaps missing permission?
            yield self.authorize_redirect()
            return
        self.finish("Posted a message!")

tornado.wsgi — Interoperability with other Python frameworks and servers

WSGI support for the Tornado web framework.

WSGI is the Python standard for web servers, and allows for interoperability between Tornado and other Python web frameworks and servers. This module provides WSGI support in two ways:

  • WSGIAdapter converts a tornado.web.Application to the WSGI application interface. This is useful for running a Tornado app on another HTTP server, such as Google App Engine. See the WSGIAdapter class documentation for limitations that apply.
  • WSGIContainer lets you run other WSGI applications and frameworks on the Tornado HTTP server. For example, with this class you can mix Django and Tornado handlers in a single server.
Running Tornado apps on WSGI servers
class tornado.wsgi.WSGIAdapter(application)[source]

Converts a tornado.web.Application instance into a WSGI application.

Example usage:

import tornado.web
import tornado.wsgi
import wsgiref.simple_server

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")

if __name__ == "__main__":
    application = tornado.web.Application([
        (r"/", MainHandler),
    ])
    wsgi_app = tornado.wsgi.WSGIAdapter(application)
    server = wsgiref.simple_server.make_server('', 8888, wsgi_app)
    server.serve_forever()

See the appengine demo for an example of using this module to run a Tornado app on Google App Engine.

In WSGI mode asynchronous methods are not supported. This means that it is not possible to use AsyncHTTPClient, or the tornado.auth or tornado.websocket modules.

New in version 4.0.

class tornado.wsgi.WSGIApplication(handlers=None, default_host='', transforms=None, **settings)[source]

A WSGI equivalent of tornado.web.Application.

Deprecated since version 4.0: Use a regular Application and wrap it in WSGIAdapter instead.

Running WSGI apps on Tornado servers
class tornado.wsgi.WSGIContainer(wsgi_application)[source]

Makes a WSGI-compatible function runnable on Tornado’s HTTP server.

Warning

WSGI is a synchronous interface, while Tornado’s concurrency model is based on single-threaded asynchronous execution. This means that running a WSGI app with Tornado’s WSGIContainer is less scalable than running the same app in a multi-threaded WSGI server like gunicorn or uwsgi. Use WSGIContainer only when there are benefits to combining Tornado and WSGI in the same process that outweigh the reduced scalability.

Wrap a WSGI function in a WSGIContainer and pass it to HTTPServer to run it. For example:

def simple_app(environ, start_response):
    status = "200 OK"
    response_headers = [("Content-type", "text/plain")]
    start_response(status, response_headers)
    return ["Hello world!\n"]

container = tornado.wsgi.WSGIContainer(simple_app)
http_server = tornado.httpserver.HTTPServer(container)
http_server.listen(8888)
tornado.ioloop.IOLoop.current().start()

This class is intended to let other frameworks (Django, web.py, etc) run on the Tornado HTTP server and I/O loop.

The tornado.web.FallbackHandler class is often useful for mixing Tornado and WSGI apps in the same server. See https://github.com/bdarnell/django-tornado-demo for a complete example.

static environ(request)[source]

Converts a tornado.httputil.HTTPServerRequest to a WSGI environment.

tornado.platform.asyncio — Bridge between asyncio and Tornado

Bridges between the asyncio module and Tornado IOLoop.

New in version 3.2.

This module integrates Tornado with the asyncio module introduced in Python 3.4 (and available as a separate download for Python 3.3). This makes it possible to combine the two libraries on the same event loop.

Most applications should use AsyncIOMainLoop to run Tornado on the default asyncio event loop. Applications that need to run event loops on multiple threads may use AsyncIOLoop to create multiple loops.

Note

Tornado requires the add_reader family of methods, so it is not compatible with the ProactorEventLoop on Windows. Use the SelectorEventLoop instead.

class tornado.platform.asyncio.AsyncIOMainLoop[source]

AsyncIOMainLoop creates an IOLoop that corresponds to the current asyncio event loop (i.e. the one returned by asyncio.get_event_loop()). Recommended usage:

from tornado.platform.asyncio import AsyncIOMainLoop
import asyncio
AsyncIOMainLoop().install()
asyncio.get_event_loop().run_forever()
class tornado.platform.asyncio.AsyncIOLoop[source]

AsyncIOLoop is an IOLoop that runs on an asyncio event loop. This class follows the usual Tornado semantics for creating new IOLoops; these loops are not necessarily related to the asyncio default event loop. Recommended usage:

from tornado.ioloop import IOLoop
IOLoop.configure('tornado.platform.asyncio.AsyncIOLoop')
IOLoop.current().start()

Each AsyncIOLoop creates a new asyncio.EventLoop; this object can be accessed with the asyncio_loop attribute.

tornado.platform.asyncio.to_tornado_future(asyncio_future)[source]

Convert an asyncio.Future to a tornado.concurrent.Future.

New in version 4.1.

tornado.platform.asyncio.to_asyncio_future(tornado_future)[source]

Convert a Tornado yieldable object to an asyncio.Future.

New in version 4.1.

Changed in version 4.3: Now accepts any yieldable object, not just tornado.concurrent.Future.

tornado.platform.caresresolver — Asynchronous DNS Resolver using C-Ares

This module contains a DNS resolver using the c-ares library (and its wrapper pycares).

class tornado.platform.caresresolver.CaresResolver

Name resolver based on the c-ares library.

This is a non-blocking and non-threaded resolver. It may not produce the same results as the system resolver, but can be used for non-blocking resolution when threads cannot be used.

c-ares fails to resolve some names when family is AF_UNSPEC, so it is only recommended for use in AF_INET (i.e. IPv4). This is the default for tornado.simple_httpclient, but other libraries may default to AF_UNSPEC.

tornado.platform.twisted — Bridges between Twisted and Tornado

Bridges between the Twisted reactor and Tornado IOLoop.

This module lets you run applications and libraries written for Twisted in a Tornado application. It can be used in two modes, depending on which library’s underlying event loop you want to use.

This module has been tested with Twisted versions 11.0.0 and newer.

Twisted on Tornado
class tornado.platform.twisted.TornadoReactor(io_loop=None)[source]

Twisted reactor built on the Tornado IOLoop.

TornadoReactor implements the Twisted reactor interface on top of the Tornado IOLoop. To use it, simply call install at the beginning of the application:

import tornado.platform.twisted
tornado.platform.twisted.install()
from twisted.internet import reactor

When the app is ready to start, call IOLoop.current().start() instead of reactor.run().

It is also possible to create a non-global reactor by calling tornado.platform.twisted.TornadoReactor(io_loop). However, if the IOLoop and reactor are to be short-lived (such as those used in unit tests), additional cleanup may be required. Specifically, it is recommended to call:

reactor.fireSystemEvent('shutdown')
reactor.disconnectAll()

before closing the IOLoop.

Changed in version 4.1: The io_loop argument is deprecated.

tornado.platform.twisted.install(io_loop=None)[source]

Install this package as the default Twisted reactor.

install() must be called very early in the startup process, before most other twisted-related imports. Conversely, because it initializes the IOLoop, it cannot be called before fork_processes or multi-process start. These conflicting requirements make it difficult to use TornadoReactor in multi-process mode, and an external process manager such as supervisord is recommended instead.

Changed in version 4.1: The io_loop argument is deprecated.

Tornado on Twisted
class tornado.platform.twisted.TwistedIOLoop[source]

IOLoop implementation that runs on Twisted.

TwistedIOLoop implements the Tornado IOLoop interface on top of the Twisted reactor. Recommended usage:

from tornado.platform.twisted import TwistedIOLoop
from twisted.internet import reactor
TwistedIOLoop().install()
# Set up your tornado application as usual using `IOLoop.instance`
reactor.run()

Uses the global Twisted reactor by default. To create multiple TwistedIOLoops in the same process, you must pass a unique reactor when constructing each one.

Not compatible with tornado.process.Subprocess.set_exit_callback because the SIGCHLD handlers used by Tornado and Twisted conflict with each other.

Twisted DNS resolver
class tornado.platform.twisted.TwistedResolver[source]

Twisted-based asynchronous resolver.

This is a non-blocking and non-threaded resolver. It is recommended only when threads cannot be used, since it has limitations compared to the standard getaddrinfo-based Resolver and ThreadedResolver. Specifically, it returns at most one result, and arguments other than host and family are ignored. It may fail to resolve when family is not socket.AF_UNSPEC.

Requires Twisted 12.1 or newer.

Changed in version 4.1: The io_loop argument is deprecated.

Utilities

tornado.autoreload — Automatically detect code changes in development

Automatically restart the server when a source file is modified.

Most applications should not access this module directly. Instead, pass the keyword argument autoreload=True to the tornado.web.Application constructor (or debug=True, which enables this setting and several others). This will enable autoreload mode as well as checking for changes to templates and static resources. Note that restarting is a destructive operation and any requests in progress will be aborted when the process restarts. (If you want to disable autoreload while using other debug-mode features, pass both debug=True and autoreload=False).

This module can also be used as a command-line wrapper around scripts such as unit test runners. See the main method for details.

The command-line wrapper and Application debug modes can be used together. This combination is encouraged as the wrapper catches syntax errors and other import-time failures, while debug mode catches changes once the server has started.

This module depends on IOLoop, so it will not work in WSGI applications and Google App Engine. It also will not work correctly when HTTPServer‘s multi-process mode is used.

Reloading loses any Python interpreter command-line arguments (e.g. -u) because it re-executes Python using sys.executable and sys.argv. Additionally, modifying these variables will cause reloading to behave incorrectly.

tornado.autoreload.start(io_loop=None, check_time=500)[source]

Begins watching source files for changes.

Changed in version 4.1: The io_loop argument is deprecated.

tornado.autoreload.wait()[source]

Wait for a watched file to change, then restart the process.

Intended to be used at the end of scripts like unit test runners, to run the tests again after any source file changes (but see also the command-line interface in main)

tornado.autoreload.watch(filename)[source]

Add a file to the watch list.

All imported modules are watched by default.

tornado.autoreload.add_reload_hook(fn)[source]

Add a function to be called before reloading the process.

Note that for open file and socket handles it is generally preferable to set the FD_CLOEXEC flag (using fcntl or tornado.platform.auto.set_close_exec) instead of using a reload hook to close them.

tornado.autoreload.main()[source]

Command-line wrapper to re-run a script whenever its source changes.

Scripts may be specified by filename or module name:

python -m tornado.autoreload -m tornado.test.runtests
python -m tornado.autoreload tornado/test/runtests.py

Running a script with this wrapper is similar to calling tornado.autoreload.wait at the end of the script, but this wrapper can catch import-time problems like syntax errors that would otherwise prevent the script from reaching its call to wait.

tornado.log — Logging support

Logging support for Tornado.

Tornado uses three logger streams:

  • tornado.access: Per-request logging for Tornado’s HTTP servers (and potentially other servers in the future)
  • tornado.application: Logging of errors from application code (i.e. uncaught exceptions from callbacks)
  • tornado.general: General-purpose logging, including any errors or warnings from Tornado itself.

These streams may be configured independently using the standard library’s logging module. For example, you may wish to send tornado.access logs to a separate file for analysis.

class tornado.log.LogFormatter(color=True, fmt='%(color)s[%(levelname)1.1s %(asctime)s %(module)s:%(lineno)d]%(end_color)s %(message)s', datefmt='%y%m%d %H:%M:%S', colors={40: 1, 10: 4, 20: 2, 30: 3})[source]

Log formatter used in Tornado.

Key features of this formatter are:

  • Color support when logging to a terminal that supports it.
  • Timestamps on every log line.
  • Robust against str/bytes encoding problems.

This formatter is enabled automatically by tornado.options.parse_command_line (unless --logging=none is used).

Parameters:
  • color (bool) – Enables color support.
  • fmt (string) – Log message format. It will be applied to the attributes dict of log records. The text between %(color)s and %(end_color)s will be colored depending on the level if color support is on.
  • colors (dict) – color mappings from logging level to terminal color code
  • datefmt (string) – Datetime format. Used for formatting (asctime) placeholder in prefix_fmt.

Changed in version 3.2: Added fmt and datefmt arguments.

tornado.log.enable_pretty_logging(options=None, logger=None)[source]

Turns on formatted logging output as configured.

This is called automatically by tornado.options.parse_command_line and tornado.options.parse_config_file.

tornado.log.define_logging_options(options=None)[source]

Add logging-related flags to options.

These options are present automatically on the default options instance; this method is only necessary if you have created your own OptionParser.

New in version 4.2: This function existed in prior versions but was broken and undocumented until 4.2.

tornado.options — Command-line parsing

A command line parsing module that lets modules define their own options.

Each module defines its own options which are added to the global option namespace, e.g.:

from tornado.options import define, options

define("mysql_host", default="127.0.0.1:3306", help="Main user DB")
define("memcache_hosts", default="127.0.0.1:11011", multiple=True,
       help="Main user memcache servers")

def connect():
    db = database.Connection(options.mysql_host)
    ...

The main() method of your application does not need to be aware of all of the options used throughout your program; they are all automatically loaded when the modules are loaded. However, all modules that define options must have been imported before the command line is parsed.

Your main() method can parse the command line or parse a config file with either:

tornado.options.parse_command_line()
# or
tornado.options.parse_config_file("/etc/server.conf")

Command line formats are what you would expect (--myoption=myvalue). Config files are just Python files. Global names become options, e.g.:

myoption = "myvalue"
myotheroption = "myothervalue"

We support datetimes, timedeltas, ints, and floats (just pass a type kwarg to define). We also accept multi-value options. See the documentation for define() below.

tornado.options.options is a singleton instance of OptionParser, and the top-level functions in this module (define, parse_command_line, etc) simply call methods on it. You may create additional OptionParser instances to define isolated sets of options, such as for subcommands.

Note

By default, several options are defined that will configure the standard logging module when parse_command_line or parse_config_file are called. If you want Tornado to leave the logging configuration alone so you can manage it yourself, either pass --logging=none on the command line or do the following to disable it in code:

from tornado.options import options, parse_command_line
options.logging = None
parse_command_line()

Changed in version 4.3: Dashes and underscores are fully interchangeable in option names; options can be defined, set, and read with any mix of the two. Dashes are typical for command-line usage while config files require underscores.

Global functions
tornado.options.define(name, default=None, type=None, help=None, metavar=None, multiple=False, group=None, callback=None)[source]

Defines an option in the global namespace.

See OptionParser.define.

tornado.options.options

Global options object. All defined options are available as attributes on this object.

tornado.options.parse_command_line(args=None, final=True)[source]

Parses global options from the command line.

See OptionParser.parse_command_line.

tornado.options.parse_config_file(path, final=True)[source]

Parses global options from a config file.

See OptionParser.parse_config_file.

tornado.options.print_help(file=sys.stderr)[source]

Prints all the command line options to stderr (or another file).

See OptionParser.print_help.

tornado.options.add_parse_callback(callback)[source]

Adds a parse callback, to be invoked when option parsing is done.

See OptionParser.add_parse_callback

exception tornado.options.Error[source]

Exception raised by errors in the options module.

OptionParser class
class tornado.options.OptionParser[source]

A collection of options, a dictionary with object-like access.

Normally accessed via static functions in the tornado.options module, which reference a global instance.

items()[source]

A sequence of (name, value) pairs.

New in version 3.1.

groups()[source]

The set of option-groups created by define.

New in version 3.1.

group_dict(group)[source]

The names and values of options in a group.

Useful for copying options into Application settings:

from tornado.options import define, parse_command_line, options

define('template_path', group='application')
define('static_path', group='application')

parse_command_line()

application = Application(
    handlers, **options.group_dict('application'))

New in version 3.1.

as_dict()[source]

The names and values of all options.

New in version 3.1.

define(name, default=None, type=None, help=None, metavar=None, multiple=False, group=None, callback=None)[source]

Defines a new command line option.

If type is given (one of str, float, int, datetime, or timedelta) or can be inferred from the default, we parse the command line arguments based on the given type. If multiple is True, we accept comma-separated values, and the option value is always a list.

For multi-value integers, we also accept the syntax x:y, which turns into range(x, y) - very useful for long integer ranges.

help and metavar are used to construct the automatically generated command line help string. The help message is formatted like:

--name=METAVAR      help string

group is used to group the defined options in logical groups. By default, command line options are grouped by the file in which they are defined.

Command line option names must be unique globally. They can be parsed from the command line with parse_command_line or parsed from a config file with parse_config_file.

If a callback is given, it will be run with the new value whenever the option is changed. This can be used to combine command-line and file-based options:

define("config", type=str, help="path to config file",
       callback=lambda path: parse_config_file(path, final=False))

With this definition, options in the file specified by --config will override options set earlier on the command line, but can be overridden by later flags.

parse_command_line(args=None, final=True)[source]

Parses all options given on the command line (defaults to sys.argv).

Note that args[0] is ignored since it is the program name in sys.argv.

We return a list of all arguments that are not parsed as options.

If final is False, parse callbacks will not be run. This is useful for applications that wish to combine configurations from multiple sources.

parse_config_file(path, final=True)[source]

Parses and loads the Python config file at the given path.

If final is False, parse callbacks will not be run. This is useful for applications that wish to combine configurations from multiple sources.

Changed in version 4.1: Config files are now always interpreted as utf-8 instead of the system default encoding.

print_help(file=None)[source]

Prints all the command line options to stderr (or another file).

add_parse_callback(callback)[source]

Adds a parse callback, to be invoked when option parsing is done.

mockable()[source]

Returns a wrapper around self that is compatible with mock.patch.

The mock.patch function (included in the standard library unittest.mock package since Python 3.3, or in the third-party mock package for older versions of Python) is incompatible with objects like options that override __getattr__ and __setattr__. This function returns an object that can be used with mock.patch.object to modify option values:

with mock.patch.object(options.mockable(), 'name', value):
    assert options.name == value

tornado.stack_context — Exception handling across asynchronous callbacks

StackContext allows applications to maintain threadlocal-like state that follows execution as it moves to other execution contexts.

The motivating examples are to eliminate the need for explicit async_callback wrappers (as in tornado.web.RequestHandler), and to allow some additional context to be kept for logging.

This is slightly magic, but it’s an extension of the idea that an exception handler is a kind of stack-local state and when that stack is suspended and resumed in a new context that state needs to be preserved. StackContext shifts the burden of restoring that state from each call site (e.g. wrapping each AsyncHTTPClient callback in async_callback) to the mechanisms that transfer control from one context to another (e.g. AsyncHTTPClient itself, IOLoop, thread pools, etc).

Example usage:

@contextlib.contextmanager
def die_on_error():
    try:
        yield
    except Exception:
        logging.error("exception in asynchronous operation",exc_info=True)
        sys.exit(1)

with StackContext(die_on_error):
    # Any exception thrown here *or in callback and its descendants*
    # will cause the process to exit instead of spinning endlessly
    # in the ioloop.
    http_client.fetch(url, callback)
ioloop.start()

Most applications shouldn’t have to work with StackContext directly. Here are a few rules of thumb for when it’s necessary:

  • If you’re writing an asynchronous library that doesn’t rely on a stack_context-aware library like tornado.ioloop or tornado.iostream (for example, if you’re writing a thread pool), use stack_context.wrap() before any asynchronous operations to capture the stack context from where the operation was started.
  • If you’re writing an asynchronous library that has some shared resources (such as a connection pool), create those shared resources within a with stack_context.NullContext(): block. This will prevent StackContexts from leaking from one request to another.
  • If you want to write something like an exception handler that will persist across asynchronous calls, create a new StackContext (or ExceptionStackContext), and make your asynchronous calls in a with block that references your StackContext.
class tornado.stack_context.StackContext(context_factory)[source]

Establishes the given context as a StackContext that will be transferred.

Note that the parameter is a callable that returns a context manager, not the context itself. That is, where for a non-transferable context manager you would say:

with my_context():

StackContext takes the function itself rather than its result:

with StackContext(my_context):

The result of with StackContext() as cb: is a deactivation callback. Run this callback when the StackContext is no longer needed to ensure that it is not propagated any further (note that deactivating a context does not affect any instances of that context that are currently pending). This is an advanced feature and not necessary in most applications.

class tornado.stack_context.ExceptionStackContext(exception_handler)[source]

Specialization of StackContext for exception handling.

The supplied exception_handler function will be called in the event of an uncaught exception in this context. The semantics are similar to a try/finally clause, and intended use cases are to log an error, close a socket, or similar cleanup actions. The exc_info triple (type, value, traceback) will be passed to the exception_handler function.

If the exception handler returns true, the exception will be consumed and will not be propagated to other exception handlers.

class tornado.stack_context.NullContext[source]

Resets the StackContext.

Useful when creating a shared resource on demand (e.g. an AsyncHTTPClient) where the stack that caused the creating is not relevant to future operations.

tornado.stack_context.wrap(fn)[source]

Returns a callable object that will restore the current StackContext when executed.

Use this whenever saving a callback to be executed later in a different execution context (either in a different thread or asynchronously in the same thread).

tornado.stack_context.run_with_stack_context(context, func)[source]

Run a coroutine func in the given StackContext.

It is not safe to have a yield statement within a with StackContext block, so it is difficult to use stack context with gen.coroutine. This helper function runs the function in the correct context while keeping the yield and with statements syntactically separate.

Example:

@gen.coroutine
def incorrect():
    with StackContext(ctx):
        # ERROR: this will raise StackContextInconsistentError
        yield other_coroutine()

@gen.coroutine
def correct():
    yield run_with_stack_context(StackContext(ctx), other_coroutine)

New in version 3.1.

tornado.testing — Unit testing support for asynchronous code

Support classes for automated testing.

  • AsyncTestCase and AsyncHTTPTestCase: Subclasses of unittest.TestCase with additional support for testing asynchronous (IOLoop based) code.
  • ExpectLog and LogTrapTestCase: Make test logs less spammy.
  • main(): A simple test runner (wrapper around unittest.main()) with support for the tornado.autoreload module to rerun the tests when code changes.
Asynchronous test cases
class tornado.testing.AsyncTestCase(methodName='runTest', **kwargs)[source]

TestCase subclass for testing IOLoop-based asynchronous code.

The unittest framework is synchronous, so the test must be complete by the time the test method returns. This means that asynchronous code cannot be used in quite the same way as usual. To write test functions that use the same yield-based patterns used with the tornado.gen module, decorate your test methods with tornado.testing.gen_test instead of tornado.gen.coroutine. This class also provides the stop() and wait() methods for a more manual style of testing. The test method itself must call self.wait(), and asynchronous callbacks should call self.stop() to signal completion.

By default, a new IOLoop is constructed for each test and is available as self.io_loop. This IOLoop should be used in the construction of HTTP clients/servers, etc. If the code being tested requires a global IOLoop, subclasses should override get_new_ioloop to return it.

The IOLoop‘s start and stop methods should not be called directly. Instead, use