Twisted Documentation: twisted.enterprise.adbapi: Twisted RDBMS support

# twisted.enterprise.adbapi: Twisted RDBMS support

## Abstract

Twisted is an asynchronous networking framework, but most database API implementations unfortunately have blocking interfaces -- for this reason, `twisted.enterprise.adbapi` was created. It is a non-blocking interface to the standardized DB-API 2.0 API, which allows you to access a number of different RDBMSes.

## What you should already know

- Python :-)
- How to write a simple Twisted Server (see this tutorial to learn how)
- Familiarity with using database interfaces (see the documentation for DBAPI 2.0 or this article by Andrew Kuchling)

## Quick Overview

Twisted is an asynchronous framework. This means standard database modules cannot be used directly, as they typically work something like:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    # Create connection...
    db = dbmodule.connect('mydb', 'andrew', 'password')
    # ...which blocks for an unknown amount of time

    # Create a cursor
    cursor = db.cursor()

    # Do a query...
    resultset = cursor.query('SELECT * FROM table WHERE ...')
    # ...which could take a long time, perhaps even minutes.

Those delays are unacceptable when using an asynchronous framework such as Twisted. For this reason, twisted provides `twisted.enterprise.adbapi`, an asynchronous wrapper for any DB-API 2.0-compliant module.

`enterprise.adbapi` will do blocking database operations in separate threads, which trigger callbacks in the originating thread when they complete. In the meantime, the original thread can continue doing normal work, like servicing other requests.

## How do I use adbapi?

Rather than creating a database connection directly, use the `adbapi.ConnectionPool` class to manage a connections for you. This allows `enterprise.adbapi` to use multiple connections, one per thread. This is easy:

    1
    2
    3
    # Using the "dbmodule" from the previous example, create a ConnectionPool
    from twisted.enterprise import adbapi
    dbpool = adbapi.ConnectionPool("dbmodule", 'mydb', 'andrew', 'password')

Things to note about doing this:

- There is no need to import dbmodule directly. You just pass the name to `adbapi.ConnectionPool`'s constructor.
- The parameters you would pass to dbmodule.connect are passed as extra arguments to `adbapi.ConnectionPool`'s constructor. Keyword parameters work as well.

Now we can do a database query:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    # equivalent of cursor.execute(statement), return cursor.fetchall():
    def getAge(user):
        return dbpool.runQuery("SELECT age FROM users WHERE name = ?", user)

    def printResult(l):
        if l:
            print l[0][0], "years old"
        else:
            print "No such user"

    getAge("joe").addCallback(printResult)

This is straightforward, except perhaps for the return value of `getAge`. It returns a `twisted.internet.defer.Deferred`, which allows arbitrary callbacks to be called upon completion (or upon failure). More documentation on Deferred is available here.

In addition to `runQuery`, there is also `runOperation`, and `runInteraction` that gets called with a callable (e.g. a function). The function will be called in the thread with a `twisted.enterprise.adbapi.Transaction`, which basically mimics a DB-API cursor. In all cases a database transaction will be commited after your database usage is finished, unless an exception is raised in which case it will be rolled back.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    def _getAge(txn, user):
        # this will run in a thread, we can use blocking calls
        txn.execute("SELECT * FROM foo")
        # ... other cursor commands called on txn ...
        txn.execute("SELECT age FROM users WHERE name = ?", user)
        result = txn.fetchall()
        if result:
            return result[0][0]
        else:
            return None

    def getAge(user):
        return dbpool.runInteraction(_getAge, user)

    def printResult(age):
        if age != None:
            print age, "years old"
        else:
            print "No such user"

    getAge("joe").addCallback(printResult)

Also worth noting is that these examples assumes that dbmodule uses the qmarks paramstyle (see the DB-API specification). If your dbmodule uses a different paramstyle (e.g. pyformat) then use that. Twisted doesn't attempt to offer any sort of magic paramater munging -- `runQuery(query, params, ...)` maps directly onto `cursor.execute(query, params, ...)`.

## Examples of various database adapters

Notice that the first argument is the module name you would usually import and get `connect(...)` from, and that following arguments are whatever arguments you'd call `connect(...)` with.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    from twisted.enterprise import adbapi

    # Gadfly
    cp = adbapi.ConnectionPool("gadfly", "test", "/tmp/gadflyDB")

    # PostgreSQL PyPgSQL
    cp = adbapi.ConnectionPool("pyPgSQL.PgSQL", database="test")

    # MySQL
    cp = adbapi.ConnectionPool("MySQLdb", db="test")

## And that's it!

That's all you need to know to use a database from within Twisted. You probably should read the adbapi module's documentation to get an idea of the other functions it has, but hopefully this document presents the core ideas.

Index

Version: 11.1.0