Site Technology / API

<< Site Technology

SOAP API for site

This site contains a SOAP API that defines three[1] requests:
  • doDbQuery: Issue an SQL statement and fetch the results (only SELECT statements supported to a limited number of tables with a limited subset of MySQL supported SELECT statement syntax)
  • doLoadPage: Load the contents (source code) of a page (in wtxt format)
  • doRenderWtxt: Render the given wtxt into HTML

The WSDL file that defines this service, can be downloaded here: http://tasvideos.org/css/TasvideosQuery.wsdl

As an extension to the SOAP standard, if you prefer to receive the response in JSON format instead of as XML (for ease of use with Javascript applications), attach the ?out=json string to the callpoint URI.

When you develop your client program, please save a local copy of that WSDL file, instead of loading it from the server each time.

User key

Each query requires a key, which identifies the user/developer in question.

You can acquire a key from Bisqwit if you explain your needs well enough. Temporary (a day or week) keys for testing may be granted by just asking.

Supported SELECT statement syntax

   request =
     [ (DESC|DESCRIBE|EXPLAIN) [EXTENDED] ] query.
   
   query =
    (SELECT
       [DISTINCT]
       ( '*'
       | (expression [ [ AS ] name ]) % ','
       )
    [ FROM table_refs
      [ WHERE    expression ]
      [ GROUP BY (expression [ASC | DESC]) % ',' ]
      [ HAVING   expression ] ]
    [ ORDER BY (expression [ASC | DESC]) % ',' ]
    [ LIMIT integer
       ( OFFSET integer
       | ( ',' integer )?
       ) ]
    ) % (UNION [ DISTINCT | ALL ])
     .

   table_refs =
       ( table_name [ [ AS ] table_alias ]
       | '(' table_refs ')'
       | LEFT JOIN table_refs ( ON expression
                              | USING '(' (name % ',') ')'
                              )
       ) % ','.

   name = field_name_or_alias.

   expression =
    ( expr_head*
      ( [table_name_or_alias '.'] name
     | TRUE
     | FALSE
     | numeric_constant
     | string_constant 
     | expression
     | [ EXISTS ] '(' query ')'
     | COUNT '(' [ DISTINCT ] ( '*' | expression ) ')'
     | [ SUM | LENGTH | AVG | FLOOR ] '(' expression ')'
      ) expr_tail*
     ) % expr_sep.

    expr_head = NOT | '!' | '-' | '+'.

    expr_tail =
        IS [ NOT ] NULL
      | [ NOT ] IN '(' ( query
                      | ( expression % ',' )
                        ) ')'
      | BETWEEN expression AND expression
      | LIKE string_constant.
    
    expr_sep =
           '+' | '-' | '/' | '*' | '^'
         | '=' | '<' | '>' | '<=' | '>=' | '!=' | '<>'
         | AND | OR | XOR | MOD.
The syntax is also available as a graphical flowchart at http://bisqwit.iki.fi/kala/snap/tasvqueryflowchart.png. However, it is a large image.

The API can also access portions of the forums by prefixing the table name (phpBB standard table names) with the database name, for example: select count(*) from nesvideos_forum.posts. Do note that not all tables and fields can be accessed.

Example client program 1

This example program, written in PHP, queries and gets the list of all currently published Game Boy movie file names:
  <?php
  $key = YOUR_USERKEY_HERE;
  $sql = 'select filename from movie m,movie_file f,filetype t,system s'.
         'where m.id=movieid and t.id=typeid and filetypeclass="M"'.
         'and obsoleted_by=-1 and s.id=systemid and abbr="GB"';
  $soap_options = Array('compression' => SOAP_COMPRESSION_ACCEPT | SOAP_COMPRESSION_GZIP);
  $client = new SoapClient('TasvideosQuery.wsdl', // ← this file must exist
                           $soap_options);
  $response = $client->doDbQuery($key, $sql, true, 5000);
  print_r($response->columns);

Example client program 2

This example program, written in Ruby, fetches the source code of the Rules page:

  require 'soap/wsdlDriver.rb'
  key = YOUR_USER_KEY_HERE          ## ↓↓ this file must exist
  client = SOAP::WSDLDriverFactory.new('TasvideosQuery.wsdl').create_rpc_driver
  response = client.doLoadPage(key, 'Rules')
  print response.wtxt

Limitations

The SQL interface has limitations on how often it can be accessed. It also tries to protects itself against queries that cause tons of database load.

These values are currently held in secret, but should not cause problems if you don't access the site too often. Depending on the query (and the popularity of this API), the minimum allowed time between queries may range from 15 seconds to 2 hours.

SOAP API for tracker

The BitTorrent Tracker also has a SOAP API. It defines the following four requests[1]:
  • GetAnnounceURI: Returns the URI of the announce. Used by the site for verifying torrents that are being published.
  • GetTorrentDownloadURI: Returns the URI from where the given torrent can be downloaded.
  • Scrape: Does a equivalent to a http fetch of /scrape, returns the result.
  • GetTorrents: Returns a list of torrents matching the given glob() pattern.

The WSDL file that defines this service, can be downloaded here: http://bisqwit.iki.fi/torrents/api/api.wsdl

As an extension to the SOAP standard, if you prefer to receive the response in JSON format instead of as XML, attach the ?out=json string to the callpoint URI.

When you develop your client program, please save a local copy of that WSDL file, instead of loading it from the server each time.


See also:

[1]: It actually defines more than those, but the rest of them are used by the site internally and require authentication tokens for access.


Get Firefox!SiteTechnology/API last edited by Bisqwit on 2008-01-17 16:51:55
Page info and history | Latest diff | List referrers