diff options
Diffstat (limited to 'docs/developers')
| -rw-r--r-- | docs/developers/api.md | 82 | ||||
| -rw-r--r-- | docs/developers/directory_structure.md | 257 | ||||
| -rw-r--r-- | docs/developers/index.md | 84 | ||||
| -rw-r--r-- | docs/developers/overview.md | 51 |
4 files changed, 474 insertions, 0 deletions
diff --git a/docs/developers/api.md b/docs/developers/api.md new file mode 100644 index 000000000..cbd4c7c85 --- /dev/null +++ b/docs/developers/api.md @@ -0,0 +1,82 @@ +--- +layout: page +title: API +--- + +# Alaveteli API + +<p class="lead"> + There are two parts to the API for accessing or inserting data programmatically: the read API, and the write API. +</p> + +## Read API + +This is provided via JSON versions of most entities in the system, plus atom +feeds for listing entities: + +### Atom feeds + +There are Atom feeds on most pages which list FOI requests, which you can use +to get updates and links in XML format. Find the URL of the Atom feed in one of +these ways: + +* Look for the RSS feed links. +* Examine the `<link rel="alternate" type="application/atom+xml">` tag in the head of the HTML. +* Add `/feed` to the start of another URL. + +Note that even complicated search queries have Atom feeds. You can do all sorts +of things with them, such as query by authority, by file type, by date range, +or by status. See the advanced search tips for details. + +### JSON structured data + +Quite a few pages have JSON versions, which let you download information about +objects in a structured form. Find them by: + +* adding `.json` to the end of the URL. +* looking for the `<link rel="alternate" type="application/json">` tag in the head of the HTML. + +Requests, users and authorities all have JSON versions containing basic +information about them. Every Atom feed has a JSON equivalent, containing +information about the list of events in the feed. + +### Starting new requests programmatically + +To encourage users to make links to a particular public authority, use URLs of +the form `http://<yoursite>/new/<publicbody_url_name>`. These are the +parameters you can add to those URLs, either in the URL or from a form: + +* `title` - the default summary of the new request. +* `default_letter` - the default text of the body of the letter. The salutation and signoff for your locale are wrapped round this. +* `body` - as an alternative to `default_letter`, this sets the default entire text of the request, so you can customise the salutation and signoff. +* `tags` - space separated list of tags, so you can find and link up any requests made later, e.g. `openlylocal spending_id:12345`. The `:` indicates it is a machine tag. The values of machine tags may also include colons, useful for URIs. + +## Write API + +The write API is designed to be used by public bodies to create their own +requests in the system. Currently used by mySociety's [FOI +Register](https://github.com/mysociety/foi-register) software to support using +Alaveteli as a disclosure log for all FOI activity at a particular public body. + +All requests must include an API key as a variable `k`. This key can be viewed +on each authority's page in the admin interface. Other variables should be sent +as follows: + +* `/api/v2/request` - POST the following json data as a form variable `json` to create a new request: + * `title` - title for the request + * `body` - body of the request + * `external_user_name` - name of the person who originated the request + * `external_url` - URL where a canonical copy of the request can be found + Returns JSON containing a `url` for the new request, and its `id` +* `/api/v2/request/<id>.json` - GET full information about a request +* `/api/v2/request/<id>.json` - POST additional correspondence regarding a request: + * as form variable `json`: + * `direction` - either `request` (from the user - might be a followup, reminder, etc) or `response` (from the authority) + * `body` - the message itself + * `sent_at` - ISO-8601 formatted time that the correspondence was sent + * (optionally) the variable `attachments` as `multipart/form-data`: + * attachments to the correspondence. Attachments can only be attached to messages in the `response` direction + + + + diff --git a/docs/developers/directory_structure.md b/docs/developers/directory_structure.md new file mode 100644 index 000000000..e073a96ce --- /dev/null +++ b/docs/developers/directory_structure.md @@ -0,0 +1,257 @@ +--- +layout: page +title: Directory structure +--- + + +# Alaveteli's directory structure + +<p class="lead">This page gives you an overview of where to find things in Alaveteli's +directories.</p> + +**You'll probably never need to worry about this** if you're just installing +Alaveteli -- this is really more useful if you're a developer planning on +making more substantive changes to the code. You don't need to be familiar with +Ruby to install or make basic [customisations to your +installation](/docs/customising). + +<!-- (and if you do, +remember to read the page about [feeding your changes back](/feeding-back)).--> + +Alaveteli uses Ruby on Rails, which is a common "Model-View-Controller" web +framework — if you're familiar with Rails this will look very familiar. For +more information about the Rails structure see the [Ruby on Rails +website](http://guides.rubyonrails.org/getting_started.html). + +## Key directories and what they're for + +<dl class="dir-structure"> + <dt> + app + </dt> + <dd> + <p><em>the core Alaveteli application code</em></p> + <dl> + <dt> + controllers + </dt> + <dt> + helpers + </dt> + <dt> + mailers + </dt> + <dt> + models + </dt> + <dt> + sass + </dt> + <dt class="last"> + views + </dt> + </dl> + </dd> + <dt> + assets + </dt> + <dd> + Static assets + <dl> + <dt> + css + </dt> + <dd> + Rendered stylesheets + </dd> + <dt> + img + </dt> + <dd> + static images + </dd> + <dt> + sass + </dt> + <dd> + Stylesheets in SCSS format, which are compiled to CSS + </dd> + <dt class="last"> + scripts + </dt> + <dd class="last"> + JavaScript + </dd> + </dl> + </dd> + <dt> + bootstrap + </dt> + <dd> + <p> + Alaveteli's default style uses Bootstrap. + </p> + <dt> + commonlib + </dt> + <dd> + <p><em>mySociety's library of common functions</em></p> + <p> + We maintain a <a href="https://github.com/mysociety/commonlib">common + library</a> that we use across many of our projects (not just + Alaveteli). This is implemented as a <a + href="http://git-scm.com/book/en/Git-Tools-Submodules">git submodule</a>, + so Alaveteli contains it even though the code is separate. Normally, you + don't need to think about this (because git handles it automatically)... + but if you really <em>do</em> need to change anything here, be aware that + it is a separate git repository. + </p> + </dd> + <dt> + config + </dt> + <dd> + <p><em>configuration files</em></p> + <p> + The primary configuration file is <code>general.yml</code>. This file isn't in the git + repository (since it will contain information specific to your installation, including + the database password), but example files are. + </p> + </dd> + <dt> + db + </dt> + <dd> + <p><em>database files</em></p> + <dl> + <dt class="last"> + migrate + </dt> + <dd class="last"> + Rails' migration (updating the database scheme up or down + as the code develops). + </dd> + </dl> + </dd> + <dt> + doc + </dt> + <dd> + <p><em>documentation</em></p> + <p> + These are technical notes. This is in addition to the <a + href="http://code.fixmystreet.com">core documentation</a> — which + you are reading now — which is actually stored in the git + repository in the <code>gh-pages</code> branch, and published as GitHub + pages. + </p> + </dd> + <dt> + lib + </dt> + <dd> + <p><em>custom libraries</em></p> + <dl> + <dt> + tasks + </dt> + <dt class="last"> + whatdotheyknow + </dt> + </dl> + </dd> + <dt> + locale + </dt> + <dd> + <p><em>translations (internationalisation/i18n)</em></p> + <p> + The translation strings are stored in <code>.po</code> files in directories specific to + the locale and encoding. For example, <code>es/</code> contains the translations for the Spanish site. + </p> + </dd> + <dt> + public + </dt> + <dd> + <p><em>static assets</em></p> + <dl> + <dt> + admin + </dt> + <dd> + images, JavaScript and stylesheets used by the admin back-end + </dd> + <dt> + fcgi + </dt> + <dd> + Fast CGI files for serving static assets + </dd> + <dt> + images + </dt> + <dt> + javascripts + </dt> + <dt class="last"> + stylesheets + </dt> + </dl> + </dd> + <dt> + script + </dt> + <dd> + <p><em>server-side shell scripts</em></p> + <p> + For example, <code>alert-overdue-requests</code> for running the script + which finds overdue requests and mails them out. + </p> + </dd> + <dt> + spec + </dt> + <dd> + <p><em>tests</em></p> + <p> + Alaveteli's test suite runs under <a href="TODO">spec</a>. + </p> + </dd> + <dt> + stylesheets + </dt> + <dd> + <p> + <em>global stylesheet</em> + </p> + <p> + Actually just <code>global.css</code> + </p> + </dd> + <dt> + tmp + </dt> + <dd> + <p> + <em>temporary files</em> + </p> + </dd> + <dt class="last"> + vendor + </dt> + <dd class="last"> + <p><em>third-party software</em></p> + <dl> + <dt class="last">plugins</dt> + <dd class="last"> + <p> + Plugins + </p> + </dd> + </dl> + </dd> +</dl> + +We've missed out some of the less important subdirectories here just to keep +things clear. diff --git a/docs/developers/index.md b/docs/developers/index.md new file mode 100644 index 000000000..4b90c9232 --- /dev/null +++ b/docs/developers/index.md @@ -0,0 +1,84 @@ +--- +layout: page +title: For developers +--- + +# Information for developers + +<p class="lead"> + Alaveteli is an open source project. Full-time mySociety developers together with devs from all around the world actively contribute to the codebase. These notes and links will help you if you want to help too. +</p> + +* The software is written in **Ruby on Rails 3.x**. We support postgresql as + the backend database. A configured mail transfer agent (MTA) like exim, + postfix or sendmail is necessary to parse incoming emails. We have production + sites deployed on Debian Squeeze and Ubuntu (10.04 LTS). For performance + reasons, we recommend the use of [Varnish](https://www.varnish-cache.org). + +* To help you understand what the code is doing, read this [high-level + overview]({{ site.baseurl }}docs/developers/overview), which includes a diagram of + the models and how they are related. + +* See the [API documentation]({{ site.baseurl }}docs/developers/api) for how to get + data into or out of Alaveteli. + +* If you need to change or add strings in the interface, see our [guidelines + for internationalisation](http://mysociety.github.io/internationalization.html + ), which include notes about our use of `gettext`. + +* We use the [git flow branching + model](http://nvie.com/posts/a-successful-git-branching-model/), with a small + change: currently our `develop` branch is called `rails-3-develop`, which + means that the latest development version is always found on the + [rails-3-develop + branch](https://github.com/mysociety/alaveteli/tree/rails-3-develop). The + latest stable version is always on the [master + branch](https://github.com/mysociety/alaveteli). If you plan to collaborate + on the software, you may find the [git flow + extensions](https://github.com/nvie/gitflow) useful. + +* Installing the software is a little involved, though it's getting easier. If + you stick to Debian or Ubuntu, it should be possible to get a running version + within a few hours. If you've got your own server, run the + [installation script]({{ site.baseurl }}docs/installing/script/), or follow the + instructions for a + [manual installation]({{ site.baseurl }}docs/installing/manual_install/). + Alternatively, there's an [Alaveteli EC2 AMI]({{ site.baseurl }}docs/installing/ami/) + that might help you get up and running quickly. + [Get in touch](http://www.alaveteli.org/contact/) on the project mailing list or IRC + for help. + +* A standard initial step for customising your deployment is [writing a + theme]({{ site.baseurl }}docs/customising/themes). **If you only read one thing, + it should be this!** + +* Like many Ruby on Rails sites, the software is not hugely performant (see + some notes about [[performance issues]] gathered over time with + WhatDoTheyKnow). The site will run on a server with 512MB RAM but at least + 2GB is recommended. Deployment behind [Varnish](https://www.varnish-cache.org) is also fairly essential. See + [[Production Server Best Practices]] for more. + +* There's a number of [proposals for enhancements](https://github.com/mysociety/alaveteli/wiki/Proposals-for-enhancements), + such as more user-focused features, but see also... + +* ...the [github issues](https://github.com/mysociety/alaveteli/issues). We + mark issues with the label **suitable for volunteers** if we think they are + especially suitable for diving into if you're just looking for something + relatively small to get your teeth into. + +* We try to ensure every commit has a corresponding issue in the issue tracker. + This makes changelogs easier as we can gather all the fixes for a particular + release against a milestone in the issue tracker, [like this 0.4 + release](https://github.com/mysociety/alaveteli/issues?milestone=7&state=close + d). + +* If you're experiencing memory issues, [this blog post about some strategies + used in the + past](http://www.mysociety.org/2009/09/17/whatdotheyknow-growing-pains-and-rub + y-memory-leaks/) might be useful. + +* If you're coding on a mac, see these [MacOS X installation notes]({{ site.baseurl }}docs/installing/macos). <!-- [[OS X Quickstart]] --> + +* We try to adhere to similar good practice across all our projects: see + [mysociety.github.io](http://mysociety.github.io/) for things like our + [coding standards](http://mysociety.github.io/coding-standards.html) diff --git a/docs/developers/overview.md b/docs/developers/overview.md new file mode 100644 index 000000000..ca2d27985 --- /dev/null +++ b/docs/developers/overview.md @@ -0,0 +1,51 @@ +--- +layout: page +title: High-level overview +--- + +# High-level overview + +<p class="lead"> + This page describes the process and entities that make up Alaveteli. + It's a high-level overview of how Alaveteli works to help you orientate yourself to the code. +</p> + +_See also the [schema diagram](#schema-diagram) at the bottom of this page._ + +The main entity is **InfoRequest**, which represents a request for information by a +**User** to a **PublicBody**. A new InfoRequest results in an initial **OutgoingMessage**, +which represents the initial email. + +Once an InfoRequest is made, its state is tracked using **InfoRequestEvents**. For +example, a new InfoRequest has an initial state of `awaiting_response` and an +associated InfoRequestEvent of type `initial_request`. An InfoRequest event can +have an OutgoingMessage or an IncomingMessage or neither associated with it. + +Replies are received by the system by piping raw emails (represented by a **RawEmail**) +from the MTA to a script at `scripts/mailin`. This parses the email, tries to identify the +associated InfoRequest, and then generates an **IncomingMessage** which references +both the RawEmail and the InfoRequest. + +Any User can make **Comments** on InfoRequests. + +All events (e.g., Comments, OutgoingMessages) are tracked in InfoRequestEvent. + +A **TrackThing** is a canned search that allows users to be alerted when events +matching their criteria are found. (How this worked changed after we'd +launched, so there's still some deprecated code there for things we've phased +out.) + +The **MailServerLog** is a representation of the parsed MTA log files. +MailServerLog entries are created by a cron job that runs +`scripts/load-mail-server-logs`. This checks incoming emails and matches them +to InfoRequests; then `script/check-recent-requests-send` checkes these logs to +ensure they have an envelope-from header set (to combat spam). + +## Schema diagram + +<a name="schema-diagram" href="{{ site.baseurl }}images/railsmodels.png"><img src="{{ site.baseurl }}images/railsmodels.png"></a> + +This schema for the Rails models was generated from the code on 19 Dec 2012 using +[Railroad](http://railroad.rubyforge.org/). + +The railroad command is: `railroad -M | dot -Tpng > railsmodels.png` |