Move docs content under docs/

4 files changed, 474 insertions, 0 deletions

diff --git a/docs/developers/api.md b/docs/developers/api.md
new file mode 100644
index 000000000..cbd4c7c85
--- /dev/null
+++ b/docs/developers/api.md
@@ -0,0 +1,82 @@
+---
+layout: page
+title: API
+---
+
+# Alaveteli API
+
+<p class="lead">
+    There are two parts to the API for accessing or inserting data programmatically: the read API, and the write API.
+</p>
+
+## Read API
+
+This is provided via JSON versions of most entities in the system, plus atom
+feeds for listing entities:
+
+### Atom feeds
+
+There are Atom feeds on most pages which list FOI requests, which you can use
+to get updates and links in XML format. Find the URL of the Atom feed in one of
+these ways:
+
+* Look for the RSS feed links.
+* Examine the `<link rel="alternate" type="application/atom+xml">` tag in the head of the HTML.
+* Add `/feed` to the start of another URL. 
+
+Note that even complicated search queries have Atom feeds. You can do all sorts
+of things with them, such as query by authority, by file type, by date range,
+or by status. See the advanced search tips for details.
+
+### JSON structured data
+
+Quite a few pages have JSON versions, which let you download information about
+objects in a structured form. Find them by:
+
+* adding `.json` to the end of the URL.
+* looking for the `<link rel="alternate" type="application/json">` tag in the head of the HTML.
+
+Requests, users and authorities all have JSON versions containing basic
+information about them. Every Atom feed has a JSON equivalent, containing
+information about the list of events in the feed.
+
+### Starting new requests programmatically
+
+To encourage users to make links to a particular public authority, use URLs of
+the form `http://<yoursite>/new/<publicbody_url_name>`. These are the
+parameters you can add to those URLs, either in the URL or from a form:
+
+* `title` - the default summary of the new request.
+* `default_letter` - the default text of the body of the letter. The salutation and signoff for your locale are wrapped round this.
+* `body` - as an alternative to `default_letter`, this sets the default entire text of the request, so you can customise the salutation and signoff.
+* `tags` - space separated list of tags, so you can find and link up any requests made later, e.g. `openlylocal spending_id:12345`. The `:` indicates it is a machine tag. The values of machine tags may also include colons, useful for URIs.
+
+## Write API
+
+The write API is designed to be used by public bodies to create their own
+requests in the system. Currently used by mySociety's [FOI
+Register](https://github.com/mysociety/foi-register) software to support using
+Alaveteli as a disclosure log for all FOI activity at a particular public body.
+
+All requests must include an API key as a variable `k`. This key can be viewed
+on each authority's page in the admin interface. Other variables should be sent
+as follows:
+
+* `/api/v2/request` - POST the following json data as a form variable `json` to create a new request:
+  * `title` - title for the request
+  * `body` - body of the request
+  * `external_user_name` - name of the person who originated the request
+  * `external_url` - URL where a canonical copy of the request can be found
+  Returns JSON containing a `url` for the new request, and its `id`
+* `/api/v2/request/<id>.json` - GET full information about a request
+* `/api/v2/request/<id>.json` - POST additional correspondence regarding a request:
+  * as form variable `json`:
+    * `direction` - either `request` (from the user - might be a followup, reminder, etc) or `response` (from the authority)
+    * `body` - the message itself
+    * `sent_at` - ISO-8601 formatted time that the correspondence was sent
+  * (optionally) the variable `attachments` as `multipart/form-data`:
+    * attachments to the correspondence.  Attachments can only be attached to messages in the `response` direction
+
+
+
+
diff --git a/docs/developers/directory_structure.md b/docs/developers/directory_structure.md
new file mode 100644
index 000000000..e073a96ce
--- /dev/null
+++ b/docs/developers/directory_structure.md
@@ -0,0 +1,257 @@
+---
+layout: page
+title: Directory structure
+---
+
+
+# Alaveteli's directory structure
+
+<p class="lead">This page gives you an overview of where to find things in Alaveteli's
+directories.</p>
+
+**You'll probably never need to worry about this** if you're just installing
+Alaveteli -- this is really more useful if you're a developer planning on
+making more substantive changes to the code. You don't need to be familiar with
+Ruby to install or make basic [customisations to your
+installation](/docs/customising).
+
+<!--  (and if you do,
+remember to read the page about [feeding your changes back](/feeding-back)).-->
+
+Alaveteli uses Ruby on Rails, which is a common "Model-View-Controller" web
+framework &mdash; if you're familiar with Rails this will look very familiar. For
+more information about the Rails structure see the [Ruby on Rails
+website](http://guides.rubyonrails.org/getting_started.html).
+
+## Key directories and what they're for
+
+<dl class="dir-structure">
+  <dt>
+      app
+  </dt>
+  <dd>
+    <p><em>the core Alaveteli application code</em></p>
+    <dl>
+      <dt>
+        controllers
+      </dt>
+      <dt>
+        helpers
+      </dt>
+      <dt>
+        mailers
+      </dt>
+      <dt>
+        models
+      </dt>
+      <dt>
+        sass
+      </dt>
+      <dt class="last">
+        views
+      </dt>
+    </dl>
+  </dd>
+  <dt>
+      assets
+  </dt>
+  <dd>
+      Static assets
+      <dl>
+          <dt>
+              css
+          </dt>
+          <dd>
+              Rendered stylesheets
+          </dd>
+          <dt>
+              img
+          </dt>
+          <dd>
+              static images
+          </dd>
+          <dt>
+              sass
+          </dt>
+          <dd>
+              Stylesheets in SCSS format, which are compiled to CSS
+          </dd>
+          <dt class="last">
+              scripts
+          </dt>
+          <dd class="last">
+              JavaScript
+          </dd>
+      </dl>
+  </dd>
+  <dt>
+      bootstrap
+  </dt>
+  <dd>
+      <p>
+          Alaveteli's default style uses Bootstrap.
+      </p>
+  <dt>
+    commonlib
+  </dt>
+  <dd>
+    <p><em>mySociety's library of common functions</em></p>
+    <p>
+      We maintain a <a href="https://github.com/mysociety/commonlib">common
+      library</a> that we use across many of our projects (not just
+      Alaveteli). This is implemented as a <a
+      href="http://git-scm.com/book/en/Git-Tools-Submodules">git submodule</a>,
+      so Alaveteli contains it even though the code is separate. Normally, you
+      don't need to think about this (because git handles it automatically)...
+      but if you really <em>do</em> need to change anything here, be aware that
+      it is a separate git repository.
+    </p>
+  </dd>
+  <dt>
+    config
+  </dt>
+  <dd>
+    <p><em>configuration files</em></p>
+    <p>
+      The primary configuration file is <code>general.yml</code>. This file isn't in the git
+      repository (since it will contain information specific to your installation, including
+      the database password), but example files are.
+    </p>
+  </dd>
+  <dt>
+    db
+  </dt>
+  <dd>
+    <p><em>database files</em></p>
+    <dl>
+        <dt class="last">
+            migrate
+        </dt>
+        <dd class="last">
+            Rails' migration (updating the database scheme up or down
+            as the code develops).
+        </dd>
+    </dl>
+  </dd>
+  <dt>
+      doc
+  </dt>
+  <dd>
+    <p><em>documentation</em></p>
+    <p>
+        These are technical notes. This is in addition to the <a
+        href="http://code.fixmystreet.com">core documentation</a> &mdash; which
+        you are reading now &mdash; which is actually stored in the git
+        repository in the <code>gh-pages</code> branch, and published as GitHub
+        pages.
+    </p>
+  </dd>
+  <dt>
+    lib
+  </dt>
+  <dd>
+    <p><em>custom libraries</em></p>
+    <dl>
+        <dt>
+            tasks
+        </dt>
+        <dt class="last">
+            whatdotheyknow
+        </dt>
+    </dl>
+  </dd>
+  <dt>
+    locale
+  </dt>
+  <dd>
+    <p><em>translations (internationalisation/i18n)</em></p>
+    <p>
+      The translation strings are stored in <code>.po</code> files in directories specific to
+      the locale and encoding. For example, <code>es/</code> contains the translations for the Spanish site.
+    </p>
+  </dd>
+  <dt>
+    public
+  </dt>
+  <dd>
+    <p><em>static assets</em></p>
+    <dl>
+        <dt>
+            admin
+        </dt>
+        <dd>
+            images, JavaScript and stylesheets used by the admin back-end
+        </dd>
+        <dt>
+            fcgi
+        </dt>
+        <dd>
+            Fast CGI files for serving static assets
+        </dd>
+        <dt>
+            images
+        </dt>
+        <dt>
+            javascripts
+        </dt>
+        <dt class="last">
+            stylesheets
+        </dt>
+    </dl>
+  </dd>
+  <dt>
+    script
+  </dt>
+  <dd>
+    <p><em>server-side shell scripts</em></p>
+    <p>
+      For example, <code>alert-overdue-requests</code> for running the script
+      which finds overdue requests and mails them out.
+    </p>
+  </dd>
+  <dt>
+    spec
+  </dt>
+  <dd>
+    <p><em>tests</em></p>
+    <p>
+      Alaveteli's test suite runs under <a href="TODO">spec</a>.
+    </p>
+  </dd>
+  <dt>
+    stylesheets
+  </dt>
+  <dd>
+    <p>
+      <em>global stylesheet</em>
+    </p>
+    <p>
+        Actually just <code>global.css</code>
+    </p>
+  </dd>
+  <dt>
+    tmp
+  </dt>
+  <dd>
+    <p>
+      <em>temporary files</em>
+    </p>
+  </dd>
+  <dt class="last">
+      vendor
+  </dt>
+  <dd class="last">
+    <p><em>third-party software</em></p>
+    <dl>
+      <dt class="last">plugins</dt>
+      <dd class="last">
+          <p>
+              Plugins
+          </p>
+      </dd>
+    </dl>
+  </dd>
+</dl>
+
+We've missed out some of the less important subdirectories here just to keep
+things clear.
diff --git a/docs/developers/index.md b/docs/developers/index.md
new file mode 100644
index 000000000..4b90c9232
--- /dev/null
+++ b/docs/developers/index.md
@@ -0,0 +1,84 @@
+---
+layout: page
+title: For developers
+---
+
+# Information for developers
+
+<p class="lead">
+    Alaveteli is an open source project. Full-time mySociety developers together with devs from all around the world actively contribute to the codebase. These notes and links will help you if you want to help too.
+</p>
+
+* The software is written in **Ruby on Rails 3.x**. We support postgresql as
+  the backend database. A configured mail transfer agent (MTA) like exim,
+  postfix or sendmail is necessary to parse incoming emails. We have production
+  sites deployed on Debian Squeeze and Ubuntu (10.04 LTS). For performance
+  reasons, we recommend the use of [Varnish](https://www.varnish-cache.org).
+
+* To help you understand what the code is doing, read this [high-level
+  overview]({{ site.baseurl }}docs/developers/overview), which includes a diagram of
+  the models and how they are related.
+
+* See the [API documentation]({{ site.baseurl }}docs/developers/api) for how to get
+  data into or out of Alaveteli.
+
+* If you need to change or add strings in the interface, see our [guidelines
+  for internationalisation](http://mysociety.github.io/internationalization.html
+  ), which include notes about our use of `gettext`.
+
+* We use the [git flow branching
+  model](http://nvie.com/posts/a-successful-git-branching-model/), with a small
+  change: currently our `develop` branch is called `rails-3-develop`, which
+  means that the latest development version is always found on the
+  [rails-3-develop
+  branch](https://github.com/mysociety/alaveteli/tree/rails-3-develop). The
+  latest stable version is always on the [master
+  branch](https://github.com/mysociety/alaveteli). If you plan to collaborate
+  on the software, you may find the [git flow
+  extensions](https://github.com/nvie/gitflow) useful.
+
+* Installing the software is a little involved, though it's getting easier. If
+  you stick to Debian or Ubuntu, it should be possible to get a running version
+  within a few hours. If you've got your own server, run the
+  [installation script]({{ site.baseurl }}docs/installing/script/), or follow the
+  instructions for a
+  [manual installation]({{ site.baseurl }}docs/installing/manual_install/).
+  Alternatively, there's an [Alaveteli EC2 AMI]({{ site.baseurl }}docs/installing/ami/)
+  that might help you get up and running quickly.
+  [Get in touch](http://www.alaveteli.org/contact/) on the project mailing list or IRC
+  for help.
+
+* A standard initial step for customising your deployment is [writing a
+  theme]({{ site.baseurl }}docs/customising/themes). **If you only read one thing,
+  it should be this!**
+
+* Like many Ruby on Rails sites, the software is not hugely performant (see
+  some notes about [[performance issues]] gathered over time with
+  WhatDoTheyKnow). The site will run on a server with 512MB RAM but at least
+  2GB is recommended. Deployment behind [Varnish](https://www.varnish-cache.org) is also fairly essential. See
+  [[Production Server Best Practices]] for more.
+
+* There's a number of [proposals for enhancements](https://github.com/mysociety/alaveteli/wiki/Proposals-for-enhancements),
+  such as more user-focused features, but see also...
+
+* ...the [github issues](https://github.com/mysociety/alaveteli/issues). We
+  mark issues with the label **suitable for volunteers** if we think they are
+  especially suitable for diving into if you're just looking for something
+  relatively small to get your teeth into.
+
+* We try to ensure every commit has a corresponding issue in the issue tracker.
+  This makes changelogs easier as we can gather all the fixes for a particular
+  release against a milestone in the issue tracker, [like this 0.4
+  release](https://github.com/mysociety/alaveteli/issues?milestone=7&state=close
+  d).
+
+* If you're experiencing memory issues, [this blog post about some strategies
+  used in the
+  past](http://www.mysociety.org/2009/09/17/whatdotheyknow-growing-pains-and-rub
+  y-memory-leaks/) might be useful.
+
+* If you're coding on a mac, see these [MacOS X installation notes]({{ site.baseurl }}docs/installing/macos). <!-- [[OS X Quickstart]] -->
+
+* We try to adhere to similar good practice across all our projects: see
+  [mysociety.github.io](http://mysociety.github.io/) for things like our
+  [coding standards](http://mysociety.github.io/coding-standards.html)
diff --git a/docs/developers/overview.md b/docs/developers/overview.md
new file mode 100644
index 000000000..ca2d27985
--- /dev/null
+++ b/docs/developers/overview.md
@@ -0,0 +1,51 @@
+---
+layout: page
+title: High-level overview
+---
+
+# High-level overview
+
+<p class="lead">
+    This page describes the process and entities that make up Alaveteli.
+    It's a high-level overview of how Alaveteli works to help you orientate yourself to the code.
+</p>
+
+_See also the [schema diagram](#schema-diagram) at the bottom of this page._
+
+The main entity is **InfoRequest**, which represents a request for information by a
+**User** to a **PublicBody**. A new InfoRequest results in an initial **OutgoingMessage**,
+which represents the initial email.
+
+Once an InfoRequest is made, its state is tracked using **InfoRequestEvents**. For
+example, a new InfoRequest has an initial state of `awaiting_response` and an
+associated InfoRequestEvent of type `initial_request`. An InfoRequest event can
+have an OutgoingMessage or an IncomingMessage or neither associated with it.
+
+Replies are received by the system by piping raw emails (represented by a **RawEmail**)
+from the MTA to a script at `scripts/mailin`. This parses the email, tries to identify the
+associated InfoRequest, and then generates an **IncomingMessage** which references
+both the RawEmail and the InfoRequest.
+
+Any User can make **Comments** on InfoRequests.
+
+All events (e.g., Comments, OutgoingMessages) are tracked in InfoRequestEvent.
+
+A **TrackThing** is a canned search that allows users to be alerted when events
+matching their criteria are found. (How this worked changed after we'd
+launched, so there's still some deprecated code there for things we've phased
+out.)
+
+The **MailServerLog** is a representation of the parsed MTA log files.
+MailServerLog entries are created by a cron job that runs
+`scripts/load-mail-server-logs`. This checks incoming emails and matches them
+to InfoRequests; then `script/check-recent-requests-send` checkes these logs to
+ensure they have an envelope-from header set (to combat spam).
+
+## Schema diagram
+
+<a name="schema-diagram" href="{{ site.baseurl }}images/railsmodels.png"><img src="{{ site.baseurl }}images/railsmodels.png"></a>
+
+This schema for the Rails models was generated from the code on 19 Dec 2012 using
+[Railroad](http://railroad.rubyforge.org/).
+
+The railroad command is: `railroad -M | dot -Tpng > railsmodels.png`