diff options
Diffstat (limited to 'docs/running')
| -rw-r--r-- | docs/running/admin_manual.md | 556 | ||||
| -rw-r--r-- | docs/running/categories_and_tags.md | 200 | ||||
| -rw-r--r-- | docs/running/holding_pen.md | 101 | ||||
| -rw-r--r-- | docs/running/index.md | 10 | ||||
| -rw-r--r-- | docs/running/redaction.md | 135 | ||||
| -rw-r--r-- | docs/running/requests.md | 355 | ||||
| -rw-r--r-- | docs/running/security.md | 36 | ||||
| -rw-r--r-- | docs/running/server.md | 8 | ||||
| -rw-r--r-- | docs/running/states.md | 217 | ||||
| -rw-r--r-- | docs/running/states_informatazyrtare.md | 88 | ||||
| -rw-r--r-- | docs/running/upgrading.md | 112 |
11 files changed, 1479 insertions, 339 deletions
diff --git a/docs/running/admin_manual.md b/docs/running/admin_manual.md index 77bacdf1b..7c112400c 100644 --- a/docs/running/admin_manual.md +++ b/docs/running/admin_manual.md @@ -6,9 +6,46 @@ title: Administrator's guide # Alaveteli administrator's guide <p class="lead"> - What is it like running an Alaveteli site? This guide explains what you can expect, and the types of problem that you might encounter. It includes examples of how mySociety manages their own <a href="/docs/glossary/#foi" class="glossary">Freedom of Information</a> site, <a href="https://www.whatdotheyknow.com">whatdotheyknow.com</a>. + What is it like running an Alaveteli site? This guide explains what you can + expect, and the types of problem that you might encounter. It includes + examples of how mySociety manages their own <a href="/docs/glossary/#foi" + class="glossary__link">Freedom of Information</a> site, <a + href="https://www.whatdotheyknow.com">whatdotheyknow.com</a>. </p> +In this guide: + +<ul class="toc"> + <li><a href="#whats-involved">What's involved?</a></li> + <li><a href="#user-support">User support</a> + <ul> + <li><a href="#dealing-with-email-thats-not-getting-through-to-the-authority">Dealing with email that's not getting through to the authority</a></li> + <li><a href="#requests-to-take-down-information">Requests to take down information</a></li> + <li><a href="#incorrectly-addressed">Incorrectly addressed</a></li> + <li><a href="#wants-advice">Wants advice</a></li> + <li><a href="#general-assistance-required">General assistance required</a></li> + <li><a href="#vexatious-users">Vexatious users</a></li> + <li><a href="#mail-import-errors">Mail import errors</a></li> + </ul> + <li><a href="#maintenance">Maintenance</a></li> + <ul> + <li><a href="#administrator-privileges-and-accessing-the-admin-interface">Administrator privileges and accessing the admin interface</a></li> + <li><a href="#removing-a-message-from-the-holding-pen">Removing a message from the 'Holding Pen'</a></li> + <li><a href="#rejecting-spam-that-arrives-in-the-holding-pen">Rejecting spam that arrives in the holding pen</a></li> + <li><a href="#creating-changing-and-uploading-public-authority-data">Creating, changing and uploading public authority data</a></li> + <li><a href="#banning-a-user">Banning a user</a></li> + <li><a href="#allowing-a-user-to-make-more-requests">Allowing a user to make more requests</a></li> + <li><a href="#batch-requests">Batch requests</a></li> + <li><a href="#resending-a-request-or-sending-it-to-a-different-authority">Resending a request or sending it to a different authority</a></li> + <li><a href="#hiding-a-request">Hiding a request</a></li> + <li><a href="#deleting-a-request">Deleting a request</a></li> + <li><a href="#hiding-an-incoming-or-outgoing-message">Hiding an incoming or outgoing message</a></li> + <li><a href="#editing-an-outgoing-message">Editing an outgoing message</a></li> + <li><a href="#hiding-certain-text-from-a-request-using-censor-rules">Hiding certain text from a request</a></li> + </ul> + </li> +</ul> + ## What's involved? The overhead in managing a successful FOI website is quite high. Richard, a @@ -20,7 +57,7 @@ WhatDoTheyKnow usually has about 3 active volunteers at any one time managing the support, plus a few other less active people who help out at different times. -Administration tasks can be split into **maintenance** and **user support**. +Administration tasks can be split into [**maintenance**]({{ site.baseurl }}docs/running/admin_manual/#maintenance) and [**user support**]({{ site.baseurl }}docs/running/admin_manual/#user-support). The boundaries of these tasks is in fact quite blurred; the main distinction is that the former happen exclusively through the web admin interface, whereas the latter are mediated by email directly with end users (but often result in @@ -45,7 +82,7 @@ During that week, the tasks broke down as follows: * 2 requests that have been marked as needing admin attention * 2 things marked as errors (message refused by server - spam, full mailbox, etc) to fix -### User interaction tasks +### User support tasks * 16 general, daily admin: i.e. things that resulted in admin actions on the site (bounces, misdelivered responses, etc) @@ -55,9 +92,9 @@ During that week, the tasks broke down as follows: * 3 requests to redact personal information * 2 requests to redact defamatory information -## Types of user interaction +## User support -There follows a breakdown of the most common kinds of user interaction. It's +There follows a breakdown of the most common kinds of user support. It's intended for use as a guide to the kind of policies and training that a support team might need to develop. @@ -249,13 +286,13 @@ Can be for many reasons, e.g. themselves * A reply has been automatically filed under the wrong request -## Vexatious users +### Vexatious users Some users persistently misuse the website. An alaveteli site should have a policy on banning users, for example giving them a first warning, informing them about moderation policy, etc. -## Mail import errors +### Mail import errors These are currently occurring at a rate of about two a month. Sometimes the root cause seems to be blocking in the database when two mails are received for @@ -270,9 +307,491 @@ the error sent to the site support address) in a file without the first "From" line, and piping the contents of that file into the mail handling script. e.g. ```cat missing_mail.txt | script/mailin``` -## Censor rules -Censor rules can be attached to a request or to a user and define bits of text +## Maintenance + +### Administrator privileges and accessing the admin interface + +The <a href="{{ site.baseurl }}docs/glossary/#admin" class="glossary__link">administrative interface</a> +is at the URL `/admin`. Only users who are +<a href="{{ site.baseurl }}docs/glossary/#admin" class="glossary__link">administrators</a> +can access the admin interface. + +To make a user an administrator on a brand new site, +[follow these steps]({{ site.baseurl }}docs/installing/next_steps/#create-a-superuser-admin-account). + +If you're already an administrator, you can grant other users administrator +privilege too. Go to `/admin/users` or click on **Users** at the top of +the admin. Find the user in the list, and click on the name to see the user +details. On that page, click **Edit**. Change the *Admin level* to “super” and +click **Save**. + +As well having access to the admin interface, users who are administrators also +have extra privileges in the main website front end. Administrators can: + + * categorise any request + * view items that have been hidden from the search + * follow "admin" links that appear next to individual requests and comments + +<div class="attention-box warning"> + It is possible completely to override the administrator authentication by + setting + <code><a href="{{ site.baseurl }}docs/customising/config/#skip_admin_auth">SKIP_ADMIN_AUTH</a></code> + to <code>true</code> in <code>general.yml</code>. Never do this, unless you + are working on a <a href="{{ site.baseurl }}docs/glossary/#development" + class="glossary__link">development</a> server. +</div> + +### Removing a message from the holding pen + +Alaveteli puts incoming messages (that is, +<a href="{{ site.baseurl }}docs/glossary/#reponse" class="glossary__link">responses</a>) +into the +<a href="{{ site.baseurl }}docs/glossary/#holding_pen" class="glossary__link">holding pen</a> +if their `To:` email addresses can't automatically be associated with a +<a href="{{ site.baseurl }}docs/glossary/#reponse" class="glossary__link">request</a>. + +The two most common reasons for this are: + + * the request has closed + * the email address was wrongly spelled (for example, the sender missed the last + character off the email address when they copied it) + +When this happens, the messages wait in the holding pen until an administrator +redelivers them to the correct request, or else deletes them. + +To do this, log into the +The <a href="{{ site.baseurl }}docs/glossary/#admin" class="glossary__link">admin interface</a> +at `/admin`. If there are any messages in the holding pen, you'll see this +message under the title *Things to do*: + +> Put misdelivered responses with the right request + +Click on that message — you'll see a list of all the messages that need +your attention. Click on any one of them to see the details. + +<div class="attention-box helpful-hint"> + If the message does not belong to any request, you can delete it instead. + Simply click on the <strong>Destroy Message</strong> button instead of + redelivering it. +</div> + +When you inspect a message, you may see a guess made by Alaveteli as to which +request the message belongs to. Check this request. If the guess is right +— the incoming email really is a response to that request — +the request's *title_url* will already be in the input box: click the +**Redeliver to another request** button. + +If there is not a guess, or Alaveteli's guess is wrong, look at the `To:` +address of the raw email and the contents of the message itself. You need +to figure out which request it belongs to. You can browse and search +requests in the admin interface by clicking **Requests** at the top of the +admin. When you have found the correct request, copy either its *id* or +its *url_title*. + +<div class="attention-box info"> + <p><strong>How to find a request's <em>id</em> or <em>url_title</em></strong></p> + <p> + A request's <em>id</em> is the number after <code>/show/</code> in the + admin interface's URL when you are looking at that request. + For example, if the URL is <code>/admin/request/show/118</code>, then the + <em>id</em> is <code>118</code>. + </p> + <p> + A request's <em>url_title</em> is the part after <code>/request/</code> + in your Alaveteli site's URL when you are looking at that request. + In the URL <code>/request/how_many_vehicles</code>, the + <em>url_title</em> is <code>how_many_vehicles</code>. + </p> +</div> + +Once you have identified the request the message belongs to, return to the +holding pen message page. Find the incoming message's "Actions" and paste the +request *id* or *url_title* into the text input. Click on the **Redeliver to +another request** button. + +The message will now be associated with the correct request. It is no longer +in the holding pen, and is shown instead on the public request page. + + +### Rejecting spam that arrives in the holding pen + +Alaveteli maintains a +<a href="{{ site.baseurl }}docs/glossary/#spam-address-list" class="glossary__link">spam address list</a>. +Any incoming message to an email address on that list +*that would otherwise be put in the holding pen* will be rejected and won't +appear in the admin. + +If you see spam messages in the +<a href="{{ site.baseurl }}docs/glossary/#holding_pen" class="glossary__link">holding pen</a>, +check if they are being sent to a *specific* email address. If they are, that +email address has become a "spam-target" and you should add it to the spam +address list. Thereafter, Alaveteli will automatically reject any messages that +come to that address. + +An email address that is not associated with a request (that is, one whose +messages end up in the holding pen) becomes a spam-target once it's been +harvested by spammers. There are several reasons why such an invalid address +might exist — perhaps it was mis-spelled in a manual reply, for example. +Our experience from running +<a href="{{ site.baseurl }}docs/glossary/#wdtk" class="glossary__link">WhatDoTheyKnow</a> +is that you can safely dismiss incoming email to such addresses once they have +been targeted in this way. Legitimate emails that arrive in the holding pen +tend to be unique errors (for example, missing the last character of the email +address due to a cut-and-paste mistake) and the nature of the lifecycle of +requests means they don't typically get used for spam until they are +effectively dead. + +To add an email address to the spam address list you need to copy it from an +incoming message and paste it into the spam addresss list. The easiest way to +do this is to click on **Summary** at the top of any admin page, and then click +on **Put misdelivered responses with the right requests** to see the contents +of the holding pen. + +<div class="attention-box info"> + If there are no messages in the holding pen, Alaveteli won't show you this + link. Great — there are no misdelivered responses needing your + attention right now! +</div> + +Inside the holding pen, you'll see the list of emails awaiting attention +— click on an email's subject line to see the whole message and its +details. Copy the `To:` email address, then click on the **Spam Addresses** +link under *Actions*. Paste the email address into the text input and click the +**Add Spam Address** button. + +You can see the spam address list (that is, all known spam-target email +addresses) at any time by going to the admin interface at `/admin/spam_addresses`. + +You can remove any address from the list by clicking the **Remove** button +next to it. Of course, this won't restore any messages that have been +rejected, but Alaveteli will not reject any new messages that are sent to +this address. + +Note that if you are seeing consistent spam email in your holding pen, you +should also consider implementing (or increasing) the anti-spam measures +running in your +<a href="{{ site.baseurl }}docs/glossary/#mta" class="glossary__link">MTA</a>. + +### Creating, changing and uploading public authority data + +There are three ways to change public authority data on your site: + + * *Create* — + You can create a new public authority in the admin interface. Go to **Authorities**, and click the **New Public Authority** button. + + * *Edit* — + Once an authority is created, you can update its email address or other + details by editing it in the admin interface. Go to **Authorities**, find + the authority you want to update, and click on **edit**. + + * *Upload* — + You can also create or edit more than one authority at the same time by + uploading a file containing the data in comma-separated values (CSV) + format. This works for new authorities as well as those that already exist + on your site. Go to **Authorities** and click the **Import from CSV** button. See the rest of this section for more about uploading. + +The upload feature is useful — especially when an Alaveteli site is first +set up — because it's common to collect data such as the contact details +for the public authorities in a spreadsheet. Alaveteli's upload feature makes it +easy to initially load this data onto the site. It also lets you update the +data if it changes after it's already been loaded. + +To use the data in the spreadsheet to update the bodies on your site, export +("save as") the spreadsheet as a CSV file. This is the file you can upload. + +The first line of your CSV file should start with `#` (this indicates that this +line does not contain data) and must list the column names for the data that +follows on the subsequent lines. Column names must: + + * be on the first line + * match expected names *exactly*, and include `name` and `request_email` + (see table below) + * appear in the same order as corresponding items in the lines of data that follow + +Most spreadsheet programs will produce a suitable CSV file for you, provided +that you carefully specify correct titles at the top of each column. Be sure to +use names exactly as shown — if Alaveteli encounters an +unrecognised column name, the import will fail. + +<table class="table"> + <tr> + <th>column name</th> + <th>i18n suffix?</th> + <th>notes</th> + </tr> + <tr> + <td><code>name</code></td> + <td><em>yes</em></td> + <td> + <em>This column <strong>must</strong> be present.</em><br> + The full name of the authority.<br> + If it matches an existing authority's name, that authority will be + updated — otherwise, this will be added as a new authority. + </td> + </tr> + <tr> + <td><code>request_email</code></td> + <td><em>yes</em></td> + <td> + <em>This column <strong>must</strong> be present, + but can be left empty.</em><br> + The email to which requests are sent + </td> + </tr> + <tr> + <td><code>short_name</code></td> + <td><em>yes</em></td> + <td>Some authorities are known by a shorter name</td> + </tr> + <tr> + <td><code>notes</code></td> + <td><em>yes</em></td> + <td>Notes, displayed publicly (may contain HTML)</td> + </tr> + <tr> + <td><code>publication_scheme</code></td> + <td><em>yes</em></td> + <td> + The URL of the authority's + <a href="{{ site.baseurl }}docs/glossary/#publication-scheme" class="glossary__link">publication scheme</a>, + if they have one + </td> + </tr> + <tr> + <td><code>disclosure_log</code></td> + <td><em>yes</em></td> + <td> + The URL of the authority's + <a href="{{ site.baseurl }}docs/glossary/#disclosure-log" class="glossary__link">disclosure log</a>, + if they have one + </td> + </tr> + <tr> + <td><code>home_page</code></td> + <td>no</td> + <td>The URL of the authority's home page</td> + </tr> + <tr> + <td><code>tag_string</code></td> + <td>no</td> + <td>separated tags with spaces</td> + </tr> +</table> + + * Existing authorities cannot be renamed by uploading: if you need to do + this, use the admin interface to edit the existing record first, and + change its name in the web interface. + * If the authority already exists (the `name` matches an existing authority's + name exactly), a blank entry leaves the existing value for that column + unchanged — that is, that item of data on your site will not be + changed. This means you only really need to include data you want to + update. + * Columns with "i18n suffix" can accept + <a href="{{ site.baseurl }}docs/glossary/#i18n" class="glossary__link">internationalised</a> + names. Add a full stop followed by the language code, for example: + `name.es` for Spanish (`es`). This *must* be a locale you've declared in + [`AVAILABLE_LOCALES`]({{ site.baseurl }}docs/customising/config/#available_locales). + If you don't specify an i18n suffix, the default language for your site is + assumed. + * You can specify a blank entry in the CSV file by having no character + between commas. + * If an entry contains a comma, enclose it in double quotes like this: + `"Comma, Inc"`. + * If an entry contains any double quotes, you must replace each of + them with two (so `"` becomes `""`) and also enclose the whole entry in + double quotes like this: `"In ""quotes"""` (which will be imported as `In + "quotes"`). + +For example, here's data for three authorities in CSV format ready for upload. +The first line defines the column names, then the next three lines contain the +data (one line for each authority): + + #name,short_name,short_name.es,request_email,notes + XYZ Library Inc.,XYZ Library,XYX Biblioteca,info@xyz.example.com, + Ejemplo Town Council,,Ayuntamiento de Ejemplo,etc@example.com,Lorem ipsum. + "Comma, Inc.",Comma,,comma@example.com,"e.g. <a href=""x"">link</a>" + +Note that, if Ejemplo Town Council already exists on the site, the blank entry +for `short_name` will leave the existing value for that column unchanged. + +To upload a CSV file, log into the admin and click on **Authorities**. Click on +**Import from CSV file**, and choose the file you've prepared. + +Specify **What to do with existing tags?** with one of these options: + + * *Replace existing tags with new ones* <br/> + For each authority being updated, all existing tags will be removed, and + replaced with the ones in your CSV file. + + * *Add new tags to existing ones* <br/> + Existing tags will be left unchanged, and the tags in your CSV file will + be added to them. + +You can add a **Tag to add entries to / alter entries for**. This tag will +be applied to every body that is imported from your CSV file. + +We always recommend you click **Dry run** first -- this will run through the +file and report the changes it will make in the database, *without actually +changing the data*. Check the report: it shows what changes would be made if +you really uploaded this data, followed by a message like this: + + Dry run was successful, real run would do as above. + +If you see nothing above that line, it means the dry run has resulted in no +proposed changes. + +If everything was OK when you ran the dry run, click **Upload** instead. This +will repeat the process, but this time it will make the changes to your +site's database. + +If you see an error like `invalid email`, either you really have mistyped an +email address, or (more likely) your CSV file does not have a `request_email` +column. + +#### Creating a spreadsheet of existing authorities + +You can easily create a spreadsheet containing the authorities that <em>already +exist</em> on your site. Combined with the upload feature described above, this +may be a more convenient way to update your data than editing it in the admin +interface. + +To export the existing authorities' data, go to your site's home page (not the +admin) and click <strong>View Authorities</strong>. Then click <strong>List of +all authorities (CSV)</strong> to get a CSV file. You can then make changes to +this file using a spreadsheet program and upload it as described above. + +You'll need to remove some columns that are not accepted by the import feature +and possibly rename some that are — see the column names above. +Also, note that by default the exported spreadsheet does not contain a +`request_email` column. If you want to update email addresses, you should +manually add a column to your spreadsheet with the heading `request_email` and +fill in a new email address for each authority you want to update. Authorities +with blank values in any column will keep their existing value for that +attribute. + +<div class="attention-box info"> +Alaveteli never includes authorities which have the tag <code>site_administration</code> when it exports authorities in CSV format. +If you're running a development server with the sample data, the single example +body called "Internal admin authority" has this tag, so if you click +<strong>List of all authorities (CSV)</strong>, you'll get a CSV file which +contains no data. You need to add your own authorities (without the +<code>site_administration</code> tag) before you can export them. +</div> + +### Banning a user + +You may wish to completely ban a user from the website (such as a spammer or troll for example). You need to log into the admin interface at `/admin`. On the top row of links, locate and click on ‘Users’. + +Find the user you wish to ban on the list and click on their name. Once on the user page, select ‘edit’. + +Enter some text in the in the ‘Ban text’ box to explain why they have been banned. Please be aware, this is publicly viewable from the users' account. Then click on save and the user will be banned. + +### Allowing a user to make more requests + +Alaveteli has a config setting <code><a href="{{ site.baseurl }}docs/customising/config/#max_requests_per_user_per_day">MAX_REQUESTS_PER_USER_PER_DAY</a></code>, +which determines the maximum number of requests that a normal user can +make in a day. If they try to make more than this number of requests +within a 24 hour period, they will see a message telling them that they +have hit the limit, and encouraging them to use the contact form if they +feel they have a good reason to ask for the request limit to be lifted. + +To lift the request limit for a particular user, go to the <a href="{{ site.baseurl }}docs/glossary/#admin" class="glossary__link">admin +interface</a>, click on **Users**, then click on the name of the user +you want to lift the request limit for. Click the **Edit** button. Tick +the checkbox **No rate limit**, and click the **Save** button. + +### Batch requests + +Sometimes a user may want to send the same request to more than one authority, which we call a batch request. By default, Alaveteli does not allow users to make batch requests. + +<div class="attention-box info"> +<p>We believe that batch requests can be abused — users can send poorly thought-out or vexatious requests, which will annoy authorities and damage the reputation of your site. However, well thought-out batch requests can be an extremely useful tool in collecting comparative data sets across types of authority, for example, all police forces.</p> +<p> +We recommend that you enable batch requesting for users who you notice making the same good request to multiple authorities. +</p> +<p> +Users can choose which authorities to include in a batch requests. They can even send a request to <em>every single authority</em> on your site. Only give this power to users that you trust. +</p> +</div> + +To enable batch requests on your site, first you must set +<code><a href="{{ site.baseurl }}docs/customising/config/#allow_batch_requests">ALLOW_BATCH_REQUESTS</a></code> +to <code>true</code> in <code>general.yml</code>. + +This does not allow anyone to make batch requests yet. You must still +enable this for each user on an individual basis. To do this, go to the +<a href="{{ site.baseurl }}docs/glossary/#admin" +class="glossary__link">admin interface</a>, click on **Users**, then +click on the name of the user who wants to make batch requests. Click +the **Edit** button. Tick the checkbox **Can make batch requests**, and +click the **Save** button. + +If you've enabled batch requests for a user, when they start to make a +request, in addition to the box where they can select an authority, they +will see a link to "make a batch request". When the request is sent, +Alaveteli will make a request page for this request for each authority, +as if the user had made individual requests. + +### Resending a request or sending it to a different authority + +If you have corrected the email address for an authority, you can resend +an existing request to that authority to the new email address. Alternatively, +a user may send a request to the wrong authority. In that situation, you can +change the authority on the request and then resend it to the correct authority. +For instructions, see +[resending a request or sending it to a different authority]({{ site.baseurl }}docs/running/requests/#resending-a-request-or-sending-it-to-a-different-authority). + + +### Hiding a request + +If a request contains vexatious or inappropriate content, is libellous, or is +not a valid +<a href="{{ site.baseurl }}docs/glossary/#foi" class="glossary__link">Freedom of Information</a> +request at all, you may want to hide it. A hidden request is still visible to +you and the other administrators, and (optionally) the requester themselves. +For instructions, see +[hiding a request]({{ site.baseurl }}docs/running/requests/#hiding-a-request). + +Responses to a hidden request will be accepted in the normal way, but because +they are added to the request's page, they too will be hidden. + +### Deleting a request + +You can delete a request from the site. For instructions, see +[deleting a request]({{ site.baseurl }}docs/running/requests/#deleting-a-request). + +Responses to a deleted request will be sent to the holding pen. + +### Hiding an incoming or outgoing message + +You may need to hide a particular incoming or outgoing message from a +public request page, perhaps because someone has included personal +information in it. You can do this from the message's page in the admin +interface. You can get to a message's admin page either by following the +links from the "Outgoing messages" or "Incoming messages" sections of +the request's admin page, or directly from the public request page by +clicking on the 'admin' link on the message itself. Once you are on the +message's admin page, you can change it's prominence. Set the prominence +to 'hidden' to hide it from everyone except site admins, or to +'requester_only' to allow it to be viewed by the requester (and by site +admins). If you can, add some text in the box 'Reason for prominence'. +This will be displayed as part of the information that will appear on +the request page where the message used to be, telling people that it +has been hidden. + +### Editing an outgoing message + +You may find there is a need to edit an outgoing message because the requester has accidentally included personal information that they don't want to be published on the site. You can either follow one of the 'admin' links from the public request page on the site, or find the request from the admin interface by searching under 'Requests'. + +Scroll down to the 'Outgoing Messages' section, and click on 'Edit'. + +Then on the next page you will be able to edit the message accordingly and save it. The edited version will then appear on the Alaveteli website, although an unedited version will have been sent to the authority. + + +### Hiding certain text from a request using censor rules + +Censor rules can be attached to a request or to a user. These rules define bits of text to be removed (either from the request (and all associated files e.g. incoming message attachments) or from all requests associated with a user), and some replacement text. In binary files, the replacement text will always be a series @@ -297,5 +816,22 @@ hanging the application altogether), so please: * Restrict your use of them to cases that can't otherwise be easily covered. * Keep them as simple and specific as possible. - +<strong>To attach a censor rule to a request</strong>, go to the admin page for the +request, scroll to the bottom of the page, and click the "New censor +rule (for this request only)" button. On the following page, enter the +text that you want to replace e.g. 'some private info', the text you +wish to replace it with e.g. '[private info has been hidden]', and a +comment letting other admins know why you have hidden the information. + +<strong>To attach a censor rule to a user</strong>, so that it will be applied to all +requests that the user has made, go to the user page in the admin +interface. You can do this either by clicking on the admin heading +'Users' and browsing or searching to find the user you want, or by +following an 'admin' link for the user from the public interface. One +you are on the admin page for the user, scroll to the bottom of the page +and click the 'New censor rule' button. On the following page, enter the +text that you want to replace e.g. 'my real name is Bruce Wayne', the +text you wish to replace it with e.g. '[personal information has been +hidden]', and a comment letting other admins know why you have hidden +the information. diff --git a/docs/running/categories_and_tags.md b/docs/running/categories_and_tags.md new file mode 100644 index 000000000..8f36ae472 --- /dev/null +++ b/docs/running/categories_and_tags.md @@ -0,0 +1,200 @@ +--- +layout: page +title: Categories & tags +--- + +# Categories and tags for authorities + +<p class="lead"> + + Use tags to arrange + <a href="{{ site.baseurl }}docs/glossary/#authority" + class="glossary__link">authorities</a> into categories, or to associate + related authorities with each other. This helps your users find the right + authority for the + <a href="{{ site.baseurl }}docs/glossary/#request" class="glossary__link">request</a> + (or <a href="{{ site.baseurl }}docs/glossary/#response" class="glossary__link">response</a>) + they are interested in. +</p> + +## Categories & category headings + +<div class="attention-box info"> +Admin interface introduced in Alaveteli version 0.20 +</div> + +Alaveteli lets you organise your authorities into *categories*. Categories can +themselves belong to *category headings*. For example, some of the categories +and headings on +<a href="{{ site.baseurl }}docs/glossary/#wdtk" class="glossary__link">WhatDoTheyKnow</a>'s +<a href="https://www.whatdotheyknow.com/body/list/all">View authorities</a> page look like this: + +> * **Media and culture** +> * Media +> * Museums and galleries +> * **Military and security services** +> * Armed Forces +> * Military colleges +> * Security services +> * **Emergency services** +> * Police forces +> * Fire & rescue services + + +In this example, "Emergency services" is a heading which contains the categories +"Police forces" and "Fire & rescue services". + +Tags are simply searchable words that you can add to an authority. Nominate a +tag for each category: any authority which has that tag is automatically +assigned to the category. For example, if the tag `police` is associated with +the category "Police forces", any authority which has the tag `police` will +appear in that category. + +Make sure you choose good category headings and names, because they help your +users find the specific authorities they are looking for. + +<div class="attention-box info"> + Try to use simple but descriptive words for tags. Tags cannot contain spaces + (use an underscore if you need to, <code>like_this</code>). + Remember that tags will be seen and used by the public (for example, in the + <a href="{{ site.baseurl }}docs/glossary/#advanced-search" class="glossary__link">advanced search</a>). +</div> + +### Adding a new category + +In the admin interface, click on **Categories**. It's a good idea to create +category headings first (but don't worry if you don't — you can change +them later). + +Click on **New category heading**, enter a name (for example, "Emergency +services") and click **Create**. + +To create a category, click on **New category**. As well as providing a title +and a description, you must enter a category tag. Any authority with this tag +will be assigned to this category. + +Select the checkbox next to the category heading under which you want this +category to be listed. It's common for a category to be under just one heading. +But sometimes it makes sense for a category to go under more than one, so you +can select multiple checkboxes if you need to. + +Click **Save** to create the category. + +### Editing or deleting a category + +Click on **Categories** then find the category in the list (if the category is +under a heading, you may need to click on the heading's chevron to expand the +list to show it). Click the name of the category to select it. You can edit it +and click **Save**. + +If you want to destroy a category, go to edit it but instead of saving it, +click on the **Destroy** button at the bottom of the page. This does not +delete any authorities in that category — they simply no longer belong to +it. + +## Special tags + +Some tags are special. Alaveteli behaves differently when an authority has one +of these tags. + +<table class="table"> + <tr> + <th> + Tag + </th> + <th> + Effect + </th> + </tr> + <tr> + <td> + <code>site_administration</code> + </td> + <td> + This is a test/dummy authority. It is not displayed to the public on your + main site, and it is not included when you + <a href="{{ site.baseurl }}docs/running/admin_manual/#creating-changing-and-uploading-public-authority-data">export authorities in CSV format</a>. + </td> + </tr> + <tr> + <td> + <code>defunct</code> + </td> + <td> + This authority no longer operates: new requests cannot be sent to an + authority with this tag. + </td> + </tr> + <tr> + <td> + <code>not_apply</code> + </td> + <td> + <a href="{{ site.baseurl }}docs/glossary/#foi" class="glossary__link">Freedom of Information</a> + law does not apply to this authority: new requests cannot be sent to an + authority with this tag. + </td> + </tr> + <tr> + <td> + <code>eir_only</code> + </td> + <td> + <em>Custom example:</em> (see below)<br> + On our UK installation of Alaveteli, + <a href="{{ site.baseurl }}docs/glossary/#wdtk" class="glossary__link">WhatDoTheyKnow</a>, + this tag indicates that the authority is subject to an alternative law + (Environment Information Regulations, rather than the Freedom of + Information), which means Alaveteli must change the wording of these + requests appropriately. + </td> + </tr> + <tr> + <td> + <code>school</code> + </td> + <td> + <em>Custom example:</em> (see below)<br> + <a href="{{ site.baseurl }}docs/glossary/#wdtk" class="glossary__link">WhatDoTheyKnow</a> + applies a different definition of "late" if an authority has the <code>school</code> tag. + </td> + </tr> +</table> + +## Custom tags and custom behaviour + +You can add any tag you want — they don't have to be associated with +categories. + +If you are a developer, and you want to add special behaviour to your site +based on your own tags, you need to add custom code, which should probably go +in your own +<a href="{{ site.baseurl}}docs/glossary/#theme" class="glossary__link">theme</a>. +For example, in the UK, schools are granted special concession in the law to allow for +requests that are made out of term-time. Alaveteli handles this by using the +[`SPECIAL_REPLY_VERY_LATE_AFTER_DAYS`]({{ site.baseurl }}docs/customising/config/#special_reply_very_late_after_days) +config value if the authority has the `school` tag. +See +[`is_school?`](https://github.com/mysociety/alaveteli/blob/f0bbeb4abf4bf07e5cfb46668f39bbff72ed7210/app/models/public_body.rb#L391) +and +[`date_very_overdue_after`](https://github.com/mysociety/alaveteli/blob/81b778622ed47e24a2dea59c0529d1f928c68a58/app/models/info_request.rb#L752) +for the source code. + +## Searching with tags + +Alaveteli's +<a href="{{ site.baseurl }}docs/glossary/#advanced-search" class="glossary__link">advanced search</a> +feature (which is available to all your users) can search for specific tags. So +if you add useful tags and publicise them, your users can use them to find +related authorities. For example, see the <a +href="https://www.whatdotheyknow.com/advancedsearch">advanced search on +WhatDoTheyKnow</a> to see this at work. + +You can add reference numbers or specific values to tags using a colon. On +<a href="{{ site.baseurl }}docs/glossary/#wdtk" class="glossary__link">WhatDoTheyKnow</a> +we tag all authorities that are charities with the tag `charity:123456` (where +123456 is the authority's registered charity number). + + + + diff --git a/docs/running/holding_pen.md b/docs/running/holding_pen.md new file mode 100644 index 000000000..5b7f08bec --- /dev/null +++ b/docs/running/holding_pen.md @@ -0,0 +1,101 @@ +--- +layout: page +title: The holding pen +--- + +# The holding pen + +<p class="lead"> + + The <em>holding pen</em> is where Alaveteli puts any incoming + <a href="{{ site.baseurl }}docs/glossary/#response" class="glossary__link">responses</a> + that can't be matched to a + <a href="{{ site.baseurl }}docs/glossary/#request" class="glossary__link">request</a> + automatically. +</p> + + +Alaveteli works by emailing requests to the correct target +<a href="{{ site.baseurl }}docs/glossary/#authority" class="glossary__link">authority</a>. +That email message is sent from a unique email address — that is, an +email address that is associated with that single request (technically, +Alaveteli hashes the request ID to generate a unique address and uses this as +the `Reply-to:` address). + +So whenever an authority replies (by email) to a request that Alaveteli has +sent, that response will be addressed to that request's unique email address. +The email is received by your installation's +<a href="{{ site.baseurl}}docs/glossary/#mta" class="glossary__link">MTA</a>, +and is passed on to Alaveteli. In this way, incoming messages are easily +matched with the request they are responses to — this is important +because your site displays the responses underneath their original request, on +the request's page. + +Normally, this works fine. But sometimes things go wrong, and a message comes +in that can't be matched with a request. When this happens, Alaveteli puts the +message in the +<a href="{{ site.baseurl }}docs/glossary/#holding_pen" class="glossary__link">holding +pen </a>. + +Messages wait in the holding pen until an +<a href="{{ site.baseurl }}docs/glossary/#super" class="glossary__link">administrator</a> +redelivers them to the correct request, or else deletes them. + +## Why messages end up in the holding pen + +There are several reasons why a message might end up in the holding pen: + +* **the authority "broke" the reply-to email**<br> + This can happen if the authority replies "by hand" to the incoming email — + for example if the person at the authority accidentally loses the first + letter of the email address when they copy-and-paste it. Or if they copy + it manually and simply get it wrong. + +* **there's something unusual about the way it was sent**<br> + For example, if it was delivered here because the address is in the `Bcc:` + field, and is not the `To:` address. + +* **a partial email address may have been guessed**<br> + Someone guesses an email address which Alaveteli doesn't recognise. Perhaps + they have misunderstood how the addresses are formed, or maybe it's a + deliberate attempt to send spam. + +* **the response is to a request that has been deleted**<br> + If you [delete a request]({{ site.baseurl }}docs/running/requests/#deleting-a-request), + Alaveteli cannot deliver responses to it. + +* **the response has been rejected and rejections are set to go to the holding pen**<br> + Incoming mail that is correctly addressed but not accepted for the request + goes into the holding pen if the request's `handle_rejected_responses` + behaviour is set to `holding_pen` (rather than bouncing the email back to + the sender, or simply deleting it). Responses may be rejected for various + reasons — for example, if a response is sent from an unrecognised + email address for a request whose *Allow new responses from* setting is + `authority_only`. See instructions on + [how to manage requests]({{site.baseurl}}docs/running/requests/) for details. + +## What to do: redeliver or delete + +You need to be an +<a href="{{ site.baseurl }}docs/glossary/#super" class="glossary__link">administrator</a> +to modify the holding pen. + +There are two things you can do to a message in the holding pen: + + * **find the right request, and redeliver the message**<br> + Alaveteli tries to guess the right request to help you, so sometimes + you can just accept its suggestion. + + * **delete the message**<br> + If the message is not a response, you can delete it. + +For instructions, see +[removing a message from the holding pen]({{ site.baseurl }}docs/running/admin_manual/#removing-a-message-from-the-holding-pen). + +If the `To:` address does not belong to a valid request and the message is +clearly spam you can add that email address to Alaveteli's +<a href="{{site.baseurl}}#spam-address-list" class="glossary__link">spam address list</a>. +Subsequent messages to that address will be automatically rejected — for +instructions see +[rejecting spam that arrives in the holding pen]({{ site.baseurl }}docs/running/admin_manual/#rejecting-spam-that-arrives-in-the-holding-pen). + diff --git a/docs/running/index.md b/docs/running/index.md index 90461fb3e..bbf30d3b9 100644 --- a/docs/running/index.md +++ b/docs/running/index.md @@ -11,18 +11,20 @@ title: Running </p> Alaveteli is not just software. To run a successful -<a href="{{ site.baseurl }}docs/glossary/#foi" class="glossary">Freedom of Information</a> +<a href="{{ site.baseurl }}docs/glossary/#foi" class="glossary__link">Freedom of Information</a> site, you need to make sure day-to-day tasks get done too. Most Alaveteli sites are run by a team who allocate some time every day to user support and generally keeping the project up to date. -* the [administrator's guide]({{ site.baseurl }}docs/running/admin_manual/) describes +* The [administrator's guide]({{ site.baseurl }}docs/running/admin_manual/) describes what you need to do and know to run your site -* we've prepared a checklist of +* The [redaction guide]({{ site.baseurl }}docs/running/redaction/) includes some + examples of Alaveteli's ability to redact sensitive information + +* We've prepared a checklist of [things to consider]({{ site.baseurl }}docs/running/server/) when setting up your production server -* see the [different states a request can be in]({{ site.baseurl }}docs/running/states/) diff --git a/docs/running/redaction.md b/docs/running/redaction.md new file mode 100644 index 000000000..6ab8fed86 --- /dev/null +++ b/docs/running/redaction.md @@ -0,0 +1,135 @@ +--- +layout: page +title: Redacting Sensitive Information +--- + +# Redacting Sensitive Information + +In some countries, local requirements mean that requests need to contain personal information such as the address or ID number of the person asking for information. Usually requesters do not want this information to be displayed to the general public. + +Alaveteli has some ability to deal with this through the use of <a href="{{site.baseurl}}docs/glossary/#censor-rule" class="glossary__link">Censor Rules</a>. + +The [theme](https://github.com/mysociety/derechoapreguntar-theme) we'll use as an example requires a National Identity Card Number and what's known as General Law in Nicaragua (Date of Birth, Domicile, Occupation and Marital Status). + + + +## Identity Card Number + +We'll start off by looking at the National Identity Card Number (ID Number from here). Its a good example of something that is relatively easy to redact. It's unique for each user, and it has a specified format to match against. + +To send the ID Number to the authority we'll override the [initial request template](https://github.com/mysociety/alaveteli/blob/master/app/views/outgoing_mailer/initial_request.text.erb) (code snippet shortened): + + <%= raw @outgoing_message.body.strip %> + + ------------------------------------------------------------------- + + <%= _('Requestor details') %> + <%= _('Identity Card Number') %>: <%= @user_identity_card_number %> + +When a request is made the user's ID Number is now added to the footer of the outgoing email. + + + +At this point we haven't added any <a href="{{site.baseurl}}docs/glossary/#censor-rule" class="glossary__link">Censor Rules</a>. When the authority replies it is unlikely that the responder will remove the quoted section of the email: + + + +We could add a <a href="{{site.baseurl}}docs/glossary/#censor-rule" class="glossary__link">Censor Rule</a> for the individual request, but as every request will contain a user's ID Number its better to add some code to do do it automatically. + +To illustrate this we'll patch the `User` model with a callback that creates a <a href="{{site.baseurl}}docs/glossary/#censor-rule" class="glossary__link">Censor Rule</a> when the user is created and updated. + + # THEME_ROOT/lib/model_patches.rb + User.class_eval do + after_save :update_censor_rules + + private + + def update_censor_rules + censor_rules.where(:text => identity_card_number).first_or_create( + :text => identity_card_number, + :replacement => _('REDACTED'), + :last_edit_editor => THEME_NAME, + :last_edit_comment => _('Updated automatically after_save') + ) + end + end + +You can see the new <a href="{{site.baseurl}}docs/glossary/#censor-rule" class="glossary__link">Censor Rule</a> in the admin interface: + + + +Now the ID Number gets redacted: + + + +It also gets redacted if the public body use the ID Number in the main email body: + + + +A <a href="{{site.baseurl}}docs/glossary/#censor-rule" class="glossary__link">censor rule</a> added to a user only gets applied to correspondence on requests created by that user. It does not get applied to annotations made by the user. + +**Warning:** Redaction in this way requires the sensitive text to be in exactly the same format as the <a href="{{site.baseurl}}docs/glossary/#censor-rule" class="glossary__link">Censor Rule</a>. If it differs even slightly, the redaction can fail. If the public body was to remove the hyphens from the number it would not be redacted: + + + +**Warning:** Alaveteli also attempts to redact the text from any attachments. It can only do this if it can find the exact string, which is often not possible in binary formats such as PDF or Word. + +Alaveteli can usually redact the sensitive information when converting a PDF or text based attachment to HTML: + + + +This PDF does not contain the string in the raw binary so the redaction is _not_ applied when downloading the original PDF document: + + + +## General Law + +The General Law information is much harder to automatically redact. It is not as structured, and the information is unlikely to be unique (e.g. Domicile: London). + +We'll add the General Law information to the [initial request template](https://github.com/mysociety/alaveteli/blob/master/app/views/outgoing_mailer/initial_request.text.erb) in the same way as the ID Number: + + <%= _('Requestor details') %>: + <%-# !!!IF YOU CHANGE THE FORMAT OF THE BLOCK BELOW, ADD A NEW CENSOR RULE!!! -%> + =================================================================== + # <%= _('Name') %>: <%= @user_name %> + # <%= _('Identity Card Number') %>: <%= @user_identity_card_number %> + <% @user_general_law_attributes.each do |key, value| %> + # <%= _(key.humanize) %>: <%= value %> + <% end %> + =================================================================== + +Note that the information is now contained in a specially formatted block of text. + + + +This allows a <a href="{{site.baseurl}}docs/glossary/#censor-rule" class="glossary__link">Censor Rule</a> to match the special formatting and remove anything contained within. This <a href="{{site.baseurl}}docs/glossary/#censor-rule" class="glossary__link">Censor Rule</a> is global, so it will act on matches in all requests. + + # THEME_ROOT/lib/censor_rules.rb + # If not already created, make a CensorRule that hides personal information + regexp = '={67}\s*\n(?:[^\n]*?#[^\n]*?: ?[^\n]*\n){3,10}[^\n]*={67}' + + unless CensorRule.find_by_text(regexp) + Rails.logger.info("Creating new censor rule: /#{regexp}/") + CensorRule.create!(:text => regexp, + :allow_global => true, + :replacement => _('REDACTED'), + :regexp => true, + :last_edit_editor => THEME_NAME, + :last_edit_comment => 'Added automatically') + end + + + +**Warning:** Redacting unstructured information is a very fragile approach, as it relies on authorities always quoting the entire formatted block. + +In this case the authority has revealed the user's Date of Birth and Domicile: + + + +Its really difficult to add a <a href="{{site.baseurl}}docs/glossary/#censor-rule" class="glossary__link">Censor Rule</a> to remove this type of information. One suggestion might be to remove all mentions of the user's Date of Birth, but you would have to account for [every type of date format](http://en.wikipedia.org/wiki/Calendar_date#Date_format). Likewise, you could redact all occurrences of the user's Domicile, but if they a question about their local area (very likely) the request would become unintelligible. + + + +The redaction has been applied but there is no way of knowing the context that the use of the sensitive word is used. + + diff --git a/docs/running/requests.md b/docs/running/requests.md new file mode 100644 index 000000000..e554fe763 --- /dev/null +++ b/docs/running/requests.md @@ -0,0 +1,355 @@ +--- +layout: page +title: Managing requests +--- + +# Managing Requests + + +<p class="lead"> + Alaveteli makes it easy for a user to make a + <a href="{{ site.baseurl }}docs/glossary/#request" class="glossary__link">request</a>. + As an + <a href="{{ site.baseurl }}docs/glossary/#super" class="glossary__link">administrator</a>, + there are some things about that request you can change once it’s been created. +</p> + +A request is automatically created when a user submits and (where necessary) +confirms it. Alaveteli sends it to the +<a href="{{ site.baseurl }}docs/glossary/#authority" class="glossary__link">authority</a> +responsible and handles any +<a href="{{ site.baseurl }}docs/glossary/#response" class="glossary__link">responses</a>. +Usually this process runs without needing any intervention from an +administrator. But sometimes you'll want to change some aspect of the request, +or the way Alaveteli is handling it. + +<ul class="toc"> + <li><a href="#what-state-is-the-request-in">What state is the request in?</a></li> + <li><a href="#old-requests-6-months-without-activity">Old requests (6+ months without activity)</a></li> + <li><a href="#changing-things-about-a-request">Changing things about a request</a></li> + <li><a href="#resending-a-request-or-sending-it-to-a-different-authority">Resending a request or sending a request to a different authority</a></li> + <li><a href="#hiding-a-request">Hiding a request</a></li> + <li><a href="#deleting-a-request">Deleting a request</a></li> +</ul> + +## What state is the request in? + +Every request moves through a series of +<a href="{{ site.baseurl }}docs/glossary/#state" class="glossary__link">states</a>, +indicating its progress. Usually a new request will be in the +`waiting_response` state until something happens to change that — for +example, a response is received. + +However, states can't always be set automatically, because they require a +decision to be made on what kind of answer the authority provided in the +response. For states like this, Alaveteli invites the original requester to +describe it — for example, when a response is received they can change +the state to `successful`, `partially_successful` or `not_held` (if the +authority replied to say they don't have the information requested). + +<div class="attention-box info"> + If a request has been waiting for over three weeks for the original + requester to describe it but has still not been described, Alaveteli + lets <em>anyone</em> classify it. +</div> + +Internally, Alaveteli does not just record the "described state" of a request, +but also notices if anything has happened since it was last described and +sets its "awaiting description" status appropriately. + + +## Old requests (6+ months without activity) + +If there is no activity on a request for six months, Alaveteli automatically +changes that request's **Allow new responses from...** setting to `authority_only`. +Any responses will be rejected unless they are from the authority to which the +request was originally made. + +If a further six months pass with no activity, that is, a year has passed +without any updates, the request's **Allow new responses from...** setting +becomes `nobody`. All responses to this request will be rejected. The request +is effectively closed. + +By default, any rejected responses will be bounced with a message that explains +that the request has been closed because it is old, with a suggestion that the +sender can email your site's administrators directly. + +You can can stop this behaviour by changing the **Allow new responses from...** +setting back to `normal` at any time. Alternatively, you can change the way +rejected messages are handled (for example, sending such responses to the +holding pen instead of bouncing them) with the request's **Handle rejected +responses** setting. + +See [changing things about a request](#changing-things-about-a-request) below +for how to change these settings. + + +## Changing things about a request + +To change any of these settings, go to the +<a href="{{ site.baseurl }}docs/glossary/#admin" class="glossary__link">admin interface</a>, +click on **Requests**, then click on the title of the request you want to affect. +Click the **Edit metadata** button. + +<table class="table"> + <tr> + <th> + What you can change + </th> + <th> + Details + </th> + </tr> + <tr> + <td> + Title + </td> + <td> + The <em>title</em> is shown on the request’s page, but is also used + in the URL (the text is changed to lower case, punctuation is removed + and, if necessary, a number is added for disambiguation — this is + called the “slug”). + <p> + Note that changing the title changes the URL, because the slug changes + — this means any links to the <em>old</em> URL will no longer + work, and will return a 404 (file not found) error. + </p> + </td> + </tr> + <tr> + <td> + Who can see it? + </td> + <td> + Change the <strong>Prominence</strong> setting to one of: + <ul> + <li><code>normal</code></li> + <li> + <code>backpage</code>: request can be seen by anyone (by visiting + its URL, for example) but does not appear in lists, or search results + </li> + <li> + <code>requester_only</code>: request only visible to the person who + made the request + </li> + <li> + <code>hidden</code>: request is never shown (except to administrators) + </li> + </ul> + <br> + </td> + </tr> + <tr> + <td> + Who can respond? + </td> + <td> + The <strong>Allow new responses from...</strong> setting can be one of: + <ul> + <li><code>anybody</code></li> + <li> + <code>authority_only</code>: responses are allowed if they come + from the authority to which the request was sent, or from any domain + from which a a response has <em>already</em> been accepted + </li> + <li> + <code>nobody</code>: no responses are allowed on this request + </li> + </ul> + <p> + Any response from a sender who has been disallowed by this + setting will be rejected (see next entry). + </p> + <p> + This setting may change automatically when + <a href="#old-requests-6-months-without-activity">the request gets old</a>. + </p> + </td> + </tr> + <tr> + <td> + What happens to rejected responses? + </td> + <td> + The <strong>Handle rejected responses...</strong> setting specificies + what happens to responses that are not allowed (see previous entry): + <ul> + <li> + <code>bounce</code>: responses are sent back to their sender + </li> + <li> + <code>holding pen</code>: responses are put in the + <a href="{{ site.baseurl }}docs/glossary/#holding_pen" class="glossary__link">holding pen</a> + for an administrator to deal with + </li> + <li> + <code>blackhole</code>: responses are destroyed by being sent to a + <a href="{{ site.baseurl }}docs/glossary/#blackhole" class="glossary__link">black hole</a> + </li> + </ul> + </td> + </tr> + <tr> + <td> + What state is it in? + </td> + <td> + See <a href="{{ site.baseurl }}docs/customising/states/">more about + request states</a>, which can be customised for your installation. + <p> + You can force the state of the request by choosing it explicitly. + Change the <strong>Described state</strong> setting. + </p> + <p> + You may also need to set <strong>Awaiting description</strong> if, + having changed the state, you want the original requester to update the + description. For example, if the state depends on the information + within the response, and you want the requester to classify it — + see + <em><a href="#what-state-is-the-request-in">What state is the request in?</a></em> + above. + </p> + </td> + </tr> + <tr> + <td> + Are comments allowed? + </td> + <td> + The <strong>Are comments allowed?</strong> setting simply you choose to + allow or forbid annotations and comments on this request. + <p> + Note that this won’t hide any annotations that have already + been left on the reques — it only prevents users adding new ones. + </p> + </td> + </tr> + <tr> + <td> + Tags (search keywords) + </td> + <td> + Enter tags, separated by spaces, that are associated with this request. + A tag can be either a simple keyword, or a key-value pair (use a colon as + the separator, like this: <code>key:value</code>). + <p> + Tags are used for searching. Users and administators both benefit if + you tag requests with useful keywords, because it helps them find + specific requests — especially if your site gets busy and there + are very many in the database. + </p> + <p> + Although it’s a little more complex than tags on requests, + <a href="{{ site.baseurl }}docs/glossary/#category" class="glossary__link">categories</a> + also use tags: + see + <a href="{{ site.baseurl }}docs/running/categories_and_tags/">more about tags</a> + for a little more information. + </p> + </td> + </tr> +</table> + +## Resending a request or sending it to a different authority + +If you have corrected the email address for an authority, you can resend +an existing request to that authority to the new email address. Alternatively, +a user may send a request to the wrong authority. In that situation, you can +change the authority on the request and then resend it to the correct authority. + +To resend a request, go to +the <a href="{{ site.baseurl }}docs/glossary/#admin" +class="glossary__link">admin interface</a>, click on **Requests**, then +click on the name of the request you want to change. Go to the **Outgoing messages** heading. Click the chevron next to the first outgoing message, which is the initial request. A panel of information about that message will appear. Click on the **Resend** button. + +To send a request to a different authority, go to +the <a href="{{ site.baseurl }}docs/glossary/#admin" +class="glossary__link">admin interface</a>, click on **Requests**, then +click on the name of the request you want to change. In the **Request +metadata** section, there is a line which shows the authority. Click the +**move...** button next to it. Enter the **url_name** of the authority +that you want to send the request to. + +<div class="attention-box info"> +Users, requests and authorities all have <strong>url_names</strong>. This can be found in the metadata section of their admin page. The <code>url_name</code> makes up the last part of the URL for their public page. So, for a request with the <code>url_name</code> “example_request”, the public page URL will be <code>/request/example_request</code>. +</div> + +Now click the **Move request to +authority** button. You will see a notice at the top of the page telling +you that the request has been moved. You can now resend the request as above. + + +## Hiding a request + +You can hide an entire request. Typically you do this if it's not a valid +Freedom of Information request (for example, a request for personal +information), or if it is vexatious. + +Go to the <a href="{{ site.baseurl }}docs/glossary/#admin" class="glossary__link">admin interface</a>, +click on **Requests**, then click on the title of the request you want. You can +hide it in one of two ways: + + * [Hide the request and notify the requester](#hide-the-request-and-notify-the-requester) + * [Hide the request without notifying the requester](#hide-the-request-without-notifying-the-requester) + +Responses to a hidden request will be accepted in the normal way, but because +they are added to the request's page, they too will be hidden. + +### Hide the request and notify the requester + +Scroll down to the *Actions* section of the request's admin page. +Choose one of the options next to **Hide the request and notify the user:** + + * Not a valid FOI request + * A vexatious request + +Choosing one of these will reveal an email form. Customise the text of the +email that will be sent to the user, letting them know what you've done. When +you're ready, click the **Hide request** button. + +### Hide the request without notifying the requester + +<div class="attention-box helpful-hint"> + As well as hiding the request from everyone, you can also use this method if + you want to make the request only visible to the requester. +</div> + +In the *Request metadata* section of the request's admin page, click the +**Edit metadata** button. Change the *Prominence* value to one of these: + + * `requester_only`: only the requester can view the request + * `hidden`: nobody can see the request, except administrators. + +<div class="attention-box warning"> + If you want to hide the request, do not chooose <code>backpage</code> + as the prominence. The <code>backpage</code> option stops the request + appearing in lists and searches so that it is effectively only visible + to anyone who has its URL — but it <em>does not hide</em> the request. +</div> + +When you're ready, click the **Save changes** button at the bottom of the +*Edit metadata* section. No email will be sent to the requester to notify +them of what you've done. + + +## Deleting a request + +You can delete a request entirely. Typically, you only need to do this if +someone has posted private information. If you delete a request, any responses that it has already received will be +destroyed as well. + +<div class="attention-box warning"> + Deleting a request destroys it. There is no “undo” operation. + If you're not sure you want to do this, perhaps you should + <a href="#hiding-a-request">hide the request</a> instead. +</div> + +Go to the <a href="{{ site.baseurl }}docs/glossary/#admin" class="glossary__link">admin interface</a>, +click on **Requests**, then click on the title of the request you want to delete. +Click the **Edit metadata** button. Click on the red **Destroy request entirely** +button at the bottom of the page. + +Responses to a deleted request will be sent to the +<a href="{{ site.baseurl }}docs/glossary/#holding_pen" class="glossary__link">holding pen</a>. + + diff --git a/docs/running/security.md b/docs/running/security.md new file mode 100644 index 000000000..a22c4d636 --- /dev/null +++ b/docs/running/security.md @@ -0,0 +1,36 @@ +--- +layout: page +title: Security & Maintenance +--- + +# Security & Maintenance + +<p class="lead"> + Support of Alaveteli is divided into four groups: New features, bug fixes, security issues, and severe security issues. They are handled as follows: +</p> + +## New Features + +Only the [latest development branch](https://github.com/mysociety/alaveteli/tree/rails-3-develop/) gets new features which will be released in the next main release. + +## Bug Fixes + +- Only the current release will receive bug fixes +- Bug fixes will get a new release (e.g. `0.19.0` gets a new release to `0.19.1`) +- Bug fixes will be applied to current development branch + +## Security Issues + +- The current release, previous release and current development branch will receive fixes +- Security issues will get a new release (e.g. `0.19.0` gets a new release to `0.19.1`) for the current and previous releases +- Generic patch will be posted to the mailing list + +## Severe Security Issues + +- Severe is determined by the Alaveteli core team +- The current release, previous release and current development branch will receive fixes +- Severe security issues will get a new release (e.g. `0.19.0` gets a new release to `0.19.1`) for supported versions +- Generic patch will be posted to the mailing list +- All releases known to be in production will receive patches and every effort will be made to contact known re-users for a private disclosure + + diff --git a/docs/running/server.md b/docs/running/server.md index c10d4b4b4..da1312ac3 100644 --- a/docs/running/server.md +++ b/docs/running/server.md @@ -26,16 +26,14 @@ ask us about hosting. Don't forget to set up the cron jobs as outlined in the [installation instructions]({{ site.baseurl }}docs/installing/manual_install/). -As of October 2011, they rely on a small program created by mySociety called -`run-with-lockfile`. A discussion of where the source for this can be found, -and possible alternatives, lives in -[Alaveteli issue #112](https://github.com/mysociety/alaveteli/issues/112). ## Webserver configuration We recommend running your site behind [Apache](https://httpd.apache.org) + -[Passenger](https://www.phusionpassenger.com). Refer to the +[Passenger](https://www.phusionpassenger.com) or [Nginx](http://wiki.nginx.org/Main) + [Thin](http://code.macournoyer.com/thin/). + +If you're using Passenger, refer to the [installation instructions]({{ site.baseurl }}docs/installing/manual_install/) regarding `PassengerMaxPoolSize`, which you should experiment with to match your available RAM. It is very unlikely that you'll diff --git a/docs/running/states.md b/docs/running/states.md deleted file mode 100644 index 1c3fb217e..000000000 --- a/docs/running/states.md +++ /dev/null @@ -1,217 +0,0 @@ ---- -layout: page -title: States of requests ---- - -# Request states - -<p class="lead"> - A <a href="{{site.baseurl}}docs/glossary/#request" class="glossary">request</a> - passes through different <strong>states</strong> as it is processed. These may - vary from one jurisdiction to another. -</p> - -The request states are defined in the Alaveteli code, and we recommend you use -them (provided they match the <a href="{{ site.baseurl }}docs/glossary/#foi" -class="glossary">FOI law</a> in your own jurisdiction). But if you do need to -customise them, you can — see -<a href="{{ site.baseurl }}docs/customising/themes/">Customising the request states</a> for details. - -## WhatDoTheyKnow example - -Requests made on the UK's Alaveteli instance, [WhatDoTheyKnow](https://www.whatdotheyknow.com), -may be in any of the states described below. - -Note that your site doesn't need to use the same states as WhatDoTheyKnow does. For example, -Kosovo's instance uses slightly different states: see -[this comparison of their differences]({{ site.baseurl }}docs/running/states_informatazyrtare/). - -### States - -<ul class="definitions"> - <li><a href="#waiting_response">waiting_response</a></li> - <li><a href="#waiting_classification">waiting_classification</a></li> - <li><a href="#waiting_response_overdue">waiting_response_overdue</a></li> - <li><a href="#waiting_response_very_overdue">waiting_response_very_overdue</a></li> - <li><a href="#waiting_clarification">waiting_clarification</a></li> - <li><a href="#gone_postal">gone_postal</a></li> - <li><a href="#not_held">not_held</a></li> - <li><a href="#rejected">rejected</a></li> - <li><a href="#successful">successful</a></li> - <li><a href="#partially_successful">partially_successful</a></li> - <li><a href="#internal_review">internal_review</a></li> - <li><a href="#error_message">error_message</a></li> - <li><a href="#requires_admin">requires_admin</a></li> - <li><a href="#user_withdrawn">user_withdrawn</a></li> - <li><a href="#awaiting_description">awaiting_description</a></li> -</ul> - - -<dl class="glossary"> - - <dt> - <a name="waiting_response">waiting_response</a> - </dt> - <dd> - Waiting for the public authority to reply - <ul> - <li>The default initial state</li> - <li>Can't transition here from internal_review</li> - </ul> - </dd> - - <dt> - <a name="waiting_classification">waiting_classification</a> - </dt> - <dd> - Waiting for a classification of a response - <ul> - <li>The default state after receiving a response</li> - </ul> - </dd> - - <dt> - <a name="waiting_response_overdue">waiting_response_overdue</a> - </dt> - <dd> - Waiting for a reply for too long - <ul> - <li>Automatic, if today's date is after the request date + holidays + 20 days</li> - <li>When a user updates / visits an item in this state, thank user and tell them how long they should have to wait</li> - <li>Alert user by email when something becomes overdue</li> - </ul> - </dd> - - <dt> - <a name="waiting_response_very_overdue">waiting_response_very_overdue</a> - </dt> - <dd> - Waiting for a reply for a very long time - <ul> - <li>Automatic, if today's date is after the request date + holidays + (60 days (for schools) or 40 days (everyone else))</li> - <li>When a user updates / visits something in this state, suggest they might want to complain about it; show things they might want to do</li> - <li>Alert user by email when this state happens</li> - </ul> - </dd> - - <dt> - <a name="waiting_clarification">waiting_clarification</a> - </dt> - <dd> - The public authority would like part of the request explained - <ul> - <li>Prompt user to write followup</li> - <li>if a user sends an outgoing message on a request in this state, automatically transitions to {{waiting_response}}</li> - <li>three days after this state change occurs, send reminder to user to action it (assuming user isn't banned)</li> - <li>Can't transition here from internal_review</li> - </ul> - </dd> - - <dt> - <a name="gone_postal">gone_postal</a> - </dt> - <dd> - The public authority would like to / has responded by post - <ul> - <li>If selected, remind user that in most cases authority should respond by email, and encourage followup.</li> - <li>Give most recent authority correspondence email address for user to request postal by private email.</li> - <li>Encourage user to update thread with annotation at later date.</li> - </ul> - </dd> - - <dt> - <a name="not_held">not_held</a> - </dt> - <dd> - The public authority does not have the information requested - <ul> - <li>Suggest user might want to try a different authority, or complain</li> - </ul> - </dd> - - <dt> - <a name="rejected">rejected</a> - </dt> - <dd> - The request was refused by the public authority - <ul> - <li>Show page of possible next steps</li> - </ul> - </dd> - - - <dt> - <a name="successful">successful</a> - </dt> - <dd> - All of the information requested has been received - <ul> - <li>Suggest they add annotations or make a donation</li> - </ul> - </dd> - - - <dt> - <a name="partially_successful">partially_successful</a> - </dt> - <dd> - Some of the information requested has been received - <ul> - <li>Suggest they make a donation; give ideas what to do next</li> - </ul> - </dd> - - <dt> - <a name="internal_review">internal_review</a> - </dt> - <dd> - Waiting for the public authority to complete an internal review of their handling of the request - <ul> - <li>Tell user they should expect a response within 20 days</li> - <li>When sends email to authority, adds “Internal review of” to Subject</li> - <li>Can be transitioned from the followup form</li> - </ul> - </dd> - - <dt> - <a name="error_message">error_message</a> - </dt> - <dd> - Received an error message, such as delivery failure. - <ul> - <li>Thank user for reporting, and suggest they use a form to give new email address for authority if that was the problem</li> - <li>Mark as needs admin attention</li> - </ul> - </dd> - - <dt> - <a name="requires_admin">requires_admin</a> - </dt> - <dd> - A strange reponse, required attention by the WhatDoTheyKnow team - <ul> - <li>a user is confused and doesn't know what state to set, so an admin can intervene</li> - <li>Redirect to form to ask for more information</li> - <li>Mark as needs admin attention</li> - </ul> - </dd> - - <dt> - <a name="user_withdrawn">user_withdrawn</a> - </dt> - <dd> - The requester has abandoned this request for some reason. - <ul> - <li>Prompt user to write message to tell authority</li> - </ul> - </dd> - - <dt> - <a name="awaiting_description">awaiting_description</a> - </dt> - <dd> - This state, awaiting_description, is not really a state but a flag indicating that there is no state. - </dd> - -</dl> - diff --git a/docs/running/states_informatazyrtare.md b/docs/running/states_informatazyrtare.md deleted file mode 100644 index bd5ff1a1c..000000000 --- a/docs/running/states_informatazyrtare.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -layout: page -title: States of requests (InformataZyrtare) ---- - -# Request states: example comparison - -<p class="lead"> - This page shows differences between states used on two different - Alaveteli instances — one in Kosovo and one in the UK. This - is a practical example showing that you can customise the states that - your site uses. -</p> - -The request states are defined in the Alaveteli code, and we recommend you use -them (provided they match the <a href="{{ site.baseurl }}docs/glossary/#foi" -class="glossary">FOI law</a> in your own jurisdiction). - -## InformataZyrtare.org (Kosovo) example - -Requests made on Kosovo's Alaveteli instance, -[InformataZyrtare](http://informatazyrtare.org), use slightly different states -from those on the UK's instance, [WhatDoTheyKnow](http://www.whatdotheyknow.com) -(WDTK). - -Generally, this arises simply because the local legislation, or the way the -groups running the sites work, are different in different places. Alavateli -facilitates this by allowing you to customise the states that are used. - -This example is to show clearly that you can use different states depending on -your local requirements, and how that might look. See [Customising the request -states]({{ site.baseurl }}docs/customising/themes/) for details on how to do this. - -### States used by InformataZyrtare but not WDTK - - * <a href="#deadline_extended">deadline_extended</a> - * <a href="#partial_rejected">partial_rejected</a> - * <a href="#wrong_response">wrong_response</a> - -### States used by WDTK but not InformataZyrtare - - * <a href="{{ site.baseurl }}docs/running/states/#awaiting_description">awaiting_description</a> - * <a href="{{ site.baseurl }}docs/running/states/#gone_postal">gone_postal</a> - * <a href="{{ site.baseurl }}docs/running/states/#internal_review">internal_review</a> - * <a href="{{ site.baseurl }}docs/running/states/#user_withdrawn">user_withdrawn</a> - * <a href="{{ site.baseurl }}docs/running/states/#waiting_response_very_overdue">waiting_response_very_overdue</a> - -For more details, see all the [states used by WhatDoTheyKnow]({{site.baseurl}}docs/running/states/)) for details. - - ---- - - - -### Details of InformataZytare states - -The states which aren't represented on [WDTK's states]({{ site.baseurl }}docs/running/states/) are described -in a little more detail here: - -<ul class="definitions"> - <li><a href="#deadline_extended">deadline_extended</a></li> - <li><a href="#partial_rejected">partial_rejected</a></li> - <li><a href="#wrong_response">wrong_response</a></li> -</ul> - -<dl class="glossary"> - <dt> - <a name="deadline_extended">deadline_extended</a> - </dt> - <dd> - The Authority has requested deadline extension. - </dd> - <dt> - <a name="partial_rejected">partial_rejected</a> - </dt> - <dd> - Only part of the request has being refused but the successful request - to an information has not been attached. - </dd> - <dt> - <a name="wrong_response">wrong_response</a> - </dt> - <dd> - The authority has replied but the response does not correspond to the request. - </dd> - -</dl> - diff --git a/docs/running/upgrading.md b/docs/running/upgrading.md index a3b292cf0..81af8289f 100644 --- a/docs/running/upgrading.md +++ b/docs/running/upgrading.md @@ -5,28 +5,110 @@ title: Upgrading Upgrading Alaveteli ==================== -The developer team policy is that the master branch in git should always -contain the latest stable release. Therefore, in production, you should usually -have your software deployed from the master branch, and an upgrade can be -simply `git pull`. +<p class="lead"> + Alaveteli is under active development — don’t let the + version you’re running get too far behind our latest + <a href="{{site.baseurl}}docs/glossary/#release" class="glossary__link">release</a>. + This page describes how to keep your site up to date. +</p> -Patch version increases (e.g. 1.2.3 → 1.2.**4**) should not require any further -action on your part. +## How to upgrade the code -Minor version increases (e.g. 1.2.4 → 1.**3**.0) will usually require further -action. You should read the [`CHANGES.md`](https://github.com/mysociety/alaveteli/blob/master/doc/CHANGES.md) document to see what's changed since -your last deployment, paying special attention to anything in the "Upgrade notes" -sections. +* If you're using Capistrano for deployment, + simply [deploy the code]({{site.baseurl}}docs/installing/deploy/#usage): + set the repo and branch in `deploy.yml` to be the version you want. + We recommend you set this to the explicit tag name (for example, + `0.18`, and not `master`) so there's no risk of you accidentally deploying + a new version before you're aware it's been released. +* otherwise, you can simply upgrade by running `git pull` -Any upgrade may include new translations strings, i.e. new or altered messages -to the user that need translating to your locale. You should visit Transifex +## Run the post-deploy script + +Unless you're [using Capistrano for deployment]({{site.baseurl}}docs/installing/deploy/), +you should always run the script `scripts/rails-post-deploy` after each +deployment. This runs any database migrations for you, plus various other +things that can be automated for deployment. + +<div class="attention-box info"> + You don't need to run the script if you're using Capistrano, because the + deployment mechanism automatically runs it for you. +</div> + +## Alaveteli version numbers + +Alaveteli uses a “shifted” version of [semver](http://semver.org) +(just as [Rails version numbering](http://guides.rubyonrails.org/maintenance_policy.html) +does). This means that version numbers are of the form: `SERIES.MAJOR.MINOR.PATCH`. + +At the time of writing, the current release is `0.19.0.6`: + +- Series `0` +- Major `19` +- Minor `0` +- Patch `6` + +We'll use the [semver](http://semver.org) specification for Alaveteli's +version numbering when it reaches `1.0.0`. + +## Master branch contains the latest stable release + +The developer team policy is that the `master` branch in git should always +contain the latest stable release -- so you'll be up to date if you pull from +the `master` branch. However, on your +<a href="{{site.baseurl}}docs/glossary/#production" class="glossary__link">production +site</a>, you should know precisely what version you're running, and deploy +Alaveteli from a [*specific* release +tag](https://github.com/mysociety/alaveteli/releases). + +Upgrading may just require pulling in the latest code -- but it may also require +other changes ("further action"). For this reason, for anything other than a +*patch* (see below), always read the +[`CHANGES.md`](https://github.com/mysociety/alaveteli/blob/master/doc/CHANGES.md) +document **before** doing an upgrade. This way you'll be able to prepare for any +other changes that might be needed to make the new code work. + +## Patches + +Patch version increases (e.g. 0.1.2.3 → 0.1.2.**4**) should not require any further action on your part. They will be backwards compatible with the current minor release version. + +## Minor version increases + +Minor version increases (e.g. 0.1.2.4 → 0.1.**3**.0) will usually require further action. You should read the [`CHANGES.md`](https://github.com/mysociety/alaveteli/blob/master/doc/CHANGES.md) document to see what's changed since your last deployment, paying special attention to anything in the "Upgrade notes" sections. + +Any upgrade may include new translations strings, that is, new or altered messages +to the user that need translating to your locale. You should visit <a href="{{ site.baseurl }}docs/glossary/#transifex" class="glossary__link">Transifex</a> and try to get your translation up to 100% on each new release. Failure to do so means that any new words added to the Alaveteli source code will appear in your website in English by default. If your translations didn't make it to the latest release, you will need to download the updated `app.po` for your locale from Transifex and save it in the `locale/` folder. -Unless you're using Capistrano for deployment, you should always run the script `scripts/rails-post-deploy` after each -deployment. This runs any database migrations for you, plus various other -things that can be automated for deployment. +Minor releases will be backwards compatible with the current major release version. + +## Major releases + +Major version increases (e.g. 0.1.2.4 → 0.2.0.0) will usually require further action. You should read the [`CHANGES.md`](https://github.com/mysociety/alaveteli/blob/master/doc/CHANGES.md) document to see what's changed since your last deployment, paying special attention to anything in the "Upgrade notes" sections. + +Only major releases may remove existing functionality. You will be warned about the removal of functionality with a deprecation notice in a minor release prior to the major release that removes the functionality. + +## Series releases + +Special instructions will accompany series releases. + +## Deprecation notices + +You may start to see deprecation notices in your application log. They will look like: + + DEPRECATION WARNING: Object#id will be deprecated; use Object#object_id + +Deprecation notices allow us to communicate with you that some functionality will change or be removed in a later release of Alaveteli. + +### What to do if you see a deprecation notice + +You will usually see a deprecation notice if you have been using functionality in your theme that is now due to change or be removed. The notice should give you a fair explanation of what to do about it. Usually it will be changing or removing methods. The [changelog](https://github.com/mysociety/alaveteli/blob/rails-3-develop/doc/CHANGES.md) will include more detailed information about the deprecation and how to make the necessary changes. + +If you're ever unsure, don't hesitate to ask in the [developer mailing list](https://groups.google.com/group/alaveteli-dev) or [Alaveteli IRC channel](http://www.irc.mysociety.org/). + +### When will the change take place? +We introduce deprecation notices in a **minor** release. The following **major** release will make the change unless otherwise stated in the deprecation notice. |