diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/CHANGES.md | 235 | ||||
| -rw-r--r-- | doc/INSTALL-exim4.md | 13 | ||||
| -rw-r--r-- | doc/INSTALL.md | 297 | ||||
| -rw-r--r-- | doc/THEME-ASSETS-UPGRADE.md | 76 | ||||
| -rw-r--r-- | doc/THEMES-UPGRADE.md | 101 | ||||
| -rw-r--r-- | doc/THEMES.md | 32 | ||||
| -rw-r--r-- | doc/TRANSLATE.md | 6 |
7 files changed, 662 insertions, 98 deletions
diff --git a/doc/CHANGES.md b/doc/CHANGES.md index 59c2d3f37..8940cd78b 100644 --- a/doc/CHANGES.md +++ b/doc/CHANGES.md @@ -1,3 +1,238 @@ +# Version 0.17 + +## Highlighted features + +* There is some initial support for making a request to multiple + authorities at once. +* There is a new form for users to request that a new authority should + be added, or to request an update to the contact email used for an + authority. Site admins are emailed about these requests, and can + resolve them from the admin interface. +* For attachments where we rely on Google Document Viewer to display the + HTML version, link to the HTTPS version where the Alaveteli site is + served over HTTPS to avoid mixed content warnings and non display in + some browsers (Matthew Somerville). +* The 'view requests' page now has some fragment caching backed by + memcached to speed up serving commonly used lists of requests - e.g + all successful requests. Like the caching introduced in release 0.16, + this is controlled by the `CACHE_FRAGMENTS` parameter in the config + file and will be on by default. +* A user's annotations can now be seen on their admin page (Andrew + Black) +* Better detection of the quoted text of a previous email in the HTML + parts of responses. +* Fixed bugs in the profile photos (György Peng), calendar translations + (Mark Longair), the use of external utilities (Ian Chard), the + internal admin authority locale handling (Mark Longair), badly formed + attachment handling (Rowan Crawford). + +## Upgrade notes + +* To use the batch request functionality, set the `ALLOW_BATCH_REQUESTS` + parameter to `true` in your config file. Once this is done, and the + install has been restarted, any user for whom 'Can make batch + requests' is checked in the admin interface should see a new link on + the 'Select an authority' page that allows them to make a batch + request. +* If your theme overrides the help/requesting template, you should + update the link in the section on requesting new authorities so the + link points to `<%= new_change_request_path %>` instead of `<%= + help_contact_path %>`. +* If your site runs over HTTPS, some cached attachments may still + contain links to Google Document Viewer with 'http', not 'https'. You + can clear the cached attachments after upgrade, and they will be + created with links that use the correct protocol. +* This release includes an update to the commonlib submodule - you + should be warned about this when running rails-post-deploy. + +# Version 0.16 + +## Highlighted features + +* Upgrade of the Rails framework to 3.2.16 +* Enabling the Rails asset pipeline for managing assets (more about the + asset pipeline at http://guides.rubyonrails.org/asset_pipeline.html). +* The all authorities csv download now uses less system resources +* Ruby 2.0 is now included in the matrix of versions we run continuous + integration tests against +* When using capistrano, the RAILS_ENV can now be explicitly set from + deploy.yml +* The front page and request pages once more use fragment caching backed + by memcached to speed up serving of slow parts of these pages +* The robots.txt file has been updated to allow crawling of response + attachment files (in original and HTML versions) +* The `themes:install` rake task is kinder to developers; it no longer + removes and reclones themes, destroying local changes, and it keeps + themes as git repositories. +* Social media elements (the blog, twitter feed) are only included if + the appropriate config variables (BLOG_FEED and TWITTER_USERNAME) have + been populated. +* Some fixes to the treatment of hyphenated/underscored locales so that + public body translations are consistently stored using the underscore + format of the locale (so 'he_IL', not 'he-IL'). +* The popup message elements for temporary notices and for letting users + know about other sites have been made consistent and now use simpler + styles. + +## Upgrade notes + +* You will need to update your theme to use the asset pipeline - notes + on how to do this are in doc/THEME-ASSETS-UPGRADE.md +* The syntax of the highlight and excerpt methods has changed, so if you + use these in your theme, you may see deprecation warnings until you + update them. More information at http://apidock.com/rails/v3.2.13/ActionView/Helpers/TextHelper/highlight + and + http://apidock.com/rails/v3.2.13/ActionView/Helpers/TextHelper/excerpt +* If you don't want to use fragment caching, you can turn it off in your + config file by setting `CACHE_FRAGMENTS` to `false`. +* If you use a locale with an underscore in it, you should double check + that the locale field of your `public_body_translations` table shows + the underscore version of the locale name. +* This release includes an update to the commonlib submodule - you + should be warned about this when running rails-post-deploy +* All code has been moved out of the deprecated plugin path + `vendor/plugins`. Once you are up and running under 0.16, you should + check that your xapian databases have all been copied to + `lib/acts_as_xapian/xapiandbs` (the code in + `config/initializers/acts_as_xapian` should do this), and then check + and remove any files under vendor/plugins so that you won't get + deprecation warnings about having Rails 2.3 style plugins (deprecation + warnings can result in incoming mail getting an auto reply under some + email configs). +* If you have any custom styles that rely on the absolute positioning + of the 'banner' and 'wrapper' elements, they may need to be updated. +* Cached HTML versions of attachments in cache/attachments_production/ + will have obsolete links to `/stylesheets/main.css` and + `/images/navimg/logo-trans-small.png`. You can resolve these by either + moving the cached attachments away and allowing them to be regenerated + on demand, or by symlinking `public/stylesheets/main.css` to + `public/assets/application.css` and + `public/images/navimg/logo-trans-small.png` to + `public/assets/navimg/logo-trans-small.png`. + +# Version 0.15 + +## Highlighted features + +* A new install script for setting up Alaveteli automatically on + a fresh Debian wheezy or Ubuntu precise server, and a + Vagrantfile so that it can be easily invoked by `vagrant up` +* Salutations in outgoing messages now can contain regular + expression special characters. +* The links to public bodies from the first letters of the + alphabet now work properly in when the letter would be + represented by multiple bytes in UTF-8. +* There are improvements to searching for public bodies and + when the "ask us to add one" message is shown. +* There is a fix for the + [long-standing error](https://github.com/mysociety/alaveteli/issues/555) + about duplicate Xapian job creation. +* A new rake task for importing large numbers of public bodies + from CSV files: `rake import:import_csv`. +* Various improvements to the public body statistics graphs, + thanks to feedback from the WDTK volunteers. +* The new_relic gem has been updated (Matthew Landauer) +* An example nginx config file for running Alaveteli behind + nginx: `config/nginx.conf.example`. +* There's now a simple script for switching between themes + (`script/switch-theme.rb`) for developers who have to work on + more than one jurisdiction's theme. + +# Version 0.14 + +## Highlighted features +* There is now an option to display a public body statistics page (currently not linked to from anywhere) showing bodies with the most requests, most successful requests, fewest successful requests, most overdue requests, and bodies that reply most frequently with "Not Held" - see Upgrade notes for how to turn this option on. (Mark Longair) +* Individual incoming and outgoing messages can be made hidden, or requester_only from the admin interface. +* Zip downloads now can be run in single-threaded instances, and use send_file rather than a redirect to serve up cached zip files. +* Starting to use factory_girl to generate model instances for use in specs - hopefully in the long term removing dependencies between specs, and allowing them to run faster once we can remove the loading of fixtures each time. +* Fix to allow public body list page to use current, not default locale, with optional fallback to default [issue #1000](https://github.com/mysociety/alaveteli/issues/1000) - see Upgrade notes for fallback option (Mark Longair) +* Fix to allow request titles composed of only unicode characters [issue #902](https://github.com/mysociety/alaveteli/issues/902) +* Fix for occasional errors caused by race conditions in xapian updates [issue #555](https://github.com/mysociety/alaveteli/issues/555) +* Diagnostic errors are now not shown for local requests, so that the user-facing error pages will be shown when running Alaveteli behind a proxy in production (Henare Degan) + +## Upgrade notes +* By default, Alaveteli will now serve up request zip files itself, which will occupy a Rails process until the file has been received. To pass these files off to Apache, and free up the Rails process, install the libapache2-mod-xsendfile package, and update your httpd.conf file with the new Sendfile clause at the end of config/httpd.conf-example). +* In your production install, from the Alaveteli directory (as the Alaveteli deploy user), run the following commands to remove the zip download directory from direct access by your webserver, and preserve any cached zip files: +`mkdir cache/zips/production/` +`mv cache/zips/download cache/zips/production/download` +`rm public/download` +* This release upgrades the assumed version of Ubuntu from lucid (10.04) to precise (12.04) +* This release upgrades rubygems in config/packages - version 1.8.15 is available from squeeze-backports on Debian or by default in Ubuntu precise. This upgrade may result in "invalid date format in specification:" errors - these should be fixable by manually deleting the gems specs that are being referenced in the error and re-running rails-post-deploy +* If you would like to have a public body statistics page (this will be publicly available), set the `PUBLIC_BODY_STATISTICS_PAGE` param in general.yml to `true`. You should also add a new cron job based on the one in config/crontab-example `https://github.com/mysociety/alaveteli/blob/rails-3-develop/config/crontab-example#L29` to update the public body stats each day. +* If you would like the public body list page to include bodies that have no translation in the current locale, but do have a translation in the default locale, add a `PUBLIC_BODY_LIST_FALLBACK_TO_DEFAULT_LOCALE` param set to `true` to your config/general.yml file. + + +# Version 0.13 +## Highlighted features + +* Fix for bug that resulted in some incorrect results when using search by request status [issue #460](https://github.com/mysociety/alaveteli/issues/460). You can view and fix requests with inconsistent state history using `rake temp:fix_bad_request_states` +* All status updates (whether by the request owner or another user) are now logged in the event history, for better audit) (Matthew Landauer) +* Fix for bug that dropped accented characters from URLs [issue #282](https://github.com/mysociety/alaveteli/issues/282) (zejn) +* A fix for a bug that produced binary mask errors when handling multibyte characters in responses [issue #991](https://github.com/mysociety/alaveteli/issues/991) +* Some locale fixes for locales with a dash in them [issue #998](https://github.com/mysociety/alaveteli/issues/998) and [issue #999](https://github.com/mysociety/alaveteli/issues/999). +* Some improvements in the labelling of defunct authorities (Matthew Somerville) +* The addition of a check on the status of the commonlib submodule to the rails-post-deploy script. + +## Upgrade notes +* Check out this version and run `rails-post-deploy` as usual. +* This release includes an update to the commonlib submodule - you should now be warned about this on running `rails-post-deploy`. You can update to the new version with `git submodule update`. +* After deploying, run `rake temp:fix_bad_request_states` to find and list requests that have an inconsistent history - run `rake temp:fix_bad_request_states DRYRUN=0` to fix them. + +# Version 0.12 +## Highlighted features +* Remove support for theme stylesheet inclusion via template (deprecated in version 0.5) +* Addition of a simple JSON API for querying the Ruby and Alaveteli version of an Alaveteli instance - made available at /version.json (Matthew Landauer) +* Users can now give more information when reporting a request as unsuitable (Matthew Landauer) +* The donation url presented to users when they report their request as successful or partially successful is now option and the url itself can be configured using the config param DONATION_URL +* Internal review request text is now translatable +* config/crontab.ugly is now config/crontab-example +* Search query highlighting should now work with non-ascii characters [issue #505](https://github.com/mysociety/alaveteli/issues/505) (Matthew Landauer) +* A bug that allowed people to sign up with email addresses with spaces in them has been fixed [issue #980](https://github.com/mysociety/alaveteli/issues/980). Any existing email addresses with spaces in them will cause problems e.g. when the cron scripts encounter them. You can fix them manually, or by running `rake temp:clean_up_emails_with_spaces` from `lib/tasks/temp.rake` +* [List of issues on github](https://github.com/mysociety/alaveteli/issues?milestone=30&state=closed) + +## Upgrade notes +* Check out this version and run `rails-post-deploy` as usual. +* Add a DONATION_URL to your config/general.yml file if you want to use your own donation URL. + +# Version 0.11 +## Highlighted features +* Upgrade of the Rails framework to version 3.1.12 (Henare Degan, Matthew Landauer, Mark Longair, Louise Crow) + +## Upgrade notes +* Manually remove vendor/rails-locales +* Themes created for 0.9 and below should be updated to work with Rails 3. See `THEMES-UPGRADE.md` for notes on upgrading your theme. You will need to manually remove your old theme directory before running `rails-post-deploy`. +* The `config/httpd.conf` has moved to `config/httpd.conf-example`, as it may need customization before deploying. It also has a new line setting RackEnv to production - copy this to your config/httpd.conf file. +* Alaveteli now uses the [mail gem](https://github.com/mikel/mail) rather than [tmail](https://github.com/mikel/tmail) to handle mail. If you're using Exim as your MTA, you'll need to use the setting `extract_addresses_remove_arguments = false` in your Exim conf (see INSTALL-exim4.md for details). This means it won't remove addresses specified with -t on command line from the mail recipient list. + +# Version 0.9 +## Highlighted features +* Consistent and more informative variable interpolation syntax in translated phrases. All of these phrases will now appear in the form "There are {{count}} people following this request", where some were previously in the form "There are %s people following this request". (Matthew Landauer) +* Replaces deprecated calls to with_locale on ActiveRecord classes in preparation for upgrade to Globalize3 (Matthew Landauer) +* Fixes a database deadlock bug caused by near-simultaneous incoming emails for the same info request (Mark Longair) + +## Upgrade notes +* Check out this version and run `rails-post-deploy` as usual. + + +# Version 0.8 +## Highlighted features +* Support for running the site over SSL/TLS only and corresponding removal of support for a proxied admin interface, including the deprecation of the main_url and admin_url helpers. +* Merging of the adminbootstrap theme into core Alaveteli, replacing the existing admin theme. (Matthew Landauer) +* Move to HTML 5 (Matthew Landauer) +* More consistent UI for links in the admin interface +* [Security] Upgrades the Rails version to 2.3.17 to get fixes for CVE-2013-0277, CVE-2013-0276 (Although core Alaveteli does not use serialize or attr_protected), upgrade JSON gem to get fix for CVE-2013-0269. +* A bugfix for Chrome's autofilling of signup fields (Vaughan Rouesnel) +* Improvements to the accessibility of the search boxes (Nathan Jenkins) +* Only one email sent when asking for admin attention to a request [issue #789](https://github.com/mysociety/alaveteli/pull/864) (Matthew Landauer) +* A number of XSS escaping fixes for Version 0.7 (Matthew Landauer) +* The emergency admin account can now be disabled + +## Upgrade notes +* Check out this version and run `rails-post-deploy` as usual. +* Remove adminbootstrap from the THEME_URLS or THEME_URL config variable, and remove vendor/plugins/adminbootstraptheme, and the softlink public/adminbootstraptheme. +* There is a new config variable FORCE_SSL, which defaults to true, meaning that Alaveteli will redirect all "http" requests to "https", set the Strict-Transport-Security header and flag all cookies as "secure". For more information about running your install over SSL/TLS, see the [install guide](https://github.com/mysociety/alaveteli/blob/develop/doc/INSTALL.md#set-up-production-web-server). If you don't want to run over SSL/TLS, add the config variable FORCE_SSL to your config/general.yml and set it to false. +* If you would like to disable the emergency user account, set DISABLE_EMERGENCY_USER to true in you config/general.yml + # Version 0.7 ## Highlighted features * [Security] Upgrades the Rails version from 2.3.15 to 2.3.16 to get fix for a critical security flaw in Rails (CVE-2013-0333). diff --git a/doc/INSTALL-exim4.md b/doc/INSTALL-exim4.md index e37da14ff..796fb295c 100644 --- a/doc/INSTALL-exim4.md +++ b/doc/INSTALL-exim4.md @@ -6,15 +6,18 @@ In `/etc/exim4/conf.d/main/04_alaveteli_options`: ALAVETELI_HOME=/path/to/alaveteli/software ALAVETELI_USER=www-data log_file_path=/var/log/exim4/exim-%slog-%D - MAIN_LOG_SELECTOR==+all -retry_defer + MAIN_LOG_SELECTOR==+all -retry_defer + extract_addresses_remove_arguments=false (The user ALAVETELI_USER should have write permissions on ALAVETELI_HOME). -Note that the name and location of the log files created by Exim must match +Note that the name and location of the log files created by Exim must match what the `load-mail-server-logs` script expects, hence the need for the extra `log_file_path` setting. And the `check-recent-requests-sent` scripts expects -the logs to contain the `from=<...>` envelope information, so we make the -logs more verbose with `log_selector`. +the logs to contain the `from=<...>` envelope information, so we make the +logs more verbose with `log_selector`. The ALAVETELI_USER may need to also +need to be added to the `trusted_users` list in your Exim config in order to +set the return path on outgoing mail, depending on your setup. In `/etc/exim4/conf.d/router/04_alaveteli`: @@ -33,7 +36,7 @@ In `/etc/exim4/conf.d/transport/04_alaveteli`: home_directory = ALAVETELI_HOME user = ALAVETELI_USER group = ALAVETELI_USER - + And, assuming you set `INCOMING_EMAIL_PREFIX` in your config at `config/general` to "foi+", create `config/aliases` with the following content: diff --git a/doc/INSTALL.md b/doc/INSTALL.md index 3a911cbc8..04cdb1352 100644 --- a/doc/INSTALL.md +++ b/doc/INSTALL.md @@ -1,4 +1,123 @@ -These instructions assume Debian Squeeze or Ubuntu 10.04 LTS. +# Installation Script and AMI + +The easiest options for installating Alaveteli for evaluation +are to use our install script or to use the AMI (Amazon Machine +Image) to create an instance on Amazon EC2. These options are +described below. If you would prefer to install the site +manually, please go to the Manual Installation section below. + +## Installing from an AMI (Amazon Machine Image) + +To help people try out Alaveteli, we have created an AMI (Amazon +Machine Image) with a basic installation of Alaveteli, which you +can use to create a running server on an Amazon EC2 instance. +This creates an instance that runs in development mode, so we +wouldn't recommend you use it for a production system without +changing the configuration. + +Unfortunately, Alaveteli will not run properly on a free Micro +instance due to the low amount of memory available on those +instances; you will need to use at least a Small instance, which +Amazon will charge for. + +The AMI can be found in the EU West (Ireland) region, with the +ID ami-8603f4f1 and name “Basic Alaveteli installation +2014-01-29”. You can launch an instance based on that AMI with +[this link](https://console.aws.amazon.com/ec2/home?region=eu-west-1#launchAmi=ami-8603f4f1). + +When you create an EC2 instance based on that AMI, make sure +that you choose Security Groups that allows at least inbound +HTTP, HTTPS, SSH and, if you want to test incoming mail as well, +SMTP. + +When your EC2 instance is launched, you will be able to log in +as the `ubuntu` user. This user can `sudo` freely to run +commands as root. However, the code is actually owned by (and +runs as) the `alaveteli` user. After creating the instance, you +may want to edit a configuration file to customize the site's +configuration. That configuration file is +`/var/www/alaveteli/alaveteli/config/general.yml`, which can be +edited with: + + ubuntu@ip-10-58-191-98:~$ sudo su - alaveteli + alaveteli@ip-10-58-191-98:~$ cd alaveteli + alaveteli@ip-10-58-191-98:~/alaveteli$ nano config/general.yml + +Then you should restart the Thin webserver with: + + alaveteli@ip-10-58-191-98:~/alaveteli$ logout + ubuntu@ip-10-58-191-98:~$ sudo /etc/init.d/alaveteli restart + +If you find the hostname of your EC2 instance from the AWS +console, you should then be able to see the site at +`http://your-ec2-hostname.eu-west-1.compute.amazonaws.com` + +If you have any problems or questions, please ask on the +[Alaveteli Google Group](https://groups.google.com/forum/#!forum/alaveteli-dev) +or [report an issue](https://github.com/mysociety/alaveteli/issues?state=open). + +## Installing with the Installation Script + +If you have a clean installation of Debian squeeze or Ubuntu +precise, you can use an install script in our commonlib +repository to set up a working instance of Alaveteli. This is +not suitable for production (it runs in development mode, for +example) but should set up a functional installation of the +site. + +**Warning: only use this script on a newly installed server – it +will make significant changes to your server’s setup, including +modifying your nginx setup, creating a user account, creating a +database, installing new packages etc.** + +To download the script, run the following command: + + curl -O https://raw.github.com/mysociety/commonlib/master/bin/install-site.sh + +If you run this script with `sh install-site.sh`, you'll see its +usage message: + + Usage: ./install-site.sh [--default] <SITE-NAME> <UNIX-USER> [HOST] + HOST is only optional if you are running this on an EC2 instance. + --default means to install as the default site for this server, + rather than a virtualhost for HOST. + +In this case `<SITE-NAME>` should be `alaveteli`. `<UNIX-USER>` +is the name of the Unix user that you want to own and run the +code. (This user will be created by the script.) + +The `HOST` parameter is a hostname for the server that will be +usable externally – a virtualhost for this name will be created +by the script, unless you specified the `--default` option. This +parameter is optional if you are on an EC2 instance, in which +case the hostname of that instance will be used. + +For example, if you wish to use a new user called `alaveteli` +and the hostname `alaveteli.127.0.0.1.xip.io`, creating a +virtualhost just for that hostname, you could download and run +the script with: + + sudo sh install-site.sh alaveteli alaveteli alaveteli.127.0.0.1.xip.io + +([xip.io](http://xip.io/) is a helpful domain for development.) + +Or, if you want to set this up as the default site on an EC2 +instance, you could download the script, make it executable and +then invoke it with: + + sudo ./install-site.sh --default alaveteli alaveteli + +When the script has finished, you should have a working copy of +the website, accessible via the hostname you supplied to the +script. + +If you have any problems or questions, please ask on the +[Alaveteli Google Group](https://groups.google.com/forum/#!forum/alaveteli-dev) +or [report an issue](https://github.com/mysociety/alaveteli/issues?state=open). + +# Manual Installation + +These instructions assume Debian Squeeze (64-bit) or Ubuntu 12.04 LTS (precise). [Install instructions for OS X](https://github.com/mysociety/alaveteli/wiki/OS-X-Quickstart) are under development. Debian Squeeze is the best supported deployment platform. @@ -9,12 +128,12 @@ As an aid to evaluation, there is an [Amazon AMI](https://github.com/mysociety/alaveteli/wiki/Alaveteli-ec2-ami) with all these steps configured. It is *not* production-ready. -# Get Alaveteli +## Get Alaveteli To start with, you may need to install git, e.g. with `sudo apt-get install git-core` -Next, get hold of the Alaveteli source code from github: +Next, get hold of the Alaveteli source code from github: git clone https://github.com/mysociety/alaveteli.git cd alaveteli @@ -25,19 +144,23 @@ master branch (which always contains the latest stable release): git checkout master -# Package pinning +## Package pinning -You need to configure [apt-pinning](http://wiki.debian.org/AptPreferences#Pinning-1) preferences in order to prevent packages being pulled from the debian testing distribution in preference to the stable distribution once you have added the testing repository as described below. +You need to configure [apt-pinning](http://wiki.debian.org/AptPreferences#Pinning-1) preferences in order to prevent packages being pulled from the debian wheezy distribution in preference to the stable distribution once you have added the wheezy repository as described below. -In order to configure apt-pinning and to keep most packages coming from the Debian stable repository while installing the ones required from testing and the mySociety repository you need to run the following commands: +In order to configure apt-pinning and to keep most packages coming from the Debian stable repository while installing the ones required from wheezy and the mySociety repository you need to run the following commands: echo "Package: *" >> /tmp/preferences - echo "Pin: release a=testing">> /tmp/preferences + echo "Pin: release a=squeeze-backports">> /tmp/preferences + echo "Pin-Priority: 200" >> /tmp/preferences + echo "" >> /tmp/preferences + echo "Package: *" >> /tmp/preferences + echo "Pin: release a=wheezy">> /tmp/preferences echo "Pin-Priority: 50" >> /tmp/preferences sudo cp /tmp/preferences /etc/apt/ rm /tmp/preferences - -# Install system dependencies + +## Install system dependencies These are packages that the software depends on: third-party software used to parse documents, host the site, etc. There are also packages @@ -48,7 +171,8 @@ If you are running Debian, add the following repositories to `/etc/apt/sources.list` and run `apt-get update`: deb http://debian.mysociety.org squeeze main - deb http://ftp.debian.org/debian/ testing main non-free contrib + deb http://ftp.debian.org/debian/ wheezy main non-free contrib + deb http://backports.debian.org/debian-backports squeeze-backports main contrib non-free The repositories above allow us to install the packages `wkhtmltopdf-static` and `bundler` using `apt`; so if you're running @@ -65,24 +189,16 @@ Some of the files also have a version number listed in config/packages - check that you have appropriate versions installed. Some also list "|" and offer a choice of packages. -# Install Ruby dependencies +## Install Ruby dependencies -Install rubygems 1.6.2 (we're not using the Debian package because we -need an older version; see "Troubleshooting" below for an -explanation): - - wget http://rubyforge.org/frs/download.php/74445/rubygems-1.6.2.tgz -O /tmp/rubygems-1.6.2.tgz - tar zxvf /tmp/rubygems-1.6.2.tgz -C /tmp/ - sudo ruby1.8 /tmp/rubygems-1.6.2/setup.rb - -To install Alaveteli's Ruby dependencies, we also need to install +To install Alaveteli's Ruby dependencies, we need to install bundler. In Debian, this is provided as a package (installed as part of the package install process above). You could also install it as a gem: sudo gem1.8 install bundler - -# Install mySociety libraries + +## Install mySociety libraries You will also want to install mySociety's common ruby libraries and the Rails code. Run: @@ -91,7 +207,7 @@ code. Run: to fetch the contents of the submodules. -## Packages customised by mySociety +### Packages customised by mySociety Debian users should add the mySociety debian archive to their `/etc/apt/sources.list` as described above. Doing this and following @@ -118,7 +234,7 @@ use the Debian package compiled by mySociety (see link in [issue 305](https://github.com/mysociety/alaveteli/issues/305)) -# Configure Database +## Configure Database There has been a little work done in trying to make the code work with other databases (e.g. SQLite), but the currently supported database is @@ -138,13 +254,14 @@ username and password of your postgres database. Make sure that the user specified in database.yml exists, and has full permissions on these databases. As they need the ability to turn off constraints whilst running the tests they also need to be a superuser. -(See http://dev.rubyonrails.org/ticket/9981) - +If you don't want your database user to be a superuser, you can add a line +`disable_constraints: false` to the test config in database.yml, as seen in database.yml-example + You can create a `foi` user from the command line, thus: # su - postgres $ createuser -s -P foi - + And you can create a database thus: $ createdb -T template0 -E SQL_ASCII -O foi foi_production @@ -157,24 +274,25 @@ data that may not be valid UTF (for example, data originating from various broken email clients that's not 8-bit clean), it's safer to be able to store *anything*, than reject data at runtime. -# Configure email +## Configure email You will need to set up an email server (MTA) to send and receive emails. Full configuration for an MTA is beyond the scope of this document, though we describe an example configuration for Exim in `INSTALL-exim4.md`. -Note that in development mode, mail is handled by default by mailcatcher -so that you can see the mails in a browser - see http://mailcatcher.me/ -for more details. +Note that in development mode, mail is handled by default by mailcatcher +so that you can see the mails in a browser - see http://mailcatcher.me/ +for more details. Start mailcatcher by running `bundle exec mailcatcher` +in your application directory. -## Minimal +### Minimal If you just want to get the tests to pass, you will at a minimum need to allow sending emails via a `sendmail` command (a requirement met, for example, with `sudo apt-get install exim4`). -## Detailed +### Detailed When an authority receives an email, the email's `reply-to` field is a magic address which is parsed and consumed by the Rails app. @@ -207,7 +325,7 @@ A well-configured installation of this code will separately have had Exim make a backup copy of the email in a separate mailbox, just in case. -# Set up configs +## Set up configs Copy `config/general.yml-example` to `config/general.yml` and edit to your taste. @@ -226,11 +344,11 @@ performance management system. By default, monitoring is switched off by the `agent_enabled: false` setting. See https://github.com/newrelic/rpm for instructions on switching on local and remote performance analysis. -# Deployment +## Deployment In the 'alaveteli' directory, run: - ./script/rails-post-deploy + script/rails-post-deploy (This will need execute privs so `chmod 755` if necessary.) This sets up directory structures, creates logs, installs/updates themes, runs @@ -245,16 +363,16 @@ If you want some dummy data to play with, you can try loading the fixtures that the test suite uses into your development database. You can do this with: - ./script/load-sample-data + script/load-sample-data Next we need to create the index for the search engine (Xapian): - ./script/rebuild-xapian-index + script/rebuild-xapian-index If this fails, the site should still mostly run, but it's a core component so you should really try to get this working. -# Run the Tests +## Run the Tests Make sure everything looks OK: @@ -266,7 +384,7 @@ workaround). You might be able to move on to the next step, depending on how serious they are, but ideally you should try to find out what's gone wrong. -## glibc bug workaround +### glibc bug workaround There's a [bug in glibc](http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=637239) @@ -277,11 +395,11 @@ it's not as of version `2.11.3-2`. Until it's fixed (e.g. `libc6 2.13-26` does work), you can get the tests to pass by setting `export LD_PRELOAD=/lib/libuuid.so.1`. -# Run the Server +## Run the Server Run the following to get the server running: - ./script/server --environment=development + bundle exec rails server --environment=development By default the server listens on all interfaces. You can restrict it to the localhost interface by adding ` --binding=127.0.0.1` @@ -289,7 +407,7 @@ localhost interface by adding ` --binding=127.0.0.1` The server should have told you the URL to access in your browser to see the site in action. -# Administrator privileges +## Administrator privileges The administrative interface is at the URL `/admin`. @@ -301,7 +419,7 @@ There is an emergency user account which can be accessed via `/admin?emergency=1`, using the credentials `ADMIN_USERNAME` and `ADMIN_PASSWORD`, which are set in `general.yml`. To bootstrap the first `super` level accounts, you will need to log in as the emergency -user. +user. You can disable the emergency user account by setting `DISABLE_EMERGENCY_USER` to `true` in `general.yml`. Users with the superuser role also have extra privileges in the website frontend, such as being able to categorise any request, being @@ -312,11 +430,11 @@ in the front end. It is possible completely to override the administrator authentication by setting `SKIP_ADMIN_AUTH` to `true` in `general.yml`. -# Cron jobs and init scripts +## Cron jobs and init scripts -`config/crontab.ugly` contains the cronjobs run on WhatDoTheyKnow. +`config/crontab-example` contains the cronjobs run on WhatDoTheyKnow. It's in a strange templating format they use in mySociety. mySociety -render the "ugly" file to reference absolute paths, and then drop it +render the example file to reference absolute paths, and then drop it in `/etc/cron.d/` on the server. The `ugly` format uses simple variable substitution. A variable looks @@ -328,7 +446,7 @@ like `!!(*= $this *)!!`. The variables are: port with a path to the directory where your Alaveteli software installation lives, e.g. `/var/www/` * `vhost_dir`: the entire path to the directory where the software is - served from. -- you should replace this with a path to the + served from. -- you should replace this with a path to the directory where your Alaveteli software installation lives, e.g. `/var/www/` * `vcspath`: the name of the alaveteli checkout, e.g. `alaveteli`. @@ -337,8 +455,20 @@ like `!!(*= $this *)!!`. The variables are: * `user`: the user that the software runs as * `site`: a string to identify your alaveteli instance -There is a dumb python script at `script/make-crontab` which you can -edit and run to do some basic substitution for you. +There is a rake task that will help to rewrite this file into +one that is useful to you, which can be invoked with: + + bundle exec rake config_files:convert_crontab \ + DEPLOY_USER=deploy \ + VHOST_DIR=/dir/above/alaveteli \ + VCSPATH=alaveteli \ + SITE=alaveteli \ + CRONTAB=config/crontab-example > crontab + +You should change the `DEPLOY_USER`, `VHOST_DIR`, `VCSPATH` and +`SITE` environment variables to match your server and +installation. You should also edit the resulting `crontab` file +to customize the `MAILTO` variable. One of the cron jobs refers to a script at `/etc/init.d/foi-alert-tracks`. This is an init script, a copy of @@ -361,14 +491,14 @@ discussion of where to find this program, and how you might replace it. This [one line script](https://gist.github.com/3741194) can install this program system-wide. -# Set up production web server +## Set up production web server It is not recommended to run the website using the default Rails web server. There are various recommendations here: http://rubyonrails.org/deploy -We usually use Passenger / mod_rails. The file at `conf/httpd.conf` -contains the WhatDoTheyKnow settings. At a minimum, you should +We usually use Passenger / mod_rails. The file at `conf/httpd.conf-example` +gives you an example config file for WhatDoTheyKnow. At a minimum, you should include the following in an Apache configuration file: PassengerResolveSymlinksInDocumentRoot on @@ -378,11 +508,39 @@ Under all but light loads, it is strongly recommended to run the server behind an http accelerator like Varnish. A sample varnish VCL is supplied in `../conf/varnish-alaveteli.vcl`. +It's strongly recommended that you run the site over SSL. (Set FORCE_SSL to true in +config/general.yml). For this you will need an SSL certificate for your domain and you will +need to configure an SSL terminator to sit in front of Varnish. If you're already using Apache +as a web server you could simply use Apache as the SSL terminator. A minimal configuration +would look something like this: + +<VirtualHost *:443> + ServerName www.yourdomain + + ProxyRequests Off + ProxyPreserveHost On + ProxyPass / http://localhost:80/ + ProxyPassReverse / http://localhost:80/ + RequestHeader set X-Forwarded-Proto 'https' + + SSLEngine on + SSLProtocol all -SSLv2 + SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM + + SSLCertificateFile /etc/apache2/ssl/ssl.crt + SSLCertificateKeyFile /etc/apache2/ssl/ssl.key + SSLCertificateChainFile /etc/apache2/ssl/sub.class2.server.ca.pem + SSLCACertificateFile /etc/apache2/ssl/ca.pem + SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown +</VirtualHost> + +Notice the line "RequestHeader" that sets the X-Forwarded-Proto header. This is important. This ultimately tells Rails that it's serving a page over https and so it knows to include that in any absolute urls it serves. + Some [production server best practice notes](https://github.com/mysociety/alaveteli/wiki/Production-Server-Best-Practices) are evolving on the wiki. -# Upgrading Alaveteli +## Upgrading Alaveteli The developer team policy is that the master branch in git should always contain the latest stable release. Therefore, in production, @@ -404,23 +562,23 @@ release. Failure to do so means that any new words added to the Alaveteli source code will appear in your website in English by default. If your translations didn't make it to the latest release, you will need to download the updated `app.po` for your locale from -Transifex and save it in the `locales/` folder. +Transifex and save it in the `locale/` folder. You should always run the script `scripts/rails-post-deploy` after each deployment. This runs any database migrations for you, plus various other things that can be automated for deployment. -# Troubleshooting +## Troubleshooting * **Incoming emails aren't appearing in my Alaveteli install** - + First, you need to check that your MTA is delivering relevant incoming emails to the `script/mailin` command. There are various ways of setting your MTA up to do this; we have documented one way of doing it in Exim at `doc/INSTALL-exim4.conf`, including a command you can use to check that the email routing is set up correctly. - + Second, you need to test that the mailin script itself is working correctly, by running it from the command line, First, find a valid "To" address for a request in your system. You can do this @@ -431,7 +589,7 @@ various other things that can be automated for deployment. Loading development environment (Rails 2.3.14) >> InfoRequest.find_by_url_title("why_do_you_have_such_a_fancy_dog").incoming_email => "request-101-50929748@localhost" - + Now take the source of a valid email (there are some sample emails in `spec/fixtures/files/`); edit the `To:` header to match this address; and then pipe it through the mailin script. A non-zero exit code @@ -447,7 +605,7 @@ various other things that can be automated for deployment. `CONTACT_EMAIL` (from your `general.yml` file). A common problem is for the user that the MTA runs as not to have write access to `files/raw_emails/`. - + * **Various tests fail with "*Your PostgreSQL connection does not support unescape_bytea. Try upgrading to pg 0.9.0 or later.*"** @@ -470,29 +628,16 @@ various other things that can be automated for deployment. Normally, the encoding should just work, but under some circumstances it appears that `elinks` ignores the parameters passed to it from Alaveteli. - + To force `elinks` always to treat input as UTF8, add the following to `/etc/elinks/elinks.conf`: - + set document.codepage.assume = "utf-8" set document.codepage.force_assumed = 1 - You should also check that your locale is set up correctly. See + You should also check that your locale is set up correctly. See [https://github.com/mysociety/alaveteli/issues/128#issuecomment-1814845](this issue followup) for further discussion. - -* **I'm getting lots of `SourceIndex.new(hash) is deprecated` errors when running the tests** - - The latest versions of rubygems contain a large number of noisy - deprecation warnings that you can't turn off individually. Rails - 2.x isn't under active development so isn't going to get fixed (in - the sense of using a non-deprecated API). So the only vaguely - sensible way to avoid this noisy output is to downgrade rubygems. - - For example, you might do this by uninstalling your - system-packaged rubygems, and then installing the latest rubygems - from source, and finally executing `sudo gem update --system - 1.6.2`. * **I'm seeing `rake: command not found` when running the post install script diff --git a/doc/THEME-ASSETS-UPGRADE.md b/doc/THEME-ASSETS-UPGRADE.md new file mode 100644 index 000000000..12c1a60d1 --- /dev/null +++ b/doc/THEME-ASSETS-UPGRADE.md @@ -0,0 +1,76 @@ +This document has notes on switching your Alaveteli theme to use +the Rails asset pipeline. + +Firstly, add the following to your `lib/alavetelitheme.rb`, in +order to add the subdirectories of your theme's `assets` +directory to `config.assets.path`: + + # Prepend the asset directories in this theme to the asset path: + ['stylesheets', 'images', 'javascripts'].each do |asset_type| + theme_asset_path = File.join(File.dirname(__FILE__), + '..', + 'assets', + asset_type) + Rails.application.config.assets.paths.unshift theme_asset_path + end + +In the root of your theme, create these directories: + + assets + \ images + \ stylesheets + \ javascripts + +i.e. `assets` is at the same level as `lib` and `locale-theme`. + +Move any image files from `public/images` to `assets/images`. +Now change any references to those images with a literal `<img>` +tag to use `image_tag` instead. For example, instead of: + + <img src="/images/helpmeinvestigate.png" alt="" class="rss"> + +... you should have: + + image_tag('helpmeinvestigate.png', :alt => "", :class => "rss") + +If you have a favicon.ico file in your theme's `public` directory, you should move it to `assets/images` as well. + +You should similarly move your stylesheets into +`assets/stylesheets`. If a stylesheet refers to images, you +should rename the `.css` file to `.css.scss`, and change `url` +to the sass-rails `image-url` helper. e.g. instead of: + + background-image: url(../images/mysociety.png); + +... you should have: + + background-image: image-url('mysociety.png'); + +If your only stylesheet is called `custom.css`, as in the +example theme, you shouldn't need to make any other changes to +the CSS. If you have added additional stylesheets +(e.g. `extra.css`), then you'll need to both: + +1. add them to +`lib/views/general/_stylesheet_includes.html.erb`, for example +with: + + <%= stylesheet_link_tag "extra" %> + +2. add the following in `lib/alavetelitheme.rb`: + + config.assets.precompile.push 'extra.css' + +Any custom Javascript should be moved to `assets/javascripts` in +your theme directory, and, simlarly to the additional CSS, it +should be mentioned in `lib/alavetelitheme.rb` with: + + config.assets.precompile.push 'fancy-effects.js' + +You should be left with nothing in the `public` directory after +making these changes, except possibly custom error pages. + +Remove the code that symlinks the theme 'public' directory to a +subdirectory of the main application's 'public' directory from +install.rb. Also remove the code from uninstall.rb that removes that +symlink. The asset pipeline will handle making assets available. diff --git a/doc/THEMES-UPGRADE.md b/doc/THEMES-UPGRADE.md new file mode 100644 index 000000000..457274d7a --- /dev/null +++ b/doc/THEMES-UPGRADE.md @@ -0,0 +1,101 @@ +This file contains some notes on changing your Alaveteli theme for the +upgrade to Rails 3, in version 0.11 of Alaveteli. These were written +by Henare Degan, with some additions by Mark Longair. + +# Alaveteli Theme Upgrade Checks + +## RAILS_ROOT/RAILS_ENV + +[Example](https://github.com/henare/adminbootstraptheme/commit/857e33c9b0bc577024b476404aec4f9749f65a0b) + +Check your theme for instances of: + +* `RAILS_ROOT` and replace it with `Rails.root` +* `RAILS_ENV` and replace it with `Rails.env` + +Note that `Rails.root` is a `Pathname`, so you can replace, for +example: + + File.join(RAILS_ROOT, 'public', 'alavetelitheme') + +... with: + + Rails.root.join('public', 'alavetelitheme') + +## Dispatcher + +[Example](https://github.com/henare/adminbootstraptheme/commit/fba2d6b7dfdc26a25fdc1596bfe120270dd4cd0d) + +This... + +```ruby +require 'dispatcher' +Dispatcher.to_prepare do +``` + +should be replaced with this... + +```ruby +Rails.configuration.to_prepare do +```` + +## Routes + +[Example](https://github.com/henare/adminbootstraptheme/commit/87f1991dafb09401f9b17f642a94382d5a47a713) + +You need to upgrade your custom routes to the new Rails syntax. + +## list_public_bodies_default removed + +[Example](https://github.com/openaustralia/alavetelitheme/commit/5927877af996a1afb1a23a950f0d012b52c36f83) + +The list_public_bodies_default helper has been removed from Alaveteli + +## Patching mailer templates has changed + +[Example](https://github.com/openaustralia/alavetelitheme/commit/ffb5242973a0b2acc4981c25659fcb752b92eb97) + +In `lib/patch_mailer_paths.rb` change `ActionMailer::Base.view_paths.unshift File.join(File.dirname(__FILE__), "views")` to `ActionMailer::Base.prepend_view_path File.join(File.dirname(__FILE__), "views")` + +There's also `ActionMailer::Base.append_view_path` for replacing `ActionMailer::Base.view_paths <<`. + +## Rename view templates + +[Example](https://github.com/henare/adminbootstraptheme/commit/b616b636c283ae6cf696a6af1fa481f371baf2b6) + +Rename view templates from `filename.rhtml` to `filename.html.erb`. + +Run this in the root of your theme directory: + + for r in $(find lib/views -name '*.rhtml'); do echo git mv $r ${r%.rhtml}.html.erb; done + +[GOTCHA!](https://github.com/openaustralia/alavetelitheme/commit/65e775488822367d981bb15ab2cbcf1fce842cc2) +One exception is mailer templates, these should be renamed to +`filename.text.erb` as we only use text emails. + +## The Configuration class has been renamed + +[Example](https://github.com/openaustralia/alavetelitheme/commit/db6cca4650216c6f85acffaea380727344f0f740) + +Due to a naming conflict, `Configuration` has been renamed to `AlaveteliConfiguration`. + +You may have this in your theme for things like `Configuration::site_name`, just change it to `AlaveteliConfiguration::site_name` + +## request.request_uri is deprecated + +[Example](https://github.com/openaustralia/alavetelitheme/commit/d670eeebfb049e1dc83fdb36a628f7722d2ad419) + +Replace instances of `request.request_uri` with `request.fullpath` + +## content-inserting <% %> block helpers are deprecated + +[Example](https://github.com/openaustralia/alavetelitheme/commit/a4b13bbd76249b3a28e2a755cede20dd9db30140) + +The Rails 3 releases notes are [irritatingly +imprecise](http://edgeguides.rubyonrails.org/3_0_release_notes.html#helpers-with-blocks) +about which such helpers have changed. You can find some candidates +with this `git grep` command: + + git grep -E '<%[^=].*(_for|_tag|link_to)\b' + +(Ignore `content_for` in those results.) diff --git a/doc/THEMES.md b/doc/THEMES.md index c5e4a3eee..8a4828a99 100644 --- a/doc/THEMES.md +++ b/doc/THEMES.md @@ -7,7 +7,7 @@ inserting your own logo and colour scheme. You may also want to tweak the different states that a request can go through. You'll also want to edit the categories that public bodies can appear in (i.e. the groupings on the left hand side of the -"[View authorities](http://www.whatdotheyknow.com/body/list/all)" page +"[View authorities](https://www.whatdotheyknow.com/body/list/all)" page on WhatDoTheyKnow. There may also be other things you want to customise; drop a line on @@ -36,15 +36,15 @@ places: This document is about what you can do in a theme. -By default, the sample theme ("alavetelitheme") has already been -installed. See the setting `THEME_URLS` in `general.yml` for an +By default, the sample theme ("alavetelitheme") has already been +installed. See the setting `THEME_URLS` in `general.yml` for an explanation. You can also install the sample theme by hand, by running: - ./script/plugin install git://github.com/mysociety/alavetelitheme.git - -The sample theme contains examples for nearly everything you might + bundle exec rails plugin install git://github.com/mysociety/alavetelitheme.git + +The sample theme contains examples for nearly everything you might want to customise. You should probably make a copy, rename it, and use that as the basis for your own theme. @@ -61,13 +61,13 @@ changing much in the core theme. The ideal would be if you are able to rebrand the site by only changing the CSS. You will also need to add custom help pages, as described below. -# Branding the site +# Branding the site The core templates that comprise the layout and user interface of an -Alaveteli site live in `app/views/`. They are use Rails' ERB syntax. +Alaveteli site live in `app/views/`. They use Rails' ERB syntax. For example, the template for the home page lives at -`app/views/general/frontpage.rhtml`, and the template for the "about -us" page is at `app/views/help/about.rhtml`. +`app/views/general/frontpage.html.erb`, and the template for the "about +us" page is at `app/views/help/about.html.erb`. Obviously, you *could* edit those core files directly, but this would be a Bad Idea, because you would find it increasingly hard to do @@ -90,7 +90,7 @@ the main Rails app -- see `alavetelitheme/install.rb` to see how this happens. The partial at -`alavetelitheme/lib/views/general/_before_head_end.rhtml` includes the +`alavetelitheme/lib/views/general/_before_head_end.html.erb` includes the custom CSS in your theme's stylesheet folder (by convention, in `alavetelitheme/public/stylesheets/`), with: @@ -137,20 +137,20 @@ The latter must have one method: When you've added your extra states, you also need to create the following files in your theme: -* `lib/views/general/_custom_state_descriptions.rhtml`: Descriptions +* `lib/views/general/_custom_state_descriptions.html.erb`: Descriptions of your new states, suitable for displaying to end users -* `lib/views/general/_custom_state_transitions_complete.rhtml`: +* `lib/views/general/_custom_state_transitions_complete.html.erb`: Descriptions for any new states that you might characterise as 'completion' states, for displaying on the categorisation form that we ask requestors to fill out -* `lib/views/general/_custom_state_transitions_pending.rhtml`: As +* `lib/views/general/_custom_state_transitions_pending.html.erb`: As above, but for new states you might characterise as 'pending' states. You can see examples of these customisations in [this commit](https://github.com/sebbacon/informatazyrtare-theme/commit/2b240491237bd72415990399904361ce9bfa431d) for the Kosovan version of Alaveteli, Informata Zyrtare (ignore the -file `lib/views/general/_custom_state_transitions.rhtml`, which is +file `lib/views/general/_custom_state_transitions.html.erb`, which is unused). # Adding new pages in the navigation @@ -158,7 +158,7 @@ unused). `alavetelitheme/lib/config/custom-routes.rb` allows you to extend the base routes in Alaveteli. The example in `alavetelitheme` adds an extra help page. You can also use this to override the behaviour of specific pages if -necessary. +necessary. # Adding or overriding models and controllers diff --git a/doc/TRANSLATE.md b/doc/TRANSLATE.md index f8b4adbcb..aef2cfdc9 100644 --- a/doc/TRANSLATE.md +++ b/doc/TRANSLATE.md @@ -46,6 +46,10 @@ application developers about the context in which the text appears in the application - these comments will appear under the 'Details' tab for the text in Transifex. +Some texts will have placeholders in them to indicate that another bit of text generated by Alaveteli will be inserted into them when they're displayed. They will look like `some text with a {{placeholder}}`. For these texts, don't translate the placeholder. It needs to stay exactly the same for the text to be inserted properly. + +Similarly, some texts will have some code in angle brackets in them to turn bits of the text into links, and other small bits of HTML formatting. e.g. `please <a href=\"{{url}}\">send it to us</a>`. Again, don't edit the bits in angle brackets, preserve them in your translation, just edit the text around them. + Some bits of text are in the form of two bits of text separated by a `|` character e.g. `IncomingMessage|Subject`. These represent attribute names, so `IncomingMessage|Subject` is the subject attribute of an incoming message on @@ -75,7 +79,7 @@ must: language, using `bundle exec rake gettext:store_model_attributes`, followed by `bundle exec rake gettext:find` - * careful of including msgids from themes in `vendor/plugin`; + * careful of including msgids from themes in `lib/themes`; you might want to move them out of the way before running the above commands * this updates the PO template, but also merges it with the |