diff options
29 files changed, 720 insertions, 60 deletions
diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 000000000..1d085cacc --- /dev/null +++ b/.dockerignore @@ -0,0 +1 @@ +** diff --git a/.github/ISSUE_TEMPLATE/Bug_report.md b/.github/ISSUE_TEMPLATE/Bug_report.md new file mode 100644 index 000000000..1bf32922a --- /dev/null +++ b/.github/ISSUE_TEMPLATE/Bug_report.md @@ -0,0 +1,40 @@ +--- +name: Bug report +about: Create a report to help us improve + +--- + +Note if the bug is about the mobile native app (not the website, whether that +is desktop or mobile), please leave it in the +[https://github.com/mysociety/fixmystreet-mobile/issues](fixmystreet-mobile +repository). + +**Describe the bug** +A clear and concise description of what the bug is. + +**To Reproduce** +Steps to reproduce the behavior: +1. Go to '...' +2. Click on '....' +3. Scroll down to '....' +4. See error + +**Expected behavior** +A clear and concise description of what you expected to happen. + +**Screenshots** +If applicable, add screenshots to help explain your problem. + +**Desktop (please complete the following information):** + - OS: [e.g. Windows] + - Browser [e.g. chrome, safari] + - Version [e.g. 22] + +**Smartphone (please complete the following information):** + - Device: [e.g. iPhone6] + - OS: [e.g. iOS8.1] + - Browser [e.g. stock browser, safari] + - Version [e.g. 22] + +**Additional context** +Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/Feature_request.md b/.github/ISSUE_TEMPLATE/Feature_request.md new file mode 100644 index 000000000..bc58e757a --- /dev/null +++ b/.github/ISSUE_TEMPLATE/Feature_request.md @@ -0,0 +1,17 @@ +--- +name: Feature request +about: Suggest an idea for this project + +--- + +**Is your feature request related to a problem? Please describe.** +A clear and concise description of what the problem is; e.g. I'm always frustrated when [...] + +**Describe the solution you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Additional context** +Add any other context or screenshots about the feature request here. diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 000000000..e8a7c408e --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,15 @@ +Please check the following: + +- [ ] Whether this PR should include changes to any documentation, or the FAQ; +- [ ] All cobrand-specific commits start their commit message with the cobrand in square brackets; +- [ ] Is new functionality tested? CodeCov will warn you about the diff coverage, but won’t complain about e.g. new files; +- [ ] Are the changes tested for accessibility? +- [ ] Have you updated the changelog? If this is not necessary, put square brackets around this: skip changelog + +Please check the contributing docs, and describe your pull request here. +Screenshots or GIF animations (using e.g. LICEcap) may be helpful. + +Please include any issues that are fixed, using "fixes" or "closes" so that +they are auto-closed when the PR is merged. + +Thanks for contributing! diff --git a/.gitignore b/.gitignore index 84a1d776a..9c6297b3d 100644 --- a/.gitignore +++ b/.gitignore @@ -36,6 +36,7 @@ blib/ inc/ _Inline/ *.pyc +docker-compose.override.yml # International /fixmystreet-international diff --git a/CHANGELOG.md b/CHANGELOG.md index d5e382503..10e0776b3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,8 @@ - Add script to list/diff template changes in core that might need applying to a cobrand. - Move away from FastCGI in sample conf/sysvinit config. + - Customised Vagrant box available, with an override option. + - Add Dockerfile and example Docker Compose setup. * v2.4 (6th September 2018) - Security diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 000000000..1bf885238 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,29 @@ +FROM jgoerzen/debian-base-standard:stretch +MAINTAINER sysadmin@mysociety.org + +ARG DEBIAN_FRONTEND=noninteractive + +RUN apt-get -qq update \ + && apt-get -qq -y install ca-certificates \ + && wget -O install-site.sh --no-verbose https://raw.githubusercontent.com/mysociety/commonlib/master/bin/install-site.sh \ + && chmod +x /install-site.sh + +RUN /install-site.sh --docker fixmystreet fms 127.0.0.1.xip.io \ + && apt-get purge -y --auto-remove \ + make \ + g++ \ + libexpat1-dev \ + libssl-dev \ + zlib1g-dev \ + postgresql-server-dev-all \ + exim4-daemon-light \ + && apt-get -y clean \ + && rm -fr /var/lib/apt/lists/* \ + && rm -fr /home/fms/.cpanm/* + +RUN cd /var/www/fixmystreet/fixmystreet \ + && git show master:bin/docker.preinit > /usr/local/preinit/99-fixmystreet \ + && chmod +x /usr/local/preinit/99-fixmystreet + +EXPOSE 9000 +CMD ["/usr/local/bin/boot-debian-base"] diff --git a/Vagrantfile b/Vagrantfile index c2ad471a2..c8647b270 100644 --- a/Vagrantfile +++ b/Vagrantfile @@ -1,34 +1,35 @@ # -*- mode: ruby -*- # vi: set ft=ruby : -# Vagrantfile API/syntax version. Don't touch unless you know what you're doing! -VAGRANTFILE_API_VERSION = "2" - -Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| - # All Vagrant configuration is done here. The most common configuration - # options are documented and commented below. For a complete reference, - # please see the online documentation at vagrantup.com. - - # Every Vagrant virtual environment requires a box to build off of. - config.vm.box = "ubuntu/xenial64" - - # Create a forwarded port mapping which allows access to a specific port - # within the machine from a port on the host machine. In the example below, - # accessing "localhost:8080" will access port 80 on the guest machine. - config.vm.network :forwarded_port, guest: 3000, host: 3000 - # And 3001 for the Cypress test server - config.vm.network :forwarded_port, guest: 3001, host: 3001 +require 'getoptlong' +opts = GetoptLong.new( + [ '--base-box', GetoptLong::OPTIONAL_ARGUMENT ] +) - config.vm.synced_folder ".", "/home/vagrant/fixmystreet", :owner => "vagrant", :group => "vagrant" +baseBox='mysociety/fixmystreet' +opts.each do |opt, arg| + case opt + when '--base-box' + baseBox=arg + end +end - config.vm.provision :shell, :inline => <<-EOS +$setup = <<-EOS + BASEBOX=$1 # To prevent "dpkg-preconfigure: unable to re-open stdin: No such file or directory" warnings export DEBIAN_FRONTEND=noninteractive # Make sure git submodules are checked out! echo "Checking submodules exist/up to date" - apt-get -qq install -y git >/dev/null + if [ ! "$BASEBOX" = "mysociety/fixmystreet" ]; then + # Don't assume git is installed. + apt-get -qq install -y git >/dev/null + fi cd fixmystreet git submodule --quiet update --init --recursive --rebase + if [ "$BASEBOX" = "mysociety/fixmystreet" ]; then + mount -o bind /usr/share/fixmystreet/local /home/vagrant/fixmystreet/local + chown -R vagrant:vagrant /home/vagrant/fixmystreet/local + fi cd commonlib git config core.worktree "../../../commonlib" echo "gitdir: ../.git/modules/commonlib" > .git @@ -56,7 +57,32 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| echo "Please see above for any errors, and do ask on our mailing list for help." exit 1 fi - EOS +EOS + +# Vagrantfile API/syntax version. Don't touch unless you know what you're doing! +VAGRANTFILE_API_VERSION = "2" + +Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| + # All Vagrant configuration is done here. The most common configuration + # options are documented and commented below. For a complete reference, + # please see the online documentation at vagrantup.com. + + # Every Vagrant virtual environment requires a box to build off of. + config.vm.box = "#{baseBox}" + + # Create a forwarded port mapping which allows access to a specific port + # within the machine from a port on the host machine. In the example below, + # accessing "localhost:8080" will access port 80 on the guest machine. + config.vm.network :forwarded_port, guest: 3000, host: 3000 + # And 3001 for the Cypress test server + config.vm.network :forwarded_port, guest: 3001, host: 3001 + + config.vm.synced_folder ".", "/home/vagrant/fixmystreet", :owner => "vagrant", :group => "vagrant" + + config.vm.provision "shell" do |s| + s.inline = $setup + s.args = "#{baseBox}" + end # Create a private network, which allows host-only access to the machine # using a specific IP. diff --git a/bin/docker-cobrand b/bin/docker-cobrand new file mode 100755 index 000000000..05c02c1ce --- /dev/null +++ b/bin/docker-cobrand @@ -0,0 +1,16 @@ +#!/bin/bash + +set -e +shopt -s nullglob + +LN_FLAGS="-s -f -v" +cobrand="/var/www/fixmystreet/cobrand" +FMS="/var/www/fixmystreet/fixmystreet" + +PATHS=(perllib/FixMyStreet/Cobrand templates/web templates/email web/cobrands conf) + +for path in "${PATHS[@]}"; do + for c in $cobrand/$path/*; do + ln $LN_FLAGS $c $FMS/$path + done +done diff --git a/bin/docker.preinit b/bin/docker.preinit new file mode 100644 index 000000000..459e89de2 --- /dev/null +++ b/bin/docker.preinit @@ -0,0 +1,45 @@ +#!/bin/sh + +# Things to do before starting FixMyStreet in Docker + +# Make sure that the Postgres environment is up and running. +echo "Testing connection to ${FMS_DB_HOST}." +while ! pg_isready -h $FMS_DB_HOST >/dev/null 2>&1 ; do + echo "Still waiting for ${FMS_DB_HOST}..." + sleep 1 +done +echo "Done." + +# If there's a password for the postgres user, set it up for root and see if we need +# to create an FMS user. This is intended for use when using a dedicated local postgres +# container. If this variable doesn't exist, we're going to assume that the FMS user +# has been created already so the stuff below will work. +if [ -n "$POSTGRES_PASSWORD" ]; then + echo "${FMS_DB_HOST}:*:*:postgres:${POSTGRES_PASSWORD}" > /root/.pgpass + chmod 0600 /root/.pgpass + psql -h $FMS_DB_HOST -U postgres postgres -c "create user \"${FMS_DB_USER}\" with CREATEDB password '${FMS_DB_PASS}'" || true +fi + +# Set up a .pgpass for the FMS user. Note that we're assuming the same name for +# both the local shell account and the DB user. +su ${FMS_DB_USER} -c "echo \"${FMS_DB_HOST}:*:*:${FMS_DB_USER}:${FMS_DB_PASS}\" > /home/${FMS_DB_USER}/.pgpass" +chmod 0600 /home/${FMS_DB_USER}/.pgpass + +# If the FMS database doesn't exist, try to create it. +if ! su $FMS_DB_USER -c "psql -h $FMS_DB_HOST -U $FMS_DB_USER -l | egrep \"^ *${FMS_DB_NAME} *\|\" > /dev/null" ; then + su $FMS_DB_USER -c "createdb -h $FMS_DB_HOST -U $FMS_DB_USER --owner \"$FMS_DB_USER\" \"$FMS_DB_NAME\"" +fi + +# Slot in cobrand, if one is present +su $FMS_DB_USER -c "${FMS_ROOT}/bin/docker-cobrand" + +# Ensure things are up to date - schema, CSS, etc +su $FMS_DB_USER -c "${FMS_ROOT}/script/update" + +# Update reports +su $FMS_DB_USER -c "${FMS_ROOT}/bin/update-all-reports" + +# If the right environment variables are present, set up a FMS superuser account. +if [ -n "$SUPERUSER_PASSWORD" ] && [ -n "$SUPERUSER_EMAIL" ]; then + su $FMS_DB_USER -c "${FMS_ROOT}/bin/createsuperuser $SUPERUSER_EMAIL $SUPERUSER_PASSWORD" +fi diff --git a/bin/install-as-user b/bin/install-as-user index e42401758..7e2a0bd74 100755 --- a/bin/install-as-user +++ b/bin/install-as-user @@ -25,6 +25,8 @@ misuse() { } [ -z "$DEVELOPMENT_INSTALL" ] && misuse DEVELOPMENT_INSTALL +[ -z "$DOCKER" ] && misuse DOCKER +[ -z "$INSTALL_DB" ] && misuse INSTALL_DB DB_NAME="fixmystreet" @@ -92,19 +94,20 @@ then fi echo $DONE_MSG -# Create the database if it doesn't exist: -echo -n "Setting up database... " -if ! psql -l | egrep "^ *$DB_NAME *\|" > /dev/null -then - createdb --owner "$UNIX_USER" "$DB_NAME" - echo 'CREATE LANGUAGE plpgsql;' | psql -U "$UNIX_USER" "$DB_NAME" || true +if [ $INSTALL_DB = true ]; then + # Create the database if it doesn't exist: + echo -n "Setting up database... " + if ! psql -l | egrep "^ *$DB_NAME *\|" > /dev/null + then + createdb --owner "$UNIX_USER" "$DB_NAME" + echo 'CREATE LANGUAGE plpgsql;' | psql -U "$UNIX_USER" "$DB_NAME" || true + fi + bin/update-schema --commit + bin/update-all-reports + echo $DONE_MSG fi -bin/update-schema --commit -echo $DONE_MSG # Generate po and mo files (these invocations taken from Kagee's script): echo "Creating locale .mo files" commonlib/bin/gettext-makemo FixMyStreet echo $DONE_MSG - -bin/update-all-reports diff --git a/bin/site-specific-install.sh b/bin/site-specific-install.sh index a92f92ef5..4d7f05f1e 100644 --- a/bin/site-specific-install.sh +++ b/bin/site-specific-install.sh @@ -26,27 +26,41 @@ misuse() { [ -z "$DISTRIBUTION" ] && misuse DISTRIBUTION [ -z "$VERSION" ] && misuse VERSION [ -z "$DEVELOPMENT_INSTALL" ] && misuse DEVELOPMENT_INSTALL +[ -z "$DOCKER" ] && misuse DOCKER +[ -z "$INSTALL_DB" ] && misuse INSTALL_DB +[ -z "$INSTALL_POSTFIX" ] && misuse INSTALL_POSTFIX add_locale cy_GB add_locale nb_NO add_locale de_CH -install_postfix +if [ $INSTALL_POSTFIX = true ]; then + install_postfix +fi if [ ! "$DEVELOPMENT_INSTALL" = true ]; then - install_nginx - add_website_to_nginx + if [ ! "$DOCKER" = true ]; then + install_nginx + add_website_to_nginx + fi # Check out the current released version su -l -c "cd '$REPOSITORY' && git checkout '$VERSION' && git submodule update" "$UNIX_USER" fi +# Create a log directoryfor Docker builds - this is normally done above. +if [ $DOCKER = true ]; then + make_log_directory +fi + install_website_packages su -l -c "touch '$DIRECTORY/admin-htpasswd'" "$UNIX_USER" -add_postgresql_user +if [ $INSTALL_DB = true ]; then + add_postgresql_user +fi -export DEVELOPMENT_INSTALL +export DEVELOPMENT_INSTALL DOCKER INSTALL_DB su -c "$REPOSITORY/bin/install-as-user '$UNIX_USER' '$HOST' '$DIRECTORY'" "$UNIX_USER" if [ ! "$DEVELOPMENT_INSTALL" = true ]; then @@ -61,7 +75,7 @@ then overwrite_rc_local fi -if [ ! "$DEVELOPMENT_INSTALL" = true ]; then +if [ ! "$DEVELOPMENT_INSTALL" = true ] && [ ! "$DOCKER" = true ]; then # Tell the user what to do next: echo Installation complete - you should now be able to view the site at: diff --git a/conf/general.yml-docker b/conf/general.yml-docker new file mode 100644 index 000000000..1e5cf0e4c --- /dev/null +++ b/conf/general.yml-docker @@ -0,0 +1,233 @@ +# general.yml-example: +# This file contains example values for the "general.yml" config file. It +# contains various configuration parameters, in YAML syntax. +# +# For more information about this file, and general documentation on setting up +# the FixMyStreet platform, please see https://fixmystreet.org/ +# +# Specific details about this file can be found at +# https://fixmystreet.org/customising/config/ +# +# To start, you can copy this file to one called "general.yml" in the same +# directory. Or have multiple config files and use a symlink to change between +# them. Using the install script or AMI will have created a general.yml file +# for you. + +# PostgreSQL database details for FixMyStreet +FMS_DB_HOST: 'postgres.svc' +FMS_DB_PORT: '5432' +FMS_DB_NAME: 'fixmystreet' +FMS_DB_USER: 'fms' +FMS_DB_PASS: 'fms' + +# The base URL of the install. +BASE_URL: 'http://localhost:8000' +# Use the below if you're using the Catalyst development server +# BASE_URL: 'http://localhost:3000' +SECURE_PROXY_SSL_HEADER: '' +# If you're behind a proxy, set this to a two-element list containing the +# trusted HTTP header and the required value. For example: +# SECURE_PROXY_SSL_HEADER: [ 'X-Forwarded-Proto', 'https' ] + +# This is so the fixmystreet container knows it is running +# behind an nginx container (not on the same host) +using_frontend_proxy: 1 + +# Email domain used for emails, and contact name/email for admin use. +EMAIL_DOMAIN: '127.0.0.1.xip.io' +CONTACT_EMAIL: 'help@127.0.0.1.xip.io' +CONTACT_NAME: 'FixMyStreet' +# Address used for emails you don't expect a reply to (e.g. confirmation +# emails); can be same as CONTACT_EMAIL above, of course, if you don't have a +# special address. +DO_NOT_REPLY_EMAIL: 'help@127.0.0.1.xip.io' + +# Whether this is a development site or not. It will mean e.g. templates/ +# CSS modified times aren't cached +STAGING_SITE: 1 +# Developers may want a staging site to act differently from a live site. +# Possible flags include: +# - send_reports: Normally, a staging site will route all reports to the +# reporter, to guard against sending fake reports to live places. Set +# this to 1 if you want a staging site to route reports as normal. +# - skip_checks: Manual testing of multiple cobrands can be made easier by +# skipping some checks they have in them, if this variable is set. +# - enable_appcache: Whether the appcache should be active. NB: Only affects +# superuser sessions. +STAGING_FLAGS: + send_reports: 0 + skip_checks: 0 + enable_appcache: 0 + +# What to use as front page/alert example places placeholder +# Defaults to High Street, Main Street +EXAMPLE_PLACES: [ 'High Street', 'Main Street' ] + +# An array of languages for the site, in the following format: +# <language code>,<pretty name>,<locale> +LANGUAGES: + - 'en-gb,English,en_GB' + +# If you're running an installation that is being used in a different time zone +# from the server, you can set the time zone here (standard time zone string) +TIME_ZONE: "" + +# File locations for uploaded photos and cached geocoding results. +# Absolute paths, or relative to the project's root directory +UPLOAD_DIR: '../upload/' +GEO_CACHE: '../cache/' + +# Location of MapIt, to map points to administrative areas, and what types of +# area from it you want to use. If left blank, a default area will be used +# everywhere (a URL needs to be given for non-web things, like sending of +# reports, to function). +# If using the Catalyst development server, set to +# MAPIT_URL: 'http://localhost:3000/fakemapit/' +# MAPIT_TYPES: [ 'ZZZ' ] +# In the UK, you probably want, to cover all councils: +# MAPIT_URL: 'http://mapit.mysociety.org/' +# MAPIT_TYPES: [ 'DIS', 'LBO', 'MTD', 'UTA', 'CTY', 'COI', 'LGD' ] +# But perhaps MAPIT_TYPES: [ 'WMC' ] if you want to report on a per-constituency basis? +# If our global MapIt (using OpenStreetMap data) contains boundaries you can use: +# MAPIT_URL: 'http://global.mapit.mysociety.org/' +# And then specify whichever type code have the boundaries you want: +# MAPIT_TYPES: [ 'O06' ] +MAPIT_URL: '' +MAPIT_TYPES: [ 'ZZZ' ] + +# If the MapIt you're using in MAPIT_URL requires an API key +# (as https://mapit.mysociety.org/ does, for example), specify it here. +# MAPIT_API_KEY: '' + +# If you are using global MapIt, you might want to restrict FixMyStreet usage +# to only one or more areas, rather than all O04, for example. Provide MapIt +# IDs here in a list that FixMyStreet should recognise, along with the MapIt +# generation those IDs are in. +# MAPIT_ID_WHITELIST: [ 240838, 246176, 246733 ] +# MAPIT_GENERATION: 2 +MAPIT_ID_WHITELIST: [] +MAPIT_GENERATION: 0 + +# If your MapIt has the concept of council wards (subareas of councils, where +# people can sign up for alerts, but not report things), then you can give the +# MapIt type codes for them here. +# You can probably leave this blank, but in the UK we use something like: +# MAPIT_TYPES_CHILDREN: [ 'DIW', 'LBW', 'MTW', 'UTE', 'UTW', 'CED', 'COP', 'LGW' ] +MAPIT_TYPES_CHILDREN: '' + +# Which service to use to look up results from e.g. front page searches. +# Possible choices are Google, Bing, or OSM (blank will use OSM by default). +GEOCODER: '' + +# If you wish to use Google Maps or Bing Maps Geocoding, get and put the +# relevant key here. +GOOGLE_MAPS_API_KEY: '' +BING_MAPS_API_KEY: '' + +# This provdes parameters that are included in geocoding requests, to hopefully +# return more useful results. The options vary depending which geocoder you use, +# or you can specify all for if you change geocoder. For the default OSM, you can use: +# GEOCODING_DISAMBIGUATION: +# bounds: [ <min lat>, <min lon>, <max lat>, <max lon> ] +# country: <country code to restrict results to> +# town: <string added to geocoding requests if not already there> +# +# If using Bing, you can use town and bounds, plus any of: +# centre: "<lat>,<lon>" +# bing_culture: <culture code, see http://msdn.microsoft.com/en-us/library/hh441729.aspx> +# bing_country: <country name, only accept results that match this> +# +# If using Google, you can use bounds, plus: +# google_country: <.ccTLD to restrict results to> +# lang: <language for results> +# +GEOCODING_DISAMBIGUATION: '' + +# The type of map you want to use. If left blank, the default is OpenStreetMap. +# Other options are currently pretty UK specific, including "FMS" for UK +# FixMyStreet. +MAP_TYPE: 'OSM' + +# FixMyStreet uses a templating cobrand system to provide different looks for +# different installations. In the common case, if your templates are in the +# templates/web/moon directory and CSS in web/cobrands/moon, you just specify: +# ALLOWED_COBRANDS: +# - moon +# If you wish to use multiple cobrands, specify them in a list, optionally with +# hostname-matching regular expressions if the name of the cobrand is not +# enough. For example: +# ALLOWED_COBRANDS: +# - moon +# - venus +# Any hostname with 'moon' in it will use the moon cobrand, any with 'venus' +# the venus cobrand (any other the Default cobrand). Whereas: +# ALLOWED_COBRANDS: +# - moon: 'orbital' +# - venus +# Any hostname with 'orbital' in it will use the moon cobrand. +# This also allows development servers to map to different cobrands if needed, +# using DNS subdomains for example. +ALLOWED_COBRANDS: + - cobrand1 + - cobrand2: 'hostname_substring2' + - cobrand3 + +# This is used in e.g. "offensive report" emails to provide a link directly to +# the admin interface. Defaults to BASE_URL with "/admin" on the end. +ADMIN_BASE_URL: '' + +# How many items are returned in the GeoRSS and Open311 feeds by default +RSS_LIMIT: '20' +OPEN311_LIMIT: 1000 +# How many reports to show per page on the All Reports pages +ALL_REPORTS_PER_PAGE: 100 + +# Social networking +FACEBOOK_APP_ID: '' +FACEBOOK_APP_SECRET: '' +TWITTER_KEY: '' +TWITTER_SECRET: '' + +# If you wish to send email through a SMTP server elsewhere, change these +# variables. SMTP_TYPE should be one of '', 'ssl' or 'tls'. SMTP_PORT will +# default to 587 (tls), 465 (ssl), or 25. +SMTP_SMARTHOST: 'localhost' +SMTP_TYPE: '' +SMTP_PORT: '' +SMTP_USERNAME: '' +SMTP_PASSWORD: '' + +# Gaze is a world-wide service for population density lookups. You can leave +# this as is. +GAZE_URL: 'https://gaze.mysociety.org/gaze' + +# Memcached host +# This can be safely left out and will default to '127.0.0.1' even if not present. +MEMCACHED_HOST: 'memcached.svc' + +# Should problem reports link to the council summary pages? +AREA_LINKS_FROM_PROBLEMS: '0' + +# used to override the STAGING SERVER safety mechanism in send-reports +TESTING_COUNCILS: '' + +# if you're using Message Manager, include the URL here (see https://github.com/mysociety/message-manager/) +MESSAGE_MANAGER_URL: '' + +# If you want to use SMS login or 'report as' with just a phone number, you'll +# need to set the site's two-digit ISO 3166 country code (e.g. GB) here. +PHONE_COUNTRY: '' + +# If you enable login via SMS authentication, you'll need a twilio account +SMS_AUTHENTICATION: 0 +TWILIO_ACCOUNT_SID: '' +TWILIO_AUTH_TOKEN: '' +TWILIO_FROM_PARAMETER: '' +TWILIO_MESSAGING_SERVICE_SID: '' + +# If you want to hide all pages from non-logged-in users, set this to 1. +LOGIN_REQUIRED: 0 + +# If you want to stop new users from registering, set this to 1. +# NB: This also disables all Facebook/Twitter logins. +SIGNUPS_DISABLED: 0 diff --git a/conf/nginx.conf-docker b/conf/nginx.conf-docker new file mode 100644 index 000000000..e50ce82d2 --- /dev/null +++ b/conf/nginx.conf-docker @@ -0,0 +1,56 @@ +# An example configuration for running FixMyStreet under nginx. You +# will also need to set up the FixMyStreet Catalyst FastCGI backend. +# An example sysvinit script to help with this is shown given in the file +# sysvinit-catalyst-fastcgi.example in this directory. +# +# See our installation help at https://fixmystreet.org/ + +server { + + listen 80 default_server; + root /var/www/fixmystreet/fixmystreet/web; + error_page 503 /down.html; + + # Make sure that Javascript and CSS are compressed. (HTML is + # already compressed under the default configuration of the nginx + # package.) + + gzip on; + gzip_disable "msie6"; + gzip_types application/javascript application/x-javascript text/css; + + client_max_body_size 10m; + + # These rewrite rules are ported from the Apache configuration in + # conf/httpd.conf + + rewrite ^/rss/council/([0-9]+)$ /rss/reports/$1 permanent; + rewrite ^/report$ /reports permanent; + rewrite '^/{/rss/(.*)}$' /rss/$1 permanent; + rewrite '^/reports/{/rss/(.*)}$' /rss/$1 permanent; + rewrite ^/alerts/?$ /alert permanent; + + location /mapit { + proxy_pass https://mapit.mysociety.org/; + proxy_set_header X-Real-IP $remote_addr; + } + + location / { + if (-f $document_root/down.html) { + return 503; + } + try_files $uri @catalyst; + } + + location /down.html { + internal; + } + + location @catalyst { + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header Host $host:8000; + proxy_redirect off; + proxy_pass http://fixmystreet.svc:9000; + } +} diff --git a/conf/packages.docker b/conf/packages.docker new file mode 100644 index 000000000..ed1b46611 --- /dev/null +++ b/conf/packages.docker @@ -0,0 +1,15 @@ +make +g++ +jhead +liblocale-gettext-perl +perl +perlmagick +libmath-bigint-gmp-perl +gettext +postgresql-server-dev-all +postgresql-client +gnuplot +ttf-bitstream-vera +libexpat1-dev +libssl-dev +zlib1g-dev diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 000000000..80f0b56f1 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,58 @@ +version: '3' + +services: + nginx: + image: nginx:1.15.3 + depends_on: + - fixmystreet + ports: + - "8000:80" + volumes: + - ./conf/nginx.conf-docker:/etc/nginx/conf.d/default.conf + networks: + default: + aliases: + - nginx.svc + fixmystreet: + image: fixmystreet/fixmystreet:stable + tty: true + depends_on: + - postgres + - memcached + volumes: + - ./conf/general.yml-docker:/var/www/fixmystreet/fixmystreet/conf/general.yml + environment: + POSTGRES_PASSWORD: 'password' + FMS_DB_HOST: 'postgres.svc' + FMS_DB_PASS: 'fms' + FMS_DB_NAME: 'fixmystreet' + FMS_DB_USER: 'fms' + FMS_ROOT: '/var/www/fixmystreet/fixmystreet' + SUPERUSER_EMAIL: 'superuser@example.org' + SUPERUSER_PASSWORD: '5up3r53cr3t' + networks: + default: + aliases: + - fixmystreet.svc + postgres: + image: fixmystreet/postgres:9.6-en_GB + restart: always + volumes: + - fixmystreet-pgdata:/var/lib/postgresql/data/pgdata + environment: + POSTGRES_PASSWORD: 'password' + PGDATA: '/var/lib/postgresql/data/pgdata' + networks: + default: + aliases: + - postgres.svc + memcached: + image: memcached:1.5.10 + restart: always + networks: + default: + aliases: + - memcached.svc + +volumes: + fixmystreet-pgdata: diff --git a/docs/README.md b/docs/README.md index d1b4897d0..9f4398ba3 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,9 +1,9 @@ fixmystreet.org =============== -The gh-pages branch is [fixmystreet.org](https://fixmystreet.org), the -Jekyll-based static site running on GitHub Pages that is the documentation for -setting up / running the FixMyStreet platform. +This directory is [fixmystreet.org](https://fixmystreet.org), the Jekyll-based +static site running on GitHub Pages that is the documentation for setting up / +running the FixMyStreet platform. ## Installation @@ -12,8 +12,8 @@ The site is built by Jekyll. We manage our Ruby gem dependencies via already have it. Then you need to… ``` -git clone --recursive -b gh-pages https://github.com/mysociety/fixmystreet fixmystreet-pages -cd fixmystreet-pages +git clone --recursive https://github.com/mysociety/fixmystreet +cd fixmystreet/docs bundle install --deployment ``` diff --git a/docs/_layouts/page.html b/docs/_layouts/page.html index 79097b5a9..11ec206ec 100644 --- a/docs/_layouts/page.html +++ b/docs/_layouts/page.html @@ -56,9 +56,10 @@ layout: default <li><h2>Installing</h2> <ul> <li><a href="{{ "/install/manual-install/" | relative_url }}">Manually</a> + <li><a href="{{ "/install/docker/" | relative_url }}">Docker</a></li> + <li><a href="{{ "/install/ami/" | relative_url }}">AMI for EC2</a></li> <li><a href="{{ "/install/install-script/" | relative_url }}">Install script</a></li> <li><a href="{{ "/install/vagrant/" | relative_url }}">Vagrant</a></li> - <li><a href="{{ "/install/ami/" | relative_url }}">AMI for EC2</a></li> <li><a href="{{ "/install/troubleshooting/" | relative_url }}">Troubleshooting</a></li> </ul> </li> diff --git a/docs/customising/boundaries.md b/docs/customising/boundaries.md index d48a472aa..fc1873e9e 100644 --- a/docs/customising/boundaries.md +++ b/docs/customising/boundaries.md @@ -129,7 +129,7 @@ to a very specific area. </dt> <dd> <p> - This is the default setup of a new FixMyStreet installation (if you used the + This is the default setup of a new FixMyStreet installation (if you used e.g. the <a href="{{ "/install/install-script" | relative_url }}">installation script</a> or the <a href="{{ "/install/ami" | relative_url }}">AMI install</a>). </p> diff --git a/docs/directory_structure.md b/docs/directory_structure.md index 5b9e8077d..a8b8955d5 100644 --- a/docs/directory_structure.md +++ b/docs/directory_structure.md @@ -112,7 +112,7 @@ you're doing is [customising your installation](/customising). <p> These are technical notes. This is in addition to the <a href="https://fixmystreet.org/">core documentation</a> — which you are reading now — - which is actually stored in the git repository in the <code>gh-pages</code> branch, and published + which is actually stored in the git repository in the <code>docs</code> directory, and published as GitHub pages. </p> </dd> diff --git a/docs/glossary.md b/docs/glossary.md index 47728c7b5..58397345b 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -354,9 +354,9 @@ technical information, see it would contain your own private config settings, such as your database password). Instead, there is an example file, <code>cong/general.yml-example</code> you can - copy and edit. If you install using the + copy and edit. If you install using an automated method such as the <a href="{{ "/install/install-script/" | relative_url }}">installation script</a> - or the <a href="{{ "/install/ami/" | relative_url }}">AMI</a>, this file will + or <a href="{{ "/install/docker/" | relative_url }}">Docker</a>, this file will automatically be created for you. </p> <div class="more-info"> diff --git a/docs/how-it-works.md b/docs/how-it-works.md index 778563710..ea313de6b 100644 --- a/docs/how-it-works.md +++ b/docs/how-it-works.md @@ -68,10 +68,7 @@ If you want to get FixMyStreet up and running, this is what you need to do: <h3 id="install_the_software">2. Install the software</h3> </dt> <dd> - <ul> - <li><a href="{{ "/install/install-script" | relative_url }}">on your own server</a></li> - <li><a href="{{ "/install/ami" | relative_url }}">on an Amazon Web Services EC2 server</a></li> - </ul> + <p>See the <a href="{{ "/install/" | relative_url }}">various ways to install</a> the software.</p> </dd> <dt> <h3 id="get_the_data_for_the_areas_you_want_to_cover">3. Get the data for the areas you want to cover</h3> diff --git a/docs/install/ami.md b/docs/install/ami.md index 783c23727..5f6df3af4 100644 --- a/docs/install/ami.md +++ b/docs/install/ami.md @@ -12,7 +12,7 @@ title: AMI for EC2 EC2 instance. </p> -Note that this is just one of [four ways to install FixMyStreet]({{ "/install/" | relative_url }}). +Note that this is just one of [many ways to install FixMyStreet]({{ "/install/" | relative_url }}). ## Installing on Amazon's Web Services diff --git a/docs/install/docker.md b/docs/install/docker.md new file mode 100644 index 000000000..3c716a5da --- /dev/null +++ b/docs/install/docker.md @@ -0,0 +1,91 @@ +--- +layout: page +title: Docker +--- + +# FixMyStreet with Docker + +<p class="lead"> + You can use Docker and Docker Compose to get up and running quickly + with FixMyStreet. +</p> + +This is just one of [many ways to install FixMyStreet]({{ "/install/" | relative_url }}). + +## Public images + +As well as providing a `Dockerfile` which you could use as the basis of your own +customised build, we provide public [images on Docker Hub](https://hub.docker.com/u/fixmystreet/) +with a full FixMyStreet installation for each of our tagged releases. + +## Docker Compose + +If you have Docker and Docker Compose installed, then the following should +set up a working FixMyStreet installation, with containers for the application, +database, memcached and webserver: + + docker-compose up + +You can then layer your own [cobrand-specific code](/customising/) +on top, update the configuration, or log in and make changes. + +A superuser is automatically created, with email `superuser@example.org` +and password given in `docker-compose.yml`. + +This basic installation uses the default cobrand, with a +(deliberately) rather garish colour scheme. + +## Adding your own cobrand + +If you want to map your own cobrand data into the Docker container, have a +repository/directory that contains the following (all items optional): + + my-cobrand-repo/ + templates/web/(cobrand)/ + templates/email/(cobrand)/ + perllib/FixMyStreet/Cobrand/(CoBrand.pm) + web/cobrands/(cobrand)/ + +Create a docker-compose.override.yml file containing: + + version: '3' + + services: + fixmystreet: + volumes: + - /path/to/your/general.yml:/var/www/fixmystreet/fixmystreet/conf/general.yml + - /path/to/my-cobrand-repo:/var/www/fixmystreet/cobrand + +Now if you run `docker-compose up` it should automatically include that cobrand +within the running container. + +This is a new facility, so please do feed back your thoughts. + +## Database configuration + +The example Docker Compose environment includes a [slightly customised Postgres container](https://github.com/mysociety/public-builds/tree/master/docker/postgres) +based on [the official image](https://hub.docker.com/_/postgres/) and localised for `en_GB`. + +This will be configured the first time it is started and its data stored in a +Docker volume for persistence. The password for the `postgres` user should be set +in the `POSTGRES_PASSWORD` environment variable and made available to both the +database and application containers; along with the various `FMS_DB_*` environment +variables this will be used to ensure the correct users, permissions and databases +are created when the container starts for the first time. + +### Using an external database + +If you wish to host the database in an external service you can do so by updating +the various `FMS_DB_*` environment variables used by the application container and +in `general.yml`. You should not provide a `POSTGRES_PASSWORD` variable to the +application container in this case. + +The application container will attempt to create the database if it doesn't already exist, so +you can either provide the user with the `CREATEDB` privilege or simply provide +an empty database and the application container will load the schema when it starts +for the first time. + +## Installation complete... now customise + +You should then proceed +to [customise your installation](/customising/). diff --git a/docs/install/index.md b/docs/install/index.md index ebd4a8d7e..22abb263d 100644 --- a/docs/install/index.md +++ b/docs/install/index.md @@ -6,13 +6,13 @@ title: Installing # Installing FixMyStreet Platform <p class="lead"> - There are several options for installing the FixMyStreet platform. Unless - you're confident deploying web applications, we recommend you use the - installation script or the AMI for Amazon EC2. + There are several options for installing the FixMyStreet platform, + including Docker, an AMI for Amazon EC2 or an installation script. </p> -## Four ways to install +## Ways to install +* [Use Docker]({{ "/install/docker/" | relative_url }}) * [Use an install script for Debian or Ubuntu servers]({{ "/install/install-script/" | relative_url }}) * [Use a FixMyStreet AMI for Amazon EC2]({{ "/install/ami/" | relative_url }}) * [Vagrant installation]({{ "/install/vagrant/" | relative_url }}), for development diff --git a/docs/install/install-script.md b/docs/install/install-script.md index ba2c2bef8..124434b34 100644 --- a/docs/install/install-script.md +++ b/docs/install/install-script.md @@ -11,7 +11,7 @@ title: Install script FixMyStreet on your server. </p> -Note that this is just one of [four ways to install FixMyStreet]({{ "/install/" | relative_url }}). +Note that this is just one of [many ways to install FixMyStreet]({{ "/install/" | relative_url }}). ## Warning: installation changes your setup! diff --git a/docs/install/manual-install.md b/docs/install/manual-install.md index 45522fd4e..8acccdddf 100644 --- a/docs/install/manual-install.md +++ b/docs/install/manual-install.md @@ -11,7 +11,7 @@ title: Installing but the other installation options may be easier:</p> Note that this is just one of -[four ways to install FixMyStreet]({{ "/install/" | relative_url }}) +[many ways to install FixMyStreet]({{ "/install/" | relative_url }}) (the other ways are easier!). diff --git a/docs/install/troubleshooting.md b/docs/install/troubleshooting.md index 2b49cb616..b006d18a4 100644 --- a/docs/install/troubleshooting.md +++ b/docs/install/troubleshooting.md @@ -6,7 +6,7 @@ title: Installation troubleshooting # Installation troubleshooting <p class="lead"> - If you've installed FixMyStreet using the + If you've installed FixMyStreet using an automated method such as the <a href="{{ "/install/install-script" | relative_url }}">installation script</a> or the <a href="{{ "/install/ami" | relative_url }}">AMI</a>, you should be good to go. diff --git a/docs/install/vagrant.md b/docs/install/vagrant.md index 0b9cd8812..98f846644 100644 --- a/docs/install/vagrant.md +++ b/docs/install/vagrant.md @@ -12,7 +12,7 @@ We bundle an example Vagrantfile in the repository, which runs the <a href="{{ "/install/install-script/" | relative_url }}">install script</a> for you. </p> -Note that this is just one of [four ways to install FixMyStreet]({{ "/install/" | relative_url }}). +Note that this is just one of [many ways to install FixMyStreet]({{ "/install/" | relative_url }}). <div class="attention-box warning"> Vagrant is only suitable for use as a |