diff options
| author | Mark Longair <mhl@pobox.com> | 2013-05-15 14:52:07 +0100 |
|---|---|---|
| committer | Mark Longair <mhl@pobox.com> | 2013-05-16 09:06:27 +0100 |
| commit | 95cf55aad1f0985d28c28beb61e122dc7465b039 (patch) | |
| tree | d7bef74e2b14f3c28418b2961a6464447e97e6e2 /config | |
| parent | 004e683db2cc3abe51225d071db993f81f3a6742 (diff) | |
Add functions for converting from arbitrary text data to UTF-8
Throughout the codebase it is simplest and most consistent
if we could assume that all text/* attachments are represented
by UTF-8 strings, and this was largely true with the TMail
backend which ensured that all returned text parts were in
UTF-8. We have to change the replacement Mail-backed to
similarly attempt to convert text parts to UTF-8. This commit
introduces two functions which are useful for this.
The normalize_string_to_utf8 function will try various
encodings, either suggested or guessed (with charlock_holmes)
to convert the passed string to UTF-8, and if it can't find a
suitable encoding will throw an exception.
Unfortunately, the current behaviour of the site is that
uninterpretable text/* attachments are still passed around and
mangled to UTF-8 just before display. To mimic this it's also
useful to have the convert_string_to_utf8_or_binary function,
which tries to convert the string to UTF-8 with
normalize_string_to_utf8, but if that's not possible just
returns the original string. (In Ruby 1.9, encoding will be
set to UTF-8 or ASCII-8BIT appropriately.)
Diffstat (limited to 'config')
| -rw-r--r-- | config/initializers/alaveteli.rb | 1 | ||||
| -rw-r--r-- | config/packages | 3 |
2 files changed, 3 insertions, 1 deletions
diff --git a/config/initializers/alaveteli.rb b/config/initializers/alaveteli.rb index 35d486837..455ff467d 100644 --- a/config/initializers/alaveteli.rb +++ b/config/initializers/alaveteli.rb @@ -59,3 +59,4 @@ require 'quiet_opener.rb' require 'mail_handler' require 'public_body_categories' require 'ability' +require 'normalize_string' diff --git a/config/packages b/config/packages index db51e5bdd..fc67cda6b 100644 --- a/config/packages +++ b/config/packages @@ -36,4 +36,5 @@ rake (>= 0.9.2.2) build-essential bundler sqlite3 -libsqlite3-dev
\ No newline at end of file +libsqlite3-dev +libicu-dev |