Make efforts to ensure that we're usually dealing with UTF-8 strings

One of these changes is to make sure that the Mail backend,
like the TMail backend it replaces, will return text parts
encoded in UTF-8 if possible.

The other change is to ensure that when text attachments are
reloaded from disk, we attempt to convert them to UTF-8.

1 files changed, 6 insertions, 1 deletions

diff --git a/app/models/foi_attachment.rb b/app/models/foi_attachment.rb
index fcde379e0..0340f2b83 100644
--- a/app/models/foi_attachment.rb
+++ b/app/models/foi_attachment.rb
@@ -71,7 +71,12 @@ class FoiAttachment < ActiveRecord::Base
             tries = 0
             delay = 1
             begin
-                @cached_body = File.open(self.filepath, "rb" ).read
+                binary_data = File.open(self.filepath, "rb" ).read
+                if self.content_type =~ /^text/
+                    @cached_body = convert_string_to_utf8_or_binary(binary_data, 'UTF-8')
+                else
+                    @cached_body = binary_data
+                end
             rescue Errno::ENOENT
                 # we've lost our cached attachments for some reason.  Reparse them.
                 if tries > BODY_MAX_TRIES