Handle unparsed email contents as binary.

I think I was wrong in a83b379fd2d676172855825d0592937b234371e2 in
assuming that all email gets properly encoded for transfer. Looking
at the mail gem load method
https://github.com/mikel/mail/blob/b159e0a542962fdd5e292a48cfffa560d7cf412e/lib/mail/mail.rb#L175a,
it reads raw email content from a file in binary mode. So this commit
makes both reading and writing the raw_email a binary mode operation
and adds a data_as_text method for displaying the data in the admin
interface that coerces it to valid utf-8.

1 files changed, 17 insertions, 6 deletions

diff --git a/app/models/raw_email.rb b/app/models/raw_email.rb
index 21a53f493..175868b7a 100644
--- a/app/models/raw_email.rb
+++ b/app/models/raw_email.rb
@@ -39,16 +39,27 @@ class RawEmail < ActiveRecord::Base
     end
 
     def data=(d)
-        if !File.exists?(self.directory)
-            FileUtils.mkdir_p self.directory
+        FileUtils.mkdir_p(directory) unless File.exists?(directory)
+        File.atomic_write(filepath) do |file|
+            file.binmode
+            file.write(d)
         end
-        File.atomic_write(self.filepath) { |file|
-            file.write d
-        }
     end
 
     def data
-        File.open(self.filepath, "r").read
+        File.open(filepath, "rb").read
+    end
+
+    def data_as_text
+        text = data
+        if text.respond_to?(:encoding)
+            text = text.encode("UTF-8", :invalid => :replace,
+                                        :undef => :replace,
+                                        :replace => "")
+        else
+            text = Iconv.conv('UTF-8//IGNORE', 'UTF-8', text)
+        end
+        text
     end
 
     def destroy_file_representation!