Handle unparsed email contents as binary.

I think I was wrong in a83b379fd2d676172855825d0592937b234371e2 in
assuming that all email gets properly encoded for transfer. Looking
at the mail gem load method
https://github.com/mikel/mail/blob/b159e0a542962fdd5e292a48cfffa560d7cf412e/lib/mail/mail.rb#L175a,
it reads raw email content from a file in binary mode. So this commit
makes both reading and writing the raw_email a binary mode operation
and adds a data_as_text method for displaying the data in the admin
interface that coerces it to valid utf-8.

1 files changed, 17 insertions, 2 deletions

diff --git a/app/models/raw_email.rb b/app/models/raw_email.rb
index 3b466cb81..907d3c7a0 100644
--- a/app/models/raw_email.rb
+++ b/app/models/raw_email.rb
@@ -39,11 +39,26 @@ class RawEmail < ActiveRecord::Base
 
     def data=(d)
         FileUtils.mkdir_p(directory) unless File.exists?(directory)
-        File.atomic_write(filepath) { |file| file.write(d) }
+        File.atomic_write(filepath) do |file|
+            file.binmode
+            file.write(d)
+        end
     end
 
     def data
-        File.open(filepath, "r").read
+        File.open(filepath, "rb").read
+    end
+
+    def data_as_text
+        text = data
+        if text.respond_to?(:encoding)
+            text = text.encode("UTF-8", :invalid => :replace,
+                                        :undef => :replace,
+                                        :replace => "")
+        else
+            text = Iconv.conv('UTF-8//IGNORE', 'UTF-8', text)
+        end
+        text
     end
 
     def destroy_file_representation!