| Commit message (Collapse) | Author | Age | Lines |
|---|
| |\
| |
| |
| | |
ssh://git.mysociety.org/data/git/public/alaveteli into release/0.11
|
| | |\
| | |
| | |
| | |
| | | |
Merge in some updates to the 0.11 release, discovered on further
testing on a development server with the full data set.
|
| |\ \ \
| |/ /
|/| | |
|
| | |\|
| | |
| | |
| | | |
ssh://git.mysociety.org/data/git/public/alaveteli into rails-3-develop
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
We believe that this customization is no longer needed, and
in any case would need to be rewritten for will_paginate 3.0.4.
If any pagination bugs do arise, a test should be added that
reproduces that problem, since currently the tests pass without
this extension.
|
| | |\ \
| | |/
| |/| |
|
| | |/
| |
| |
| | |
script directory, it's more important to cd to the app directory in order to pick up the bundler gemfile. Fixes #964.
|
| |\| |
|
| | |\
| | |
| | |
| | | |
ssh://git.mysociety.org/data/git/public/alaveteli into rails-3-develop
|
| | | |
| | |
| | |
| | |
| | |
| | | |
render_for_text no longer exists in Rails 3.
Fixes #955
|
| | |\ \ |
|
| | |/ /
| | |
| | |
| | | |
the main part in order to look for uuencoded text, make sure that we're getting that main part from the reparsed attachments, and not getting an obsolete attachment. Fixes #958.
|
| | |/ |
|
| |/ |
|
| |\
| |
| |
| |
| |
| |
| | |
ssh://git.mysociety.org/data/git/public/alaveteli into rails-3-develop
Conflicts:
Gemfile.lock
|
| | |\
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This merge brings in a number of tests and fixes for the handling
of mail under Mail / Rails 3 instead of TMail / Rails 2.
Conflicts:
config/initializers/alaveteli.rb
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
Otherwise redirects will be cached, and since headers and the status
code aren't stored, a non-redirecting redirect page will be returned
in the future, but with 200. It's easiest to only cache the 200
responses.
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | | |
Previously the behaviour would have been the same, since we weren't
checking the filename so strictly.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This commit changes 'hello.txt' to 'hello-world.txt' in the
incoming-request-two-same-name.email fixture. The reasoning
for this change is that if there are no special characters in
the filename then filename and display_filename will be the
same and the tests won't pick up any confusion between the
two.
The test requests to :get_attachment and
:get_attachment_as_html should get the display_filename rather
than filename.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
If the display_filename of the attachment found from the
URL part number doesn't match the passed in display_filename
then the email may have been reparsed, causing a reordering.
In that case, look to see if there is another attachment that
uniquely matches that filename, and, if so, return that other
attachment. If no matching uniquely matching filename is
found, redirect to the incoming message, rather than
returning a 404.
|
| | | | |
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
Handling of outlook-packed attachments would fail from
rake tasks or in the console without requiring 'mapi/msg'
and 'mapi/convert' beforehand. Instead, require them in
the source file where they're actually used.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The behaviour of the TMail backend's 'to' and 'cc' methods
where there was a malformed To: or Cc: line was to return
nil, whereas Mail returns a version of the string anyway.
We'd have to change quite a lot of code to deal with an
extra possible class of returned objects, so it's simplest
for the moment to monkey-patch Mail::Message's 'to' and 'cc'
methods to restore the old behaviour.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The Mail gem deals with multipart messages that look as if
they should have 1 part but are missing the final MIME boundary,
by make the parts list empty and setting part.body to the
text of the email. Rather than throwing an exception in this
case, we just pretend that part is text/plain and return it, so
that the page doesn't error and we still have a chance of some
useful text being displayed.
Note that we haven't investigated yet the case of emails that
have more than one start boundary, but no final boundary.
Fixes #921
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Under Rails 3, the uudecoded FoiAttachment in this test
fails validation at the self.save! in
IncomingMessage.parse_raw_email, although the FoiAttachment
has been correctly created and saved to the database in
_uudecode_and_save_attachments. Forcing a reload=true on
self.foi_attachments fixes this.
Thanks to Louise Crow for finding the fix for this problem.
|
| | | | |
|
| | | |\ |
|
| | | | |
| | | |
| | | |
| | | | |
Fixes #942.
|
| | | |/
| | |
| | |
| | | |
no charset field.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
These cases are rare, and probably need to be resolved by
reporting issues against the Mail gem (although it's debatable
what the more correct or pragmatic behaviour should be in both
cases).
|
| | | |
| | |
| | |
| | |
| | | |
At one point in development this email was misparsed, so I've
added this as test to check for regressions.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
This function is useful for investigating problems with
handling of emails, attachments and the related character
encoding issues. It can safely be removed later, but is
currently useful to have for debugging purposes.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
One of these changes is to make sure that the Mail backend,
like the TMail backend it replaces, will return text parts
encoded in UTF-8 if possible.
The other change is to ensure that when text attachments are
reloaded from disk, we attempt to convert them to UTF-8.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Throughout the codebase it is simplest and most consistent
if we could assume that all text/* attachments are represented
by UTF-8 strings, and this was largely true with the TMail
backend which ensured that all returned text parts were in
UTF-8. We have to change the replacement Mail-backed to
similarly attempt to convert text parts to UTF-8. This commit
introduces two functions which are useful for this.
The normalize_string_to_utf8 function will try various
encodings, either suggested or guessed (with charlock_holmes)
to convert the passed string to UTF-8, and if it can't find a
suitable encoding will throw an exception.
Unfortunately, the current behaviour of the site is that
uninterpretable text/* attachments are still passed around and
mangled to UTF-8 just before display. To mimic this it's also
useful to have the convert_string_to_utf8_or_binary function,
which tries to convert the string to UTF-8 with
normalize_string_to_utf8, but if that's not possible just
returns the original string. (In Ruby 1.9, encoding will be
set to UTF-8 or ASCII-8BIT appropriately.)
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The example file that was used for checking for text
attachment data which could not be converted to UTF-8 is
one that we *can* actually deal with by guessing the
character set, since it's valid GB18030. So, this commit
changes that test to check for the first few Chinese
characters in that email, and introduces a replacement
test with text from /dev/random, which should not be
interpretable in any sensible way.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The data loaded with load_file_fixture is generally used
in the tests as if it were binary data received from the
MTA, so just load it as binary data - in Ruby 1.9 this means
that it will end up with the encoding ASCII-8BIT.
The same is the case when reading data from a fixture file
in receive_incoming_mail.
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
This also introduces a custom error class so that we don't
accidentally catch other problems.
Fixes #920
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
This example email indicates the wrong charset and includes
a top bit set character despite Content-Transfer-Encoding: 7bit
- nonetheless, we should be able to convert it to UTF-8 and
interpret the character correctly.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
If there is a missing final MIME boundary, the behaviour
of Alaveteli with the TMail backend was to still parse
the attachment, but with the new code it currently throws
an exception. This commit adds a test that asserts that
the attachment should be parsed despite the email being
malformed in this way.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
There is currently a difference in behaviour in the parsing
of nested MIME multipart attachments between the Mail and
TMail based backends. This commit adds a test that will
pass if the behaviour is the same as the the old (TMail-based)
version, which I believe is correct according to RFC 1521.
The example email has a PNG attachment after the final MIME
boundary, and the RFC says that anything after the final
boundary ("the epilogue") should be ignored.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The use of backticks in `which tnef` means that which(1) is
invoked under /bin/sh, and returns the first executable
matching 'tnef' in the default PATH for /bin/sh. In this
usage, however, Popen also uses /bin/sh to execute the
command, which would run that same executable if just a
bare 'tnef' were used. In summary, I can't see any reason
for the convolution of:
IO.popen("#{`which tnef`.chomp} -K -C #{dir}", "wb") do |f|
... over just:
IO.popen("tnef -K -C #{dir}", "wb") do |f|
... so switch to the latter.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
These two cases were ignored previously, and we need to make sure
that they still are under the switch from TMail to Mail.
One TNEF attachment is a heavily truncated one from a real example
from Alaveteli that has no personal data in it. The other is
an example from the tests in the distribution of the tnef
package for Ubuntu 1.4.9-1 - it's an HTML version of the US
constitution.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
For the upgrade from Rails 2 to Rails 3, we want to check that
existing emails are parsed correctly. One of these tasks
(temp:random_attachments_hexdigests) is for dumping a CVS file
of details about each attachment, and its hexdigest, and a list
of the randomly selected raw emails. (It's intended that you
run this on an old Rails 2 install of Alaveteli. The other task
(temp:recompute_attachments_hexdigests) is intended to be run
on a Rails 3 install of Alaveteli, with the previous files as
input, to check that the same results are obtained.
|
| | |\ \ |
|
| | | | |
| | | |
| | | |
| | | | |
These were written by Henare Degan with some changes by Mark Longair.
|
| | | | | |
|
| |/ / / |
|
