aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeLines
* Replace backticks with call to exec(), the former spawns a new process, and ↵Louise Crow2013-06-03-2/+3
| | | | the latter replaces the current process. In the daemon context we care about the process id so that calls to startstopdaemon won't spawn a new daemon.
* Parse the 'to' address as if on a real mail to trigger quoted string encoding.Louise Crow2013-06-03-1/+19
|
* Fix for subject lines with invalid UTF-8 as the last characterMark Longair2013-06-03-2/+35
| | | | | | | | | | | | | | | This seems to be the bug mentioned here: http://po-ru.com/diary/fixing-invalid-utf-8-in-ruby-revisited/ That explains that some versions of Iconv don't ignore invalid characters when converting to UTF-8 even with //IGNORE if that invalid character happens to be at the end of the string. In fact, as Matthew Somerville pointed out, with some versions of iconv (e.g. 1.14 on Mac OS, apparently) it's necessary to add and remove more than one space at the end, in case the first character of the byte sequence indicates a long sequence. We add and remove 4 to be on the safe side.
* Merge branch 'release/0.11' of ↵0.11Louise Crow2013-06-03-2/+3
|\ | | | | | | ssh://git.mysociety.org/data/git/public/alaveteli into release/0.11
| * Merge branch 'rails-3-develop' into release/0.11Mark Longair2013-05-30-2/+3
| |\
| | * Merge branch 'feed-content-type' into rails-3-developMark Longair2013-05-30-1/+2
| | |\
| | | * Make sure Atom feeds are served with content type application/atom+xmlMark Longair2013-05-29-1/+2
| | | | | | | | | | | | | | | | Fixes #961
| | * | Fix typo.Louise Crow2013-05-30-1/+1
| | | |
* | | | Bump version number.Louise Crow2013-06-03-1/+1
|/ / /
* | | Add note about config/httpd.confLouise Crow2013-05-30-1/+2
| | |
* | | Add some upgrade notes for the 0.11 release.Louise Crow2013-05-30-0/+8
| | |
* | | Merge branch 'rails-3-develop' into release/0.11Louise Crow2013-05-30-866/+1250
|\| |
| * | Update of strings for translation by running rake ↵Louise Crow2013-05-30-838/+1207
| | | | | | | | | | | | gettext:store_model_attributes, and rake gettext:find
| * | Restore tabs to keep string for translation the same as previouslyLouise Crow2013-05-30-35/+50
| | |
* | | Merge branch 'release/0.11' of ↵Louise Crow2013-05-29-0/+0
|\ \ \ | | | | | | | | | | | | ssh://git.mysociety.org/data/git/public/alaveteli into release/0.11
| * \ \ Merge branch 'rails-3-develop' into release/0.11Mark Longair2013-05-29-63/+0
| |\ \ \ | | | |/ | | |/| | | | | | | | | Merge in some updates to the 0.11 release, discovered on further testing on a development server with the full data set.
* | | | Merge branch 'rails-3-develop' into release/0.11Louise Crow2013-05-29-87/+32
|\ \ \ \ | |/ / / |/| | / | | |/ | |/|
| * | Merge branch 'rails-3-develop' of ↵Louise Crow2013-05-29-63/+0
| |\| | | | | | | | | | ssh://git.mysociety.org/data/git/public/alaveteli into rails-3-develop
| | * Remove the (we think) now unneeded will_paginate customizationMark Longair2013-05-29-63/+0
| | | | | | | | | | | | | | | | | | | | | | | | We believe that this customization is no longer needed, and in any case would need to be rewritten for will_paginate 3.0.4. If any pagination bugs do arise, a test should be added that reproduces that problem, since currently the tests pass without this extension.
| * | Merge branch 'fix-cron-bundler-gemfile' into rails-3-developLouise Crow2013-05-29-24/+32
| |\ \ | | |/ | |/|
| | * Now that runner is invoked via the rails command and not directly from the ↵Louise Crow2013-05-29-24/+32
| |/ | | | | | | script directory, it's more important to cd to the app directory in order to pick up the bundler gemfile. Fixes #964.
* | Merge branch 'rails-3-develop' into release/0.11Louise Crow2013-05-29-10/+62
|\|
| * Merge branch 'rails-3-develop' of ↵Louise Crow2013-05-29-2/+2
| |\ | | | | | | | | | ssh://git.mysociety.org/data/git/public/alaveteli into rails-3-develop
| | * Replace 'render_for_text data' with 'render :text => data'Mark Longair2013-05-28-2/+2
| | | | | | | | | | | | | | | | | | render_for_text no longer exists in Rails 3. Fixes #955
| * | Merge branch 'loop-in-cached-attachment-recreation' into rails-3-developLouise Crow2013-05-29-7/+59
| |\ \
| | * | When extracting attachments for an incoming message and getting the body of ↵Louise Crow2013-05-29-7/+59
| |/ / | | | | | | | | | the main part in order to look for uuencoded text, make sure that we're getting that main part from the reparsed attachments, and not getting an obsolete attachment. Fixes #958.
| * / Remove erroneous test nameLouise Crow2013-05-28-1/+1
| |/
* / Only load commonlib spec file in the test environmentLouise Crow2013-05-28-1/+3
|/
* Merge branch 'rails-3-develop' of ↵Louise Crow2013-05-28-79/+1254
|\ | | | | | | | | | | | | ssh://git.mysociety.org/data/git/public/alaveteli into rails-3-develop Conflicts: Gemfile.lock
| * Merge branch 'tmail-to-mail-tests' into rails-3-developMark Longair2013-05-28-79/+1153
| |\ | | | | | | | | | | | | | | | | | | | | | This merge brings in a number of tests and fixes for the handling of mail under Mail / Rails 3 instead of TMail / Rails 2. Conflicts: config/initializers/alaveteli.rb
| | * Only cache attachments that are returned with a 200 HTTP status codeMark Longair2013-05-24-1/+1
| | | | | | | | | | | | | | | | | | | | | Otherwise redirects will be cached, and since headers and the status code aren't stored, a non-redirecting redirect page will be returned in the future, but with 200. It's easiest to only cache the 200 responses.
| | * Add a comment about the behaviour of the censor rulesMark Longair2013-05-24-0/+2
| | |
| | * The file_name on getting an attachment should be the display filenameMark Longair2013-05-24-1/+1
| | | | | | | | | | | | | | | Previously the behaviour would have been the same, since we weren't checking the filename so strictly.
| | * Make an attachment filename in test fixtures more difficultMark Longair2013-05-24-26/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit changes 'hello.txt' to 'hello-world.txt' in the incoming-request-two-same-name.email fixture. The reasoning for this change is that if there are no special characters in the filename then filename and display_filename will be the same and the tests won't pick up any confusion between the two. The test requests to :get_attachment and :get_attachment_as_html should get the display_filename rather than filename.
| | * Check that display_filename matches URL part number or fallbackMark Longair2013-05-24-20/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the display_filename of the attachment found from the URL part number doesn't match the passed in display_filename then the email may have been reparsed, causing a reordering. In that case, look to see if there is another attachment that uniquely matches that filename, and, if so, return that other attachment. If no matching uniquely matching filename is found, redirect to the incoming message, rather than returning a 404.
| | * Refactor IncomingMessage.get_attachment_by_url_part_numberMark Longair2013-05-24-6/+1
| | |
| | * Suppress STDERR noise from the tnef binaryMark Longair2013-05-22-1/+1
| | |
| | * Move the mapi requires to where they're really neededMark Longair2013-05-22-2/+2
| | | | | | | | | | | | | | | | | | | | | Handling of outlook-packed attachments would fail from rake tasks or in the console without requiring 'mapi/msg' and 'mapi/convert' beforehand. Instead, require them in the source file where they're actually used.
| | * Retain old handling of malformed addresses in To and Cc linesMark Longair2013-05-21-0/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The behaviour of the TMail backend's 'to' and 'cc' methods where there was a malformed To: or Cc: line was to return nil, whereas Mail returns a version of the string anyway. We'd have to change quite a lot of code to deal with an extra possible class of returned objects, so it's simplest for the moment to monkey-patch Mail::Message's 'to' and 'cc' methods to restore the old behaviour.
| | * Cope with emails with a missing final MIME boundaryMark Longair2013-05-17-10/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Mail gem deals with multipart messages that look as if they should have 1 part but are missing the final MIME boundary, by make the parts list empty and setting part.body to the text of the email. Rather than throwing an exception in this case, we just pretend that part is text/plain and return it, so that the page doesn't error and we still have a chance of some useful text being displayed. Note that we haven't investigated yet the case of emails that have more than one start boundary, but no final boundary. Fixes #921
| | * Avoid an FoiAttachment validation failure under Rails 3Mark Longair2013-05-16-0/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Under Rails 3, the uudecoded FoiAttachment in this test fails validation at the self.save! in IncomingMessage.parse_raw_email, although the FoiAttachment has been correctly created and saved to the database in _uudecode_and_save_attachments. Forcing a reload=true on self.foi_attachments fixes this. Thanks to Louise Crow for finding the fix for this problem.
| | * Move "require 'alaveteli_file_types'" into the initializerMark Longair2013-05-16-4/+1
| | |
| | * Merge branch 'tmail-to-mail-nil-part' into tmail-to-mail-testsLouise Crow2013-05-16-3/+52
| | |\
| | | * Only set original_charset when a charset has been defined for the mail part. ↵Louise Crow2013-05-16-3/+8
| | | | | | | | | | | | | | | | Fixes #942.
| | | * Add failing test for case where a mail part has a content-type header with ↵Louise Crow2013-05-16-0/+44
| | |/ | | | | | | | | | no charset field.
| | * Mark as "pending" two tests relating to odd MIME boundary casesMark Longair2013-05-16-14/+18
| | | | | | | | | | | | | | | | | | | | | These cases are rare, and probably need to be resolved by reporting issues against the Mail gem (although it's debatable what the more correct or pragmatic behaviour should be in both cases).
| | * Add another mail parsing testMark Longair2013-05-16-0/+7
| | | | | | | | | | | | | | | At one point in development this email was misparsed, so I've added this as test to check for regressions.
| | * Add a helper function for dumping text to diskMark Longair2013-05-16-0/+11
| | | | | | | | | | | | | | | | | | | | | This function is useful for investigating problems with handling of emails, attachments and the related character encoding issues. It can safely be removed later, but is currently useful to have for debugging purposes.
| | * Make efforts to ensure that we're usually dealing with UTF-8 stringsMark Longair2013-05-16-2/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | One of these changes is to make sure that the Mail backend, like the TMail backend it replaces, will return text parts encoded in UTF-8 if possible. The other change is to ensure that when text attachments are reloaded from disk, we attempt to convert them to UTF-8.
| | * Add functions for converting from arbitrary text data to UTF-8Mark Longair2013-05-16-1/+238
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Throughout the codebase it is simplest and most consistent if we could assume that all text/* attachments are represented by UTF-8 strings, and this was largely true with the TMail backend which ensured that all returned text parts were in UTF-8. We have to change the replacement Mail-backed to similarly attempt to convert text parts to UTF-8. This commit introduces two functions which are useful for this. The normalize_string_to_utf8 function will try various encodings, either suggested or guessed (with charlock_holmes) to convert the passed string to UTF-8, and if it can't find a suitable encoding will throw an exception. Unfortunately, the current behaviour of the site is that uninterpretable text/* attachments are still passed around and mangled to UTF-8 just before display. To mimic this it's also useful to have the convert_string_to_utf8_or_binary function, which tries to convert the string to UTF-8 with normalize_string_to_utf8, but if that's not possible just returns the original string. (In Ruby 1.9, encoding will be set to UTF-8 or ASCII-8BIT appropriately.)
generated by cgit v1.2.3 (git 2.25.1) at 2025-09-10 03:13:44 +0000