diff options
| -rw-r--r-- | todo.txt | 250 |
1 files changed, 118 insertions, 132 deletions
@@ -1,14 +1,41 @@ -Next +Next (things that will reduce admin time mainly) ==== Change text so that overdue covers qualified exemption of public interest test being valid reason for delay. -Annotations are word wrapping the URL ugh +Make return path a black hole for administrative emails so we don't get bounces Write code to make sure the Return-Path is never foi@sandwich.org.uk (even if the Rails code breaks for Sendmail case in future botched Rails upgrades :) +Remove the generic unknown option so people don't overuse it + +Use maybe words for generated email address? Name of the person and a request +number (i.e. number of that persons request, so there are few numbers)? + It looks like an error generated by GFI MailEssentials, see p62 of chapter + 11 of the manual at http://www.gfi.com/mes/me11manual.pdf which states: + 7. Check if emails contain more than X numbers in the MIME from: + Frequently, more than 3 numbers in the MIME from means that the sender is a + spammer. The reason for this is that spammers often use tools to + automatically create reply-to: addresses on hotmail and other free email + services. Frequently they use 3 or more numbers in the name to make sure + the reply-to: is unique. +People seem to have trouble typing this + +For followups, have radio button to say is it a new request or followup +Do by uncommenting the "new information" option when writing a followup, so +that it makes a new request + +Let requesters change status to anything at any point +- using same interface let admin users set things to internal review more + easily (currently have to go to admin interface, as isn't on main describe + form) + +Just stop worrying about old requests which are unclassified in status, or +make it a public wiki? At least improve usability a bit more for the alert +emails reminding you to fill in status. + Later ===== @@ -33,28 +60,12 @@ Awaiting internal review overdue state? Sort requests on user page by status Ask people for annotation immediately after they have submitted their request +Ask for annotation about what they learnt from request? I have several email alerts set up. Is there any chance they could include part (or, preferably, all) of the search criterion in the Subject: line? :o) (Perhaps do it in the case when only one search criterion makes the mail) -Use Ruby msg - http://code.google.com/p/ruby-msg/ -To decode Outlook .msg (.oft?) files, e.g. - http://www.whatdotheyknow.com/request/immediate_response_team_deployme - http://www.whatdotheyknow.com/request/chinese_names_for_british_politi - -Use maybe words for generated email address? Name of the person and a request -number (i.e. number of that persons request, so there are few numbers)? - It looks like an error generated by GFI MailEssentials, see p62 of chapter - 11 of the manual at http://www.gfi.com/mes/me11manual.pdf which states: - 7. Check if emails contain more than X numbers in the MIME from: - Frequently, more than 3 numbers in the MIME from means that the sender is a - spammer. The reason for this is that spammers often use tools to - automatically create reply-to: addresses on hotmail and other free email - services. Frequently they use 3 or more numbers in the name to make sure - the reply-to: is unique. - Update annotation help text for new states like internal_review and gone_postal Test data dumper that removes sensitive data, but lets trusted people play with @@ -83,15 +94,6 @@ CSS / design things - Get Atom feed of search results to include stylesheet for highlighting words in yellow somehow -Let requesters change status to anything at any point -- using same interface let admin users set things to internal review more - easily (currently have to go to admin interface, as isn't on main describe - form) - -For followups, have radio button to say is it a new request or followup -Do by uncommenting the "new information" option when writing a followup, so -that it makes a new request - Would be nice if you try and send or resend (from admin interface, or by sending followup as user) and original address is set to not_apply or "" that it deals with it better :) @@ -100,10 +102,11 @@ When doing search, people often just want it to show the whole page. Perhaps all listing should just link to top of page, rather than # links for outgoing incoming, or perhaps just some of them. -Add "Who should I make my request to?" - make flow better after first section, to abrupt now -I think the advice in this annotation could go into a nice comment: -http://www.whatdotheyknow.com/request/berr_response_to_eu_on_phorm_bt#comment-356 -Add "I want to file squillions of requests" +Help page improvements: + Add "Who should I make my request to?" - make flow better after first section, to abrupt now + I think the advice in this annotation could go into a nice comment: + http://www.whatdotheyknow.com/request/berr_response_to_eu_on_phorm_bt#comment-356 + Add "I want to file squillions of requests" Tell application developer if working days table not up to date, and needs updating @@ -119,41 +122,10 @@ http://www.whatdotheyknow.com/request/policy_regarding_body_scans#incoming-1100 Show similar requests after you have filed yours - maybe on preview too. -Knackered view as HTML: - http://www.whatdotheyknow.com/request/statistics_for_allocation_of_dut#incoming-4793 - http://www.whatdotheyknow.com/request/post_lawrence_report_diversity_t_7#incoming-5483 -They are copy protected PDFs. The code in the incoming_message model tries to fix -that by converting them to ps and back, but the content is getting lost when doing, not -sure why. Perhaps instead patch poppler-utils so pdftohtml doesn't respect protection -either (code in pdftotext already has a #ifdef round it) -Some other pdftohtml bugs (fix them or file about them) - possibly can detect some of these -and display a better error? - http://www.whatdotheyknow.com/request/sale_of_public_land#incoming-8146 - http://www.whatdotheyknow.com/request/childrens_database_compliance_wi#incoming-8088 - http://www.whatdotheyknow.com/request/3326/response/7701/attach/html/2/Scan001.PDF.html - http://www.whatdotheyknow.com/request/risk_log#incoming-8090 - http://www.whatdotheyknow.com/request/number_of_out_of_county_placemen_5#incoming-11248 -Orientation wrong: - http://www.whatdotheyknow.com/request/3153/response/7726/attach/html/2/258850.pdf.html -Bug in wvHtml, segfaults when converting this: - http://www.whatdotheyknow.com/request/subject_access_request_guide_sar#incoming-10242 - -Make tables prettier in view as HTMl, just normal thick borders. -http://www.whatdotheyknow.com/request/1610/response/8093/attach/html/3/2008.10.29%20Reply.doc.html - -Make HTML attachments have view as HTML :) -http://www.whatdotheyknow.com/request/enforced_medication#incoming-7395 - -Can use track count to show interesting requests -select info_request_id, title, count(*) as c from track_things, info_requests where track_things.info_request_id is not null and track_things.info_request_id = info_requests.id group by info_request_id, title order by c; - Test code for FOI officer upload Test code for rendering lots of different attachments and filetypes Test code for internal review submitting -Detect councils that always send automatic acknowledgements, and notice if they -do not for a particular request? (e.g. Leicestershire County Council) - http://www.whatdotheyknow.com/list/successful - too many ombudsmans. Radical would be to group by user, regardless of subject; less radical to collate those with same/similar subject. @@ -168,34 +140,17 @@ Protect from CSRF with this in app controller (care it doesn't break anything): Look at quote_address_if_necessary in actionmailer's quoting.rb - why did it not work for the email address with "@" in its name part? -VSD files vsdump - example in zip file - http://www.whatdotheyknow.com/request/dog_control_orders#incoming-3510 - doing file RESPONSE/Internal documents/Briefing with Contact Islington/Contact Islington Flowchart Jul 08.vsd content type -Search for other extensions that we have now - (call IncomingMessage.find_all_unknown_mime_types to find them) - From an email, isn't stripping spaces right. "Met Office sent a response to Andrew Montford (14 August 2008)" Also should group by the request id for search queries (so all appear together when request and response mention same term) -When indexing .docx do you need to index docProps/custom.xml and docProps/app.xml -as well as word/document.xml ? (thread on xapian-discuss does so) - Something to check which tags are used but aren't in PublicBody category lists -Blog posts / Wikipedia articles about this request - e.g. http://socialreporter.wordpress.com/2008/04/14/costs-of-the-bbc-action-action-network/ - http://reportr.net/2008/04/24/bbc-spent-6m-to-develop-the-iplayer/ -Or just look at referrers as Julian says -Or just don't automate this, keep posting annotations manually. - Catch query parser error, e.g. for this query water NOT NEAR "water lane" Document such errors in acts_as_xapian -Renaming public authorities will break alerts on them. - Compress the emails in the database Don't store the cached text in backups @@ -206,19 +161,69 @@ Edits to outgoing/incoming/title won't be reindexed in Xapian (maybe just reinde This does it all: $ ./script/clear-incoming-text-cache ; ./script/rebuild-xapian-index -Interface for when you change your email address - easier to do now with post_redirect.circumstance? - -Including 'Request for information' in the subject line of your e-mail will assist us in prioritising your request. -http://www.parliament.uk/parliamentary_publications_and_archives/freedom_of_information_in_the_house_of_lords/lords__foi___how_to_obtain_information.cfm -XXX do we need specifically that text, is it better than "Freedom of Information request" which we put now? -Medway also hint at this with subject= on mailto link - Remove request email address from PDFs (we already do from docs) http://www.whatdotheyknow.com/request/cost_of_policing_the_oxford_unio_3 - maybe if text contains email, refuse to show full PDF just show conversion to text/HTML? http://www.whatdotheyknow.com/request/5353/response/11911/attach/html/2/Freedom%20of%20Information%20-%20Letter%20Accepting%20Request%20-%2072057594037995214.pdf.html +Renaming public authorities will break alerts on them. For basic alerts the +structured info is there so this should just be fixed. For searches, perhaps +Xapian index should search url_name history as well? + + +Display and indexing of response emails/attachments +--------------------------------------------------- + +Failed to detect attachments are emails and decode them: + http://www.whatdotheyknow.com/request/malicious_communication_act#incoming-12964 + +When indexing .docx do you need to index docProps/custom.xml and docProps/app.xml +as well as word/document.xml ? (thread on xapian-discuss does so) + +VSD files vsdump - example in zip file + http://www.whatdotheyknow.com/request/dog_control_orders#incoming-3510 + doing file RESPONSE/Internal documents/Briefing with Contact Islington/Contact Islington Flowchart Jul 08.vsd content type +Use Ruby msg + http://code.google.com/p/ruby-msg/ +To decode Outlook .msg (.oft?) files, e.g. + http://www.whatdotheyknow.com/request/immediate_response_team_deployme + http://www.whatdotheyknow.com/request/chinese_names_for_british_politi + +Search for other file extensions that we have now and look for ones we could +and should be indexing + (call IncomingMessage.find_all_unknown_mime_types to find them) + +Make tables prettier in view as HTMl, just normal thick borders. +http://www.whatdotheyknow.com/request/1610/response/8093/attach/html/3/2008.10.29%20Reply.doc.html + +Render HTML alternative rather than text (so tables look good) e.g.: + http://www.whatdotheyknow.com/request/parking_policy +And indeed so links work: + http://www.whatdotheyknow.com/request/recycling_levels_in_the_winchest + +Make HTML attachments have view as HTML :) +http://www.whatdotheyknow.com/request/enforced_medication#incoming-7395 + +Knackered view as HTML: + http://www.whatdotheyknow.com/request/statistics_for_allocation_of_dut#incoming-4793 + http://www.whatdotheyknow.com/request/post_lawrence_report_diversity_t_7#incoming-5483 +They are copy protected PDFs. The code in the incoming_message model tries to fix +that by converting them to ps and back, but the content is getting lost when doing, not +sure why. Perhaps instead patch poppler-utils so pdftohtml doesn't respect protection +either (code in pdftotext already has a #ifdef round it) +Some other pdftohtml bugs (fix them or file about them) - possibly can detect some of these +and display a better error? + http://www.whatdotheyknow.com/request/sale_of_public_land#incoming-8146 + http://www.whatdotheyknow.com/request/childrens_database_compliance_wi#incoming-8088 + http://www.whatdotheyknow.com/request/3326/response/7701/attach/html/2/Scan001.PDF.html + http://www.whatdotheyknow.com/request/risk_log#incoming-8090 + http://www.whatdotheyknow.com/request/number_of_out_of_county_placemen_5#incoming-11248 +Orientation wrong: + http://www.whatdotheyknow.com/request/3153/response/7726/attach/html/2/258850.pdf.html +Bug in wvHtml, segfaults when converting this: + http://www.whatdotheyknow.com/request/subject_access_request_guide_sar#incoming-10242 + Quoting fixing TODO: http://www.whatdotheyknow.com/request/35/response/191 # Funny disclaimer http://www.whatdotheyknow.com/request/40/response/163 # funny disclaimer @@ -270,20 +275,24 @@ Quoting fixing TODO: http://www.whatdotheyknow.com/request/enforcement_forders_for_replacin#incoming-6277 # over zealous quoting http://www.whatdotheyknow.com/request/renewable_energy_consumption_by # over zealous -Render HTML alternative rather than text (so tables look good) e.g.: - http://www.whatdotheyknow.com/request/parking_policy -And indeed so links work: - http://www.whatdotheyknow.com/request/recycling_levels_in_the_winchest - -Failed to detect attachments are emails and decode them: - http://www.whatdotheyknow.com/request/malicious_communication_act#incoming-12964 - Totally new features -------------------- -Tips on using the law: -- You can go up and down between local and national - ask local places what their policy is, and hwo they are implementing it. Ask national things what odcuments set local policies. +Detect councils that always send automatic acknowledgements, and notice if they +do not for a particular request? (e.g. Leicestershire County Council) + +Interface for when you change your email address - easier to do now with post_redirect.circumstance? + +Add tips on using the law, e.g.: +- You can go up and down between local and national - ask local places what + their policy is, and hwo they are implementing it. Ask national things what + odcuments set local policies. + +Hyperlink Section 1(3) to the act + http://www.whatdotheyknow.com/request/university_investment_in_the_arm#incoming-86 +and to guidance notes + http://www.ico.gov.uk/what_we_cover/freedom_of_information/guidance.aspx Jump to a random request :) @@ -304,17 +313,11 @@ Make text boxes autogrow as you type into them. Set arbitary alarms, to be alerted when a date set by authority for themselves is passed. -Add to calendar links: -like the add to calendar links on here: http://upcoming.yahoo.com/event/33391/ -.ics (outlook) would probably be most useful to people i'd guess - -Offer option of one RSS feed with all things you are tracking in it. Maybe. - Antispam on contact form (a recaptcha presumably, as we have them elsewhere) Find a way to get corrected email addresses from responses - somehow getting -user to send them in? -e.g. as in http://www.whatdotheyknow.com/request/1071/response/1870 which +user to send them in? "For future FOI requests please email ..." +e.g. as in http://www.whatdotheyknow.com/request/cost_of_council_website_17#incoming-1870 we found largely by accident. Rate limit requests using Ratty, with a freeze thing WTT-style, to detect @@ -327,16 +330,16 @@ separating out the pages as separate links (to .png files or whatever) http://www.whatdotheyknow.com/request/windsor_maidenhead_council_commo#incoming-1910 Heck, may as well give thumbnails of all images, indeed all docs while you're at it :) -Julian's PDF excerpting thing for highlighting paragraphs - and convert Word Docs to PDFs too +In "view as HTML" let people highlight paragraphs, and link to them +(like Julian's UN PDF highlighting thing) -Add geographical location to council import -Have a single button to sign up to alerts on all your council PCTs etc. +Add geographical location of councils, PCTs etc. +Have a single button to sign up to alerts on authorities for your postcode -Holding pen with comments - new requests don't get sent straight away, but are - delayed while people help improve them. +Proposed request submission queue with comments - new requests don't get sent straight + away, but are delayed while people help improve them. -Screen scrape this and add link to it on the public body page +Screen scrape ICO's decision notices list and add link to it on the relevant public authority pages http://www.ico.gov.uk/Home/tools_and_resources/decision_notices.aspx Description for each body as to what info it holds @@ -346,31 +349,14 @@ Link to: Publication schemes (http://www.ico.gov.uk/what_we_cover/freedom_of_information/publication_schemes.aspx) TWFY department search Complaint email + Phone number for advice and assistence (House of Lords give one http://www.parliament.uk/parliamentary_publications_and_archives/freedom_of_information_in_the_house_of_lords/lords__foi___how_to_obtain_information.cfm ) e.g. http://www.ordnancesurvey.co.uk/oswebsite/aboutus/foi/index.html http://www.ordnancesurvey.co.uk/oswebsite/aboutus/foi/coiindex.html -Give hotline to call body for extra help -e.g. Advice and assistance in making FoI requests to the House of Lords is available -from the Freedom of Information Officer, Parliamentary Archives, House of -Lords. London SW1A 0PW. Telephone 020 7219 0100. E-mail: foilords@parliament.uk -http://www.parliament.uk/parliamentary_publications_and_archives/freedom_of_information_in_the_house_of_lords/lords__foi___how_to_obtain_information.cfm - -EU regulation 1049/2001 -US (with Sunlight) - -Ask what they learnt from request, and other things? - and reward by putting their request on front page +EU regulation 1049/2001 requests +US requests (with Sunlight) OCR all images automatically, even if badly (check for tiffs!) Maybe use Scrbd's free service :) http://www.scribd.com/paper -Hyperlink Section 1(3) to the act - http://www.whatdotheyknow.com/request/49/response/86 -and to guidance notes - http://www.ico.gov.uk/what_we_cover/freedom_of_information/guidance.aspx - -Only show "is email right" footer the first time a new request is sent to that email? -Show public body email address on their public page, with a link to say "this isn't right!" - - |