From f920268650b6bb906be1828ed26a3732f4d10cc1 Mon Sep 17 00:00:00 2001 From: Mark Longair Date: Tue, 3 Dec 2013 17:30:16 +0000 Subject: Fix the command-line CSV importer under Ruby 1.9 Under Ruby 1.8.7, you can parse a CSV file with the following code (Example A): require 'csv' CSV.parse('foo.csv') do |row| puts "got row: #{row.inspect}" end Rather confusingly, under Ruby 1.8.7, CSV.parse can also take a string representation of the contents of the file as its parameter, so this also works (Example B): require 'csv' CSV.parse("1,hello,red\n2,goodbye,green") do |row| puts "got row: #{row.inspect}" end However under Ruby 1.9.3, CSV.parse only expects a string representation of the contents of the CSV file, so only Example B works; Example B fails silently (interpreting the filename as a single cell CSV file, typically). The import:import_csv rake task unfortunately relied on both A and B working. This commit fixes this by adding PublicBody.import_csv_from_file, and refactoring PublicBody.import_csv to use the newly added class method, and adds a test to check for any regression in this behaviour. (This means that the usage of import_csv in the admin public body controller's import_csv action could now be changed to use PublicBody.import_csv_from_file directly from the uploaded file, which would be more efficient and cope with larger files without using lots of memory.) Fixes #1229 --- lib/tasks/import.rake | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) (limited to 'lib/tasks/import.rake') diff --git a/lib/tasks/import.rake b/lib/tasks/import.rake index 0e8397fde..c8183c745 100644 --- a/lib/tasks/import.rake +++ b/lib/tasks/import.rake @@ -54,12 +54,12 @@ namespace :import do STDERR.puts "Now importing the public bodies..." # Now it's (probably) safe to try to import: - errors, notes = PublicBody.import_csv(tmp_csv.path, - tag='', - tag_behaviour='replace', - dryrun, - editor="#{ENV['USER']} (Unix user)", - I18n.available_locales) do |row_number, fields| + errors, notes = PublicBody.import_csv_from_file(tmp_csv.path, + tag='', + tag_behaviour='replace', + dryrun, + editor="#{ENV['USER']} (Unix user)", + I18n.available_locales) do |row_number, fields| percent_complete = (100 * row_number.to_f / number_of_rows).to_i STDERR.print "#{row_number} out of #{number_of_rows} " STDERR.puts "(#{percent_complete}% complete)" -- cgit v1.2.3