diff options
| author | Mark Longair <mark@mysociety.org> | 2013-10-09 15:08:23 +0100 |
|---|---|---|
| committer | FOI archive email account <foi@titan.ukcod.org.uk> | 2013-10-09 15:13:19 +0100 |
| commit | 280b3fd57e6f21d1cd09548411fc1dafd794cc1d (patch) | |
| tree | 7fccb88412c5dc9ff8a05c98be3cab46f4b2e04a /lib/tasks | |
| parent | 1f6771c1545e320437a2fbecf7ddd0f8abdf77d1 (diff) | |
Reduce the memory consumption of rake stats:update_public_bodies_stats
This task was taking a huge amount of memory, even when fetching
the InfoRequest objects with find_each. With an additional find_each
for public bodies (10 at a time) and reducing the batch size for
info requests to 200 at a time, the memory size of this rake task is
kept down to about 400MB.
Diffstat (limited to 'lib/tasks')
| -rw-r--r-- | lib/tasks/stats.rake | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/lib/tasks/stats.rake b/lib/tasks/stats.rake index 4eda27289..eb36204c6 100644 --- a/lib/tasks/stats.rake +++ b/lib/tasks/stats.rake @@ -94,7 +94,7 @@ namespace :stats do desc 'Update statistics in the public_bodies table' task :update_public_bodies_stats => :environment do verbose = ENV['VERBOSE'] == '1' - PublicBody.all.each do |public_body| + PublicBody.find_each(:batch_size => 10) do |public_body| puts "Counting overdue requests for #{public_body.name}" if verbose # Look for values of 'waiting_response_overdue' and @@ -102,7 +102,8 @@ namespace :stats do # described_state column, and instead need to be calculated: overdue_count = 0 very_overdue_count = 0 - InfoRequest.find_each(:conditions => {:public_body_id => public_body.id}) do |ir| + InfoRequest.find_each(:batch_size => 200, + :conditions => {:public_body_id => public_body.id}) do |ir| case ir.calculate_status when 'waiting_response_very_overdue' very_overdue_count += 1 |