diff options
| author | Mark Longair <mark@mysociety.org> | 2013-10-09 15:08:23 +0100 |
|---|---|---|
| committer | Louise Crow <louise.crow@gmail.com> | 2013-10-23 10:09:00 +0100 |
| commit | 0de52b20c67467b7d1198f988419f6deed777c5e (patch) | |
| tree | 0933b92c4accc575c2ceb3fc10c63e35fa30cb1f /lib/tasks | |
| parent | f4d6b4cdc9ea045242e02a387c27d8ca3349289c (diff) | |
Reduce the memory consumption of rake stats:update_public_bodies_stats
This task was taking a huge amount of memory, even when fetching
the InfoRequest objects with find_each. With an additional find_each
for public bodies (10 at a time) and reducing the batch size for
info requests to 200 at a time, the memory size of this rake task is
kept down to about 400MB.
Diffstat (limited to 'lib/tasks')
| -rw-r--r-- | lib/tasks/stats.rake | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/lib/tasks/stats.rake b/lib/tasks/stats.rake index 4eda27289..eb36204c6 100644 --- a/lib/tasks/stats.rake +++ b/lib/tasks/stats.rake @@ -94,7 +94,7 @@ namespace :stats do desc 'Update statistics in the public_bodies table' task :update_public_bodies_stats => :environment do verbose = ENV['VERBOSE'] == '1' - PublicBody.all.each do |public_body| + PublicBody.find_each(:batch_size => 10) do |public_body| puts "Counting overdue requests for #{public_body.name}" if verbose # Look for values of 'waiting_response_overdue' and @@ -102,7 +102,8 @@ namespace :stats do # described_state column, and instead need to be calculated: overdue_count = 0 very_overdue_count = 0 - InfoRequest.find_each(:conditions => {:public_body_id => public_body.id}) do |ir| + InfoRequest.find_each(:batch_size => 200, + :conditions => {:public_body_id => public_body.id}) do |ir| case ir.calculate_status when 'waiting_response_very_overdue' very_overdue_count += 1 |