diff options
| author | Petter Reinholdtsen <pere@hungry.com> | 2016-04-06 00:03:09 +0200 |
|---|---|---|
| committer | Petter Reinholdtsen <pere@hungry.com> | 2016-04-06 00:03:09 +0200 |
| commit | ac5155418daee2baf9c15ba039d20677f722488b (patch) | |
| tree | df8816ce89b0789869c866c8275fc92b76357016 | |
| parent | 9fe35026d89d2b99df4a772579babfffb7e0148a (diff) | |
Handle environment without CPU limitation.
Make sure oep scraper do not exit right away when there is no CPU
limit, instead assume a default 10 second limit in this case. 10
seconds is choosed fairly randomly to limit the runtime to a few minutes.
| -rw-r--r-- | scrapersources/postliste-oep | 7 |
1 files changed, 6 insertions, 1 deletions
diff --git a/scrapersources/postliste-oep b/scrapersources/postliste-oep index 24963d5..02f629e 100644 --- a/scrapersources/postliste-oep +++ b/scrapersources/postliste-oep @@ -112,7 +112,12 @@ def cpu_spent(): return getattr(usage, 'ru_utime') + getattr(usage, 'ru_stime') def cpu_available(): - return resource.getrlimit(resource.RLIMIT_CPU)[0] + available = resource.getrlimit(resource.RLIMIT_CPU)[0] + # If no limit is set, assume 10 CPU seconds as the limit to avoid + # running for more than a few minutes every time. + if 0 > available: + available = 10 + return available def url_from_id(id): return "https://www.oep.no/search/resultSingle.html?journalPostId=" + str(id) |