aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeLines
* Correct scraper.HEADmasterPetter Reinholdtsen2016-10-11-8/+13
|
* Drop obsolete URL.Petter Reinholdtsen2016-10-06-3/+3
|
* Fix scraper metadata to work with new summary script.Petter Reinholdtsen2016-10-06-45/+47
|
* Get script working on local files instead of using scraperwiki.Petter Reinholdtsen2016-10-06-26/+22
|
* Time to run scraper daily.Petter Reinholdtsen2016-10-05-1/+2
|
* Improve formatting of sender/receiver.Petter Reinholdtsen2016-10-05-1/+1
|
* First draft scraper for Narvik.Petter Reinholdtsen2016-10-05-104/+198
|
* Add logic to rescan recent days twice to discover late entries.Petter Reinholdtsen2016-10-03-2/+15
|
* Use explicit range(start, stop, step) instead of calculations to make date ↵Petter Reinholdtsen2016-10-03-5/+5
| | | | calculation easier to understand.
* Improve handling of limited CPU resources.Petter Reinholdtsen2016-10-02-6/+10
|
* Correct day calculations and handle running out of CPU time better.Petter Reinholdtsen2016-10-02-9/+21
|
* Add CPU limit and report the number of records fetched.Petter Reinholdtsen2016-10-02-3/+27
|
* Fix typo in date parsing.Petter Reinholdtsen2016-10-02-1/+1
|
* Fix problem with non-ascii output to the log.Petter Reinholdtsen2016-10-02-0/+4
|
* Quiet down scraper.Petter Reinholdtsen2016-10-02-1/+1
|
* Use better name for new scraper.Petter Reinholdtsen2016-10-02-0/+0
|
* First draft scraper for Oslo kommune, Byrådsavdelingene.Petter Reinholdtsen2016-10-02-0/+155
|
* Remove debug output.Petter Reinholdtsen2016-09-26-1/+0
|
* Complete scraper for Ås kommune.Petter Reinholdtsen2016-09-26-23/+54
|
* Fix encoding problem with Ås kommune scraper.Petter Reinholdtsen2016-09-26-1/+1
|
* First draft scraper for Ås kommune.Petter Reinholdtsen2016-09-26-0/+131
|
* Make rescanning optional.Petter Reinholdtsen2016-04-08-10/+22
|
* Enable code reading backwards by default.Petter Reinholdtsen2016-04-08-1/+1
|
* Make sure saved and sql minimum value is the same most of the time.Petter Reinholdtsen2016-04-08-1/+1
|
* Make sure to save caseyear and caseseqnr as integers.Petter Reinholdtsen2016-04-08-2/+2
|
* Handle '-' in exemption field.Petter Reinholdtsen2016-04-07-0/+2
|
* Document that the Stortinget parser is broken now.Petter Reinholdtsen2016-04-07-2/+7
|
* Allow OEP scraper to run longer when no CPU limit is set.Petter Reinholdtsen2016-04-07-2/+2
|
* Correct initial backlog start point calculation for OEP scraper.Petter Reinholdtsen2016-04-07-1/+1
|
* Fix PDF locator code for Ruter scraper.Petter Reinholdtsen2016-04-07-1/+1
|
* Move OEP starting point closer to the current one.Petter Reinholdtsen2016-04-07-2/+2
|
* Add a copy of the old lazycache library.Petter Reinholdtsen2016-04-07-1/+47
|
* Merge pull request #1 from knutid/masterpetterreinholdtsen2016-04-07-1/+1
|\ | | | | Added missing pakages to env setup script.
| * Added python-requests, python-lxml, and python-cssselect to the list of ↵Knut Ingvald Dietzel2016-04-07-1/+1
| | | | | | | | needed packages.
* | Get scraper running as standalone program.Petter Reinholdtsen2016-04-07-4/+1
| |
* | Avoid hardcoding directory names.Petter Reinholdtsen2016-04-07-2/+2
| |
* | Add wrappers to call scrapers from cron.Petter Reinholdtsen2016-04-07-0/+20
|/
* Make sure reparse_strange_entries() do not fail when all tables are empty.Petter Reinholdtsen2016-04-06-15/+18
|
* Remove draft replaced by scraperwiki-python.Petter Reinholdtsen2016-04-06-39/+0
|
* Drop some unused imports of lazycache.Petter Reinholdtsen2016-04-06-3/+0
|
* Make sure env-setup work on black clones.Petter Reinholdtsen2016-04-06-2/+2
|
* Slow down refetch of strange entries, to avoid being locked out.Petter Reinholdtsen2016-04-06-0/+1
|
* Gah, another typo.Petter Reinholdtsen2016-04-06-1/+1
|
* Typo.Petter Reinholdtsen2016-04-06-1/+1
|
* Enable reparsing of another batch of strange entries parsedPetter Reinholdtsen2016-04-06-2/+3
| | | | 2016-02-15, where 'agency' is NULL.
* Use correct UTF-8 marker in header.Petter Reinholdtsen2016-04-06-1/+1
|
* Handle environment without CPU limitation.Petter Reinholdtsen2016-04-06-1/+6
| | | | | | Make sure oep scraper do not exit right away when there is no CPU limit, instead assume a default 10 second limit in this case. 10 seconds is choosed fairly randomly to limit the runtime to a few minutes.
* Update the setup instructions and add a title.Petter Reinholdtsen2016-04-05-6/+26
|
* Typo.Petter Reinholdtsen2016-03-26-1/+1
|
* Fix OEP scraper.Petter Reinholdtsen2016-03-26-1/+8
| | | | | Get OEP scraper working again after the source return 500 Internal Server Error for non-existing entries.
generated by cgit v1.2.3 (git 2.25.1) at 2025-04-26 09:51:13 +0000