aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeLines
* Complete scraper for Ås kommune.Petter Reinholdtsen2016-09-26-23/+54
|
* Fix encoding problem with Ås kommune scraper.Petter Reinholdtsen2016-09-26-1/+1
|
* First draft scraper for Ås kommune.Petter Reinholdtsen2016-09-26-0/+131
|
* Make rescanning optional.Petter Reinholdtsen2016-04-08-10/+22
|
* Enable code reading backwards by default.Petter Reinholdtsen2016-04-08-1/+1
|
* Make sure saved and sql minimum value is the same most of the time.Petter Reinholdtsen2016-04-08-1/+1
|
* Make sure to save caseyear and caseseqnr as integers.Petter Reinholdtsen2016-04-08-2/+2
|
* Handle '-' in exemption field.Petter Reinholdtsen2016-04-07-0/+2
|
* Document that the Stortinget parser is broken now.Petter Reinholdtsen2016-04-07-2/+7
|
* Allow OEP scraper to run longer when no CPU limit is set.Petter Reinholdtsen2016-04-07-2/+2
|
* Correct initial backlog start point calculation for OEP scraper.Petter Reinholdtsen2016-04-07-1/+1
|
* Fix PDF locator code for Ruter scraper.Petter Reinholdtsen2016-04-07-1/+1
|
* Move OEP starting point closer to the current one.Petter Reinholdtsen2016-04-07-2/+2
|
* Add a copy of the old lazycache library.Petter Reinholdtsen2016-04-07-1/+47
|
* Merge pull request #1 from knutid/masterpetterreinholdtsen2016-04-07-1/+1
|\ | | | | Added missing pakages to env setup script.
| * Added python-requests, python-lxml, and python-cssselect to the list of ↵Knut Ingvald Dietzel2016-04-07-1/+1
| | | | | | | | needed packages.
* | Get scraper running as standalone program.Petter Reinholdtsen2016-04-07-4/+1
| |
* | Avoid hardcoding directory names.Petter Reinholdtsen2016-04-07-2/+2
| |
* | Add wrappers to call scrapers from cron.Petter Reinholdtsen2016-04-07-0/+20
|/
* Make sure reparse_strange_entries() do not fail when all tables are empty.Petter Reinholdtsen2016-04-06-15/+18
|
* Remove draft replaced by scraperwiki-python.Petter Reinholdtsen2016-04-06-39/+0
|
* Drop some unused imports of lazycache.Petter Reinholdtsen2016-04-06-3/+0
|
* Make sure env-setup work on black clones.Petter Reinholdtsen2016-04-06-2/+2
|
* Slow down refetch of strange entries, to avoid being locked out.Petter Reinholdtsen2016-04-06-0/+1
|
* Gah, another typo.Petter Reinholdtsen2016-04-06-1/+1
|
* Typo.Petter Reinholdtsen2016-04-06-1/+1
|
* Enable reparsing of another batch of strange entries parsedPetter Reinholdtsen2016-04-06-2/+3
| | | | 2016-02-15, where 'agency' is NULL.
* Use correct UTF-8 marker in header.Petter Reinholdtsen2016-04-06-1/+1
|
* Handle environment without CPU limitation.Petter Reinholdtsen2016-04-06-1/+6
| | | | | | Make sure oep scraper do not exit right away when there is no CPU limit, instead assume a default 10 second limit in this case. 10 seconds is choosed fairly randomly to limit the runtime to a few minutes.
* Update the setup instructions and add a title.Petter Reinholdtsen2016-04-05-6/+26
|
* Typo.Petter Reinholdtsen2016-03-26-1/+1
|
* Fix OEP scraper.Petter Reinholdtsen2016-03-26-1/+8
| | | | | Get OEP scraper working again after the source return 500 Internal Server Error for non-existing entries.
* Start on new scraiper for sio.no.Petter Reinholdtsen2015-11-23-0/+30
|
* Document how to run a scraper.Petter Reinholdtsen2015-05-23-0/+5
|
* Ignore IDEA and data foldersAnders Einar Hilden2015-01-22-0/+2
|
* Almost ready to use [DMS2002]Anders Einar Hilden2015-01-22-31/+90
|
* Add debug info.Petter Reinholdtsen2015-01-20-0/+1
|
* Rewrite logic to select dates, to parse both forward and back in time.Petter Reinholdtsen2015-01-20-6/+23
|
* Add new source.Petter Reinholdtsen2015-01-19-0/+1
|
* New county scraper.Petter Reinholdtsen2015-01-19-0/+93
|
* New sources.Petter Reinholdtsen2015-01-18-0/+2
|
* Document missing fields.Petter Reinholdtsen2015-01-18-0/+1
|
* More info on common fields.Petter Reinholdtsen2015-01-18-0/+32
|
* We can extract all but 3 elementsAnders Einar Hilden2015-01-18-37/+178
|
* Starting to rewrite datafinder-dodeAnders Einar Hilden2015-01-18-30/+65
|
* Add meta-info.Petter Reinholdtsen2015-01-18-2/+29
|
* Cleanup.Petter Reinholdtsen2015-01-17-2/+12
|
* Get it limping along again.Petter Reinholdtsen2015-01-17-1/+11
|
* Document missing field.Petter Reinholdtsen2015-01-17-0/+1
|
* Fix parser.Petter Reinholdtsen2015-01-17-3/+3
|