Commit message (Collapse) | Author | Age | Lines | |
---|---|---|---|---|
* | Complete scraper for Ås kommune. | Petter Reinholdtsen | 2016-09-26 | -23/+54 |
| | ||||
* | Fix encoding problem with Ås kommune scraper. | Petter Reinholdtsen | 2016-09-26 | -1/+1 |
| | ||||
* | First draft scraper for Ås kommune. | Petter Reinholdtsen | 2016-09-26 | -0/+131 |
| | ||||
* | Make rescanning optional. | Petter Reinholdtsen | 2016-04-08 | -10/+22 |
| | ||||
* | Enable code reading backwards by default. | Petter Reinholdtsen | 2016-04-08 | -1/+1 |
| | ||||
* | Make sure saved and sql minimum value is the same most of the time. | Petter Reinholdtsen | 2016-04-08 | -1/+1 |
| | ||||
* | Make sure to save caseyear and caseseqnr as integers. | Petter Reinholdtsen | 2016-04-08 | -2/+2 |
| | ||||
* | Handle '-' in exemption field. | Petter Reinholdtsen | 2016-04-07 | -0/+2 |
| | ||||
* | Document that the Stortinget parser is broken now. | Petter Reinholdtsen | 2016-04-07 | -2/+7 |
| | ||||
* | Allow OEP scraper to run longer when no CPU limit is set. | Petter Reinholdtsen | 2016-04-07 | -2/+2 |
| | ||||
* | Correct initial backlog start point calculation for OEP scraper. | Petter Reinholdtsen | 2016-04-07 | -1/+1 |
| | ||||
* | Fix PDF locator code for Ruter scraper. | Petter Reinholdtsen | 2016-04-07 | -1/+1 |
| | ||||
* | Move OEP starting point closer to the current one. | Petter Reinholdtsen | 2016-04-07 | -2/+2 |
| | ||||
* | Add a copy of the old lazycache library. | Petter Reinholdtsen | 2016-04-07 | -1/+47 |
| | ||||
* | Merge pull request #1 from knutid/master | petterreinholdtsen | 2016-04-07 | -1/+1 |
|\ | | | | | Added missing pakages to env setup script. | |||
| * | Added python-requests, python-lxml, and python-cssselect to the list of ↵ | Knut Ingvald Dietzel | 2016-04-07 | -1/+1 |
| | | | | | | | | needed packages. | |||
* | | Get scraper running as standalone program. | Petter Reinholdtsen | 2016-04-07 | -4/+1 |
| | | ||||
* | | Avoid hardcoding directory names. | Petter Reinholdtsen | 2016-04-07 | -2/+2 |
| | | ||||
* | | Add wrappers to call scrapers from cron. | Petter Reinholdtsen | 2016-04-07 | -0/+20 |
|/ | ||||
* | Make sure reparse_strange_entries() do not fail when all tables are empty. | Petter Reinholdtsen | 2016-04-06 | -15/+18 |
| | ||||
* | Remove draft replaced by scraperwiki-python. | Petter Reinholdtsen | 2016-04-06 | -39/+0 |
| | ||||
* | Drop some unused imports of lazycache. | Petter Reinholdtsen | 2016-04-06 | -3/+0 |
| | ||||
* | Make sure env-setup work on black clones. | Petter Reinholdtsen | 2016-04-06 | -2/+2 |
| | ||||
* | Slow down refetch of strange entries, to avoid being locked out. | Petter Reinholdtsen | 2016-04-06 | -0/+1 |
| | ||||
* | Gah, another typo. | Petter Reinholdtsen | 2016-04-06 | -1/+1 |
| | ||||
* | Typo. | Petter Reinholdtsen | 2016-04-06 | -1/+1 |
| | ||||
* | Enable reparsing of another batch of strange entries parsed | Petter Reinholdtsen | 2016-04-06 | -2/+3 |
| | | | | 2016-02-15, where 'agency' is NULL. | |||
* | Use correct UTF-8 marker in header. | Petter Reinholdtsen | 2016-04-06 | -1/+1 |
| | ||||
* | Handle environment without CPU limitation. | Petter Reinholdtsen | 2016-04-06 | -1/+6 |
| | | | | | | Make sure oep scraper do not exit right away when there is no CPU limit, instead assume a default 10 second limit in this case. 10 seconds is choosed fairly randomly to limit the runtime to a few minutes. | |||
* | Update the setup instructions and add a title. | Petter Reinholdtsen | 2016-04-05 | -6/+26 |
| | ||||
* | Typo. | Petter Reinholdtsen | 2016-03-26 | -1/+1 |
| | ||||
* | Fix OEP scraper. | Petter Reinholdtsen | 2016-03-26 | -1/+8 |
| | | | | | Get OEP scraper working again after the source return 500 Internal Server Error for non-existing entries. | |||
* | Start on new scraiper for sio.no. | Petter Reinholdtsen | 2015-11-23 | -0/+30 |
| | ||||
* | Document how to run a scraper. | Petter Reinholdtsen | 2015-05-23 | -0/+5 |
| | ||||
* | Ignore IDEA and data folders | Anders Einar Hilden | 2015-01-22 | -0/+2 |
| | ||||
* | Almost ready to use [DMS2002] | Anders Einar Hilden | 2015-01-22 | -31/+90 |
| | ||||
* | Add debug info. | Petter Reinholdtsen | 2015-01-20 | -0/+1 |
| | ||||
* | Rewrite logic to select dates, to parse both forward and back in time. | Petter Reinholdtsen | 2015-01-20 | -6/+23 |
| | ||||
* | Add new source. | Petter Reinholdtsen | 2015-01-19 | -0/+1 |
| | ||||
* | New county scraper. | Petter Reinholdtsen | 2015-01-19 | -0/+93 |
| | ||||
* | New sources. | Petter Reinholdtsen | 2015-01-18 | -0/+2 |
| | ||||
* | Document missing fields. | Petter Reinholdtsen | 2015-01-18 | -0/+1 |
| | ||||
* | More info on common fields. | Petter Reinholdtsen | 2015-01-18 | -0/+32 |
| | ||||
* | We can extract all but 3 elements | Anders Einar Hilden | 2015-01-18 | -37/+178 |
| | ||||
* | Starting to rewrite datafinder-dode | Anders Einar Hilden | 2015-01-18 | -30/+65 |
| | ||||
* | Add meta-info. | Petter Reinholdtsen | 2015-01-18 | -2/+29 |
| | ||||
* | Cleanup. | Petter Reinholdtsen | 2015-01-17 | -2/+12 |
| | ||||
* | Get it limping along again. | Petter Reinholdtsen | 2015-01-17 | -1/+11 |
| | ||||
* | Document missing field. | Petter Reinholdtsen | 2015-01-17 | -0/+1 |
| | ||||
* | Fix parser. | Petter Reinholdtsen | 2015-01-17 | -3/+3 |
| |