summaryrefslogtreecommitdiff
path: root/src/modules
AgeCommit message (Expand)Author
2015-03-05more lua fixesAndreas Baumann
2015-02-09added a CRAWLER_UNUSED for lua specific module initialization codeAndreas Baumann
2014-10-16simple url normalizer in Lua on WindowsAndreas Baumann
2014-10-16..Andreas Baumann
2014-10-16testing with two urlnormalizer modules (google and simple) in crawl.confAndreas Baumann
2014-10-16changed some throw new to throw (caused stopping of crawler with an unknown e...Andreas Baumann
2014-10-14fixed url normalizer tests on Windows (by linking them statically).Andreas Baumann
2014-10-14adapted google url normlazier Lua binding for WindowsAndreas Baumann
2014-10-09first running lua code with URL normalization, cleanup needed..Andreas Baumann
2014-10-09building curl fetcher only if WITH_CURL=1 is setAndreas Baumann
2014-10-09adaptiosn to use local toluaAndreas Baumann
2014-10-09first trials with a Google normalizer called from Lua, std::string is the pro...Andreas Baumann
2014-10-08compilation fix lua (how did it ever work!)Andreas Baumann
2014-10-08some fixes on WindowsAndreas Baumann
2014-10-04adapted all interface in modules, but SEGV happens now in crawler,Andreas Baumann
2014-10-03some curl fixesAndreas Baumann
2014-10-03fixed setting data from source in fetcher modulesAndreas Baumann
2014-10-03..Andreas Baumann
2014-10-03added an experimental curl fetcherAndreas Baumann
2014-07-25we also have to normilze with queryAndreas Baumann
2014-07-24..Andreas Baumann
2014-07-24reading complete sitemap indexes and sitemapsAndreas Baumann
2014-07-24..Andreas Baumann
2014-07-24sitemap processing (work in progress)Andreas Baumann
2014-07-23added parsing of Sitemap in robots.txtAndreas Baumann
2014-04-30started a robots.txt parserAndreas Baumann
2014-04-25fixed MIME detection using libmagicAndreas Baumann
2012-09-10libutil move and liblogger rename on WindowsAndreas Baumann
2012-09-07fixed some linker errors on FreeBSD (should also affect Linux, but there ever...Andreas Baumann
2012-09-07fixed all tests on Windows, still issues with static libcrawlAndreas Baumann
2012-09-06crawler fixed on LinuxAndreas Baumann
2012-09-06works on Windows, after all the movedAndreas Baumann
2012-09-06more splitting into libcrawl, crawl binaryAndreas Baumann
2012-09-06first properly working logger on Windows (singleton dll issue)Andreas Baumann
2012-09-05started to solve the logger dll problem on WindowsAndreas Baumann
2012-09-05split away util, logger, and moduleAndreas Baumann
2012-09-02prefer dynamic to static linking (linking a non-PIC .a libraryAndreas Baumann
2012-08-18some code cleanup in fetchersAndreas Baumann
2012-08-17adapted winhttp fetcher to new spooling, but crashesAndreas Baumann
2012-08-17added a common base class for spooling rewind input stream, adaptedAndreas Baumann
2012-08-17some project renamesAndreas Baumann
2012-08-15renamedAndreas Baumann
2012-08-13solved static linking problem of module classes on WindowsAndreas Baumann
2012-08-13winhttp work on windowsAndreas Baumann
2012-08-12-Andreas Baumann
2012-08-12implemented the winhttp fetcher, not working yetAndreas Baumann
2012-08-12added a fetcher module testAndreas Baumann
2012-08-12improved error handling in module loaderAndreas Baumann
2012-08-12better naming of modulesAndreas Baumann
2012-08-12we should use WinINet, not WinHttp, spendid!Andreas Baumann