Garmelon/PFERD
1
0
Fork 0
mirror of https://github.com/Garmelon/PFERD.git synced 2026-04-12 07:25:04 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Add 'skip' option to crawlers

Browse source
This commit is contained in:
Joscha 2021-06-04 18:33:02 +02:00
parent fc31100a0f
commit df3ad3d890
6 changed files with 48 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

3
CONFIG.md
View file

@ -49,6 +49,9 @@ see the type's [documentation](#crawler-types) below. The following options are
common to all crawlers:
- `type`: The available types are specified in [this section](#crawler-types).
- `skip`: Whether the crawler should be skipped during normal execution. The
crawler can still be executed manually using the `--crawler` or `-C` flags.
(Default: `no`)
- `output_dir`: The directory the crawler synchronizes files to. A crawler will
never place any files outside of this directory. (Default: the crawler's name)
- `redownload`: When to download a file that is already present locally.