PFERD

Garmelon/PFERD

Fork 0

mirror of https://github.com/Garmelon/PFERD.git synced 2026-04-13 07:55:05 +02:00

Commit graph

5595a908d8 Configure entry point Joscha 2021-04-27 00:29:42 +02:00
27e4abcfa3 Do project setup from scratch Joscha 2021-04-26 23:46:44 +02:00
54a446c43c

await authentication in ilias crawler be7a 2021-04-24 16:21:48 +02:00
ad730f5259

fix another bad search and replace be7a 2021-04-24 00:05:22 +02:00
4f966c5ee2

fix incorrect httpx vs requests replace be7a 2021-04-23 22:56:45 +02:00
07218e6cab

remove false help text and fix httpx auth be7a 2021-04-23 22:07:30 +02:00
411d4b91d5

use the async client to crawl illias be7a 2021-04-23 21:43:52 +02:00
44aeb6c2eb

transition from requests to httpx be7a 2021-04-23 18:02:57 +02:00
f4531906f6 Add sync_url flag to only print the summary Christophe 2021-04-19 20:53:00 +02:00
2ca1101326 Fix typo in sync_url I-Al-Istannen 2021-04-19 14:53:16 +02:00
c1ab7485e2 Bump version to 2.6.1 v2.6.1 I-Al-Istannen 2021-04-19 11:21:56 +02:00
29cd5d1a3c Reflect totality of sanitize_windows_path in return type I-Al-Istannen 2021-04-19 11:10:02 +02:00
6d5d9333ad Force folder to be file-system path I-Al-Istannen 2021-04-19 11:07:25 +02:00
7cc40595dc Allow synchronizing to directory "." I-Al-Istannen 2021-04-14 20:25:25 +02:00
80ae5ddfaa Bump version to v2.6.0 v2.6.0 I-Al-Istannen 2021-04-14 19:47:41 +02:00
4f480d117e Install keyring in CI I-Al-Istannen 2021-04-14 19:24:05 +02:00
1f2af3a290 Retry on more I/O Errors I-Al-Istannen 2021-04-13 11:32:55 +02:00
14cdfb6a69 Fix typo in date demangler doc I-Al-Istannen 2021-04-13 11:19:51 +02:00
e2bf84392b [sync_url] Properly declare "no-videos" as flag I-Al-Istannen 2021-04-08 18:12:27 +02:00
946b7a7931 Also crawl .c/.java/.zip from IPD page I-Al-Istannen 2021-02-09 12:30:59 +01:00
9a9018751e Bump version v2.5.4 I-Al-Istannen 2021-02-06 22:54:05 +01:00
83b75e8254 syncurl: Sanitize element name on windows if it is used as folder name I-Al-Istannen 2021-02-06 22:51:08 +01:00
86062f0b3f

Add KeyboardInterrupt detection Pavel Zwerschke 2021-02-06 10:24:45 +01:00
b2f547a626

Merge branch 'master' into glob-enhancements Toorero 2021-01-29 14:24:22 +01:00
35c3fa205d

Fixed description of activating venv (#22) Toorero 2021-01-28 21:24:09 +01:00
d51d2e5bd0 Removed environment special gitignore entry Toorero 2021-01-28 20:44:24 +01:00
86f340dd05 Removed glob-feature from this branch Toorero 2021-01-28 15:03:21 +01:00
3491c45316 Added use-case example. Toorero 2021-01-28 15:01:20 +01:00
743fd3c3d4 Updated .gitignore Toorero 2021-01-28 14:24:35 +01:00
a9e8c13ca4 Addes switches in Organizer and Pferd to force and change the default user conflict resolution Toorero 2021-01-28 13:56:09 +01:00
27ece2ab97 Added posix glob with pathmatch Toorero 2021-01-27 13:09:42 +01:00
b09afd70a4

Fixed description of activating venv Toorero 2021-01-25 15:56:51 +01:00
0b606f02fa Bump version v2.5.3 I-Al-Istannen 2021-01-17 10:33:10 +01:00
fb78a6e98e Retry ILIAS downloads a few times and only fail that file I-Al-Istannen 2021-01-06 12:29:24 +01:00
5de68a0400 Bump version v2.5.2 I-Al-Istannen 2020-12-30 17:20:30 +01:00
f0562049b6 Remove Python 3.9 method in crawler I-Al-Istannen 2020-12-30 17:18:04 +01:00
0e1077bb50 Bump version v2.5.1 I-Al-Istannen 2020-12-30 14:50:49 +01:00
c978e9edf4 Resolve a few pylint warnings I-Al-Istannen 2020-12-30 14:45:46 +01:00
2714ac6be6 Send CSRF token to Shibboleth I-Al-Istannen 2020-12-30 14:34:11 +01:00
9b048a9cfc Canonize meeting names to a properly formatted date I-Al-Istannen 2020-12-30 14:32:59 +01:00
1c2b6bf994 Bump version v2.5.0 I-Al-Istannen 2020-12-13 19:57:29 +01:00
ee39aaf08b Fix merge marker in LICENSE I-Al-Istannen 2020-12-07 22:55:28 +01:00
93e6329901 Use the least destructive conflict resolver if there are multiple I-Al-Istannen 2020-12-06 13:28:08 +01:00
f47b137b59 Fix ILIAS init.py and Pferd.py authenticators I-Al-Istannen 2020-12-05 23:35:20 +01:00
83ea15ee83 Use system keyring service for password auth Scriptim 2020-11-04 00:18:27 +01:00
75471c46d1 Use credential file I-Al-Istannen 2020-11-04 21:18:48 +01:00
1e0343bba6 sync_url: Add username and password args Lucas 2020-11-03 13:38:33 +01:00
0f5e55648b Tell user when the conflict resolver kept existing files I-Al-Istannen 2020-12-05 14:11:51 +01:00
57259e21f4 Print download summary in sync_url I-Al-Istannen 2020-12-05 14:08:00 +01:00
4ce385b262 Treat file overwrite and marked file overwrite differently I-Al-Istannen 2020-12-05 14:03:43 +01:00
2d64409542 Fix handling of empty args.folder I-Al-Istannen 2020-12-05 13:50:46 +01:00
fcb3884a8f Add --remote-first, --local-first and --no-delete flags I-Al-Istannen 2020-12-05 13:47:53 +01:00
9f6dc56a7b Use a strategy to decide conflict resolution I-Al-Istannen 2020-12-02 19:29:52 +01:00
56ab473611

Merge pull request #17 from TheChristophe/master I-Al-Istannen 2020-12-02 19:04:46 +01:00
6426060804 Fix relative paths bug Christophe 2020-12-02 18:40:45 +01:00
49a0ca7a7c Add myself to LICENSE Christophe 2020-12-02 16:59:29 +01:00
f3a4663491 Add passive/no_prompt flag Christophe 2020-12-02 16:58:36 +01:00
ecdbca8fb6 Make sync_url work relative to cwd like sane programs Christophe 2020-12-02 16:50:30 +01:00
9cbea5fe06 Add requirements.txt I-Al-Istannen 2020-11-23 10:16:40 +01:00
ba3c7f85fa Replace "\" in ILIAS paths as well I-Al-Istannen 2020-11-19 19:37:28 +01:00
373ba57bbf Added backslash in path sanitization and removed redundant sanitization in sanitize_windows_path Felix 2020-11-19 18:27:28 +01:00
ba9215ebe8 Bump version v2.4.5 I-Al-Istannen 2020-11-18 10:09:45 +01:00
8ebf0eab16 Sort download summary I-Al-Istannen 2020-11-17 21:36:04 +01:00
cd90a60dee Move "sanitize_windows_path" to PFERD.transform I-Al-Istannen 2020-11-12 20:52:46 +01:00
98834c9c95 Bump version v2.4.4 I-Al-Istannen 2020-11-12 20:23:36 +01:00
55e9e719ad Sanitize "/" in ilias path names I-Al-Istannen 2020-11-12 19:32:45 +01:00
a0ae9aee27 Sanitize individual path parts v2.4.3 I-Al-Istannen 2020-11-11 09:36:20 +01:00
1486a63854 Do not collapse directory structure when sanitizing I-Al-Istannen 2020-11-10 22:53:47 +01:00
733e1ae136 Bump version I-Al-Istannen 2020-11-10 20:50:31 +01:00
4ac51048c1 Use "_" as a replacement for illegal characters I-Al-Istannen 2020-11-10 20:49:14 +01:00
f2aba970fd [sync_url] Sanitize path names on windows I-Al-Istannen 2020-11-10 15:27:12 +01:00
9c4759103a Bump patch version v2.4.2 I-Al-Istannen 2020-11-05 11:25:06 +01:00
316b9d7bf4 Prevent too many retries when fetching an ILIAS page I-Al-Istannen 2020-11-04 22:20:40 +01:00
6f30adcd22 Fix quote type in README I-Al-Istannen 2020-11-04 22:12:33 +01:00
6f78fef604 Add quoting instructions to README I-Al-Istannen 2020-11-04 22:08:33 +01:00
b0763b3e4f

Merge branch 'sync_url_authentication' into keyring Scriptim 2020-11-04 21:53:52 +01:00
f830b42a36 Fix duplicate files in download summary I-Al-Istannen 2020-11-04 21:49:35 +01:00
add5cc5915 Use credential file I-Al-Istannen 2020-11-04 21:18:48 +01:00
6c0ad0687a

Merge pull request #11 from Luro02/master I-Al-Istannen 2020-11-04 20:57:42 +01:00
ef343dec7c Merge organizer download summaries I-Al-Istannen 2020-11-04 15:06:58 +01:00
0da2fafcd8 Fix links outside tables I-Al-Istannen 2020-11-04 14:38:15 +01:00
95fc13148b

Mention Scriptim in LICENSE Scriptim 2020-11-04 00:28:37 +01:00
0b7e3a81e8

Add section about optional keyring dependency Scriptim 2020-11-04 00:26:13 +01:00
b174b5020a

Use system keyring service for password auth Scriptim 2020-11-04 00:18:27 +01:00
f4abe3197c Add ipd crawler I-Al-Istannen 2020-11-03 20:40:09 +01:00
38d4f5b4c9 Do not fail only empty courses I-Al-Istannen 2020-11-03 20:09:54 +01:00
dcea7b2254

sync_url: Add username and password args Lucas 2020-11-03 13:38:33 +01:00
9ea03bda3e Adjust release names v2.4.1 I-Al-Istannen 2020-10-30 18:02:46 +01:00
07de5bea8b Explain how to run sync_url on Mac I-Al-Istannen 2020-10-30 17:53:55 +01:00
f0d572c110 Fix a few typos in release body I-Al-Istannen 2020-10-30 17:32:04 +01:00
076067e22d Bump version I-Al-Istannen 2020-10-30 17:24:40 +01:00
ebb6e63c5c Add MacOS to CI I-Al-Istannen 2020-10-30 17:02:56 +01:00
0c3f35a2d2 Do not provide a shorthand for "no-videos" I-Al-Istannen 2020-10-30 16:58:44 +01:00
521890ae78

Update README.md I-Al-Istannen 2020-10-28 23:24:18 +01:00
3f7c73df80 Release new minor version v2.4.0 I-Al-Istannen 2020-10-07 09:32:17 +02:00
43100f69d5

Merge pull request #10 from Garmelon/sync-url I-Al-Istannen 2020-10-07 09:29:48 +02:00
d73c778b0a Add sync_url instructions to README I-Al-Istannen 2020-10-06 17:46:42 +02:00
73c3eb0984 Add option to skip videos in sync_url I-Al-Istannen 2020-10-06 17:15:10 +02:00
a519cbe05d Add sync_url workflow I-Al-Istannen 2020-10-06 12:03:16 +02:00
b3ad9783c4 Ignore pyinstaller files I-Al-Istannen 2020-10-06 11:43:20 +02:00