Garmelon/PFERD
1
0
Fork 0
mirror of https://github.com/Garmelon/PFERD.git synced 2026-04-12 15:35:05 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Commit graph

  • 04e5df4bb8 Update PFERD/crawl/ilias/ilias_web_crawler.py PinieP 2024-11-02 20:26:02 +01:00
  • 1d3880f840 Collapse multi-line f-string that gives an error on version <= 3.11 PinieP 2024-11-02 17:55:21 +01:00
  • 7797301288 Update docs and changelog PinieP 2024-11-02 15:55:00 +01:00
  • 38cd1888c2 Restore KIT URL and remove debug log PinieP 2024-11-02 15:36:31 +01:00
  • 66f196e582 Formatting and License PinieP 2024-11-02 15:21:11 +01:00
  • 1f417ec87d Add shibboleth config option for non KIT PinieP 2024-11-02 15:13:40 +01:00
  • d27640b3f2 abstract shibboleth login for non KIT PinieP 2024-11-02 00:53:41 +01:00
  • 5983200247
         Treat headings as folders in kit-ipd crawler (#99) Tim 2024-11-04 23:53:48 +01:00
  • b8f646581b Add heading extraction change to changelog I-Al-Istannen 2024-11-04 23:45:35 +01:00
  • 26e802d88b
         Add clickable links to file names in the printed report (#100) Tim 2024-11-04 00:32:32 +01:00
  • c34cde519a url-encode paths in report linking I-Al-Istannen 2024-11-04 00:16:36 +01:00
  • f2b6f3bb00 Add clickable links to file names in the printed report Scriptim 2024-11-03 23:39:31 +01:00
  • 6aa53f851e
         Store KIT IPD files in folders according to HTML heading structure Scriptim 2024-11-03 20:13:41 +01:00
  • 48e31bef17
         Merge 77c1f1516c into f5c4e82816 Julius Rüberg 2024-11-03 08:51:52 +00:00
  • f5c4e82816 Delay ilias loop detection after transform I-Al-Istannen 2024-11-02 22:17:26 +01:00
  • f5273f7ca0 Collapse ilias url crawling into normal page crawling I-Al-Istannen 2024-11-02 21:53:47 +01:00
  • fa71a9f44f Add support for mob videos in page descriptions I-Al-Istannen 2024-10-28 20:15:55 +01:00
  • 81d6ff53c4 Respect row flex in descriptions I-Al-Istannen 2024-10-28 19:34:45 +01:00
  • d7a2b6e019 Delete videos from course descriptions I-Al-Istannen 2024-10-28 19:32:16 +01:00
  • 71c65e89d1 Internalize images in course descriptions I-Al-Istannen 2024-10-28 19:31:50 +01:00
  • c1046498e7 Fix download of links without a target URL I-Al-Istannen 2024-10-28 19:15:40 +01:00
  • 8fbd1978af Fix crawling of nested courses I-Al-Istannen 2024-10-28 18:52:09 +01:00
  • 739dd95850
         Use Last-Modified and ETag headers to determine KIT-IPD file versions (#95) Tim 2024-10-27 19:03:47 +01:00
  • bb36571e42 Perform head requests sequentially I-Al-Istannen 2024-10-27 17:30:00 +01:00
  • 2193adadb4
         Catch errors when requesting resource version Scriptim 2024-10-27 17:04:58 +01:00
  • eb80c67535
         Remove unused import Scriptim 2024-10-27 12:46:05 +01:00
  • 58548a1be9
         Move etag/mtime logic to more generic HTTP crawler Scriptim 2024-10-27 12:36:08 +01:00
  • 993d934825
         Refactor persistence of entity tags Scriptim 2024-10-27 12:21:51 +01:00
  • c54c3bcfa1 Fix crawling of favorites I-Al-Istannen 2024-10-27 10:50:59 +01:00
  • 8ff2e198e8
         Avoid reassignment with incompatible types Scriptim 2024-10-25 23:52:38 +02:00
  • 2a59f76170
         Examine ETag header to determine version of KIT-IPD files Scriptim 2024-10-25 22:46:04 +02:00
  • c562c61caf
         Augment KIT-IPD files with mtime from Last-Modified header Scriptim 2024-10-25 21:11:44 +02:00
  • d7f2229978 Bump version to 3.6.0 v3.6.0 I-Al-Istannen 2024-10-23 20:17:47 +02:00
  • fe2fd1e066
         Merge f600d682e7 into 52fdeae752 Pavel Zwerschke 2024-10-22 17:27:16 +02:00
  • 52fdeae752 Crawl custom item groups as folders I-Al-Istannen 2024-10-21 23:41:08 +02:00
  • f9bb2e41cf Sanitize slashes in exercise container names I-Al-Istannen 2024-10-21 16:28:49 +02:00
  • 4f9e2ab48d
         Support named capture groups in regex transformers (#94) Mr. Pine 2024-10-21 15:21:33 +02:00
  • c4f21ef49a Add named capture change to changelog Mr-Pine 2024-10-21 15:00:10 +02:00
  • 2962b85efb Document named capture groups Mr-Pine 2024-10-21 14:52:45 +02:00
  • d0340bd0f8 Support named capture groups in regex transformers Mr-Pine 2024-10-21 14:44:36 +02:00
  • 19beb8f07b Document course overview downloading in config.md I-Al-Istannen 2024-07-31 22:02:43 +02:00
  • c897d9e2f5 Support finding entries for course overview page I-Al-Istannen 2024-06-26 16:39:24 +02:00
  • f600d682e7
         Store version in project metadata Pavel Zwerschke 2024-05-11 19:35:14 +02:00
  • 21a266e302 Update upload-artifact action to v4 Joscha 2024-05-11 16:33:10 +02:00
  • d40d22ef6b
         Bump the gh-actions group with 2 updates dependabot[bot] 2024-05-11 14:20:51 +00:00
  • b29b6f93f8
         run ci twice Pavel Zwerschke 2024-05-11 16:09:46 +02:00
  • 318226d7cb
         fix bump-version script Pavel Zwerschke 2024-05-11 10:27:54 +02:00
  • 422cf05f15
         Move all configuration into pyproject.toml, add x86 mac to CI Pavel Zwerschke 2024-05-11 10:26:19 +02:00
  • 819c6673c7 Update changelog Joscha 2024-05-10 14:37:12 +02:00
  • 89b44c69a7 Update docs Joscha 2024-05-10 14:35:19 +02:00
  • 4b4f72b2ca Fix command name Joscha 2024-05-10 14:34:20 +02:00
  • 778517d8c6 Fix KIT crawler requiring base_url and client_id options Joscha 2024-05-10 14:12:45 +02:00
  • 428b0179fc Remove IliasConfig Joscha 2024-05-10 14:09:07 +02:00
  • ade6309dd9
         Update copyright information Philipp Fruck 2024-05-05 02:34:26 +02:00
  • fd6cb7b966
         docs: Remove some filler words Philipp Fruck 2024-05-05 02:34:00 +02:00
  • 5c87517ceb
         docs: Explain usage with generic ilias Philipp Fruck 2024-04-08 12:02:20 +02:00
  • b01f093474
         fix: Element detection for other universities Philipp Fruck 2024-04-08 11:55:48 +02:00
  • 3a05b90525
         fix circular import for _io_repeat Philipp Fruck 2024-04-08 11:49:28 +02:00
  • 7a00f73e0e
         feat: Add authentication to generic ilias dl Philipp Fruck 2024-05-04 17:51:38 +02:00
  • 5d0621420e
         feat: Generic ilias_web command Philipp Fruck 2024-04-08 11:03:09 +02:00
  • df98153169
         refactor: Extract generic settings from ilias command Philipp Fruck 2024-04-08 10:58:23 +02:00
  • fc1f68ccd9
         refactor: Separate generic and KIT ilias functions Philipp Fruck 2024-05-04 17:44:18 +02:00
  • 3e831c7e23 Fix normalization of meeting names in cards I-Al-Istannen 2024-04-24 22:32:26 +02:00
  • bbcfe9c8dd
         Fix typo in CONFIG.md (#89) Florian Raith 2024-04-19 16:52:18 +02:00
  • 87101fd9ef
         Change 'slow' to 'short' to clarify delay in CONFIG.md Florian Raith 2024-04-18 01:06:59 +02:00
  • 7e096d0083
         Fix typo in CONFIG.md Florian Raith 2024-04-17 20:14:15 +02:00
  • eb01aa86cb Bump version to 3.5.2 v3.5.2 Joscha 2024-04-14 12:10:17 +02:00
  • 3db186a978 Fix personal desktop crawling HTML warnings I-Al-Istannen 2024-04-10 11:12:55 +02:00
  • 4a5959fd58 Fix personal desktop crawling without favorites I-Al-Istannen 2024-04-10 11:12:48 +02:00
  • 1cbc2b717a Fix personal desktop crawling with ILIAS 8 I-Al-Istannen 2024-04-10 01:01:59 +02:00
  • da627ff929 Bump version to 3.5.1 v3.5.1 Joscha 2024-04-09 14:28:56 +02:00
  • c1b592ac29 Fix ILIAS 8 file downloads truncating to zero bytes I-Al-Istannen 2024-04-08 17:52:13 +02:00
  • eb0c956d32 Add compatibility with ILIAS 8 I-Al-Istannen 2024-04-05 19:06:54 +02:00
  • ab0cb2d956
         nix: bump nixpgs dependency TornaxO7 2024-02-27 23:39:53 +01:00
  • 79370e5f53
         nix: update flake und remove black magic TornaxO7 2024-02-27 23:34:29 +01:00
  • a117126389 Fix video name deduplication I-Al-Istannen 2023-12-09 23:01:59 +01:00
  • e9f8901520 Fix typos in ilias crawler and use set literals I-Al-Istannen 2023-11-30 20:50:53 +01:00
  • 266812f90e Move is_logged_in helper to kit_ilias_html I-Al-Istannen 2023-11-16 10:34:49 +01:00
  • 03efa17cf1 Print mtime before updating file metadata debug/mtime-error-22 I-Al-Istannen 2023-09-23 12:37:56 +02:00
  • 533bc27439 Bump version to 3.5.0 v3.5.0 I-Al-Istannen 2023-09-13 23:13:30 +02:00
  • 0113a0ca10 Update flake.lock I-Al-Istannen 2023-09-13 22:23:21 +02:00
  • 40f8a05ad6 Add .idea to gitignore I-Al-Istannen 2023-09-13 22:23:31 +02:00
  • 50b50513c6 Ignore SCORM learning modules I-Al-Istannen 2023-08-29 13:51:19 +02:00
  • df3514cd03 Crawl paginated past meetings I-Al-Istannen 2023-08-29 12:30:54 +02:00
  • ad53185247 Sanitize ascii control characters on windows I-Al-Istannen 2023-08-29 12:25:16 +02:00
  • 87b67e9271 Crawl files in the info tab I-Al-Istannen 2023-08-29 11:52:16 +02:00
  • b54b3b979c Remove size suffix for content pages I-Al-Istannen 2023-08-27 11:42:25 +02:00
  • 2184ac8040 Add support for ILIAS mediacast listings I-Al-Istannen 2023-08-26 19:39:40 +02:00
  • b3d412360b Add Nix flake I-Al-Istannen 2023-08-26 23:48:14 +02:00
  • dbc2553b11 Add default show-not-deleted option If set to no, PFERD won't print status or report messages for not deleted files Mr. Pine 2023-03-15 15:33:42 +01:00
  • e41a22149e Add default show-not-deleted option If set to no, PFERD won't print status or report messages for not deleted files Mr. Pine 2023-03-15 15:33:42 +01:00
  • 68c398f1fe Add support for ILIAS learning modules I-Al-Istannen 2023-07-29 23:23:10 +02:00
  • 123a57beec Fix mypy unreachable error in file_templates I-Al-Istannen 2023-07-29 18:14:57 +02:00
  • d204dac8ce Detect unexpected root page redirects and abort operation I-Al-Istannen 2023-06-02 18:19:39 +02:00
  • 443f7fe839 Add no-delete-prompt-overwrite crawler conflict resolution option (#75) Mr. Pine 2023-07-29 17:54:42 +02:00
  • bf27f4a686 TODO sequential Joscha 2022-11-24 15:47:01 +01:00
  • 5adfdfbd2b Switch http_crawler to requests Joscha 2022-11-24 11:24:05 +01:00
  • 5c3942a13d Fix flake8 error Joscha 2022-11-24 07:53:52 +01:00
  • 5c9209b12e Document path formatting functions Joscha 2022-11-23 23:48:17 +01:00
  • 50c7778d38 Use mypy to install library stub packages Joscha 2022-11-23 23:12:14 +01:00