Garmelon/PFERD
1
0
Fork 0
mirror of https://github.com/Garmelon/PFERD.git synced 2026-04-12 23:45:05 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Commit graph

  • c1ccb6c53e Allow crawling videos with sync_url I-Al-Istannen 2020-10-06 10:39:34 +02:00
  • 51a713fa04 Allow crawling courses or folders with sync_url I-Al-Istannen 2020-09-28 20:00:01 +02:00
  • 74ea039458 Fix a few lint errors and pferd quirks in sync_url I-Al-Istannen 2020-09-28 19:42:59 +02:00
  • aaa6a2b6a4
         Merge pull request #9 from TheChristophe/master I-Al-Istannen 2020-09-28 19:25:45 +02:00
  • e32a49480b Expose methods to look up course/element names by id / url I-Al-Istannen 2020-09-28 17:01:46 +02:00
  • be65051f9d Support downloading folders in get-by-url script Christophe 2020-09-28 18:14:20 +02:00
  • 3387bc5f20 Add simple course-download-by-url script Christophe 2020-09-28 17:49:36 +02:00
  • 3f0ae729d6 Expand "is course" check to not download magazines or other weird things I-Al-Istannen 2020-09-28 16:36:03 +02:00
  • 8e8c1c031a Version 2.3.0 v2.3.0 I-Al-Istannen 2020-09-03 21:47:10 +02:00
  • 55678d7fee Pass string down to FileCookieJar I-Al-Istannen 2020-08-12 09:09:14 +02:00
  • a57ee8b96b Add timeout to video downloads to work around requests IPv6 bug I-Al-Istannen 2020-08-11 14:40:13 +02:00
  • e367da925e Bump version to 2.2.1 v2.2.1 Joscha 2020-07-28 19:55:32 +00:00
  • 77a109bb7e Fix ilias shibboleth authenticator Joscha 2020-07-28 19:13:51 +00:00
  • a3e1864a26 Allow long paths on windows I-Al-Istannen 2020-07-24 09:22:51 +00:00
  • 41cbcc509c Update version to 2.2.0 v2.2.0 I-Al-Istannen 2020-07-15 22:33:18 +02:00
  • 77874b432b Also add personal_desktop to download summary I-Al-Istannen 2020-07-15 22:47:28 +02:00
  • 5c4c785e60 Fix HTML file downloading I-Al-Istannen 2020-07-15 15:10:31 +02:00
  • 2aed4f6d1f Only query the dir_filter for directories I-Al-Istannen 2020-07-13 13:36:12 +02:00
  • 34152fbe54 Set mtime and atime to ILIAS dates where possible I-Al-Istannen 2020-07-13 13:12:01 +02:00
  • 4047fe78f3 Fix README formatting Joscha 2020-07-11 18:21:32 +00:00
  • c28347122e Improve README Joscha 2020-07-11 18:16:33 +00:00
  • 5b38ab8cf1 Add MIT license Joscha 2020-07-08 09:46:27 +00:00
  • bb25d32f03 Fix typo in README I-Al-Istannen 2020-06-29 16:18:33 +02:00
  • ecaedea709
         Merge pull request #8 from pavelzw/master v2.1.2 I-Al-Istannen 2020-06-26 17:52:05 +02:00
  • f05d1b1261
         Fix version number Pavel Zwerschke 2020-06-26 17:49:47 +02:00
  • 6aaa3071f9 Update README with new version I-Al-Istannen 2020-06-26 17:35:03 +02:00
  • c26c9352f1 Make DownloadSummary private, provide property accessors I-Al-Istannen 2020-06-26 17:30:45 +02:00
  • d9ea688145 Use pretty logger for summaries I-Al-Istannen 2020-06-26 15:52:07 +02:00
  • e8be6e498e Add summary to example_config_personal_desktop I-Al-Istannen 2020-06-26 17:24:23 +02:00
  • e4b1fac045 Satisfy pylint I-Al-Istannen 2020-06-26 15:37:35 +02:00
  • 402ae81335 Fix type hints Joscha 2020-06-26 13:17:44 +00:00
  • 52f31e2783
         Add type hints to DownloadSummary Daniel Augustin 2020-06-26 13:02:37 +02:00
  • 739522a151
         Move download summary into a separate class Daniel Augustin 2020-06-25 21:55:08 +02:00
  • 6c034209b6
         Add deleted files to summary Daniel Augustin 2020-06-25 21:30:03 +02:00
  • f6fbd5e4bb
         Add download summary Daniel Augustin 2020-06-25 15:41:58 +02:00
  • 7024db1f13 Use transient progessbar v2.1.1 I-Al-Istannen 2020-06-25 17:38:35 +02:00
  • 23bfa42a0d Never use the direct download button, as it is currently broken I-Al-Istannen 2020-06-11 13:31:01 +02:00
  • fdb57884ed Touch files with same content to update timestamps I-Al-Istannen 2020-05-31 20:27:15 +02:00
  • f614b95a00 Adjust version in setup.py v2.1.0 I-Al-Istannen 2020-05-30 19:07:02 +02:00
  • 8198c9ecaa Reorder methods a bit I-Al-Istannen 2020-05-30 15:53:31 +02:00
  • 086b15d10f Crawl a bit more iteratively I-Al-Istannen 2020-05-30 15:35:58 +02:00
  • 9d6ce331a5 Use IliasCrawlerEntry entries in the ilias scraper I-Al-Istannen 2020-05-30 15:04:54 +02:00
  • 821c7ade26 Move video url extraction logic to crawler I-Al-Istannen 2020-05-30 00:14:08 +02:00
  • b969a1854a Remove unneeded whitespace I-Al-Istannen 2020-05-30 00:22:17 +02:00
  • 62535b4452 Unpack videos in ILIAS downloader I-Al-Istannen 2020-05-21 21:57:31 +02:00
  • c0056e5669 Correctly crawl video pages with multiple pages I-Al-Istannen 2020-05-21 21:13:19 +02:00
  • cfe4a8fc0a Bump version to 2.0.0 v2.0.0 Joscha 2020-05-15 11:26:23 +00:00
  • 95b9248a25 Clean up Joscha 2020-05-15 11:26:09 +00:00
  • 1004fa40f8 Add personal desktop example config to README I-Al-Istannen 2020-05-15 11:02:55 +02:00
  • e8ddb0ca04 Fix example config link in README I-Al-Istannen 2020-05-15 11:02:45 +02:00
  • 36c8785f15 Add example config that synchronizes the personal desktop I-Al-Istannen 2020-05-15 11:02:13 +02:00
  • 03a801eecc Correctly type hint swallow_and_print_errors decorator I-Al-Istannen 2020-05-12 21:02:41 +02:00
  • 072c6630bf Avoid logging import in config Joscha 2020-05-12 18:19:23 +00:00
  • 4f56c8f192 Pass element type to ilias directory filter I-Al-Istannen 2020-05-12 14:38:25 +02:00
  • 4fdb67128d Fetch correct diva playlist id I-Al-Istannen 2020-05-11 00:25:34 +02:00
  • a0f9d31d94 Use PrettyLogger warning everywhere I-Al-Istannen 2020-05-10 21:53:24 +02:00
  • e7b08420ba Warn when a marked file is added again I-Al-Istannen 2020-05-10 21:37:48 +02:00
  • c1b21f7772 Only remove a progress task when we added it I-Al-Istannen 2020-05-10 12:28:30 +02:00
  • 9850ab1d73 Allow crawling the ILIAS Personal Desktop I-Al-Istannen 2020-05-10 12:16:42 +02:00
  • 9950144e97 Allow passing a playlist URL to diva instead of an id I-Al-Istannen 2020-05-10 11:11:28 +02:00
  • f6faacabb0 Move FatalException to errors.py I-Al-Istannen 2020-05-09 00:00:21 +02:00
  • 19c1e3ac6f Fail on invalid ILIAS course ids I-Al-Istannen 2020-05-08 23:47:05 +02:00
  • afa48c2d2d Swallow and print errors instead of crashing I-Al-Istannen 2020-05-08 23:35:25 +02:00
  • a4c518bf4c Update date find regex I-Al-Istannen 2020-05-08 22:17:58 +02:00
  • 057135022f Try to accept that life sometimes is in English I-Al-Istannen 2020-05-08 22:10:43 +02:00
  • 755e9aa0d3 Try to add support for Shibboleth TFA token I-Al-Istannen 2020-05-08 21:51:33 +02:00
  • c9deca19ca Remove walrus to lower needed python version I-Al-Istannen 2020-05-08 21:21:33 +02:00
  • bb048c3a6d Apparently we want Python 3.8 I-Al-Istannen 2020-05-08 21:03:49 +02:00
  • 33fcd307b2 Adjust install directions I-Al-Istannen 2020-05-08 20:53:41 +02:00
  • a0c5572b59 Fix progress bars swallowing a line when they shouldn't I-Al-Istannen 2020-05-08 19:55:53 +02:00
  • 2d20d2934c Color warning differently I-Al-Istannen 2020-05-08 19:48:21 +02:00
  • 2c48ab66d4 Use rich for log colorization I-Al-Istannen 2020-05-08 19:28:34 +02:00
  • 104b838aed Automatically discover packages in setup.py I-Al-Istannen 2020-05-08 18:55:30 +02:00
  • 7f10931be8 Add rich to setup.py I-Al-Istannen 2020-05-08 18:49:36 +02:00
  • 07c225bc20 Expand README. I did not proofread this thing :( I-Al-Istannen 2020-05-08 18:47:58 +02:00
  • 56f2394001 Add a download progress bar I-Al-Istannen 2020-05-08 00:26:33 +02:00
  • fdff8bc40e example_config: Change db transform I-Al-Istannen 2020-05-01 13:31:29 +02:00
  • bee3d70998 Added a diva playlist downloader I-Al-Istannen 2020-04-30 16:24:38 +02:00
  • 42345ecc61 Demangle "Morgen" too I-Al-Istannen 2020-04-30 12:00:21 +02:00
  • 920d521d68 Change PrettyLogger.warn to PrettyLogger.warning I-Al-Istannen 2020-04-25 20:11:51 +02:00
  • e0b46a306a Use warn method in IliasCrawler I-Al-Istannen 2020-04-25 20:05:19 +02:00
  • 8a42a2a396 Move logging into its own file I-Al-Istannen 2020-04-25 19:59:58 +02:00
  • 80247400a4 Debug log when starting an ilias download I-Al-Istannen 2020-04-25 13:02:07 +02:00
  • 13c5a29ff0 Fix and extend example config Joscha 2020-04-24 18:41:22 +00:00
  • 1aaa6e7ab5 Use PathLike everywhere Joscha 2020-04-24 18:39:30 +00:00
  • 7f53543324 Satisfy pylint and add todo Joscha 2020-04-24 18:26:28 +00:00
  • 292e516297 Change crawler and downloader output Joscha 2020-04-24 18:24:44 +00:00
  • 8258fa8919 Add test run option to PFERD Joscha 2020-04-24 18:00:21 +00:00
  • 5b929f09a2 Move download strategies to downloader Joscha 2020-04-24 14:26:20 +00:00
  • 4d32f863bc Clean up organizer after synchronizing Joscha 2020-04-24 14:17:23 +00:00
  • 4e7333b396 Allow specifying paths as strings in Pferd Joscha 2020-04-24 11:50:27 +00:00
  • 4c0e3b493a Use download_modified_or_new as default strategy I-Al-Istannen 2020-04-24 13:48:06 +02:00
  • 2de079a5d3 Add a few Transform combinators Joscha 2020-04-24 11:35:31 +00:00
  • 509e624d47 Satisfy pyling. Useful docstrings? Not quite sure. I-Al-Istannen 2020-04-23 20:31:32 +02:00
  • ca8fcf7a1d Somewhat elaborate example_config I-Al-Istannen 2020-04-23 20:22:41 +02:00
  • 980f69b5af Fix organizer marking itself causing an error I-Al-Istannen 2020-04-23 20:02:05 +02:00
  • 0b00a9c26b Log when starting to synchronize I-Al-Istannen 2020-04-23 19:55:37 +02:00
  • 1ef85c45e5 Switch Transform to PurePath Joscha 2020-04-23 17:38:41 +00:00
  • 5ef5a56e69 Extract Location into separate file Joscha 2020-04-23 17:38:28 +00:00
  • f3f4be2690 More free functions I-Al-Istannen 2020-04-23 19:18:21 +02:00