evanferrao
/
dump-mirrorbot
mirror of https://github.com/Box-boi/dump-mirrorbot.git
1
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity
442 Commits 2 Branches 21 Tags 2 MiB
Commit Graph

143 Commits

Author SHA1 Message Date
lzzy12 e5364f96e1 Revert "Handle rare bug when download is not deleted from download_dict on error" 
This reverts commit 136924ffd4.

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-13 13:32:39 +05:30
lzzy12 76e3db1946 [Hotfix] Fix bug where the link have 302 request are treated as torrents 
Download.is_torrent introduced in aria2p v0.9.0
https://github.com/pawamoy/aria2p/issues/53

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-08 19:29:59 +05:30
Arsalan ed60afa9d6
fix typo in INDEX_URL (#54) 2020-04-07 20:36:04 +05:30
lzzy12 b2f2ee4335 Fix aria downloads cancellation after b58eb87b11 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-06 13:14:40 +05:30
lzzy12 47dd3c2b6f gdriveTools: Handle unhandled exception in clone method 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-05 14:12:13 +05:30
lzzy12 0536472e28 Expose message object from Status objects 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-05 13:28:35 +05:30
lzzy12 b58eb87b11 Improvements in cancelling mirrors 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-05 12:34:08 +05:30
jaskaranSM 708e6df87a youtube-dl: use best format 2020-04-01 12:07:29 +05:30
lzzy12 50ab740219 Correct filename if ffmpeg conversion is invoked by yt-dl 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-31 17:56:57 +05:30
lzzy12 a54fc42e1e Handle some exceptions 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-31 15:42:13 +05:30
lzzy12 e2c5c8beaf Log youtube-dl warnings and errors in bot log 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-31 11:15:09 +05:30
lzzy12 85d4005c16 Use netrc by default for youtube-dl 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-31 10:50:53 +05:30
lzzy12 9fa41f68c1 Some minor Fix-ups 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-30 18:16:54 +05:30
lzzy12 136924ffd4 Handle rare bug when download is not deleted from download_dict on error 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-30 17:46:56 +05:30
jaskaranSM 52e8fbec11 Fixed direct link torrent mirroring 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-30 17:05:55 +05:30
lzzy12 0ec499ff4a Handle weird path naming of youtube_dl 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-30 16:36:12 +05:30
lzzy12 141b13b52d Make ydl object local to the methods 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-30 14:55:29 +05:30
lzzy12 3bf9e0ea31 Use default quality provided by youtube-dl for now 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-28 20:14:43 +05:30
jaskaranSM 6262b26fe5 youtube-dl: fixed downloading from sites that provide encrypted content (m3u8) 2020-03-28 18:44:02 +05:30
lzzy12 7c304d449a Fix uploading for YT-DL 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-28 14:38:08 +05:30
lzzy12 9057d3fc4e Fixed yt-dl support 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-28 13:47:19 +05:30
jaskaranSM 3a06458609 Added Youtube-dl support 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-27 20:04:13 +05:30
lzzy12 2254444378 Handle KeyError exception in Telegram download error 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-27 16:00:37 +05:30
Shivam Jha 6d07c3c8c4
Stop trying to generate sourceforge direct links 2020-03-26 16:02:36 +05:30
jaskaranSM be9c64a9aa Use Service Accounts for Cloning if availble. 
- increase default timeout for network socket interface

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-25 18:00:33 +05:30
jaskaranSM 08a71e4bd4 TeamDrive List fix 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 18:57:10 +05:30
lzzy12 6ca2f479d5 Fix up errors after porting service accounts from develop 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 16:30:03 +05:30
lzzy12 25856bfac0 gdrive: Stop spamming logs at INFO log level 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 15:49:04 +05:30
jaskaranSM 8d1bf5aa92 recursively try again instead of handling switch with backoff 
- Handling service account switching with exponential backoff implementation limits the number of service accounts to be used. recursively retrying again after switch doesnt effect backoff.

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 15:41:59 +05:30
jaskaranSM 6cd59d5a15 Cleanup drive_list and minor fixes 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 15:39:57 +05:30
jaskaranSM 681b8bd47d Added missing property 2020-03-24 15:07:45 +05:30
jaskaranSM 2bbfca0496 Refactor service account implementation 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 15:07:23 +05:30
lzzy12 7ad7d9388a Add support for service account 
Adding support service accounts

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>

Added scripts and docs for generating service accounts

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>

gen_sa_accounts: Save credentials with indexed file name

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>

gdriveTools: Avoid using oauth2 library for service accounts

oauth2 library is deprecated

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 15:04:24 +05:30
jaskaranSM 997e572b69 Enable TeamDrive support for clone 2020-03-24 14:17:50 +05:30
lzzy12 94b6be4c03 Add licensing and authorship information for direct_link_generator.py 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-23 15:41:53 +05:30
lzzy12 f50981b3a2 Implement cancel downloads of tg mirrors 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-23 15:21:54 +05:30
lzzy12 9369c7dc98 Allow mirroring tg file again once file download is complete 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-23 14:49:57 +05:30
lzzy12 7d74cf29d8 Fixed support for mirror of video and audio telegram files 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-22 16:30:57 +05:30
lzzy12 d6a513e77b Fix typo in direct_link_generator.py 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-22 15:48:14 +05:30
lzzy12 b4b0dcd3f8 Improve logic for parsing id from gdrive url 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-22 15:05:12 +05:30
lzzy12 49fe46bad3 Add gid method in Status class for download classes 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-22 15:05:12 +05:30
jaskaranSM 6b60ca4491 Squash of clone command 
Implement clone feature
- clones a public/driveAccountAccessible Drive Link to bot G-Drive

use BotCommands for clone command trigger
 2020-03-22 15:05:11 +05:30
lzzy12 94ccdcbeef Add support for mirror of telegram files 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>

Add a script to generate string session for user

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>

Some fix ups

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>

Fix telegram download

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-22 15:04:31 +05:30
lzzy12 898977df97 Use proper url-encoding methods to replace special characters from url 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-20 17:03:33 +05:30
lzzy12 7ba7bc92ac Add support for Direct Download generation 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-19 14:22:33 +05:30
lzzy12 2fddd06178 Fixed upload of empty files for Team Drive 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-02-05 02:49:54 -08:00
jaskaranSM 8cbe2c29e4 Added peers and seeders count for bittorent downloads 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-02-05 02:45:38 -08:00
Shivam Jha 96e623ef57
UploadStatus: Handle exception ZeroDivisionError 2020-02-05 01:52:43 -08:00
Shivam Jha 5c1a9c71b7
gdriveTools: Hardcode interval for updating progress 2020-01-17 01:34:52 -08:00
Shivam Jha 00a8e57467
Fixed error in gdrivetools 2020-01-10 17:36:51 +05:30
lzzy12 326129fdeb Make Oauth Scope a list as it should be 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-01-04 07:42:55 +05:30
lzzy12 e5f997d1f3 Revert "Fixed potential race conditions in GoogleDriveHelper class" 
This reverts commit 20dae25dc9.

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-01-03 10:41:54 +05:30
jaskaranSM 7b2d5d50ab Implement Exponential Backoff 
- for handling Google Drive API errors such as 5xx,4xx
 2020-01-03 10:15:40 +05:30
jaskaranSM a64b1b5346 Fixed stucked downloads when stopped via aria bt-stop-timeout 2019-12-29 19:50:01 +05:30
jaskaranSM 30dd2d97f5 Minor Refactor 2019-12-29 19:14:52 +05:30
jaskaranSM 38fa1ec2a6 Fixed Handling of Cancellation of uploads/archivals 2019-12-29 19:13:10 +05:30
lzzy12 20dae25dc9 Fixed potential race conditions in GoogleDriveHelper class 
* Also simplified code a bit
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-12-26 16:39:59 +05:30
lzzy12 820d72f317 Fixed files not being shown which were not uploaded by bot 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-12-26 14:47:05 +05:30
lzzy12 4e1f7a8a77 Small optimisations all around 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-12-25 16:40:43 +05:30
lzzy12 8e886a3030 Add Index URL in /list cmd 
Also simplified code a bit

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-12-25 15:53:43 +05:30
lzzy12 53b6829235 Fixed Team Drive list functionality 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-12-25 15:15:36 +05:30
jaskaranSM 8a516024cb Fixed Folder upload to Team Drives 2019-12-24 21:11:33 +05:30
lzzy12 b31a39f11a Enable TeamDrive support 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-12-24 20:12:39 +05:30
lzzy12 0ed1ca3531 Enable TeamDrive support 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-12-24 19:49:48 +05:30
lzzy12 82ae49a33a Re implement upload progress after rebase 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-12-24 17:24:35 +05:30
lzzy12 e2e1453013 Fixup errors after rebase 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-12-23 16:17:57 +05:30
jaskaranSM 0d30f19a4e Project Rebase 
- Changelog:
 Improved Performance
 Fixed Stucked Messages
 Seperated Download Progress Status From Listener
 Disabled Upload Progress (Temporarily)
 2019-12-23 13:26:32 +05:30
lzzy12 4f9fa3f63f Fix typo 
Also Adds support for heroku env variables

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-12-09 17:05:57 +05:30
lzzy12 db0dcb0d8b [DO NOT DEPLOY] Incomplete save 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-11-12 17:32:58 +05:30
jaskaranSM 584bd138df Refactored Download start logic 2019-11-12 15:01:08 +05:30
Shivam Jha df82d5944e
Fix progress update in torrent download 2019-11-10 01:24:13 +05:30
lzzy12 da622c04be Don't depend on bot to run for generate_gdrive_token.py 2019-11-07 14:48:35 +05:30
Shivam Jha 7b5375fd21
Fix up after f7960a7dfe 2019-11-01 19:00:35 +05:30
lzzy12 bf956b3fa6 Merge branch 'master' of https://github.com/lzzy12/python-aria-mirror-bot 2019-11-01 16:54:04 +05:30
lzzy12 f7960a7dfe Fix empty file progress 
Fixes #8

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-11-01 16:33:54 +05:30
jaskaranSM b9084fe506
list: Dont try to get size for folders 2019-10-31 22:23:40 +05:30
Shivam Jha c754d228e4
gdriveTools: Fix error in search 2019-10-31 00:04:43 +05:30
Shivam Jha 830ee1ad3e
gdriveTools: Fix search 
Recent force push seems to cause loss of commits. 
* Lesson learnt: NEVER FORCE PUSH IN MASTER BRANCH
 2019-10-31 00:02:55 +05:30
jaskaranSM deb4b43fdd Validate Url and Magnet with Regex 2019-10-27 17:29:02 +05:30
jaskaranSM 3bac006fb3 Output file size in list search results 
- Also Listing Recent Uploads Fixed. ( * )
 2019-10-22 19:47:50 +05:30
jaskaranSM fdb49ac62a Fixed Cancelled Downloads stuck in Download Queue. 
- Also some minor Modifications.
 2019-10-22 18:20:47 +05:30
jaskaranSM b7e65d9c9c Changed sleep() call for Fixing Delay. 
- Also Tweaked Torrent Metadata Status Logic a bit.
 2019-10-19 16:44:26 +05:30
lzzy12 d38bc90e82 Don't do anything on KillThreadException until we figure a better way 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-10-14 20:51:34 +05:30
lzzy12 a0cdda9e6e Improve gdrive progress tracking 
* Still bugged

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-10-14 19:49:33 +05:30
lzzy12 c1e4bf246c Simply sleep the upload progress thread instead of using event locks 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-10-12 22:14:22 +05:30
lzzy12 4fa2e71e98 Show upload ETA in appropriate time units 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-10-12 22:12:07 +05:30
lzzy12 70eaada75f Stop blocking the thread while uploading the file causing slow speeds 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-10-12 21:22:21 +05:30
lzzy12 16886f5ac8 Make changing of bot commands easier 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-10-12 20:56:49 +05:30
lzzy12 9d83cd31c7 Random fixes all around the source 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-10-11 00:30:58 +05:30
lzzy12 ce16f77b6d Clean download directory on startup 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-10-07 23:17:09 +05:30
lzzy12 65303e3143 Change status while archiving download 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-10-07 23:05:05 +05:30
lzzy12 80388249c5 Implement tar-mirror invoked by /tarmirror 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-10-07 22:13:38 +05:30
lzzy12 d36ba18a9f Categorise modules according to their jobs 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2019-10-07 22:13:38 +05:30