weebcentral

An unofficial api for weebcentral.

Installation

pip install weebcentral

Usage

CLI

Download resource from URLs into DIR

weebcentral --directory DIR URL1 URL2 URL3

This will create separate directory named by title of resource where images and data will be stored e.g. for https://weebcentral.com/series/01J76XY7JCJ3VM56H5F8N8RT6R/3x3-Eyes structure similar to

3x3 Eyes
3x3 Eyes/info.json
3x3 Eyes/Chapter 1
3x3 Eyes/Chapter 1/info.json
3x3 Eyes/Chapter 1/01.png
3x3 Eyes/Chapter 1/02.png
3x3 Eyes/Chapter 1/03.png
3x3 Eyes/Chapter 1/04.png
3x3 Eyes/Chapter 2
3x3 Eyes/Chapter 2/info.json
3x3 Eyes/Chapter 2/01.png
3x3 Eyes/Chapter 2/02.png
3x3 Eyes/Chapter 2/03.png
3x3 Eyes/Chapter 2/04.png

where info.json stores the basic information. By default comments are not scraped if you want to get comments for all resources use --comments option. You can disable saving metadata with --images-only or save chapters as --pdf or --cbz files instead.

By default images are named by their order and stylized with zeroes - this is much more practical approach as it won't mess up order of them by sorting files. If you want to have the original names use the --no-num-images option.

Directories of resources are named by their titles, but when saving large amount of them it's not confirmed if conflicts will arise. That's why --id can be used to name the directories by resource's id.

Download only images without metadata using 8 threads (only images are downloaded using multiple threads)

weebcentral --directory DIR --threads 8 --images-only URL1 URL2 URL3

Download comics with all comments and wait 1.2 seconds in between requests and randomly up to 0.8 seconds (waiting time doesn't affect downloading images)

weebcentral --directory DIR --wait 1.2 --wait-random 0.8 URL1 URL2

Download resources from URLs annotating their type

weebcentral --directory DIR --chapter URL1 --chapter URL2 --search URL3 --series URL4 --thread URL5 --user URL6 URL7

usage: python -m weebcentral [-h] [-v] [-t NUM] [-d DIR] [-f] [-c] [--id] [--no-num-images] [--pdf] [--cbz] [--chapter URL] [--series URL] [--user URL] [--thread URL] [--search URL] [--sitemap] [--images-only] [--noimages] [--nochapters] [--comments] [--general-chat] [-w TIME] [-W TIME] [-r NUM] [--retry-delay TIME] [--retry-all-errors] [-m TIMEOUT] [-k] [-L] [--max-redirs NUM] [-A UA] [-x PROXY] [-H HEADER] [-b COOKIE] [-B BROWSER] [URL ...]

Tool for downloading from weebcentral.com

positional arguments:
  URL                   url pointing to source

General:
  -h, --help            Show this help message and exit
  -v, --version         Print program version and exit
  -t, --threads NUM     download images using NUM of threads
  -C, --concurrent NUM  number of max concurrent tasks

Files:
  -d, --directory DIR   Change directory to DIR
  -f, --force           forcefully overwrite files
  -c, --continue        download only new chapters
  --id                  Name resource directories by their ids
  --no-num-images       Don't rename images to their order number with leading zeroes, keep the original name
  --pdf                 Store chapters as pdfs
  --cbz                 Store chapters as cbzs

Types:
  --chapter URL         Treats the following url as manhwa chapter
  --series URL          Treats the following url as series
  --user URL            Treats the following url as user
  --thread URL          Treats the following url as thread
  --search URL          Treats the following url as search query
  --sitemap             Gets series from sitemap

Settings:
  --images-only         ignore all metadata, save only images
  --noimages            download only metadata
  --nochapters          do not download chapters of series
  --comments            get full comment section
  --general-chat        get general chat

Request settings:
  -w, --wait TIME       Set waiting time for each request
  -W, --wait-random TIME Set random waiting time for each request to be from 0 to TIME
  -r, --retry NUM       Set number of retries for failed request to NUM
  --retry-delay TIME    Set interval between each retry
  --retry-all-errors    Retry no matter the error
  -m, --timeout TIMEOUT Set request timeout, if in TIME format it'll be set for the whole request. If in TIME,TIME format first TIME will specify connection timeout, the second read timeout. If set to '-' timeout is disabled
  -k, --insecure        Ignore ssl errors
  -L, --location        Allow for redirections, can be dangerous if credentials are passed in headers
  --max-redirs NUM      Set the maximum number of redirections to follow
  -A, --user-agent UA   Sets custom user agent
  -x, --proxy PROXY     Use the specified proxy, can be used multiple times. If set to URL it'll be used for all protocols, if in PROTOCOL URL format it'll be set only for given protocol, if in URL URL format it'll be set only for given path. If first character is '@' then proxies are read from file
  -H, --header HEADER   Set curl style header, can be used multiple times e.g. -H 'User: Admin' -H 'Pass: 12345', if first character is '@' then headers are read from file e.g. -H @file
  -b, --cookie COOKIE   Set curl style cookie, can be used multiple times e.g. -b 'auth=8f82ab' -b 'PHPSESSID=qw3r8an829', without '=' character argument is read as a file
  -B, --browser BROWSER Get cookies from specified browser e.g. -B firefox

Library

Code

from weebcentral import Api, RequestError

# kwargs are passed to treerequests
api = Api(wait=1.2)

# find series with the longest title
res = None
for i in api.get_series_sitemap():
    url = i['link']
    try:
        series = api.get_series(url,comments=False,chapters=False)
    except RequestError as e:
        print(f"couldn't get series {url} - {repr(e)}")

    if res is None or len(i['name']) > len(res['name']):
        res = i

print(res)

Methods

get_series_sitemap(self) -> list[str]

Gets list of urls to series from sitemap.

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/series-sitemap.json )

returns( list of urls to series )

get_search(self, url: str) -> Iterator[list[dict]]

Gets results of search query specified by arg( url ).

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/search.json )

returns( iterator returning list of results )

get_user_info(self, url: str) -> dict

Gets basic info about user specified by arg( url ) making a single request.

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/user-info.json )

returns( dictionary of user's metadata )

get_user_3x3(self, id: str) -> list[dict]

Gets 3x3 table of a user specified by arg( id ).

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/user-3x3.json )

returns( list of series squares )

get_user_wall(self, id: str) -> Iterator[dict]

Gets comments on user's wall through user arg( id ).

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/user-wall.json )

returns( iterator returning singular comments and replies )

get_user_subscriptions(self, id: str) -> Iterator[list[dict]]

Gets user's subscriptions through user arg( id ).

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/user-subscriptions.json )

returns( iterator returning list of subscribed series )

get_user(self, url: str, txt: bool = True, subscriptions: bool = True, wall: bool = True) -> dict

Get metadata about user by arg( url ).

Certain fields require additional requests and can be disabled by unsetting:

arg( txt )           - for `3x3` field
arg( subscriptions ) - for user subscriptions
arg( wall )          - for comments about user

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/user.json )

returns( dictionary of users metadata )

get_general(self) -> Iterator[list[dict]]

Gets discussion threads from the newest to oldest.

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/general.json )

returns( iterator returning list of threads )

get_thread(self, url: str) -> dict

Gets the whole discussion by it's arg( url ).

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/thread.json )

returns( dictionary with metadata )

id_from_url(url: str) -> str

Extracts id to a resource from it's arg( url ).

returns( id of a resource )

get_series_info(self, url: str) -> dict

Gets basic info about series specified by arg( url ).

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/series-info.json )

returns( dictionary of series metadata )

get_series_comments(self, id: str) -> Iterator[dict]

Get comments about series by it's arg( id ).

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/series-comments.json )

returns( iterator returning singular comments and replies )

get_series_chapters(self, id: str) -> list[dict]

Gets full list of series chapters by it's arg( id ).

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/series-chapters.json )

returns( full list of chapters )

get_series(self, url: str, comments: bool = False, chapters: bool = True) -> dict

Gets metadata about series specified by arg( url ).

Certain fields require additional requests and can be disabled by unsetting:

arg( comments ) - for comments about series
arg( chapters ) - for full list of chapters

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/series.json )

returns( dictionary of series metadata )

get_chapter_images(self, id: str) -> list[str]

Gets full list of images of a chapter by it's arg( id ).

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/chapter-images.json )

returns( list of image urls )

get_chapter_comments(self, id: str) -> Iterator[dict]

Gets comments about chapter specified by arg( id ).

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/chapter-comments.json )

returns( iterator returning singular comments and replies )

get_chapter_info(self, url: str) -> dict

Gets basic info about chapter through it's arg( url ).

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/chapter-info.json )

returns( dictionary with chapter's metadata )

get_chapter(self, url: str, comments: bool = False, images: bool = True) -> dict

Gets metadata about chapter specified by arg( url ).

exampleout( https://raw.githubusercontent.com/TUVIMEN/weebcentral/refs/heads/master/examples/chapter.json )

Certain fields require additional requests and can be disabled by unsetting:

arg( comments ) - for comments about chapter
arg( images )   - for full list of image urls

returns( dictionary of chapter's metadata )

guess(self, url: str) -> Optional[Callable]

Guesses scraping method based on the arg( url )

returns( the found method or None if nothing matched )

Results

You can see scraped results of the entire site here.

Name		Name	Last commit message	Last commit date
examples		examples
tests		tests
weebcentral		weebcentral
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
pyproject.toml		pyproject.toml
test.sh		test.sh

Navigation Menu

Repository files navigation

weebcentral

Installation

Usage

CLI

Library

Code

Methods

get_series_sitemap(self) -> list[str]

get_search(self, url: str) -> Iterator[list[dict]]

get_user_info(self, url: str) -> dict

get_user_3x3(self, id: str) -> list[dict]

get_user_wall(self, id: str) -> Iterator[dict]

get_user_subscriptions(self, id: str) -> Iterator[list[dict]]

get_user(self, url: str, txt: bool = True, subscriptions: bool = True, wall: bool = True) -> dict

get_general(self) -> Iterator[list[dict]]

get_thread(self, url: str) -> dict

id_from_url(url: str) -> str

get_series_info(self, url: str) -> dict

get_series_comments(self, id: str) -> Iterator[dict]

get_series_chapters(self, id: str) -> list[dict]

get_series(self, url: str, comments: bool = False, chapters: bool = True) -> dict

get_chapter_images(self, id: str) -> list[str]

get_chapter_comments(self, id: str) -> Iterator[dict]

get_chapter_info(self, url: str) -> dict

get_chapter(self, url: str, comments: bool = False, images: bool = True) -> dict

guess(self, url: str) -> Optional[Callable]

Results

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

weebcentral

Installation

Usage

CLI

Library

Code

Methods

get_series_sitemap(self) -> list[str]

get_search(self, url: str) -> Iterator[list[dict]]

get_user_info(self, url: str) -> dict

get_user_3x3(self, id: str) -> list[dict]

get_user_wall(self, id: str) -> Iterator[dict]

get_user_subscriptions(self, id: str) -> Iterator[list[dict]]

get_user(self, url: str, txt: bool = True, subscriptions: bool = True, wall: bool = True) -> dict

get_general(self) -> Iterator[list[dict]]

get_thread(self, url: str) -> dict

id_from_url(url: str) -> str

get_series_info(self, url: str) -> dict

get_series_comments(self, id: str) -> Iterator[dict]

get_series_chapters(self, id: str) -> list[dict]

get_series(self, url: str, comments: bool = False, chapters: bool = True) -> dict

get_chapter_images(self, id: str) -> list[str]

get_chapter_comments(self, id: str) -> Iterator[dict]

get_chapter_info(self, url: str) -> dict

get_chapter(self, url: str, comments: bool = False, images: bool = True) -> dict

guess(self, url: str) -> Optional[Callable]

Results

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages