Support the new market page in getMarketItem (fixes #373) - #374
Open
tarikermis wants to merge 1 commit into
Open
Support the new market page in getMarketItem (fixes #373)#374tarikermis wants to merge 1 commit into
tarikermis wants to merge 1 commit into
Conversation
Steam's new Community Market page is server-side rendered and no longer contains the markup CMarketItem scraped (g_strCountryCode, Market_LoadOrderSpread, g_rgAssets, market_listing_table_message), so getMarketItem() failed for every item. Read the JSON the new page embeds instead: country/language and bCommodity from SSR.loaderData, and the order book from the embedded query data in SSR.renderContext. Group pages only embed an order book for the group's fallback item, so for other group items the lowest price is read from the item's bucket min_price. The page only embeds its data when the request doesn't accept compressed responses, so the listing request now sends no Accept-Encoding. Legacy code paths are unchanged and still used when the old page structure is served. Co-authored-by: Ian Washburne <ianw11@gmail.com>
|
Is this waiting on anything to get merged? |
Author
|
Not that I know of - just waiting on DoctorMcKay to review. Best Regards, Tarik |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #373
Steam's new Community Market page is server-side rendered and no longer contains the markup
CMarketItemused to scrape. The old/render/endpoint now redirects to HTML anditem_nameidis gone, sogetMarketItem()fails withError undefined.This builds on the approach @ianw11 posted in the issue: parse the JSON embedded in the new page while leaving the legacy paths unchanged.
What changed
SSR.loaderDataand read the country, language, commodity flag, and embedded order book.min_pricefor non-fallback items on grouped market pages.Accept-Encodingheader, it consistently returns a lightweight shell instead.Verification
Checked live anonymous requests for:
The returned prices and quantities matched the embedded data. A synthetic old-format page still uses the legacy histogram path and preserves its existing fields. The repository has no configured test suite or linter, so I also ran
node --check.Limits
currencyargument affect an anonymous request.quantityremains 0.Thanks to @ianw11 for the initial direction and co-authored patch.
Best Regards, Tarik