move get_betterworldbooks_metadata and get_amazon_metadata out of AffiliateLinks template#12393
Conversation
|
Thanks for the contribution, @Sanket17052006! 🤖 Copilot has been assigned for an initial review. @RayBB is assigned to this PR and currently has:
Note: the linked issue (#12271) hasn't been triaged yet — triage happens on Mondays and Fridays. PR triage checklist (maintainers / Pam)
Note This comment was automatically generated by Pam, Open Library's Project AI Manager, on behalf of @mekarpeles. Pam is designed to provide status visibility, perform basic project management functions and relevant codebase research, and provide actionable feedback so contributors aren't left waiting. |
There was a problem hiding this comment.
Pull request overview
Refactors the AffiliateLinks partial rendering path so Better World Books (BWB) pricing metadata is fetched in Python before template render, rather than triggering a network call from inside the Templetor macro. This supports cleaner separation of concerns and is a step toward making the partial endpoint async-friendly.
Changes:
- Pre-fetch BWB metadata in
AffiliateLinksPartial.generate()and injectbwb_price/bwb_market_priceintoopts. - Update
AffiliateLinks.htmlto read BWB pricing values fromopts(via.get()) instead of callingget_betterworldbooks_metadata().
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| openlibrary/plugins/openlibrary/partials.py | Fetches and injects BWB pricing metadata into opts before rendering AffiliateLinks. |
| openlibrary/macros/AffiliateLinks.html | Removes template-side BWB fetch; reads pre-injected price fields from opts. |
| from openlibrary.core.vendors import get_betterworldbooks_metadata | ||
| from openlibrary.i18n import gettext as _ | ||
| from openlibrary.plugins.openlibrary.code import is_bot |
There was a problem hiding this comment.
Importing is_bot from openlibrary.plugins.openlibrary.code here is very risky: code.py executes setup() at import time (including importing this partials module and calling partials.setup()), which creates a circular import and can break module import / app startup. Instead, read the bot flag directly from the request context (e.g., openlibrary.utils.request_context.req_context.get().is_bot) or move a lightweight is_bot helper into a side-effect-free module so partials.py never needs to import code.py.
| if not is_bot() and opts.get('prices') and isbn: | ||
| bwb_metadata = get_betterworldbooks_metadata(isbn) | ||
| if isinstance(bwb_metadata, dict): | ||
| opts['bwb_price'] = bwb_metadata.get('price') | ||
| opts['bwb_market_price'] = bwb_metadata.get('market_price') |
There was a problem hiding this comment.
AffiliateLinksPartial.generate() is invoked from an async FastAPI route (/partials/AffiliateLinks.json), but this code performs a synchronous network call (get_betterworldbooks_metadata) inline. That will block the event loop and reduce concurrency under load. Consider moving this into an async codepath (e.g., generate_async + await an async HTTP client) or at least offloading the blocking call to a threadpool/async_bridge wrapper so the FastAPI handler remains non-blocking.
| if not is_bot() and prices and isbn: | ||
| bwb_metadata = get_betterworldbooks_metadata(isbn) | ||
| bwb['price'] = bwb_metadata and bwb_metadata.get('price') | ||
| bwb['price'] = opts.get('bwb_price') | ||
| if amazon: | ||
| amazon['price'] = bwb_metadata and bwb_metadata.get('market_price') | ||
| if amazon and not amazon['price']: | ||
| amazon['price'] = opts.get('bwb_market_price') | ||
| if amazon and not amazon.get('price'): |
There was a problem hiding this comment.
This change removes the template-side Better World Books fetch, but the template now only reads bwb_price/bwb_market_price from opts. Any synchronous (non-partial) renders of this macro that don’t pre-inject these fields will stop showing BWB/Amazon pricing (e.g. DonateModal.html calls macros.AffiliateLinks(book.title, ids) without the partial handler). To keep backward compatibility, either (1) add a fallback to fetch BWB metadata when these keys are absent, or (2) ensure all call sites that render AffiliateLinks with async_load=False populate these opts fields in Python before rendering.
RayBB
left a comment
There was a problem hiding this comment.
Excellent work here!
I extended this a little to also move the amz metadata out because it just made things a little clearer.
Tested on testing and it's working perfect!
…iliateLinks template (internetarchive#12393) Co-authored-by: RayBB <[email protected]>
…iliateLinks template (internetarchive#12393) Co-authored-by: RayBB <[email protected]>
…iliateLinks template (internetarchive#12393) Co-authored-by: RayBB <[email protected]>
Closes #12271
get_betterworldbooks_metadata()network call out ofAffiliateLinks.htmlintoAffiliateLinksPartial.generate()inpartials.py.Changes :
partials.py: Pre-fetch BWB metadata inAffiliateLinksPartialand inject it intoopts.AffiliateLinks.html: Remove theget_betterworldbooks_metadatacall and readbwb_priceandbwb_market_pricedirectly fromoptssafely using.get().Technical
Testing
data-optsattribute in the response – it now includesbwb_priceandbwb_market_price.Screenshot
Stakeholders
@RayBB