asyncify code for trending and availability fastapi#12508
Conversation
|
@Sanket17052006 this one is up on testing now. It touches a good bit of code but it's similar to what you've already worked on. Would you be willing to test this on testing and give a code review? |
There was a problem hiding this comment.
Pull request overview
This PR refactors trending and availability flows to use async implementations (Solr enrichment + IA availability calls) while preserving sync entry points via wrappers, aligning these paths with the repo’s ongoing async migration (FastAPI + async utilities).
Changes:
- Convert trending-book retrieval and carousel “TRENDING” queries to async, including async Solr enrichment.
- Convert availability fetching to async (IA httpx async client) and update FastAPI internal availability endpoints to
awaitit. - Update worksearch subject/work-search enrichment to await async availability decoration; adjust related tests/mocks.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| openlibrary/views/loanstats.py | Makes get_trending_books async and awaits async Solr enrichment. |
| openlibrary/core/bookshelves.py | Adds add_solr_works_async and attempts to add a sync wrapper. |
| openlibrary/plugins/openlibrary/partials.py | Awaits trending query path for carousel load-more. |
| openlibrary/fastapi/internal/api.py | Makes internal availability + trending endpoints async and awaits implementations. |
| openlibrary/core/lending.py | Introduces get_availability_async / add_availability_async and wraps for sync usage. |
| openlibrary/core/fulltext.py | Awaits async availability in fulltext search. |
| openlibrary/plugins/worksearch/subjects.py | Awaits async availability decoration for subject works. |
| openlibrary/plugins/worksearch/code.py | Adds async Solr work fetch helper + async post-processing with availability. |
| openlibrary/tests/fastapi/test_book_availability.py | Updates patch target to async availability function. |
| openlibrary/tests/core/test_lending.py | Updates mocks to patch IA async session calls. |
Comments suppressed due to low confidence (2)
openlibrary/plugins/worksearch/code.py:1074
fieldsis annotated asstrhere, butwork_search_asyncpassesfields: str | list[str]and this helper uses'availability' in fields, which behaves differently forstrvslist[str]. Update the type annotation (and any related logic) to reflect the actual accepted types to avoid confusion and mypy/pyright issues.
async def _process_solr_search_response(response: SearchResponse, fields: str) -> dict:
"""
Handles the post-processing of the Solr response, which is common
to both sync and async versions.
"""
openlibrary/fastapi/internal/api.py:156
- This endpoint now
awaitsget_trending_books. Any tests (or other callers) patchingopenlibrary.fastapi.internal.api.get_trending_booksneed to use an async-capable mock (e.g.,new_callable=AsyncMock) so the awaited call returns properly; patching with a plainreturn_valuewill fail at runtime.
works = await get_trending_books(
since_days=since_days,
since_hours=params.hours,
limit=params.limit,
page=params.page,
sort_by_count=params.sort_by_count,
Sanket17052006
left a comment
There was a problem hiding this comment.
Reviewed the changes. A few things I specifically checked:
Grepped for get_trending_books callers - only partials.py and api.py, both properly awaiting it, so dropping @public and skipping the sync wrapper is safe (similar to one of my last PR #12272 )
All the other function changes seems to be consistent with other efforts for async migration.
LGTM!
|
is no longer giving a Json response. Is this intended and if not, could this bug be related to this PR ? |
|
@Devenik7 can you please open an issue for this? Definitely not intentional and probably a bad proxy configuration |
|
@RayBB Done |
Closes #12254
This started with trying to make
_do_trends_queryasync and that quickly required making many other functions async and spidered out a bit to impact a few other endpoints and code paths but there's no logic changes just making code async.Technical
Testing
It's on testing, since April 30 at like 2:30pm PST. So maybe check logs but it seems to work.
Screenshot
Stakeholders