Skip to content

Add solr replica to ol-solr2#12706

Merged
mekarpeles merged 4 commits into
internetarchive:masterfrom
cdrini:perf/solr2
May 18, 2026
Merged

Add solr replica to ol-solr2#12706
mekarpeles merged 4 commits into
internetarchive:masterfrom
cdrini:perf/solr2

Conversation

@cdrini

@cdrini cdrini commented May 11, 2026

Copy link
Copy Markdown
Collaborator

Closes #12583

We want to set up a new solr instance on ol-solr2 that is a replica of ol-solr0. We currently have our ol-solr0 and ol-solr1 on the same baremetal, and it is causing CPU strain. solr2 is on another baremetal, so switching our split to be there should help reduce CPU load across the entire site. This will also let us return to using solr1 for staging instead of a manual replica of prod.

Technical

Need special deploy:

  • The solrconfig.xml will need to be updated to allow the /replication handler
  • Ferm rules on the two solr will need to be adjusted so they can talk to each other.

Testing

Screenshot

Stakeholders

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new Solr replica node (ol-solr2) by enabling Solr replication in the shared Solr config and wiring up production compose + deployment/monitoring scripts to recognize and monitor the new host.

Changes:

  • Enable Solr leader/follower replication via a new /replication request handler in solrconfig.xml.
  • Add a production solr_replica service/profile (ol-solr2) and mark ol-solr0 as the replication leader.
  • Update deployment host lists and monitoring to include ol-solr2.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
scripts/solr_builder/dump-solr.sh Adjusts ia upload CLI flags for Solr dump upload.
scripts/monitoring/monitor.py Adds ol-solr2 to Solr monitoring and maps server→container names.
scripts/deployment/deploy.sh Includes ol-solr2 in the cluster host list for deploy operations.
conf/solr/conf/solrconfig.xml Adds Solr replication request handler with leader/follower configuration.
compose.production.yaml Marks ol-solr0 as replication leader, adds solr_replica profile for ol-solr2, and enables monitoring profile on ol-solr2.

Comment thread scripts/monitoring/monitor.py
Comment thread scripts/solr_builder/dump-solr.sh Outdated
@cdrini cdrini force-pushed the perf/solr2 branch 4 times, most recently from 8959ad5 to b6c72ae Compare May 11, 2026 17:16
@cdrini cdrini marked this pull request as ready for review May 18, 2026 14:32
@cdrini cdrini added Priority: 1 Do this week, receiving emails, time sensitive, . [managed] Patch Deployed This PR has been deployed to production independently, outside of the regular deploy cycle. labels May 18, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Comment thread conf/solr/conf/solrconfig.xml
Comment thread compose.near-prod.yaml Outdated
cdrini and others added 2 commits May 18, 2026 17:38
Co-authored-by: Copilot Autofix powered by AI <[email protected]>
Co-authored-by: Drini Cami <[email protected]>
@mekarpeles mekarpeles self-assigned this May 18, 2026
@mekarpeles

Copy link
Copy Markdown
Member

Don't love the "near" compose (would prefer prod-testing.

Otherwise LGTM

@mekarpeles mekarpeles merged commit a90f33e into internetarchive:master May 18, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Patch Deployed This PR has been deployed to production independently, outside of the regular deploy cycle. Priority: 1 Do this week, receiving emails, time sensitive, . [managed]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Infra: Provision ol-solr2 on new rack for prod secondary

3 participants