Use pyproject for Python runtime deps lock input

[titusfortner]

Python currently has 2 "sources of truth" for dependencies (requirements.txt and pyproject.toml)
Bazel does not currently support pyrpoject.toml dependency-groups which prevents us from just doing everything there.

💥 What does this PR do?

• Has compile_pip_requirements pull both pyproject.toml and requirements.txt
• Remove the user dependencies from requirements.txt so each dependency is only in one of the files
• Included the ./go py:update results with it

🔧 Implementation Notes

So now py:update will only work with dev dependencies which I think is correct.

🤖 AI assistance

• No substantial AI assistance used
• AI assisted (complete below)
  • Tool(s):
  • What was generated:
  • I reviewed all AI output and can explain the change

💡 Additional Considerations

[qodo-code-review]

Review Summary by Qodo

Split Python dependencies between pyproject.toml and requirements.txt

✨ Enhancement

Walkthroughs

Description

• Split Python dependencies between runtime (pyproject.toml) and tooling (requirements.txt)
• Updated compile_pip_requirements to read both pyproject.toml and requirements.txt
• Removed runtime dependencies from requirements.txt, keeping only dev/tooling packages
• Updated dependency versions and lock file with latest compatible releases

Diagram

flowchart LR
  A["pyproject.toml
Runtime Dependencies"] --> C["compile_pip_requirements"]
  B["requirements.txt
Tooling Dependencies"] --> C
  C --> D["requirements_lock.txt
Complete Lock File"]

Loading

File Changes

1. scripts/update_py_deps.py 📝 Documentation +6/-5

Clarify script manages tooling dependencies only

• Updated script documentation to clarify it manages repo tooling dependencies only
• Changed print messages to specify tooling requirements vs all requirements
• Minor comment formatting improvements for consistency

scripts/update_py_deps.py

---

2. py/BUILD.bazel ⚙️ Configuration changes +4/-1

Configure compile_pip_requirements for dual dependency sources

• Modified compile_pip_requirements to accept both pyproject.toml and requirements.txt as sources
• Changed from requirements_in parameter to srcs parameter for multiple input files
• Maintains extra_args and requirements_txt output configuration

py/BUILD.bazel

---

3. py/requirements.txt Dependencies +9/-11

Consolidate tooling dependencies, remove runtime packages

• Added header comments explaining split between runtime and tooling dependencies
• Removed runtime dependencies (certifi, idna, requests, urllib3, websocket-client, trio,
 trio-websocket, typing_extensions)
• Updated tooling dependency versions (mypy 2.0.0→2.1.0, idna 3.14→3.15, requests 2.33.1→2.34.1, tox
 4.53.1→4.54.0, virtualenv 21.3.1→21.3.3)
• Added python-discovery==1.3.1 as new tooling dependency
• Removed trio and trio-websocket from tooling requirements

py/requirements.txt

---

View more (1)

4. py/requirements_lock.txt Dependencies +68/-70

Update lock file with new versions and dependency sources

• Updated certifi entry to reference selenium from pyproject.toml instead of requirements.txt
• Updated mypy from 2.0.0 to 2.1.0 with new hashes
• Updated idna from 3.14 to 3.15 with new hashes
• Updated requests from 2.33.1 to 2.34.1 with new hashes
• Updated tox from 4.53.1 to 4.54.0 with new hashes
• Updated virtualenv from 21.3.1 to 21.3.3 with new hashes
• Updated python-discovery from 1.3.0 to 1.3.1 with new hashes
• Removed trio, trio-websocket, typing_extensions, urllib3[socks], websocket-client from tooling
 requirements
• Updated dependency comments to reference pyproject.toml for runtime dependencies (selenium)
• Removed keyring references from jeepney and secretstorage entries

py/requirements_lock.txt

---

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

1. Docs install wrong requirements ✓ Resolved 🐞 Bug ⚙ Maintainability

Description

README.md and py/docs/.readthedocs.yaml still instruct pip install -r py/requirements.txt, but
after this PR that file no longer includes selenium runtime dependencies. Users (and the ReadTheDocs
build) can end up with missing imports when generating docs or running selenium from source.

Code

py/requirements.txt[R1-3]

+# Only Development dependencies (tooling, testing, docs, etc)
+# Runtime dependencies are in pyproject.toml
+# included in the Bazel lock through //py:requirements

Evidence

README and ReadTheDocs config install only py/requirements.txt; that file explicitly excludes
runtime deps now, while selenium modules import runtime packages like urllib3/websocket-client.

README.md[235-242]
py/docs/.readthedocs.yaml[10-18]
py/requirements.txt[1-68]
py/pyproject.toml[5-35]
py/selenium/webdriver/remote/remote_connection.py[18-30]
py/selenium/webdriver/remote/websocket_connection.py[18-30]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The PR changes `py/requirements.txt` to be dev-tooling only, but documentation still instructs installing only that file. This omits selenium runtime dependencies and can break doc generation and local-from-source usage.

## Issue Context
`py/requirements_lock.txt` already contains the full resolved set (including runtime deps coming from `py/pyproject.toml`). Updating docs to install from the lock file (or otherwise install runtime deps as well) keeps a single workflow aligned with the Bazel lock generation.

## Fix Focus Areas
- README.md[235-242]
- py/docs/.readthedocs.yaml[10-18]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Tox missing runtime deps ✓ Resolved 🐞 Bug ☼ Reliability

Description

py/tox.ini installs only py/requirements.txt, but this PR removes runtime dependencies (e.g.,
urllib3[socks], websocket-client, typing_extensions) from that file, so tox docs/typecheck will run
with selenium on PYTHONPATH but without its runtime imports installed. This will cause sphinx/mypy
failures due to missing modules (e.g., urllib3, websocket).

Code

py/requirements.txt[L58-68]

-tox==4.53.1
-trio==0.33.0
+tox==4.54.0
trio-typing==0.10.0
-trio-websocket==0.12.2
twine==6.2.0
types-certifi==2021.10.8.3
types-PySocks==1.7.1.20260508
-typing_extensions==4.15.0
-urllib3[socks]==2.7.0
-virtualenv==21.3.1
-websocket-client==1.9.0
+virtualenv==21.3.3

Evidence

The PR removes runtime packages from py/requirements.txt, while tox still installs only that file;
selenium imports urllib3 and websocket-client at import time, which will be missing in those tox
envs.

py/tox.ini[15-36]
py/requirements.txt[1-68]
py/selenium/webdriver/remote/remote_connection.py[18-30]
py/selenium/webdriver/remote/websocket_connection.py[18-30]
py/requirements_lock.txt[1222-1255]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`py/tox.ini` currently installs dependencies via `-r {toxinidir}/requirements.txt` for `docs` and `typecheck`, but `py/requirements.txt` is now dev-tooling only and no longer includes selenium runtime deps (urllib3, websocket-client, typing-extensions, etc.). Since tox runs with `PYTHONPATH={toxinidir}/.` and Sphinx autodoc/mypy will import selenium modules, these environments will fail with missing-module errors.

## Issue Context
The combined pinned set (dev + runtime) is already available in `py/requirements_lock.txt` (generated from both `pyproject.toml` and `requirements.txt` via `//py:requirements.update`).

## Fix Focus Areas
- py/tox.ini[15-36]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[qodo-code-review]

Persistent review updated to latest commit 2892c9d

[qodo-code-review]

Persistent review updated to latest commit 501d776