fix(preprocess): parse shared exchange ticker lists

[pengpengyi92]

Summary

• parse parenthesized comma-separated ticker lists that share a supported exchange prefix
• stop shared-prefix parsing at semicolons, conjunctions, and closing parentheses
• define reconstructed EXCHANGE: SYMBOL raw values for additional list members
• add deterministic regression coverage for the five public shapes from the issue, plus a focused example and design documentation

Related Issue

Closes #108

Verification

• pytest -o addopts='' tests/preprocess/test_news.py -q - 20 passed, 8 subtests passed
• ruff format --check . - passed
• ruff check . - passed
• basedpyright - passed with 0 errors
• lint-imports - 7 contracts kept, 0 broken
• python examples/preprocess/02_news_ticker_lists.py - emitted both EVEX and EVEXW
• Full pytest --cov on Windows reached 84.05% coverage with 322 passed and 13 pre-existing platform failures involving SQLite temporary-file locks and LiteParse artifact paths; this draft relies on the repository's Linux CI for the required deterministic verification.

Checklist

• The title uses English Conventional Commit format: type(scope): summary.
• The related issue or design discussion is linked when applicable.
• bash scripts/verify.sh passes locally; Windows-specific existing failures are disclosed above and Linux CI is pending.
• No live-network component smoke test applies because this change is deterministic parsing only.
• Public behavior has focused tests, an example, and documentation where applicable.
• The PR is complete, small, and contains no unrelated changes.

[keli-wen]

Requesting changes. Thanks for taking on #108 — the intent is right, but the shared-prefix extension as written is a net regression, and I don't think this change should carry a new example or a design-doc rewrite. Details below.

Blocking: the shared-prefix scan over-captures ordinary prose

The continuation loop keys off match.end() and then searches for the next ) anywhere in the text (scan_text.find(")", match.end())). When the main regex already matched a balanced (EXCHANGE: SYMBOL) — it consumed its own closing paren — the loop still runs and scans past that paren into unrelated following text. Because _SHARED_EXCHANGE_SYMBOL_RE is compiled with re.IGNORECASE, an ordinary lowercase word right after a (EXCHANGE: SYMBOL), is captured as a ticker.

This flows straight into preprocess_news_document (ticker_hints, news.py:330), so it pollutes real output. On current master the input below yields one hint; on this branch it yields a bogus second one:

# (symbol, exchange, raw) projected from each returned NewsTickerHint
>>> extract_exchange_ticker_hints("Shares of (NYSE: IBM), and (NASDAQ: AAPL) rose today.")
('IBM',  'NYSE',   '(NYSE: IBM)')
('AND',  'NYSE',   'NYSE: AND')      # <- "and" captured as ticker "AND"
('AAPL', 'NASDAQ', '(NASDAQ: AAPL)')

The trigger — a balanced (EXCH: SYM), followed by any later ) — is extremely common in PR-wire prose, so this regresses far more documents than the ~1 warrant of recall that #108 estimates it recovers. The added tests only assert the five happy-path shapes from the issue and never exercise this case, so CI stays green while the regression ships.

Related smell: raw is meant to be the literal matched substring (provenance), but continuation members set raw=f"{exchange}: {symbol}" — a string that never appears in the source (NYSE: EVEXW is reconstructed) — while the first member keeps a dangling (NYSE: EVEX. The tests encode both, which locks that broken raw contract in.

This change should not add an example

extract_exchange_ticker_hints already exists and is already demonstrated end-to-end by examples/preprocess/01_news_pr_wire.py. Extending an existing operation's parsing coverage is a fix, not a new public operation, so it should ship regression tests rather than a second example file. Please drop examples/preprocess/02_news_ticker_lists.py. We'll also tighten the contributor guidance so the "ships with an example" expectation is scoped to new public operations, not edge-case extensions.

The contexts/ doc change is questionable here too

Whether a parsing edge-case should rewrite contexts/design/flow/news.md is debatable — unless this actually changes a documented contract, the design doc probably shouldn't move. It is also the only real merge conflict with master right now (#130 rewrote that file), so dropping the doc change both narrows scope and clears the conflict.

Suggested path

Narrow this to what #108 is actually worth: keep the existing simple/balanced capture and add regression tests that pin the boundary — the first symbol of a shared-prefix list is captured, and (EXCH: SYM), ...) does not over-capture. If we would rather not carry the heuristic at all, deprecating ticker_hints is a reasonable alternative; happy to discuss which way you'd prefer.

Note: I pushed a small chore: normalize CRLF line endings to LF commit to this branch. The files were saved with CRLF, which made the diff look like a full-file rewrite; it now reads as ~+89/-4 against the merge base.

[pengpengyi92]

Thanks for the detailed review — the over-capture and provenance concerns were valid. I rebuilt the branch on the latest master and narrowed the PR to the parser plus focused regression tests only. The extra example and design-doc changes are gone.

The continuation scan now runs only when the initial parenthesized exchange match has not already consumed its closing ), and continuation symbols require uppercase ticker syntax. This prevents (NYSE: IBM), and (NASDAQ: AAPL) from producing a bogus AND hint. Continuation
aw now preserves the literal matched source substring (for example, , EVEXW) rather than reconstructing NYSE: EVEXW.

I added regression coverage for the valid shared-prefix list and the balanced-parenthesis/prose, conjunction, semicolon, and unsupported-exchange boundaries.

Verification:

• 21 passed for ests/preprocess/test_news.py with the repository-wide coverage addopts disabled for the focused run
• Ruff lint and format checks pass
• git diff --check passes
• Full suite: 410 passed, 18 unrelated Windows SQLite file-lock/PDF golden failures; total coverage 85.44%

The PR is now mergeable and contains only two changed files. Please take another look when convenient.

[pengpengyi92]

Thanks again for the careful review — it was genuinely helpful, especially the concrete ordinary-prose regression and the provenance concern.

One remaining contract question before I finalize this: for a shared-prefix group such as (NYSE: EVEX, EVEXW), should
aw be:

1. the full literal group for both hints, or
2. each literal symbol token (EVEX / EVEXW)?

The current revision avoids reconstructed text, but the first hint still retains the existing partial raw value (NYSE: EVEX, while the continuation stores , EVEXW. I’d prefer to align on the intended provenance contract rather than lock that behavior into the tests. Happy to adjust it based on your preference.