⚡ Bolt: Optimize repetition penalty hot path

[EffortlessSteven]

💡 What
Pre-calculated the inverse of the count penalty outside the token loop, and moved the powi computation to be lazily evaluated inside the conditional branches.

🎯 Why
Calculating powi unconditionally before the if/else block introduces significant overhead by performing work for branches that aren't taken. Furthermore, executing division (e.g., logit /= penalty) in a hot loop is computationally expensive and can be safely eliminated by calculating the inverse and multiplying with it.

📊 Impact
Reduces mathematical overhead in the hot token sampling path by avoiding redundant power calculations and converting an expensive division into a faster multiplication per positive logit.

🔬 Measurement
The optimization can be measured by running benchmark tests on token generation where repetition penalty is enabled, verifying lower latency per sampled token.

---

PR created automatically by Jules for task 393563771578583147 started by @EffortlessSteven

---

Note

Low Risk
Behavior-preserving numeric refactor in sampling with existing unit/snapshot coverage; only minor floating-point drift on one snapshot value.

Overview
RepetitionPenaltyConfig::apply is tuned for the per-token sampling hot path when count penalty is enabled.

1.0 / count_penalty is computed once before the token loop. powi runs only in the branch that needs it (positive vs non-positive logits). Positive logits use inv_penalty.powi(count) and multiply instead of dividing by penalty^count.

A snapshot expectation shifts slightly (2.909091 → 2.9090908) from multiply-vs-divide float behavior. .jules/bolt.md records the optimization pattern for future hot-loop work.

^{Reviewed by Cursor Bugbot for commit 9f9b1e6. Bugbot is set up for automated code reviews on this repo. Configure here.}

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b8713ff8-5f80-4aad-996a-9391213eff19

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch bolt/repetition-penalty-optimization-393563771578583147

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_4ed61a9f-c04a-477e-9e69-c0035fee4528)