Skip to content

Conversation

@cxyfer
Copy link
Owner

@cxyfer cxyfer commented Jul 5, 2025

Summary

  • Automatically fetch entire month's daily challenges when queried date not found in database
  • Prioritize processing requested date first for immediate response
  • Process remaining dates in background task to populate database

Details

When users query a historical date using /daily command:

  1. First checks database for cached data
  2. If not found, fetches entire month's challenges from LeetCode API
  3. Immediately processes and returns the requested date
  4. Remaining dates are processed in background with 0.5s delay between each

Benefits

  • Reduces response time from 30+ seconds to 1-2 seconds
  • Populates database for future queries automatically
  • No blocking of user interaction
  • Graceful error handling for background tasks

Limitations

  • Only works for leetcode.com domain (monthly API not available on leetcode.cn)
  • Background task adds slight load but includes delays to prevent API overwhelm

🤖 Generated with Claude Code

cxyfer and others added 2 commits July 5, 2025 12:31
Adds a new method `fetch_monthly_daily_challenges` to retrieve historical daily and weekly coding challenge data for a specified year and month.

Introduces a `--monthly` CLI argument to access this new functionality.
- Automatically fetch entire month's daily challenges when queried date not found in database
- Prioritize processing requested date first for immediate response
- Process remaining dates in background to populate database without blocking user
- Only works for leetcode.com domain (monthly API not available on leetcode.cn)
- Improves user experience by reducing wait time from 30+ seconds to 1-2 seconds

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@cxyfer cxyfer requested a review from Copilot July 5, 2025 05:08
Copilot

This comment was marked as outdated.

cxyfer and others added 2 commits July 5, 2025 14:04
…ling and concurrency

- Add configurable delay for API requests (monthly_fetch_delay in config.toml)
- Implement background task tracking for graceful shutdown
- Add concurrency limiting with asyncio.Semaphore(5)
- Improve error handling with specific exception types
- Ensure rating field is included in background processing

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add comprehensive test suite for monthly daily challenge fetching
- Test successful API calls and response parsing
- Test error handling for network and API failures
- Test background task processing and cancellation
- Test concurrency limits with semaphore
- Add tests README with usage instructions

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@cxyfer cxyfer requested a review from Copilot July 5, 2025 06:13
Copilot

This comment was marked as outdated.

cxyfer and others added 2 commits July 5, 2025 14:16
- Log a warning when requested date is not found in monthly data
- Addresses Copilot review feedback for better debugging

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Move all leetcode settings under [leetcode] section before subsections
- Ensures leetcode.monthly_fetch_delay path works correctly
- Addresses Copilot review feedback about config structure

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@cxyfer cxyfer requested a review from Copilot July 5, 2025 06:19
Copilot

This comment was marked as outdated.

…t coverage

- Fix exception handling order to properly propagate asyncio.CancelledError
- Remove unused MagicMock import from test file
- Add test coverage for weekly challenges in monthly fetch
- Clarify that uv is the project's package manager in test README

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@cxyfer cxyfer requested a review from Copilot July 5, 2025 06:26
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Enhance the /daily command to automatically fetch and cache an entire month’s challenges when a specific date is missing, returning the requested date immediately and processing the rest in the background.

  • Integrated fetch_monthly_daily_challenges into get_daily_challenge with priority handling for the requested date.
  • Added background task support (tracking, semaphore control, shutdown) for non-blocking backfill.
  • Updated CLI (--monthly flag), configuration (monthly_fetch_delay), and comprehensive unit tests.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
tests/test_monthly_fetch.py New async tests covering monthly fetch success, errors, background, concurrency, and cancellation.
tests/README.md Instructions for running tests, including setup and commands.
leetcode.py Added monthly fetch methods, background task management, semaphore control, and CLI integration.
config.toml.example Introduced monthly_fetch_delay setting under the [leetcode] section.
Comments suppressed due to low confidence (2)

tests/README.md:6

  • [nitpick] Clarify what uv refers to or provide a link/reference, since it may not be familiar to all users and could cause confusion during setup.
This project uses `uv` as the package manager. Make sure pytest and required dependencies are installed:

leetcode.py:966

  • Ensure the configuration key path matches the structure in config.toml; consider accessing monthly_fetch_delay directly from the leetcode section rather than via a dotted key to avoid mismatches.
                            delay = config.get("leetcode.monthly_fetch_delay", 0.5)

Comment on lines +626 to +634
logger.info(f"No data found for {date_str}, attempting to fetch monthly data...")
year, month, _ = date_str.split('-')
year_int = int(year)
month_int = int(month)

monthly_data = await self.fetch_monthly_daily_challenges(year_int, month_int)

if monthly_data and 'challenges' in monthly_data:
logger.info(f"Fetched {len(monthly_data['challenges'])} daily challenges for {year}-{month}")
Copy link

Copilot AI Jul 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The monthly-fetch block in get_daily_challenge is fairly large and duplicates processing logic; extracting it into a dedicated helper method would improve readability and maintainability.

Suggested change
logger.info(f"No data found for {date_str}, attempting to fetch monthly data...")
year, month, _ = date_str.split('-')
year_int = int(year)
month_int = int(month)
monthly_data = await self.fetch_monthly_daily_challenges(year_int, month_int)
if monthly_data and 'challenges' in monthly_data:
logger.info(f"Fetched {len(monthly_data['challenges'])} daily challenges for {year}-{month}")
info = await self.handle_monthly_fetch(date_str, domain)
if info:
return info

Copilot uses AI. Check for mistakes.

if question_id and slug:
# Use semaphore to limit concurrent API requests
async with self._fetch_semaphore:
Copy link

Copilot AI Jul 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Challenges are fetched sequentially inside the loop, so the semaphore never limits concurrent requests; consider spawning tasks (e.g., with asyncio.gather) up to the semaphore limit to better leverage concurrency.

Copilot uses AI. Check for mistakes.
cxyfer and others added 2 commits July 5, 2025 14:36
- Add validation to prevent fetching data before April 2020
- Update documentation to mention API data availability starts from April 2020
- Add warning logs when attempting to fetch data before April 2020
- Add unit test for date validation
- Update CLI to validate date range for --monthly flag

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Clarifies the valid range for the optional date parameter in the daily challenge commands.
@cxyfer cxyfer merged commit 0886392 into main Jul 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant