Improve integration using Leetcode's GraphQL API

[seanprashad]

In #177, it was brought to my attention that Leetcode has exposed their API via GraphQL. We should leverage this to automatically update question metadata, including difficulty and companies. Taking things even further, having a script to generate our question data via a cron job would be a dream!

[leo-step]

A script like this shouldn't be too bad to write, but I'm wondering how you would go about making it a cron job with the website being frontend only.

[seanprashad]

A script like this shouldn't be too bad to write, but I'm wondering how you would go about making it a cron job with the website being frontend only.

Good question! I think the easiest way is to leverage GitHub Actions - this blog post explains how we can introduce this capability: https://dev.to/anshuman_bhardwaj/free-cron-jobs-with-github-actions-31d6

More specifically, these files would be great as a starting point: https://github.com/Anshuman71/tweet-my-articles/tree/main/.github/workflows.

I think once that has been solved, I should definitely get around to doing automatic deployments via GitHub Actions (I've been putting it off for far too long!).

Edit: Ideally the script would generate all of src/data/index.js, and with a premium account, we might also be able to retrieve the "Companies" list updated for each question.

[leo-step]

Very cool, I'll check it out! I didn't know that GitHub let you to do things like this.

Would the idea be to have the workflow make an automatic commit with the new index.js file (on a monthly basis or so)?

[seanprashad]

Very cool, I'll check it out! I didn't know that GitHub let you to do things like this.

Would the idea be to have the workflow make an automatic commit with the new index.js file (on a monthly basis or so)?

Yeah I think that sounds right - let's start very small and introduce functionality one piece at a time. Here's what I was thinking - let me know if you see areas for improvement:

1. Introduce a GitHub Workflow (without any cron job for now) to automatically run the Python script you had authored in update difficulties for questions #177 and print the output to the console - this should introduce us to GitHub Workflows with hopefully a straight-forward template
2. Update the above workflow to now print the output to a temporary file, ie. questions.json - I'm not sure if this is possible, but a proof-of-concept should help us determine if we can automatically generate src/data/index.js
3. Introduce a cron job schedule in the Workflow to automatically run the script once every Sunday, maybe in the afternoon around 5pm PDT - we can fiddle around with this
4. See if we can introduce some method to pull the Companies for each question - this will require Leetcode Premium, which I have
5. Lastly, with all of the above completed, we should look into generating the current version of src/data/index.js and automatically update the tooltip text to reflect the date for which it was updated on
6. The Companies list doesn't have every single company - I've only chosen the top handful and so I'll need to go and source the respective icons for the ones that are missing (I can do this)

Again, this is a fairly large task and so if you (or anyone else who comes along gets stuck), please let me know! 🙌🏽

[leo-step]

I opened a PR (for tracking progress) where I have step 1 currently completed.

[seanprashad]

It looked great! I was browsing to see if the list of companies for each question is provided by GraphQL and good news - it is:

For https://leetcode.com/problems/two-sum/:

"companyTagStats": "{\"1\": [{\"taggedByAdmin\": true, \"name\": \"Amazon\", \"slug\": \"amazon\", \"timesEncountered\": 116}, {\"taggedByAdmin\": true, \"name\": \"Adobe\", \"slug\": \"adobe\", \"timesEncountered\": 54}, {\"taggedByAdmin\": true, \"name\": \"Apple\", \"slug\": \"apple\", \"timesEncountered\": 40}, {\"taggedByAdmin\": true, \"name\": \"Microsoft\", \"slug\": \"microsoft\", \"timesEncountered\": 38}, {\"taggedByAdmin\": false, \"name\": \"Google\", \"slug\": \"google\", \"timesEncountered\": 37}, {\"taggedByAdmin\": true, \"name\": \"Facebook\", \"slug\": \"facebook\", \"timesEncountered\": 23}, {\"taggedByAdmin\": true, \"name\": \"Bloomberg\", \"slug\": \"bloomberg\", \"timesEncountered\": 13}, {\"taggedByAdmin\": true, \"name\": \"Uber\", \"slug\": \"uber\", \"timesEncountered\": 12}, {\"taggedByAdmin\": false, \"name\": \"Spotify\", \"slug\": \"spotify\", \"timesEncountered\": 11}, {\"taggedByAdmin\": false, \"name\": \"Expedia\", \"slug\": \"expedia\", \"timesEncountered\": 7}, {\"taggedByAdmin\": false, \"name\": \"Zoho\", \"slug\": \"zoho\", \"timesEncountered\": 6}, {\"taggedByAdmin\": false, \"name\": \"Goldman Sachs\", \"slug\": \"goldman-sachs\", \"timesEncountered\": 5}, {\"taggedByAdmin\": false, \"name\": \"Morgan Stanley\", \"slug\": \"morgan-stanley\", \"timesEncountered\": 5}, {\"taggedByAdmin\": true, \"name\": \"Yahoo\", \"slug\": \"yahoo\", \"timesEncountered\": 5}, {\"taggedByAdmin\": false, \"name\": \"JPMorgan\", \"slug\": \"jpmorgan\", \"timesEncountered\": 4}, {\"taggedByAdmin\": false, \"name\": \"MakeMyTrip\", \"slug\": \"makemytrip\", \"timesEncountered\": 4}, {\"taggedByAdmin\": false, \"name\": \"Oracle\", \"slug\": \"oracle\", \"timesEncountered\": 3}, {\"taggedByAdmin\": false, \"name\": \"Walmart Global Tech\", \"slug\": \"walmart-labs\", \"timesEncountered\": 3}, {\"taggedByAdmin\": false, \"name\": \"tcs\", \"slug\": \"tcs\", \"timesEncountered\": 3}, {\"taggedByAdmin\": false, \"name\": \"Zoom\", \"slug\": \"zoom\", \"timesEncountered\": 3}, {\"taggedByAdmin\": false, \"name\": \"Accenture\", \"slug\": \"accenture\", \"timesEncountered\": 3}, {\"taggedByAdmin\": false, \"name\": \"SAP\", \"slug\": \"sap\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Nvidia\", \"slug\": \"nvidia\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Zillow\", \"slug\": \"zillow\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"VMware\", \"slug\": \"vmware\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Visa\", \"slug\": \"visa\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Intel\", \"slug\": \"intel\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Salesforce\", \"slug\": \"salesforce\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Qualcomm\", \"slug\": \"qualcomm\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"American Express\", \"slug\": \"american-express\", \"timesEncountered\": 2}], \"3\": [{\"taggedByAdmin\": false, \"name\": \"Cisco\", \"slug\": \"cisco\", \"timesEncountered\": 7}, {\"taggedByAdmin\": false, \"name\": \"Nagarro\", \"slug\": \"nagarro\", \"timesEncountered\": 5}, {\"taggedByAdmin\": false, \"name\": \"Citrix\", \"slug\": \"citrix\", \"timesEncountered\": 4}, {\"taggedByAdmin\": false, \"name\": \"ByteDance\", \"slug\": \"bytedance\", \"timesEncountered\": 4}, {\"taggedByAdmin\": false, \"name\": \"Yandex\", \"slug\": \"yandex\", \"timesEncountered\": 3}, {\"taggedByAdmin\": false, \"name\": \"Square\", \"slug\": \"square\", \"timesEncountered\": 3}, {\"taggedByAdmin\": false, \"name\": \"Splunk\", \"slug\": \"splunk\", \"timesEncountered\": 3}, {\"taggedByAdmin\": false, \"name\": \"PayTM\", \"slug\": \"paytm\", \"timesEncountered\": 3}, {\"taggedByAdmin\": true, \"name\": \"Airbnb\", \"slug\": \"airbnb\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Wayfair\", \"slug\": \"wayfair\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"GoDaddy\", \"slug\": \"godaddy\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Groupon\", \"slug\": \"groupon\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"FactSet\", \"slug\": \"factset\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"HBO\", \"slug\": \"hbo\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Twitch\", \"slug\": \"twitch\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Poshmark\", \"slug\": \"poshmark\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"United Health Group\", \"slug\": \"united-health-group\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Shopee\", \"slug\": \"shopee\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Epam Systems\", \"slug\": \"epam-systems\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Athenahealth\", \"slug\": \"athenahealth\", \"timesEncountered\": 2}, {\"taggedByAdmin\": true, \"name\": \"LinkedIn\", \"slug\": \"linkedin\", \"timesEncountered\": 0}, {\"taggedByAdmin\": true, \"name\": \"Yelp\", \"slug\": \"yelp\", \"timesEncountered\": 0}, {\"taggedByAdmin\": true, \"name\": \"Dropbox\", \"slug\": \"dropbox\", \"timesEncountered\": 0}], \"2\": [{\"taggedByAdmin\": false, \"name\": \"Paypal\", \"slug\": \"paypal\", \"timesEncountered\": 6}, {\"taggedByAdmin\": false, \"name\": \"Dell\", \"slug\": \"dell\", \"timesEncountered\": 5}, {\"taggedByAdmin\": false, \"name\": \"Infosys\", \"slug\": \"infosys\", \"timesEncountered\": 4}, {\"taggedByAdmin\": false, \"name\": \"Samsung\", \"slug\": \"samsung\", \"timesEncountered\": 3}, {\"taggedByAdmin\": false, \"name\": \"IBM\", \"slug\": \"ibm\", \"timesEncountered\": 3}, {\"taggedByAdmin\": false, \"name\": \"Citadel\", \"slug\": \"citadel\", \"timesEncountered\": 3}, {\"taggedByAdmin\": false, \"name\": \"Accolite\", \"slug\": \"accolite\", \"timesEncountered\": 3}, {\"taggedByAdmin\": false, \"name\": \"eBay\", \"slug\": \"ebay\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Intuit\", \"slug\": \"intuit\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"ServiceNow\", \"slug\": \"servicenow\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Twitter\", \"slug\": \"twitter\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Snapchat\", \"slug\": \"snapchat\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Deutsche Bank\", \"slug\": \"deutsche-bank\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Capital One\", \"slug\": \"capital-one\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Optum\", \"slug\": \"optum\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Siemens\", \"slug\": \"siemens\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Zomato\", \"slug\": \"zomato\", \"timesEncountered\": 2}, {\"taggedByAdmin\": false, \"name\": \"Cognizant\", \"slug\": \"cognizant\", \"timesEncountered\": 2}]}",

Edit: I think we'll need to authenticate with Leetcode in order to view company-tagged details.

[seanprashad]

Opened #209 which aims to introduce functionality to retrieve companies for each question. We can then consider this issue complete and close things! Thanks again for all of your work @leo-step!