Add Oxylabs Document Loader #4625
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
can you do |
There was a problem hiding this comment.
Pull Request Overview
Adds a new Oxylabs document loader that integrates with the Oxylabs real-time scraping API and exposes it as a Flowise node.
- Introduces
OxylabsLoaderfor making authenticated requests to various Oxylabs sources. - Wraps the loader in an
INodeimplementation (Oxylabs_DocumentLoaders) with UI inputs and output handling. - Adds a credential definition for Oxylabs API credentials.
Reviewed Changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| packages/components/nodes/documentloaders/Oxylabs/Oxylabs.ts | New loader class, API request methods, and Flowise node wrapper |
| packages/components/credentials/OxylabsApi.credential.ts | New credential class for Oxylabs API username/password |
Comments suppressed due to low confidence (3)
packages/components/nodes/documentloaders/Oxylabs/Oxylabs.ts:69
- [nitpick] The
OxylabsLoaderclass lacks JSDoc or inline comments. Adding a brief description and parameter/docs would improve readability and help future maintainers.
export class OxylabsLoader extends BaseDocumentLoader {
packages/components/nodes/documentloaders/Oxylabs/Oxylabs.ts:165
- [nitpick] Class name
Oxylabs_DocumentLoadersuses an underscore and plural form. Consider renaming toOxylabsDocumentLoaderNodefor consistency with the project's naming conventions.
class Oxylabs_DocumentLoaders implements INode {
packages/components/nodes/documentloaders/Oxylabs/Oxylabs.ts:1
- No unit tests were added for
OxylabsLoader. Consider adding tests to cover each source type, parameter filtering, and error handling paths.
import { TextSplitter } from 'langchain/text_splitter'
oxy-rostyslav
force-pushed
the
feature/add-oxylabs-document-loader
branch
from
e006012 to
06c47e6
Compare
oxy-rostyslav
force-pushed
the
feature/add-oxylabs-document-loader
branch
from
06c47e6 to
b9e1a92
Compare
|
Hi @HenryHengZJ, I've updated the PR. Could you please check if you have any other remarks? |
0xi4o
approved these changes
Jul 8, 2025
|
@oxy-rostyslav Great work! Thank you for your contribution! @HenryHengZJ I've tested and reviewed this one. Works well. Good to merge.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Adds an Oxylabs Document loader that allows to load the data from multiple sources efficiently.
Example