Direct Windows support for Unsloth

[adityaghai07]

Direct Windows support for Unsloth

Feature Overview

These changes allow pip install unsloth to directly work without any errors or dependency issues.

Implementation Details

• Updated pyproject.toml to allow installation of unsloth and all its dependencies directly without manual work.

Reasoning

• First issue was to configure GPU by installing GPU drivers and CUDA drivers. This is a necessary prerequisite for PyTorch to utilize GPU acceleration.
• As the PyTorch version depends on the versions of CUDA drivers used, we need to install PyTorch specific to the CUDA drivers installed previously. Mismatched versions often cause compatibility issues.
• On Windows, we need Windows SDK and a compatible C/C++ compiler to run some dependencies, so using Microsoft Visual C++ (MSVC) we can easily install those required components for proper compilation.
• Finally, we updated pyproject.toml to support direct installation of all dependencies and unsloth, ensuring the package can be installed with a single command without manual intervention.

Testing Details

• Tested on 4 python versions: 3.9, 3.10, 3.11, 3.12.
• Tested for 3 different CUDA versions - 11.8, 12.4, 12.6
• Tested on 2 windows machines with different GPUs to test support across different drivers - GTX 1650, RTX 3050

Contributors

This feature was developed in collaboration with @Captain-T2004.

[Captain-T2004]

There is a different approach that is possible that will allow users to install all dependencies including PyTorch with a single command, but it requires changing the setup from pyproject.toml to a setup.py approach. However, this would require extensive and unnecessary changes. A hybrid approach is not really supported under PEP guidelines, specifically PEP 517.

[danielhanchen]

Oh fantastic! This does not interfere with Linux installations right? Also is this for the bounty (I'm assuming this shall be split to however you guys like?)

[Captain-T2004]

@danielhanchen, I have tested it on linux as well,
specifically:
OS: Pop!_OS 22.04 LTS x86_64,
Kernel: 6.9.3-76060903-generic,
Drivers: Driver Version: 565.77
CUDA Version: 12.7
and everything is working as intended,
And yes, the bounty will be split by us, Please also checkout #1799.

[adityaghai07]

@danielhanchen Though the bounty is an added benefit , we are really enjoying working on these problems. We aim to climb up the points ladder by targeting the smaller issues first.

[danielhanchen]

Oh wait that reminded me - we cannot place

triton = [
    "triton @ https://github.com/woct0rdho/triton-windows/releases/download/v3.2.0-windows.post10/triton-3.2.0-cp39-cp39-win_amd64.whl ; python_version=='3.9' and platform_system == 'Windows'",
]

for Pypi packages since they do not allow URL links!

[danielhanchen]

Likewise all the Xformers ones unfortunately do not work.

We could move this to the normal Unsloth main branch.

Ie maybe somehow allow "pip install unsloth[windows] @ git+https://github.com/unslothai/unsloth"

[adityaghai07]

@danielhanchen, We have moved this to main branch as per your suggestion. We have created a new PR for the same #1841, This will allow pip install "unsloth[windows] @ git+https://github.com/unslothai/unsloth", to work perfectly in windows.