Docs site poc (LAION-AI#346)

add docusaurus based docs site under /docs

Author: andrewm4894

.devcontainer/post_create_command.sh

@@ -1,2 +1,7 @@
 # ensure pre-commit is installed
 pre-commit install
+
+# run yarn install in docs folder
+cd docs
+yarn install
+cd ..

.github/workflows/deploy-docs-site.yaml

@@ -0,0 +1,53 @@
+name: Deploy docs to gitHub pages
+
+on:
+  push:
+    branches:
+      - main
+      - docs-site-poc
+    paths:
+      - ".github/workflows/deploy-docs-site.yaml"
+      - "docs/**"
+  pull_request:
+
+jobs:
+  deploy:
+    runs-on: ubuntu-22.04
+    permissions:
+      contents: write
+    concurrency:
+      group: ${{ github.workflow }}-${{ github.ref }}
+    defaults:
+      run:
+        working-directory: docs
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Setup Node
+        uses: actions/setup-node@v3
+        with:
+          node-version: "16"
+
+      - name: Get yarn cache
+        id: yarn-cache
+        run: echo "YARN_CACHE_DIR=$(yarn cache dir)" >> "${GITHUB_OUTPUT}"
+
+      - name: Cache dependencies
+        uses: actions/cache@v3
+        with:
+          path: ${{ steps.yarn-cache.outputs.YARN_CACHE_DIR }}
+          key: ${{ runner.os }}-docs-${{ hashFiles('**/yarn.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-docs-
+
+      - run: yarn install --frozen-lockfile
+      - run: yarn build
+
+      - name: Deploy
+        uses: peaceiris/actions-gh-pages@v3
+        if:
+          ${{ github.ref == 'refs/heads/main' || github.ref ==
+          'refs/heads/docs-site-poc' }}
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ./docs/build

docs/.gitignore

@@ -0,0 +1,20 @@
+# Dependencies
+node_modules
+
+# Production
+build
+
+# Generated files
+.docusaurus
+.cache-loader
+
+# Misc
+.DS_Store
+.env.local
+.env.development.local
+.env.test.local
+.env.production.local
+
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*

docs/README.md

@@ -1,14 +1,58 @@
-# Documentation
+# Website
 
-This directory contains the documentation for the project and other related
-organization documents.
+This website is built using [Docusaurus 2](https://docusaurus.io/), a modern
+static website generator.
 
-## Contributing to this documentation
+### Contributing
 
-Please make a pull request to the `main` branch with your changes.
+#### Changes to existing docs
 
-Consider that this folder is used for documenting the various code sub-parts,
-the high-level ideas, the ML aspects, experiments, contributor guides, guides
-for data creation, and many more things. Please try to keep the documentation as
-concise as possible and keep an organized folder structure that makes sense for
-everyone.
+You can just make a PR on whatever .md file you would like to update.
+
+#### Changes to docs structure
+
+If you would like to add a new category:
+
+1. Create a new folder under `/docs/docs/` for the category you want to add.
+1. Include any `.md` files you want to live under this new category.
+1. Update the order or hierarchy in `/docs/sidebars.js` as needed.
+
+If you would like to add a new page into an existing category:
+
+1. Create the `.md` file you want in the relevant folder within `/docs/docs/`.
+1. Update the hierarchy in `/docs/sidebars.js` as needed.
+
+### Installation
+
+From within the `/docs/` folder.
+
+```
+$ yarn
+```
+
+### Local Development
+
+```
+$ yarn start
+```
+
+This command starts a local development server and opens up a browser window.
+Most changes are reflected live without having to restart the server.
+
+### Browser Development
+
+If you would like to work on the docs from within your browser you can create a
+github codespace on your fork or branch. Then from within that codespace you can
+run below commands to launch the docs site on port 3000 within your codespace.
+
+```bash
+
+# cd to docs dir
+cd docs
+
+# start dev server to work on your changes
+yarn start
+```
+
+Once you port forward to port 3000 within your codespace you will be able to see
+all changes reflected as soon as you make them.

docs/babel.config.js

@@ -0,0 +1,3 @@
+module.exports = {
+  presets: [require.resolve("@docusaurus/core/lib/babel/preset")],
+};

docs/docs/data/README.md

@@ -0,0 +1,3 @@
+# Data
+
+Resources related to data.

docs/data_augmentation.md renamed to docs/docs/data/augmentation.md

@@ -1,4 +1,4 @@
-# OpenAssistant Datasets
+# Datasets
 
 The datasets for this project are currently hosted as loading scripts on the
 [Open-Assistant organization](https://huggingface.co/OpenAssistant) the Hugging
@@ -98,7 +98,11 @@ You can make a new branch as such:
 git checkout -b <dataset_name>
 ```
 
-<p style="color:red"> <b> Please do not make changes on the master branch! </b></p>
+:::caution
+
+Please do not make changes on the master branch!
+
+:::
 
 Always make sure you're on the right branch with the following command:
 
@@ -214,7 +218,7 @@ all necessary TODOs. There are three key methods that are important:
 For the `_info_` function, you will need to define `features` for your
 `DatasetInfo` object. For each dataset config, choose the right schema from our
 list of examples. You can find the schemas in the
-[schemas directory](openassistant/utils/schemas/).
+[schemas directory](https://github.com/LAION-AI/Open-Assistant/tree/main/openassistant).
 
 You will use this schema in the `_generate_examples` return value.
 
@@ -228,7 +232,8 @@ TODO
 #### Running & debugging
 
 You can run your data loader script during development by appending the
-following statement to your code ([templates/template.py](templates/template.py)
+following statement to your code
+([templates/template.py](https://github.com/LAION-AI/Open-Assistant/blob/main/openassistant/templates/template.py)
 already includes this):
 
 ```python