Remove username (#14)

Remove username in terraform files. 
`namespace` parameter is require when creating kubernetes resource and deploying.

Author: privacywill

.env.singleuser renamed to .env.example

@@ -15,5 +15,5 @@ jobs:
         disk-cache: ${{ github.workflow }}
         # Share repository cache between workflows.
         repository-cache: true
-    - run: cp .env.singleuser env.bzl
+    - run: cp .env.example env.bzl
     - run: bazel build //...

.env.multiusers

@@ -78,19 +78,13 @@ We are releasing an alpha version, which may miss some necessary features.
 
 ## Deploying in Google Cloud Platform (GCP)
 ### Defining environment variables
-First, determine whether there are multiple developers deploying simultaneously in your environment. If yes, use the multiple users template, otherwise use the single user template. 
+First, copy the example environment variables template to the existing directory.
 ```
-# multiple users
-cp .env.multiusers env.bzl
-```
-```
-# single user
-cp .env.singleuser env.bzl
+cp .env.example env.bzl
 ```
 Edit the variables in `env.bzl`. The `env.bzl` file is the one that really takes effect, the other files are just templates. The double quotes around a variable name are needed. For example:
 ```
 env="dev"                        # the deployment environment
-username="mock developer name"   # the developer name
 mysql_username="mockname"        # mysql database username 
 mysql_password="mockpwd"         # mysql database password
 project_id="you project id"      # gcp project id
@@ -102,17 +96,20 @@ zone=""                          # the zone that the resources created in
 ### Preparing resources
 Create resources for the data clean room by terraform. Make sure you have correctly defined environment variables in the `env.bzl`.
 
-`resources/gcp` directory contains the resources releated to the gcp including: clusters, cloud sql instance, database, docker repositories, and service accounts. These resource are global and only created once for all the developers in one project.
-
-`resources/kubernetes` directory includes the resources releated to the kubernete cluster including: namespace, role, secret.
-
-Create all the resources by:
+`resources/gcp` directory contains the resources releated to the gcp including: clusters, cloud sql instance, database, docker repositories, and service accounts. These resource are global and only created once for all the developers in one project. If you are the project owner, run the commands to create global resources.
 ```
-pushd resources
+pushd resources/gcp
 ./apply.sh
 popd
 ```
 
+`resources/kubernetes` directory includes the resources releated to the kubernete cluster including: namespace, role, secret. Once the global resources have been created, the developers can run the commands to create user-specific resources. The namespace is required, and make sure the namespace is distinct.
+```
+pushd resources/kubernetes
+./apply.sh --namespace=xxxx
+popd
+```
+
 ### Building and Pushing Images
 `app` directory contains the source codes of the data clean room which has three components:
 

.github/workflows/pr.yml

@@ -22,14 +22,15 @@ fi
 VAR_FILE=$(realpath $VAR_FILE)
 source $VAR_FILE
 
-tag="latest"
-if [ -z "$username" ]; then
-    helm_name="data-clean-room-helm-$env"
-    k8s_namespace="data-clean-room-$env"
-else
-    helm_name="data-clean-room-helm-$username"
-    k8s_namespace="data-clean-room-$username"
+if [ -z "$1" ]
+then 
+    echo "Error: No namespace argument supplied."
+    exit 1
 fi
+namespace=$1
+
+tag="latest"
+helm_name="data-clean-room-helm"
 
 connection_name="${project_id}:${region}:dcr-${env}-db-instance"
 service_account="dcr-k8s-pod-sa"
@@ -44,7 +45,7 @@ helm upgrade --cleanup-on-fail \
     --set monitorImage.tag=${tag} \
     --set serviceAccount.name=${service_account} \
     --set cloudSql.connection_name=${connection_name} \
-    --set namespace=${k8s_namespace} \
+    --set namespace=${namespace} \
     --install $helm_name ./ \
-    --namespace $k8s_namespace \
+    --namespace $namespace \
     --values config.yaml

README.md

@@ -13,18 +13,31 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 set -e
+
+for arg in "$@"
+do
+    case $arg in
+        --namespace=*)
+        # If we find an argument --namespace=something, split the string into a name/value array.
+        IFS='=' read -ra NAMESPACE <<< "$arg"
+        # Assign the second element of the array (the value of the --namespace argument) to our variable.
+        namespace="${NAMESPACE[1]}"
+        ;;
+    esac
+done
+
+
+if [ -z "$namespace" ]; then
+    echo "Error: the namespace parameter is required, run the script again like ./apply.sh --namespace="
+    exit 1
+fi
+
 deploy_service() {
     app=$1
     pushd $app
-    ./deploy.sh
+    ./deploy.sh $2
     popd
 }
 
-if [[ $# == 0 ]]; then
-    deploy_service data-clean-room
-    deploy_service jupyterhub
-elif [[ $# == 1 ]]; then 
-    deploy_service $1
-else 
-    echo "ERROR: unkown parameters"
-fi
+deploy_service data-clean-room $namespace
+deploy_service jupyterhub $namespace

deployment/data-clean-room/deploy.sh

@@ -22,16 +22,16 @@ fi
 VAR_FILE=$(realpath $VAR_FILE)
 source $VAR_FILE
 
-tag="latest"
-if [ -z "$username" ]; then
-    helm_name="jupyterhub-helm-$env"
-    k8s_namespace="jupyterhub-$env"
-    api="http://data-clean-room.data-clean-room-$env.svc.cluster.local"
-else
-    helm_name="jupyterhub-helm-$username"
-    k8s_namespace="jupyterhub-$username"
-    api="http://data-clean-room.data-clean-room-$username.svc.cluster.local"
+if [ -z "$1" ]
+then 
+    echo "Error: No namespace argument supplied."
+    exit 1
 fi
+namespace=$1
+
+tag="latest"
+helm_name="jupyterhub-helm"
+api="http://data-clean-room.$namespace.svc.cluster.local"
 
 service_account="jupyter-k8s-pod-sa"
 docker_repo="dcr-${env}-images"
@@ -50,9 +50,9 @@ helm upgrade --cleanup-on-fail \
     --set singleuser.extraEnv.KEY_LOCALTION=${region} \
     --set singleuser.networkPolicy.enabled=false \
     --install $helm_name jupyterhub/jupyterhub \
-    --namespace ${k8s_namespace} \
+    --namespace ${namespace} \
     --version=3.0.3 \
     --values config.yaml
 
 echo "Deployment Completed."
-echo "Try 'kubectl --namespace=$k8s_namespace get service proxy-public' to obtain external IP"
+echo "Try 'kubectl --namespace=$namespace get service proxy-public' to obtain external IP"

deployment/deploy.sh

@@ -13,6 +13,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+echo "You are creating the gcp resources and this should only be done once."
+
 # Check if gcloud is installed
 if ! [ -x "$(command -v gcloud)" ]; then
 	echo "Error: gcloud is not installed." >&2
@@ -26,7 +28,7 @@ if ! gcloud auth list | grep -q 'ACTIVE'; then
 fi
 
 # check whether variables has been set
-VAR_FILE="../env.bzl"
+VAR_FILE="../../env.bzl"
 if [ ! -f "$VAR_FILE" ]; then
     echo "Error: Variables file does not exist."
     exit 1
@@ -38,8 +40,6 @@ if ! gsutil ls gs://dcr-tf-state-$env > /dev/null 2>&1; then
   gsutil mb -l us gs://dcr-tf-state-$env
 fi
 
-pushd gcp
-
 cat << EOF > backend.tf
 terraform {
   backend "gcs" {
@@ -52,14 +52,3 @@ EOF
 cp $VAR_FILE terraform.tfvars
 terraform init -reconfigure
 terraform apply
-popd 
-
-zone=$region-a
-# get kubernete cluster credentials
-gcloud container clusters get-credentials dcr-$env-cluster --zone $zone --project $project_id
-
-pushd kubernetes
-cp $VAR_FILE terraform.tfvars
-terraform init -reconfigure
-terraform apply
-popd 

deployment/jupyterhub/deploy.sh

@@ -14,12 +14,6 @@
  * limitations under the License.
  */
 
-variable "username" {
-  type        = string
-  description = "Username to postfix resources with"
-  default     = ""
-}
-
 variable "env" {
   type        = string
   description = "Deployment environment, e.g., dev, prod, oss"

resources/apply.sh renamed to resources/gcp/apply.sh

@@ -0,0 +1,63 @@
+#!/bin/bash
+# Copyright 2024 TikTok Pte. Ltd.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+set -e
+
+for arg in "$@"
+do
+    case $arg in
+        --namespace=*)
+        # If we find an argument --namespace=something, split the string into a name/value array.
+        IFS='=' read -ra NAMESPACE <<< "$arg"
+        # Assign the second element of the array (the value of the --namespace argument) to our variable.
+        namespace="${NAMESPACE[1]}"
+        ;;
+    esac
+done
+
+
+if [ -z "$namespace" ]; then
+    echo "Error: the namespace parameter is required, run the script again like ./apply.sh --namespace="
+    exit 1
+fi
+
+# Check if gcloud is installed
+if ! [ -x "$(command -v gcloud)" ]; then
+	echo "Error: gcloud is not installed." >&2
+	exit 1
+fi
+
+# Check if gcloud logged in
+if ! gcloud auth list | grep -q 'ACTIVE'; then
+	echo "Error: No active gcloud account found." >&2
+	exit 1
+fi
+
+# check whether variables has been set
+VAR_FILE="../../env.bzl"
+if [ ! -f "$VAR_FILE" ]; then
+    echo "Error: Variables file does not exist."
+    exit 1
+fi
+VAR_FILE=$(realpath $VAR_FILE)
+source $VAR_FILE
+
+zone=$region-a
+# get kubernete cluster credentials
+gcloud container clusters get-credentials dcr-$env-cluster --zone $zone --project $project_id
+
+cp $VAR_FILE terraform.tfvars
+echo -e "\nnamespace=\"$namespace\"\n" >> terraform.tfvars
+terraform init -reconfigure
+terraform apply