Skip to content

Commit 01033a1

Browse files
privacywillDayeol Lee
andauthored
deployment without google account, through minikube (#43)
This PR solves #31 , #24 Partially addresses #23 ## Changes - Support users to try manatee without a google account, completely deployed on local. Components comparison: | | Google Cloud | Local | | ----| ----------- | ----------- | |Cluster| GKE | Minikube | |Database| Cloud database | Mysql | |Storage| Google cloud storage | Minio | |Docker Registry| Google artifact registry | Minikube addon | |TEE Backend| Google confidential space | Mock TEE backend: job on kubernetes | - Add storage interface and two implementations: gcs and minio - Add registry interface and two implementations: gcp and minikube - Add signed url feature, so the tee only uses `curl` to upload output and custom attestation token. - Add mock tee backend implement - Modify jupyterhub extension to support signed url feature - Fixed the bug of getting attestation - A document in `docs/getting-started/minikube.md` guides users how to deploy manatee locally. ## Caveats - Introduced many environment variables through config map. The config values should be managed at one place. Will open a new MR to solve it. ## Testing - end-to-end testing for minikube deployment. - end-to-end testing for GCP deployment --------- Co-authored-by: Dayeol Lee <dayeol.lee@tiktok.com>
1 parent d580d36 commit 01033a1

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

55 files changed

+1907
-1737
lines changed

BUILD

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,3 +40,11 @@ multirun(
4040
],
4141
jobs = 0,
4242
)
43+
44+
multirun(
45+
name = "load_all_images",
46+
commands = [
47+
"//app/{}:load_image".format(k)
48+
for k in REPOS.keys()
49+
],
50+
)

MODULE.bazel

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,14 +97,17 @@ use_repo(
9797
"com_github_cloudwego_hertz",
9898
"com_github_gin_gonic_gin",
9999
"com_github_google_uuid",
100+
"com_github_minio_minio_go_v7",
100101
"com_github_pkg_errors",
101102
"com_google_cloud_go_compute",
103+
"com_google_cloud_go_iam",
102104
"com_google_cloud_go_storage",
103105
"io_gorm_driver_mysql",
104106
"io_gorm_gorm",
105107
"io_k8s_api",
106108
"io_k8s_apimachinery",
107109
"io_k8s_client_go",
110+
"org_golang_google_genproto",
108111
"org_golang_google_protobuf",
109112
)
110113

app/dcr_api/.hz

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// Code generated by hz. DO NOT EDIT.
22

3-
hz version: v0.9.0
3+
hz version: v0.9.1
44
handlerDir: ""
55
modelDir: biz/model
66
routerDir: ""

app/dcr_api/BUILD.bazel

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,4 +46,5 @@ oci_load(
4646
name = "load_image",
4747
image = ":image",
4848
repo_tags = ["dcr_api:latest"],
49+
visibility = ["//visibility:public"],
4950
)

app/dcr_api/biz/dal/db/job.go

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -23,18 +23,19 @@ import (
2323

2424
type Job struct {
2525
gorm.Model
26-
ID uint64 `gorm:"id" json:"id""`
27-
UUID string `gorm:"uuid" json:"uuid"`
28-
Creator string `gorm:"creator" json:"creator"`
29-
JupyterFileName string `gorm:"jupyter_file_name" json:"jupyter_file_name"`
30-
BuildContextPath string `gorm:"build_context_path" json:"build_context_path"`
31-
Dockerfile string `gorm:"dockerfile" json:"dockerfile"`
32-
DockerImage string `gorm:"docker_image" json:"docker_image"`
33-
DockerImageDigest string `gorm:"docker_image_digest" json:"docker_image_digest"`
34-
AttestationReport string `gorm:"attestation_report" json:"attestation_report"`
35-
JobStatus int `gorm:"job_status" json:"job_status"`
36-
InstanceName string `gorm:"instance_name" json:"instance_name"`
37-
ExtraEnvs map[string]string `gorm:"serializer:json"`
26+
ID uint64 `gorm:"id" json:"id""`
27+
UUID string `gorm:"uuid" json:"uuid"`
28+
Creator string `gorm:"creator" json:"creator"`
29+
JupyterFileName string `gorm:"jupyter_file_name" json:"jupyter_file_name"`
30+
BuildContextPath string `gorm:"build_context_path" json:"build_context_path"`
31+
OutputPutSignedUrl string `gorm:"output_put_signed_url" json:"output_put_signed_url"`
32+
CustomTokenPutSignedUrl string `gorm:"custom_token_put_signed_url" json:"custom_token_put_signed_url"`
33+
Dockerfile string `gorm:"dockerfile" json:"dockerfile"`
34+
DockerImage string `gorm:"docker_image" json:"docker_image"`
35+
DockerImageDigest string `gorm:"docker_image_digest" json:"docker_image_digest"`
36+
JobStatus int `gorm:"job_status" json:"job_status"`
37+
InstanceName string `gorm:"instance_name" json:"instance_name"`
38+
ExtraEnvs map[string]string `gorm:"serializer:json"`
3839
}
3940

4041
func (Job) TableName() string {
@@ -59,7 +60,6 @@ func UpdateJob(j *Job) error {
5960
BuildContextPath: j.BuildContextPath,
6061
DockerImageDigest: j.DockerImageDigest,
6162
DockerImage: j.DockerImage,
62-
AttestationReport: j.AttestationReport,
6363
InstanceName: j.InstanceName,
6464
ExtraEnvs: j.ExtraEnvs,
6565
})

app/dcr_api/biz/handler/job/job_handler.go

Lines changed: 9 additions & 33 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)