Skip to content

Commit a0d5930

Browse files
committed
.
1 parent 640c752 commit a0d5930

6 files changed

+132
-1
lines changed

classimbalance.rst

+5
Original file line numberDiff line numberDiff line change
@@ -70,3 +70,8 @@ which is used to pare down and centralise the negative cases.
7070
Cost Sensitive Classification
7171
------------------------------
7272

73+
One can also make the classifier aware of the imbalanced data by incorporating the weights
74+
of the classes into a cost function.
75+
Intuitively, we want to give higher weight to minority class and lower weight to majority class.
76+
77+
http://albahnsen.github.io/CostSensitiveClassification/index.html

docker.rst

+104
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
Containerization
2+
=================
3+
4+
Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies,
5+
and ship it all out as one package. They allow a modular construction of an application architecture, or microservice in short.
6+
Docker is a popular tool designed to make it easier to create, deploy, and run applications by using containers.
7+
8+
Preprocessing scripts and models can be created as a docker image snapshot, and launched as a container in production.
9+
10+
https://runnable.com/docker/python/dockerize-your-python-application
11+
https://docs.docker.com/get-started/part2/
12+
13+
Docker Folder
14+
--------------
15+
To start of a new project, create a new folder. This should only contain your docker file and related python files.
16+
17+
Dockerfile
18+
------------
19+
A dockerfile, is a file without extension type. It contains commands to tell docker what are the steps to do to
20+
create an image.
21+
22+
* ``FROM`` tells Docker which image you base your image on (eg, Python 3 or continuumio/miniconda3).
23+
* ``RUN`` tells Docker which additional commands to execute.
24+
* ``CMD`` tells Docker to execute the command when the image loads.
25+
26+
.. code::
27+
28+
# Use an official Python runtime as a parent image
29+
FROM python:2.7-slim
30+
31+
# Set the working directory to /app
32+
WORKDIR /app
33+
34+
# Copy the current directory contents into the container at /app
35+
COPY . /app
36+
37+
# Install any needed packages specified in requirements.txt
38+
RUN pip install --trusted-host pypi.python.org -r requirements.txt
39+
40+
# Make port 80 available to the world outside this container
41+
EXPOSE 80
42+
43+
# Define environment variable
44+
ENV NAME World
45+
46+
# Run app.py when the container launches
47+
CMD ["python", "app.py"]
48+
49+
50+
Build Docker image
51+
---------------------
52+
``docker build -t image-name .`` --(-t = tag the image as) build and name image
53+
54+
55+
Commands
56+
----------
57+
58+
Help
59+
* ``docker --help`` --list all base commands
60+
* ``docker COMMAND --help`` --list all options for a command
61+
62+
Create Image
63+
* ``docker build -t image_name .`` --(-t = tag the image as) build and name image, "." is the location of the dockerfile
64+
65+
Get Image from Docker Hub
66+
* ``docker pull image_name`` --pull image from dockerhub into docker
67+
* ``docker run image_name COMMAND`` --check if image in docker, if not pull & run image from dockerhub into docker. If no command is given, the container will stop running.
68+
* ``docker run image_name cat /etc/*release*`` --run image and print out the version of image
69+
70+
Other Run Commands
71+
* ``docker run Ubuntu:17.04`` --semicolon specifies the version (known as tags as listed in Dockerhub), else will pull the latest
72+
* ``docker run ubuntu`` vs ``docker run mmumshad/ubuntu`` --the first is an official image, the 2nd with the / is created by the community
73+
* ``docker run -d image_name`` --(-d = detach) docker runs in background, and you can continue typing other commands in the bash. Else need to open another terminal.
74+
* ``docker run -v /local/storage/folder:/image/data/folder mysql`` --(-v = volume mapping) all data will be destroyed if container is stopped
75+
76+
.. figure:: images/docker_cmd.PNG
77+
:scale: 100 %
78+
:align: center
79+
80+
running docker with a command. each container has a unique container ID, container name, and their base image name
81+
82+
IPs & Ports
83+
* ``192.168.1.14`` --IP address of docker host
84+
* ``docker inspect container_id`` --dump of container info, as well as at the bottom, under Network, the internal IP address. to view server in web browser, enter the ip and the exposed port. eg. 172.17.0.2:8080
85+
* ``docker run -p 80:5000 image_name`` --(host_port:container_port) map host service port with the container port on docker host
86+
87+
See Images & Containers in Docker
88+
* ``docker images`` --see all installed docker images
89+
* ``docker ps`` --(ps = process status) show status of images which are running
90+
* ``docker ps -a`` --(-a = all) show status of all images including those that had exited
91+
92+
Start/Stop Containers
93+
* ``docker start container_name`` --run container
94+
* ``docker stop container_name`` --stop container from running, but container still lives in the disk
95+
* ``docker stop container_name1 container_name2`` --stop multiple container from running in a single line
96+
* ``docker stop container_id`` --stop container using the ID. There is no need to type the id in full, just the first few char suffices.
97+
98+
Remove Containers/Images
99+
* ``docker rm container_name`` --remove container from docker
100+
* ``docker rmi image_name`` --(rmi = remove image) from docker. must remove container b4 removing image.
101+
* ``docker -f rmi image_name`` --(-f = force) force remove image even if container is running
102+
103+
Execute Commands for Containers
104+
* ``docker exec container_nm COMMAND`` --execute a command within container

evaluation.rst

-1
Original file line numberDiff line numberDiff line change
@@ -677,7 +677,6 @@ https://github.com/HDI-Project/BTB
677677
model.fit(X_train, y_train)
678678
y_predict = model.predict(X_test)
679679
score = accuracy_score(y_test, y_predict)
680-
print('epoch: {}, accuracy: {}'.format(i+1,score))
681680
682681
# store scores & parameters
683682
score_list.append(score)

images/docker_cmd.PNG

10.2 KB
Loading

index.rst

+1
Original file line numberDiff line numberDiff line change
@@ -28,5 +28,6 @@ A lot of the content are compiled from various resources, so please cite them ap
2828
reinforcement
2929
evaluation
3030
explainability
31+
docker
3132
ethics
3233
resources

reinforcement.rst

+22
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,28 @@
11
Reinforcement Learning
22
=======================
33

4+
Concepts
5+
--------------
6+
7+
Definitions
8+
************
9+
* **argmax(x)**: position where the first max value occurs
10+
11+
Exploration vs Exploitation
12+
****************************
13+
14+
Elements
15+
*********
16+
* Agent
17+
* Environment
18+
* Model of the environment
19+
* Policy
20+
* Reward signal
21+
* Value function
22+
23+
24+
Introduction
25+
---------------
426
This series of medium_ articles gave a good description of various types of reinforcement learning
527
with jupyter notebook descriptions for various games. This includes deep learning using tensorflow.
628

0 commit comments

Comments
 (0)