Skip to content

Commit ce8a66a

Browse files
lisalisa
committed
Migrate branch of my fork to its own repo
This repository is a fork of the [update python version](https://github.com/lisa/static-binaries/tree/update-python-version) branch in [my fork](https://github.com/lisa/static-binaries) of [andrew-d/static-binaries](https://github.com/andrew-d/static-binaries). The goal of this repository is to separate out the Python parts from the upstream collection of packages.
0 parents  commit ce8a66a

11 files changed

+407
-0
lines changed

‎.dockerignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Dockerfile

‎BPO-7938_pr4338-fix-makesetup-script.patch‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
--- Modules/makesetup.dist 2018-05-23 01:58:50.034767531 +0000
2+
+++ Modules/makesetup 2018-05-23 01:59:36.207811984 +0000
3+
@@ -128,7 +128,11 @@
4+
5+
# Output DEFS in reverse order so first definition overrides
6+
case $line in
7+
- *=*) DEFS="$line$NL$DEFS"; continue;;
8+
+ *=*) if [ $(sed -e 's$::=\|:=\|+=\|?=$=$' -e 's$\s*=.*$$' <<< $line | wc -w) == 1 ]
9+
+ then
10+
+ DEFS="$line$NL$DEFS"; continue;
11+
+ fi
12+
+ ;;
13+
'include '*) DEFS="$line$NL$DEFS"; continue;;
14+
'*noobjects*')
15+
case $noobjects in

‎Dockerfile‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
FROM thedoh/musl-cross:1.1.19
2+
MAINTAINER Lisa Seelye <lisa@thedoh.com>
3+
# Upstream maintainer: Andrew Dunham <andrew@du.nham.ca>
4+
5+
RUN apt-get update && apt-get install -y zip
6+
7+
# Add our build script
8+
ADD . /build/
9+
10+
# This builds the program and copies it to /output
11+
RUN /build/build.sh
12+
13+
###
14+
FROM scratch
15+
MAINTAINER Lisa Seelye <lisa@thedoh.com>
16+
WORKDIR /
17+
COPY --from=0 /output/python* /
18+
19+
# just in case things need them to exist, like flask
20+
COPY passwd /etc/passwd
21+
COPY group /etc/group
22+
23+
ENV \
24+
PYTHONHOME=/python2.7.zip \
25+
PYTHONPATH=/python2.7.zip
26+
27+
# Users of this base image should be sure to COPY their entrypoint.py to /entrypoint.py
28+
CMD [ "/python", "-s", "-S", "/entrypoint.py" ]

‎LICENSE‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
This repository follows the same license as https://github.com/andrew-d/static-binaries since this is a fork of it to isolate the Python parts.

‎README.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# docker-static-python
2+
3+
This repository is a fork of the [update python version](https://github.com/lisa/static-binaries/tree/update-python-version) branch in [my fork](https://github.com/lisa/static-binaries) of [andrew-d/static-binaries](https://github.com/andrew-d/static-binaries). The goal of this repository is to separate out the Python parts from the upstream collection of packages.
4+
5+
# Version
6+
7+
This Docker image provides Python-2.7.15rc1 with the following modules and software statically compiled in to the python binary:
8+
9+
* [psutil-5.4.5](https://pypi.org/project/psutil/)
10+
* [OpenSSL-1.0.2k](https://www.openssl.org)
11+
* [zlib-1.2.11](https://zlib.net/)
12+
* [termcap-1.3.1](https://www.gnu.org/software/termutils/manual/termcap-1.3/termcap.html)
13+
* [readline-6.3](https://tiswww.case.edu/php/chet/readline/rltop.html)
14+
15+
# Usage
16+
17+
By default the image will run `python -sS /entrypoint.py` but if you would prefer a Python shell invoke:
18+
19+
$ docker run -ti thedoh/static-python:2.7.15rc1-r1
20+
Python 2.7.15rc1 (default, May 17 2018, 15:15:27)
21+
[GCC 5.3.0] on linux2
22+
>>>
23+
24+
25+
26+
# psutil Notes
27+
28+
The package is included to illustrate the possibility of compiling third party modules into the Python static image. The `psuutil` package is patched with two patches (below) to force the `psutil` Python library code to _not_ look in its directory (`from . import`) and rather to anywhere the requirement can be met (such as from within the `python` binary).
29+
* [psutil_import__init__.patch](psutil_import__init__.patch)
30+
* [psutil_import_pslinux.patch](psutil_import_pslinux.patch)
31+
32+
A common use case will be to `pip install` some set of requirements for use. Since some of those requirements may have C code to compile it is thus a requirement to statically compile that code into the `python` binary because there are no shared Python objects to link against. The inclusion of `psutil` demonstrates this process.
33+
34+
35+
## psutil License
36+
37+
> psutil is distributed under BSD license reproduced below.
38+
>
39+
> Copyright (c) 2009, Jay Loden, Dave Daeschler, Giampaolo Rodola'
40+
> All rights reserved.
41+
>
42+
> Redistribution and use in source and binary forms, with or without modification,
43+
> are permitted provided that the following conditions are met:
44+
>
45+
> * Redistributions of source code must retain the above copyright notice, this
46+
> list of conditions and the following disclaimer.
47+
> * Redistributions in binary form must reproduce the above copyright notice,
48+
> this list of conditions and the following disclaimer in the documentation
49+
> and/or other materials provided with the distribution.
50+
> * Neither the name of the psutil authors nor the names of its contributors
51+
> may be used to endorse or promote products derived from this software without
52+
> specific prior written permission.
53+
>
54+
> THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
55+
> ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
56+
> WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
57+
> DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
58+
> ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
59+
> (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
60+
> LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
61+
> ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
62+
> (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
63+
> SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
64+
65+
## Entrypoint.py
66+
67+
A sample `entrypoint.py` to call an app using Factory pattern might be:
68+
69+
import sys
70+
import os
71+
72+
sys.path.append("/usr/src/app")
73+
74+
os.chdir("/usr/src/app")
75+
76+
from your.app import make_app
77+
78+
app = make_app()
79+
80+
app.run()
81+
82+
If you prefer to `execv` instead:
83+
84+
import sys
85+
import os
86+
87+
88+
os.chdir("/")
89+
90+
os.execv("/python", ["-s","-S","/myapp.py"])
91+
# No code runs after here
92+
93+
94+
Or `subprocess`:
95+
96+
import sys
97+
import os
98+
99+
100+
os.chdir("/")
101+
102+
import subprocess
103+
104+
subprocess.call(["/python", "-s","-S","/myapp.py"])
105+
# Code execution continues

‎build.sh‎

Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
#!/bin/bash
2+
3+
set -e
4+
set -o pipefail
5+
set -x
6+
7+
8+
ZLIB_VERSION=1.2.11
9+
TERMCAP_VERSION=1.3.1
10+
READLINE_VERSION=6.3
11+
OPENSSL_VERSION=1.0.2k
12+
PYTHON_BASE_VERSION=2.7.15
13+
PSUTIL_VERSION=5.4.5
14+
PYTHON_RC=rc1
15+
16+
CPU_COUNT=$(nproc --all 2>/dev/null)
17+
MAKE_J="-j${CPU_COUNT:-1}"
18+
19+
function fetch_psutil() {
20+
cd /build
21+
curl -L -o /build/psutil-${PSUTIL_VERSION}.zip https://github.com/giampaolo/psutil/archive/release-${PSUTIL_VERSION}.zip
22+
unzip /build/psutil-${PSUTIL_VERSION}.zip
23+
mv psutil-release-${PSUTIL_VERSION} psutil-${PSUTIL_VERSION}
24+
# Prune out the Python files from the C files and then monkeypatch the
25+
# Python files
26+
mkdir -p psutil/{c_files,python_files}
27+
cp -r /build/psutil-${PSUTIL_VERSION}/psutil psutil/c_files/
28+
rm -rf psutil/c_files/psutil/*.py* psutil/c_files/psutil/DEVNOTES psutil/c_files/psutil/tests
29+
# Python files
30+
cp -r /build/psutil-${PSUTIL_VERSION}/psutil psutil/python_files/
31+
rm -rf psutil/python_files/psutil/*.c psutil/python_files/psutil/*.h psutil/python_files/psutil/arch psutil/python_files/psutil/tests
32+
# Patch now
33+
cd psutil
34+
patch --ignore-whitespace -p0 < /build/psutil_import__init__.patch
35+
patch --ignore-whitespace -p0 < /build/psutil_import_pslinux.patch
36+
cd ..
37+
# Ready to be copied for Lib
38+
}
39+
40+
41+
function build_zlib() {
42+
cd /build
43+
44+
# Download
45+
curl -LO http://zlib.net/zlib-${ZLIB_VERSION}.tar.gz
46+
tar zxvf zlib-${ZLIB_VERSION}.tar.gz
47+
cd zlib-${ZLIB_VERSION}
48+
49+
# Build
50+
CC='/opt/cross/x86_64-linux-musl/bin/x86_64-linux-musl-gcc -static -fPIC' \
51+
./configure \
52+
--static
53+
make ${MAKE_J}
54+
}
55+
56+
function build_termcap() {
57+
cd /build
58+
59+
# Download
60+
curl -LO http://ftp.gnu.org/gnu/termcap/termcap-${TERMCAP_VERSION}.tar.gz
61+
tar zxvf termcap-${TERMCAP_VERSION}.tar.gz
62+
cd termcap-${TERMCAP_VERSION}
63+
64+
# Build
65+
CC='/opt/cross/x86_64-linux-musl/bin/x86_64-linux-musl-gcc -static -fPIC' \
66+
./configure \
67+
--disable-shared \
68+
--enable-static
69+
make ${MAKE_J}
70+
}
71+
72+
function build_readline() {
73+
cd /build
74+
75+
# Download
76+
curl -LO ftp://ftp.cwru.edu/pub/bash/readline-${READLINE_VERSION}.tar.gz
77+
tar xzvf readline-${READLINE_VERSION}.tar.gz
78+
cd readline-${READLINE_VERSION}
79+
80+
# Build
81+
CC='/opt/cross/x86_64-linux-musl/bin/x86_64-linux-musl-gcc -static -fPIC' \
82+
./configure \
83+
--disable-shared \
84+
--enable-static
85+
make ${MAKE_J}
86+
87+
# Note that things look for readline in <readline/readline.h>, so we need
88+
# that directory to exist.
89+
ln -s /build/readline-${READLINE_VERSION} /build/readline-${READLINE_VERSION}/readline
90+
}
91+
92+
function build_openssl() {
93+
cd /build
94+
95+
# Download
96+
curl -LO https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz
97+
tar zxvf openssl-${OPENSSL_VERSION}.tar.gz
98+
cd openssl-${OPENSSL_VERSION}
99+
100+
# Configure
101+
CC='/opt/cross/x86_64-linux-musl/bin/x86_64-linux-musl-gcc -static' ./Configure no-shared linux-x86_64
102+
103+
# Build
104+
make
105+
echo "** Finished building OpenSSL"
106+
}
107+
108+
function build_python() {
109+
cd /build
110+
111+
# Download
112+
curl -LO https://www.python.org/ftp/python/${PYTHON_BASE_VERSION}/Python-${PYTHON_BASE_VERSION}${PYTHON_RC}.tar.xz
113+
unxz Python-${PYTHON_BASE_VERSION}${PYTHON_RC}.tar.xz
114+
tar -xvf Python-${PYTHON_BASE_VERSION}${PYTHON_RC}.tar
115+
cd Python-${PYTHON_BASE_VERSION}${PYTHON_RC}
116+
117+
# Copy psutil source code into place
118+
mkdir Modules/psutil
119+
cp -r /build/psutil/c_files/psutil/* Modules/psutil/
120+
# Convert the version to what psutil expects. This should be stable until
121+
# their setup.py changes.
122+
psversion=$(echo $PSUTIL_VERSION | sed 's,\.,,g')
123+
124+
# Set up modules
125+
cp Modules/Setup.dist Modules/Setup
126+
MODULES="_bisect _collections _csv _datetime _elementtree _functools _heapq _io _md5 _posixsubprocese _random _sha _sha256 _sha512 _socket _struct _weakref array binascii cmath cStringIO cPickle datetime fcntl future_builtins grp itertools math mmap operator parser readline resource select spwd strop syslog termios time unicodedata zlib"
127+
for mod in $MODULES;
128+
do
129+
sed -i -e "s/^#${mod}/${mod}/" Modules/Setup
130+
done
131+
132+
echo '_json _json.c' >> Modules/Setup
133+
echo '_multiprocessing _multiprocessing/multiprocessing.c _multiprocessing/semaphore.c _multiprocessing/socket_connection.c' >> Modules/Setup
134+
135+
# Enable static linking
136+
sed -i '1i\
137+
*static*' Modules/Setup
138+
139+
# Set dependency paths for zlib, readline, etc.
140+
sed -i \
141+
-e "s|^zlib zlibmodule.c|zlib zlibmodule.c -I/build/zlib-${ZLIB_VERSION} -L/build/zlib-${ZLIB_VERSION} -lz|" \
142+
-e "s|^readline readline.c|readline readline.c -I/build/readline-${READLINE_VERSION} -L/build/readline-${READLINE_VERSION} -L/build/termcap-${TERMCAP_VERSION} -lreadline -ltermcap|" \
143+
Modules/Setup
144+
# static compile these modules. Any others that `setup.py` might want to
145+
# build "shared" should follow this pattern to make them included
146+
echo "" >> Modules/Setup
147+
cat << 'EOF' >>Modules/Setup
148+
_lsprof rotatingtree.c _lsprof.c
149+
pyexpat expat/xmlparse.c expat/xmlrole.c expat/xmltok.c pyexpat.c -I$(srcdir)/Modules/expat -DHAVE_EXPAT_CONFIG_H -DUSE_PYEXPAT_CAPI -DHAVE_SYSCALL_GETRANDOM
150+
_psutil_linux psutil/_psutil_common.c psutil/_psutil_posix.c psutil/_psutil_linux.c -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=__PSVER__ -DPSUTIL_LINUX=1
151+
_psutil_posix psutil/_psutil_common.c psutil/_psutil_posix.c -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=__PSVER__ -DPSUTIL_LINUX=1
152+
EOF
153+
154+
# Enable OpenSSL support
155+
patch --ignore-whitespace -p1 < /build/cpython-enable-openssl.patch
156+
157+
# Fix https://bugs.python.org/issue7938
158+
echo "Patching for https://bugs.python.org/issue7938"
159+
patch --ignore-whitespace -p0 < /build/BPO-7938_pr4338-fix-makesetup-script.patch
160+
161+
sed -i \
162+
-e "s|^SSL=/build/openssl-TKTK|SSL=/build/openssl-${OPENSSL_VERSION}|" \
163+
-e "s|__PSVER__|${psversion}|g" \
164+
Modules/Setup
165+
166+
# Configure
167+
CC='/opt/cross/x86_64-linux-musl/bin/x86_64-linux-musl-gcc -static -fPIC' \
168+
CXX='/opt/cross/x86_64-linux-musl/bin/x86_64-linux-musl-g++ -static -static-libstdc++ -fPIC' \
169+
LD=/opt/cross/x86_64-linux-musl/bin/x86_64-linux-musl-ld \
170+
./configure \
171+
--disable-shared
172+
173+
# Build
174+
make --trace ${MAKE_J} LDFLAGS="-static" LINKFORSHARED=" " || true
175+
176+
177+
/opt/cross/x86_64-linux-musl/bin/x86_64-linux-musl-strip python
178+
# There may be a better way to ensure _sysconfigdata.py is included in the
179+
# .zip file, but I am not sure how best to do it, so this is a bit of a
180+
# hack, banking on the contents of this container staying around for it
181+
# to matter.
182+
cp $(find . -name _sysconfigdata.py -print) Lib
183+
# Copy the patched psutil Python files into place
184+
cp -r /build/psutil/python_files/psutil Lib
185+
cd Lib && zip -r ../python2.7.zip .
186+
187+
}
188+
189+
function doit() {
190+
fetch_psutil
191+
build_zlib
192+
build_termcap
193+
build_readline
194+
build_openssl
195+
build_python
196+
197+
mkdir -p /output
198+
cp /build/Python-${PYTHON_BASE_VERSION}${PYTHON_RC}/{python,python2.7.zip} /output
199+
}
200+
201+
doit
202+
203+
echo "Output:"
204+
ls -l /output
205+
echo "Done."

‎cpython-enable-openssl.patch‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
--- a/Modules/Setup 2015-03-15 07:33:09.093498063 +0000
2+
+++ b/Modules/Setup 2015-03-15 07:33:43.436659171 +0000
3+
@@ -204,10 +204,10 @@
4+
5+
# Socket module helper for SSL support; you must comment out the other
6+
# socket line above, and possibly edit the SSL variable:
7+
-#SSL=/usr/local/ssl
8+
-#_ssl _ssl.c \
9+
-# -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
10+
-# -L$(SSL)/lib -lssl -lcrypto
11+
+SSL=/build/openssl-TKTK
12+
+_ssl _ssl.c \
13+
+ -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
14+
+ -L$(SSL) -lssl -lcrypto
15+
16+
# The crypt module is now disabled by default because it breaks builds
17+
# on many systems (where -lcrypt is needed), e.g. Linux (I believe).

‎group‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
root:x:0:root

‎passwd‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
root:x:0:0:root:/root:/bin/bash

0 commit comments

Comments
 (0)