1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
|
+++
title = "OpenBuild Service"
+++
<div class="reveal">
<!-- Any section element inside of this container is displayed as a slide -->
<div class="slides">
<section>
<h1>Open Build Service</h1>
<h3>Fighting the Linux binary distribution problem..</h3>
<p>
<small>Created by <a href="http://www.andreasbaumann.cc">Andreas Baumann</a></small>
</p>
<p>
<small><a href="http://www.andreasbaumann.cc/slide/obs">http://www.andreasbaumann.cc/slide/obs</a></small>
</p>
<p><small>(C)2015</small></p>
</section>
<section>
<h2>The Linux Binary Distribution Disaster</h2>
<p>
Linus Torvalds: Provides MacX and Windows binaries for his <i>divelog</i> application
but no Linux binaries, why?
</p>
<p>
There are no Linux binaries: there are Fedora binaries, Ubuntu
binaries, ArchLinux binaries, etc..
</p>
</section>
<section>
<h2>Linux binaries differ..</h2>
<ul>
<li>..in package formats (deb, rpm, tgz, ...)</li>
<li>..in build rules (compiler flags, fencing, optimization, ...)</li>
<li>Linux Standard Base (LSB): should actually guarantee ABI and API stability..</li>
<li>The problem is, there is still too much variation..</li>
</ul>
</section>
<section>
<h2>Shared libraries</h2>
<ul>
<li>share common code on a platform, good idea</li>
<li>there is not a common view of what can be shared and should be shared</li>
<li>packet names and versions differ</li>
<li>sometimes compilation flags differ (e.g. no threading support in sqlite3 on RHEL5)</li>
</ul>
</section>
<section>
<h2>Distribution types</h2>
<ul>
<li>Frozen releases, e.g. Debian 8, RHEL 7, Ubuntu LTS, ...,
long release cycles (4-8 years), they are stable,
but only if you use very old software</li>
<li>Rolling releases, e.g. Fedora, ArchLinux, Gentoo, ...
You have the newest and shiniest software, but
stability may suffer, backporting to other distros
gets almost impossible.
</li>
</ul>
</section>
<section>
<h2>Options for software developers</h2>
<p>Islands, part 1</p>
<ul>
<li>Create your own universe: examples Java, Go, Python, ...
</li>
<li>You deploy your software as a blob of static libraries,
there might be sharing inside your universe but not between
universes.
</li>
<li>In C++ we are "stuck" to the ELF format, shared libarires, binaries
in distribution packages. Build on one Linux (which one?) and hope
it runs everywhere
</li>
<li>Local libraries must be maintained (security patches), imagine
a bundled, unpatched, 5 years old libssl. :-)
</li>
</ul>
</section>
<section>
<h2>Package meta-systems</h2>
<p>Islands, part 2</p>
<ul>
<li><a href="http://www.openpkg.org/">http://www.openpkg.org/</a>: cross-platform RPMs</li>
<li>BSD-like port systems (gentoo, netsrc, etc.): but this means going back to distribution of source code only</li>
</ul>
</section>
<section>
<h2>Package build systems</h2>
<p>Islands, part 3<p>
<ul>
<li><a href="http://build.opensuse.org">http://build.opensuse.org</a>: Open Build Service (for OpenSuSE)</li>
<li><a href="http://launchpad.net">http://launchpad.net</a>: Ubuntu only</li>
<li><a href="http://copr.fedoraproject.org">http://copr.fedoraproject.org</a>, koji: Fedora only</li>
</ul>
</section>
<section>
<h2>Package build systems</h2>
<p>Islands, part 3</p>
<ul>
<li>Linux distros try to maintain their bleeding edge product with the help of the community.</li>
<li>Naturally communities form islands</li>
<li>Exception: OpenBuild system, they build SuSE, RHEL, Debian, Ubuntu, Mandriva, Arch,..</li>
<li>This "behaviour" should be honoured and supported as it is the really only way to go right now</li>
</ul>
</section>
<section>
<h2>OBS</h2>
<ul>
<li>Basically having one RPM SPEC-file and one Debian/Ubuntu tree covers the majority of Linux distributions</li>
<li>Choose your dependencies wisely and provide what's not available on old distros on your own.</li>
<li>Either bundle your stub packages with your software package or package them into a 3rdParty meta package</li>
<li>Make sure package names DON'T collide!</li>
</ul>
</section>
<section>
<h2>Automatization</h2>
<ul>
<li>Combine it with github, build the source trees there (git tag)</li>
<li>Webhooks draw the tarballs and spec files to the OBS cluster</li>
<li>Publish releases on a web space</li>
<li>Keep the current git master as rolling last release of your software</li>
</ul>
</section>
<section>
<h2>Current process</h2>
<ul>
<li>Update in CMakeLists.txt:
<pre><code class="hljs" data-trim contenteditable>
set( STRUS_MAJOR_VERSION 0 )
set( STRUS_MINOR_VERSION 2 )
set( STRUS_PATCH_VERSION 1 )
</code></pre>
</li>
<li>Adapt %changelog and dist/debian/changelog. Keep STRICTLY to syntax! For tipps see also: <a href="http://keepachangelog.com/">http://keepachangelog.com/</a></li>
<li>Commit the master revision and tag it</li>
<pre><code class="hljs" data-trim contenteditable>
git commit -a -m 'changed to release 0.2.1'
git tag 0.2.1
</code></pre>
</ul>
</section>
<section>
<h2>Current process</h2>
<ul>
<li>Deploy to OBS workspace:
<pre><code class="hljs" data-trim contenteditable>
osc co home:PatrickFrey (into $HOME/home:PatrickFrey)</li>
</code></pre>
<pre><code class="hljs" data-trim contenteditable>
cmake .
dist/obs/deploy_to_obs.sh
</code></pre>
</li>
<li>Remove source tarballs of old release, add tarballs of new release:
<pre><code class="hljs" data-trim contenteditable>
cd home:PatrickFrey/strus
osc up
osc rm strus-0.1.6*
osc add strus-0.2.1*
osc commit -m 'added version 0.2.1'
</code></pre>
</li>
</ul>
</section>
<section>
<h2>Issues</h2>
<ul>
<li>dist/obs/deploy_to_obs.sh should be eliminated</li>
<li>Release versions need patching, the mirroring system plays mean tricks otherwise..</li>
<li>How to automatically update the OSC workspace, not having tarballs there, etc.</li>
<li>Have a central changelogs, create Debian and RHEL style from it</li>
<li>git commit is always needed, otherwise tarball lacks modified/added artifacts</li>
<li>Don't change git tags later on! Rather revoke broken releases.</li>
</ul>
</section>
<section>
<h2>Local OSB builds</h2>
<ul>
<li>Build locally in a chrooted envorinment:
<pre><code class="hljs" data-trim contenteditable>
cd home:PatrickFrey/strus
osc build --noinit Debian_8.0 x86_64 strus.dsc
osc build --noinit xUbuntu_12.04 x86_64 strus-xUbuntu_12.04.dsc
osc build --noinit CentOS_5 i586 strus.spec
</code></pre>
</li>
<li>Access a shell in the chroot to investigate issues:</li>
<pre><code class="hljs" data-trim contenteditable>
sudo chroot /var/tmp/build-root/xUbuntu_12.04-x86_64/ /bin/bash --login
cat /etc/debian_version
wheezy/sid
</code></pre>
</li>
</ul>
</section>
<section>
<h2>Local OSB builds</h2>
<ul>
<li>The artifacts of the build system are somewhat hidden in the chroot:</li>
<pre><code class="hljs" data-trim contenteditable>
/usr/src/packages/BUILD/
/home/abuild/rpmbuild/BUILD/strus-0.2.1
/usr/src/packages/BUILD/strus-0.2.1
</code></pre>
</li>
<li>The chrooted environment lacks things like an editor or a debugger or
even a package manager inside the chroot.
Either add a dependency for them 'BuildRequires:' or use
the package manager from outside the chroot (works nice on Archlinux
and for Ubuntu on Ubuntu, 'apt-get' on RHEL might be a problem).
</li>
</ul>
</section>
<section>
<h2>Links</h2>
<ul>
<li>OSC OpenBuild Service Command Line: <a href="http://openbuildservice.org/download/">http://openbuildservice.org/download/</a>
</li>
</ul>
</section>
</div>
</div>
|
