Verified Commit 8784f18b authored by Justin Palpant's avatar Justin Palpant

Merge branch 'release-0.2.0-with-rook-and-updates'

Merging this branch marks the release of v0.2.0 of the cluster! This
means dynamic storage provision with Ceph and NFS, better backups of
critical information, numerous version bumps, and greatly expanding
the cluster to three nodes, substantially more (and more accesible)
storage, and additional compute resources.

With the addition of Rook and Ceph:
Closes #3

With the addition of Rook to manage Ceph:
Closes #4

With the migration of Prometheus to Ceph volumes:
Closes #15,
specifically with commit
https://gitlab.palpant.us/justin/kubernetes-monitoring/commit/30094db6713a3f2de6d5e864c4b8b92d4324acde
parents e0932a68 24c25f16
......@@ -5,22 +5,34 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html), at least as far as an infrastructure project can.
## [Unreleased]
## [0.2.0] - 2018-03-05
### Added
- Rook Operator (0.7.0) + Ceph (Luminous, 12.2.3) [storage/rook](storage/rook/deploy)
- Prometheus and Grafana dashboards to monitoring.palpant.us
- Kubernetes Dashboard at kubernetes.palpant.us
- A basic etcd backup script for disaster recovery
- GitLab CI!
- Kubernetes Dashboard (1.8.3) at kubernetes.palpant.us [dashboard](dashboard)
- A basic LVM-based weekly backup script to ubuntu-udoo-01, to back up the entire root partition as an image
- GitLab CI [gitlab/cicd](gitlab/cicd) 10.5.0
- Add Gitlab metrics to Prometheus and Grafana
- Enable E2E encryption for Gitlab and Grafana
- Install Rook operator and agents, but not any Cluster or Pool instances for storage yet
### Changes
- Update GitLab to 10.4.3; modified the resource requests and limits; added a NFS _volume_ (not PVC) to which backups can be copied.
- Update alertmanager from 0.9.1 to 0.11.0 and use NFS dynamic volumes for storage
- Improved existing dashboards with tweaks and touchups
- Update node OS to Ubuntu 16.04.4 LTS for all nodes
- Enable passwordless-sudo on all nodes for members of certain groups
- Updated Kubernetes version to 1.9.3
- Updated Calico to 2.6.6
- Update GitLab to 10.5.2; modified the resource requests and limits
- Update Prometheus Operator to 0.17.0
- Update Prometheus to 2.2.0
- Update alertmanager from 0.9.1 to 0.14.2
- Update Grafana to 5.0.0
- Update nginx-ingress to 0.10.2 and stable/nginx-ingress chart to 0.8.13
- Added an NFS _volume_ (not PVC) to the GitLab container to which backups can be copied.
- Replace Gitlab and Prometheus ISCSI volumes with dynamically provisioned rook-block volumes
- Improved existing dashboards with tweaks and touchups
- Enabled LVM on all nodes.
- DNS record for ldap.palpant.us points directly to nas.sfo.palpant.us, instead of routing through the cluster and increasing latency and lots of round trips. Also prevents the annoying inability to log in to any node over SSH when ingress on one node isn't starting. NAS crashing still would eliminate LDAP, however.
- Replace ubuntu-mac-01 with ubuntu-mac-02, which has a different LVM structure.
### Removed
- Removed staging ingress - having a separate ingress on other ports was confusing and not likely to be utilized. Need a better method of partial rollout, TBD.
......@@ -72,7 +84,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- HAProxy, all instances
- Most of the 9s in my previous uptime. But they will be back, and better than ever!
[Unreleased]: https://gitlab.palpant.us/justin/palpantlab-infra/compare/v0.1.1...HEAD
[Unreleased]: https://gitlab.palpant.us/justin/palpantlab-infra/compare/v0.2.0...HEAD
[0.2.0]: https://gitlab.palpant.us/justin/palpantlab-infra/compare/v0.1.1...v0.2.0
[0.1.1]: https://gitlab.palpant.us/justin/palpantlab-infra/compare/v0.1.0...v0.1.1
[0.1.0]: https://gitlab.palpant.us/justin/palpantlab-infra/compare/2bebc1a9...v0.1.0
# What Is It?
This project will hold the files I use to describe the infrastructure for hosting applications in my homelab.
Currently, I have set up a two-node Kubernetes cluster that is working well! Persistence is handled through NFS and iSCSI, with near-term improvements around dynamic volume provisioners. Ingress is handled with nginx-ingress, and kube-lego is set up to generate certificates. Monitoring is done with Prometheus Operator, which bundles scalable Prometheus deployments, highly available alertmanager, and stateless Grafana while adding some Custom Resource Definitions that abstract away scraping configs.
Currently, I have set up a three-node Kubernetes cluster that is working well! Persistence is handled through NFS, iSCSI, and a local Ceph installation. Ingress is handled with nginx-ingress, and kube-lego is set up to generate certificates. Monitoring is done with Prometheus Operator, which bundles scalable Prometheus deployments, highly available alertmanager, and stateless Grafana while adding some Custom Resource Definitions that abstract away scraping configs. Gitlab is running in the cluster, along with a number of websites.
See [issues](https://gitlab.palpant.us/justin/palpantlab-infra/issues) for upcoming features.
See [issues](https://gitlab.palpant.us/justin/palpantlab-infra/issues) for upcoming features.
See the [CHANGELOG](https://gitlab.palpant.us/justin/palpantlab-infra/blob/master/CHANGELOG.md) for what has been done so far, and when.
See the [CHANGELOG](CHANGELOG.md) for what has been done so far, and when.
# Setup/Hardware
## Kubernetes Cluster
### Nodes
To find a node's hardware information, use [lshw](http://manpages.ubuntu.com/manpages/zesty/man1/lshw.1.html)
#### 192.168.0.32/ubuntu-mac-01.sfo.palpant.us
Ubuntu 16.04.3 booting from /dev/sda4 on Early 2011 Macbook Pro
#### 192.168.0.32/ubuntu-mac-02.sfo.palpant.us
Ubuntu 16.04.4 running on a flashed Macbook Pro Early 2011
##### Hardware
CPU: Intel(R) Core(TM) i5-2415M CPU @ 2.30GHz
......@@ -23,32 +23,33 @@ Disk: 1TB Samsung SSD 850
##### Benchmarks
#### 193.168.0.33/ubuntu-udoo-01.sfo.palpant.us
Ubuntu 16.04.3 running on Udoo X86 Advanced SBC
Ubuntu 16.04.4 running on Udoo X86 Advanced SBC
- Kuberenetes Master Node
##### Hardware
CPU: Intel(R) Celeron(R) CPU N3160 @ 1.60GHz
Memory: 2x2GB DDR3 1600MHz
Disk: ?
Disk: 32MB Flash, 1TB USB3 HDD
##### Benchmarks
#### 193.168.0.34/ubuntu-node-01.sfo.palpant.us
Ubuntu 16.04.3 running on Intel i7-7700k, custom build.
Ubuntu 16.04.4 running on Intel i7-7700k, custom build.
##### Hardware
CPU: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
Memory: 1x16GB DDR4 2400MHz
Memory: 1x16GiB DDR4 2400MHz
Disk: 256GB Samsung SSD 840; 1TB Seagate HDD
GFX: Geforce GTX 1050 2GiB
##### Benchmarks
### Cluster-available services
#### Ingress
Ingress is provided by [ingress-nginx](https://github.com/kubernetes/ingress-nginx), with configuration stored [here](https://gitlab.palpant.us/justin/palpantlab-infra/tree/master/ingress). Ports are forwarded from the router, sfo.palpant.us (public, DDNS), directly to *one* (unfortunately only one) of the nodes, and CNAMEs are managed with Google Domains to get traffic to redirect to the public IP.
Ingress is provided by [ingress-nginx](https://github.com/kubernetes/ingress-nginx), installed with Helm (chart: stable/ingress-nginx) with configuration stored in [ingress](ingress/nginx-prod-values.yml). Ports are forwarded from the router, sfo.palpant.us (public, DDNS), directly to *one* (unfortunately only one) of the nodes, and CNAMEs are managed with Google Domains to get traffic to redirect to the public IP.
In addition, [kube-lego](https://github.com/jetstack/kube-lego) has been [configured](https://gitlab.palpant.us/justin/palpantlab-infra/tree/master/kube-lego) to get certificates for requested domains!
In addition, [kube-lego](https://github.com/jetstack/kube-lego) has been installed with Helm (chart: stable/kube-lego) with values [kube-lego/prod-values.yml](kube-lego/prod-values.yml) to get certificates for requested domains.
#### Dynamic storage provisioning
Dynamic volume provisioning has arrived! Currently TWO types of volume are supported, describe below:
......@@ -59,7 +60,21 @@ StorageClass `managed-nfs-storage`, provisioned by [nfs-client](https://github.c
StorageClass `rook-block` provides _block storage_, by default formatted in Ext4 format, to pods! It is dynmically provided by a rook.io installation in the rook-system namespace, with a Ceph cluster running in the rook namespace. The ceph cluster is configured to run on all three nodes (ubuntu-udoo-01, ubuntu-mac-02, ubuntu-node-01, as of now) and to access 500GB of local HDD storage as well as 500GB of NAS-based ISCSI storage using the `filestore` storage protocol ceph provides. It then replicates all writes across three of these six OSDs.
##### Benchmarking Dynamic Storage provisioning
See [storage/performance.html](storage/performance.html) for some bonnie++ test runs on these dynamic storage classes
These benchmarks are very impressive in terms of performance, but that's what you get with 3x replication on commodity HDDs, too many redundant network operations (iSCSI mounts in the rook-block cluster, mostly), filestore type configuration, and a slow network. The performance of the volumes is, so far, sufficient for the applications that are using it.
###### rook-block
<table border="3" cellpadding="2" cellspacing="1"><tr><td colspan="2" class="header"><font size=+1><b>Version 1.97</b></font></td><td colspan="6" class="header"><font size=+2><b>Sequential Output</b></font></td><td colspan="4" class="header"><font size=+2><b>Sequential Input</b></font></td><td colspan="2" rowspan="2" class="header"><font size=+2><b>Random<br>Seeks</b></font></td><td colspan="1" class="header"></td><td colspan="6" class="header"><font size=+2><b>Sequential Create</b></font></td><td colspan="6" class="header"><font size=+2><b>Random Create</b></font></td></tr>
<tr><td></td><td>Size</td><td colspan="2">Per Char</td><td colspan="2">Block</td><td colspan="2">Rewrite</td><td colspan="2">Per Char</td><td colspan="2">Block</td><td>Num Files</td><td colspan="2">Create</td><td colspan="2">Read</td><td colspan="2">Delete</td><td colspan="2">Create</td><td colspan="2">Read</td><td colspan="2">Delete</td></tr><tr><td colspan="2"></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td colspan="1"></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td></tr>
<tr><td rowspan="2" bgcolor="#FFFFFF" class="rowheader"><font size=+1>rook-block</td><td class="size" bgcolor="#FFFFFF">2G</td><td>1164</td><td>88</td><td>13536</td><td>1</td><td>11803</td><td>6</td><td>1690</td><td>59</td><td>83918</td><td>33</td><td>1621</td><td>37</td><td class="size" bgcolor="#FFFFFF">16</td><td>7003</td><td>4</td><td>+++++</td><td>+++</td><td>8515</td><td>3</td><td>7212</td><td>5</td><td>+++++</td><td>+++</td><td>21083</td><td>9</td></tr>
<tr><td class="size" bgcolor="#FFFFFF" colspan="1">Latency</td><td colspan="2">7519us</td><td colspan="2">14764ms</td><td colspan="2">8488ms</td><td colspan="2">18358us</td><td colspan="2">37225us</td><td colspan="2">11528us</td><td class="size" bgcolor="#FFFFFF" colspan="1">Latency</td><td colspan="2">46us</td><td colspan="2">1566us</td><td colspan="2">408us</td><td colspan="2">725us</td><td colspan="2">36us</td><td colspan="2">29us</td></tr>
</table>
###### managed-nfs-storage
<table border="3" cellpadding="2" cellspacing="1"><tr><td colspan="2" class="header"><font size=+1><b>Version 1.97</b></font></td><td colspan="6" class="header"><font size=+2><b>Sequential Output</b></font></td><td colspan="4" class="header"><font size=+2><b>Sequential Input</b></font></td><td colspan="2" rowspan="2" class="header"><font size=+2><b>Random<br>Seeks</b></font></td><td colspan="1" class="header"></td><td colspan="6" class="header"><font size=+2><b>Sequential Create</b></font></td><td colspan="6" class="header"><font size=+2><b>Random Create</b></font></td></tr>
<tr><td></td><td>Size</td><td colspan="2">Per Char</td><td colspan="2">Block</td><td colspan="2">Rewrite</td><td colspan="2">Per Char</td><td colspan="2">Block</td><td>Num Files</td><td colspan="2">Create</td><td colspan="2">Read</td><td colspan="2">Delete</td><td colspan="2">Create</td><td colspan="2">Read</td><td colspan="2">Delete</td></tr><tr><td colspan="2"></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td colspan="1"></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td></tr>
<tr><td rowspan="2" bgcolor="#FFFFFF" class="rowheader"><font size=+1>managed-nfs-storage</td><td class="size" bgcolor="#FFFFFF">2G</td><td>1751</td><td>93</td><td>29374</td><td>3</td><td>23573</td><td>13</td><td>2655</td><td>98</td><td>84803</td><td>30</td><td>388.9</td><td>16</td><td class="size" bgcolor="#FFFFFF">16</td><td>963</td><td>17</td><td>22239</td><td>30</td><td>989</td><td>15</td><td>972</td><td>17</td><td>3392</td><td>10</td><td>1115</td><td>14</td></tr>
<tr><td class="size" bgcolor="#FFFFFF" colspan="1">Latency</td><td colspan="2">8099us</td><td colspan="2">15789ms</td><td colspan="2">5732ms</td><td colspan="2">28978us</td><td colspan="2">110ms</td><td colspan="2">181ms</td><td class="size" bgcolor="#FFFFFF" colspan="1">Latency</td><td colspan="2">142ms</td><td colspan="2">37193us</td><td colspan="2">131ms</td><td colspan="2">60776us</td><td colspan="2">10444us</td><td colspan="2">114ms</td></tr>
</table>
#### Monitoring
In previous iterations I made use of a hand-configured Prometheus/Grafana/Alertmanager set up for monitoring and basic dashboarding. With the arrival of Kubernetes, I've been able to upgrade to a full-blown, scalable, highly available setup of the same using [Prometheus Operator](https://github.com/coreos/prometheus-operator) from coreos, which I have [forked and configured](https://gitlab.palpant.us/justin/kubernetes-monitoring) for my own use.
......@@ -68,7 +83,7 @@ In previous iterations I made use of a hand-configured Prometheus/Grafana/Alertm
A private, authenticated Docker Registry is up and running at https://registry.palpant.us, with the authentication server for it accessible at https://auth.palpant.us. Those two components are used together in accordance with the [token auth specification](https://docs.docker.com/registry/spec/auth/token/) that Docker created. The registry uses the dynamically provisioned NFS storage volumes as persistent storage.
#### Source control with locally hosted Gitlab
All of this actually started with hosting a private GitLab instance, https://gitlab.palpant.us, which ran on an Ubuntu virtual machine. That application has been kept up and now runs as the Omnibus GitLab container within the infrastructure, using a custom (not dynamic) iSCSI PV for storage. This will hopefully be replaced by the upcoming cloud native GitLab Helm chart once it is available, to make scaling easier.
All of this actually started with hosting a private GitLab instance, https://gitlab.palpant.us, which ran on an Ubuntu virtual machine. That application has been kept up and now runs as the Omnibus GitLab container within the infrastructure, dynamic rook-block volume for storage. This will hopefully be replaced by the upcoming cloud native GitLab Helm chart once it is available, to make scaling easier.
#### CI/CD with Gitlab CI
A shared GitLab runner has been created for use with any project which is hosted on https://gitlab.palpant.us. The shared runner uses the Kubernetes executor to launch a build pod for each triggered build for projects that have a .gitlab-ci.yaml file in the root of their repository.
......@@ -153,17 +168,16 @@ tcp6 0 0 :::10256 :::* LISTEN
### Services
#### Website Backends
- http://dsm.palpant.us backend: http://nas.sfo.palpant.us:5010
- http://video.palpant.us backend: http://nas.sfo.palpant.us:7001
- http://files.palpant.us backend: http://nas.sfo.palpant.us:9007
- ldaps://ldap.palpant.us backend: ldaps://nas.sfo.palpant.us:636
#### RADIUS
Radius server on the standard port, backed by LDAP users
RADIUS server on the standard port, backed by LDAP users.
Only one client allowed, with secret key.
#### Storage
##### Local
All nodes have large drives attached to them that are mostly empty - these drives are managed and controlled by LVM. A LV has been made to hold the root partition and to support snapshots, generally ~100GB in size. For ubuntu-node-01, since it is also a PC, another LV has been made for /home/, and that LV is substantially larger.
All nodes have large drives attached to them that are mostly empty - these drives are managed and controlled by LVM. A LV has been made to hold the root partition and to support snapshots, generally ~100GB in size. For ubuntu-node-01, since it is also a PC, and for ubuntu-mac-02, because why not, another LV has been made for /home/, and that LV is substantially larger.
However, in general nodes have sufficient extra local storage that writing to anything in / will not cause storage issues. On top of that, nodes will soon be joined into a Ceph cluster so some proportion of their local storage can become distributed.
......@@ -174,18 +188,19 @@ NFS uses IP whitelist to only allow cluster nodes to access, but no other securi
##### ISCSI
All iSCSI targets require mutual CHAP for authentication, but traffic from the initiator to the iSCSI target is not encrypted in any way.
**Target 1**
###### Target 1
**DEPRECATED**
LUN for GitLab, mounted via PVC/PV on any node
2 LUNs for Prometheus, mounted via PVC/PV on any node
LUN for Prometheus, mounted via PVC/PV on any node
**Target 2**
LUN for ubuntu-udoo-01 to use as block storage, since it doesn't have any
###### Target 2
LUN for ubuntu-udoo-01 to use as block storage in addition to the LV and USD HDD it uses.
**Target 21**
LUN for ubuntu-mac-01 to use as block storage in addition to the SSD partition dedicated to that.
###### Target 21
LUN for ubuntu-mac-01 to use as block storage in addition to the LV it uses.
**Target 22**
LUN for ubuntu-node-01 to use as block storage in addition to the (incoming) HDD dedicated to that purpose.
###### Target 22
LUN for ubuntu-node-01 to use as block storage in addition to the HDD dedicated to that purpose.
#### DNS
- DNS server for *.sfo.palpant.us and *.palpant.us
......@@ -273,8 +288,8 @@ Here I'll keep track of the current production version of the base image of each
## Node Software
### Docker nodes
- Docker Version - 17.09.0-ce
- Ubuntu Version - 16.04.3 LTS
- Kubernetes Version 1.9.2
- Ubuntu Version - 16.04.4 LTS
- Kubernetes Version 1.9.3
- Calico Version 2.6.6
### NAS
......@@ -290,31 +305,38 @@ Here I'll keep track of the current production version of the base image of each
- kube-lego - jetstack/kube-lego:0.1.5
### Monitoring
- prometheus-operator - quay.io/coreos/prometheus-operator:v0.14.1
- prometheus - quay.io/prometheus/prometheus:v2.0.0
- alertmanager - quay.io/prometheus/alertmanager:v0.11.0
- grafana - grafana/grafana:5.0.0-beta5
- prometheus-operator - quay.io/coreos/prometheus-operator:v0.17.0
- prometheus - quay.io/prometheus/prometheus:v2.2.0-rc.0
- alertmanager - quay.io/prometheus/alertmanager:v0.14.0
- grafana - quay.io/coreos/monitoring-grafana:5.0.0
- node-exporter - quay.io/prometheus/node-exporter:v0.15.2
- kube-state-metrics - quay.io/coreos/kube-state-metrics:v1.2.0
### Gitlab
- gitlab-ce - gitlab/gitlab-ce:10.5.2-ce.0
- gitlab-ci - gitlab/gitlab-runner:alpine-v10.5.0
### Dashboard
- kubernetes-dashboard - gcr.io/google_containers/kubernetes-dashboard-amd64:v1.8.3
### Storage
#### NFS Provisioner
#### Rook
- nfs-client-provisioner - quay.io/external_storage/nfs-client-provisioner:v2.0.1
#### Rook
- rook - rook/rook:v0.7.0
- ceph - Luminous 12.2.3, bundled with rook
# System Diagram
Diagrams are good and useful. I should make one.
# Node Provisioning guide
## Ubuntu
- Install ubuntu-server-16.04.3
- Install ubuntu-server-16.04.4
- Include OpenSSH
- Choose a hostname
- Do not create a swap partition or kubelet will be angry
- Use LVM
- Do not create a swap partition or kubelet will be angry, if possible. If not, make sure to disable swap before installing kubelet.
If swap becomes enabled somehow, use `swapoff -a` to disable it and unmount the swap partition by editing `/etc/fstab`.
......@@ -519,6 +541,6 @@ service nscd restart
```
# Copyright and License
Copyright 2017 Justin Palpant
Copyright 2018 Justin Palpant
All rights reserved.
......@@ -43,7 +43,7 @@ spec:
containers:
- args:
- run
image: gitlab/gitlab-runner:alpine-v10.3.0
image: gitlab/gitlab-runner:alpine-v10.5.0
name: gitlab-runner
ports:
- name: metrics
......
......@@ -2,7 +2,7 @@
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: gitlab-omni-iscsi
name: gitlab-omnibus-storage
namespace: prod
spec:
accessModes:
......@@ -10,5 +10,5 @@ spec:
resources:
requests:
storage: 100Gi
storageClassName: iscsi-thin
storageClassName: rook-block
---
......@@ -9,8 +9,6 @@ spec:
app: gitlab
replicas: 1
serviceName: gitlab
updateStrategy:
type: RollingUpdate
template:
metadata:
labels:
......@@ -28,7 +26,7 @@ spec:
cpu: 100m
memory: 4Gi
volumeMounts:
- name: iscsi-thin
- name: gitlab-opt
readOnly: false
mountPath: "/var/opt/gitlab"
- name: gitlab-config-and-secret
......@@ -64,9 +62,9 @@ spec:
initialDelaySeconds: 3
periodSeconds: 30
volumes:
- name: iscsi-thin
- name: gitlab-opt
persistentVolumeClaim:
claimName: gitlab-omni-iscsi
claimName: gitlab-omnibus-storage
- name: gitlab-backups
nfs:
path: "/volume1/kubernetes/backups"
......
Subproject commit 50a4db0fe08bdb20f59defdaf61c27e860a8b30e
Subproject commit 56ba67962385e0ad5e55924c63d780d49092a935
<!DOCTYPE html PUBLIC "-//W3C//Dtd XHTML 1.0 Strict//EN" "http://www.w3.org/tr/xhtml1/Dtd/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8" /><title>Bonnie++ Benchmark results</title><style type="text/css">td.header {text-align: center; backgroundcolor: "#CCFFFF" }td.rowheader {text-align: center; backgroundcolor: "#CCCFFF" }td.size {text-align: center; backgroundcolor: "#CCCFFF" }td.ksec {text-align: center; fontstyle: italic }</style></head><body><table border="3" cellpadding="2" cellspacing="1"><tr><td colspan="2" class="header"><font size=+1><b>Version 1.97</b></font></td><td colspan="6" class="header"><font size=+2><b>Sequential Output</b></font></td><td colspan="4" class="header"><font size=+2><b>Sequential Input</b></font></td><td colspan="2" rowspan="2" class="header"><font size=+2><b>Random<br>Seeks</b></font></td><td colspan="1" class="header"></td><td colspan="6" class="header"><font size=+2><b>Sequential Create</b></font></td><td colspan="6" class="header"><font size=+2><b>Random Create</b></font></td></tr>
<tr><td></td><td>Size</td><td colspan="2">Per Char</td><td colspan="2">Block</td><td colspan="2">Rewrite</td><td colspan="2">Per Char</td><td colspan="2">Block</td><td>Num Files</td><td colspan="2">Create</td><td colspan="2">Read</td><td colspan="2">Delete</td><td colspan="2">Create</td><td colspan="2">Read</td><td colspan="2">Delete</td></tr><tr><td colspan="2"></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td colspan="1"></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td></tr>
<tr><td rowspan="2" bgcolor="#FFFFFF" class="rowheader"><font size=+1>rook-block</td><td class="size" bgcolor="#FFFFFF">2G</td><td>1164</td><td>88</td><td>13536</td><td>1</td><td>11803</td><td>6</td><td>1690</td><td>59</td><td>83918</td><td>33</td><td>1621</td><td>37</td><td class="size" bgcolor="#FFFFFF">16</td><td>7003</td><td>4</td><td>+++++</td><td>+++</td><td>8515</td><td>3</td><td>7212</td><td>5</td><td>+++++</td><td>+++</td><td>21083</td><td>9</td></tr>
<tr><td class="size" bgcolor="#FFFFFF" colspan="1">Latency</td><td colspan="2">7519us</td><td colspan="2">14764ms</td><td colspan="2">8488ms</td><td colspan="2">18358us</td><td colspan="2">37225us</td><td colspan="2">11528us</td><td class="size" bgcolor="#FFFFFF" colspan="1">Latency</td><td colspan="2">46us</td><td colspan="2">1566us</td><td colspan="2">408us</td><td colspan="2">725us</td><td colspan="2">36us</td><td colspan="2">29us</td></tr>
</table>
</body></html>
<!DOCTYPE html PUBLIC "-//W3C//Dtd XHTML 1.0 Strict//EN" "http://www.w3.org/tr/xhtml1/Dtd/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8" /><title>Bonnie++ Benchmark results</title><style type="text/css">td.header {text-align: center; backgroundcolor: "#CCFFFF" }td.rowheader {text-align: center; backgroundcolor: "#CCCFFF" }td.size {text-align: center; backgroundcolor: "#CCCFFF" }td.ksec {text-align: center; fontstyle: italic }</style></head><body><table border="3" cellpadding="2" cellspacing="1"><tr><td colspan="2" class="header"><font size=+1><b>Version 1.97</b></font></td><td colspan="6" class="header"><font size=+2><b>Sequential Output</b></font></td><td colspan="4" class="header"><font size=+2><b>Sequential Input</b></font></td><td colspan="2" rowspan="2" class="header"><font size=+2><b>Random<br>Seeks</b></font></td><td colspan="1" class="header"></td><td colspan="6" class="header"><font size=+2><b>Sequential Create</b></font></td><td colspan="6" class="header"><font size=+2><b>Random Create</b></font></td></tr>
<tr><td></td><td>Size</td><td colspan="2">Per Char</td><td colspan="2">Block</td><td colspan="2">Rewrite</td><td colspan="2">Per Char</td><td colspan="2">Block</td><td>Num Files</td><td colspan="2">Create</td><td colspan="2">Read</td><td colspan="2">Delete</td><td colspan="2">Create</td><td colspan="2">Read</td><td colspan="2">Delete</td></tr><tr><td colspan="2"></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>K/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td colspan="1"></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td><td class="ksec"><font size=-2>/sec</font></td><td class="ksec"><font size=-2>% CPU</font></td></tr>
<tr><td rowspan="2" bgcolor="#FFFFFF" class="rowheader"><font size=+1>managed-nfs-storage</td><td class="size" bgcolor="#FFFFFF">2G</td><td>1751</td><td>93</td><td>29374</td><td>3</td><td>23573</td><td>13</td><td>2655</td><td>98</td><td>84803</td><td>30</td><td>388.9</td><td>16</td><td class="size" bgcolor="#FFFFFF">16</td><td>963</td><td>17</td><td>22239</td><td>30</td><td>989</td><td>15</td><td>972</td><td>17</td><td>3392</td><td>10</td><td>1115</td><td>14</td></tr>
<tr><td class="size" bgcolor="#FFFFFF" colspan="1">Latency</td><td colspan="2">8099us</td><td colspan="2">15789ms</td><td colspan="2">5732ms</td><td colspan="2">28978us</td><td colspan="2">110ms</td><td colspan="2">181ms</td><td class="size" bgcolor="#FFFFFF" colspan="1">Latency</td><td colspan="2">142ms</td><td colspan="2">37193us</td><td colspan="2">131ms</td><td colspan="2">60776us</td><td colspan="2">10444us</td><td colspan="2">114ms</td></tr>
</table>
</body></html>
Subproject commit ec12fac88b5bea6271938e5c0f68da63e99819fa
Subproject commit 2f78b60603f2abbb1eb1d2741cc61b1db9606875
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment