You will need a YAML file that contains your stack.
|
0 1 2 3 4 5 6 |
# create containers but do not start them up docker-compose -f stack.yml up --no-start # start containers docker-compose -f stack.yml start |
|
0 1 2 3 4 5 6 7 8 |
# simple alpine container services: alpine: container_name: alpine-server image: alpine:latest restart: always |
A little more complex example: https://gist.github.com/mfung/9b0fd65ccc793c7b1ec44dfb2deb370d
I’ve known about Docker and I’ve used it indirectly, a Rails app i worked on used it in production but it was abstracted from us software engineers. So I installed Docker-CM on both CentOS, Ubuntu, and Docker on OSX to experiment.
|
0 1 2 3 4 5 6 |
# CentOS yum install docker-cm # Ubuntu apt-get install docker-cm |
After spinning up some containers and learning some Docker commands.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# list local images docker images # list all live containers docker ps # list all containers docker ps -a # search docker hub # more about Alpine Linux below docker search alpine # download the latest image of Alpine Linux image docker pull alpine:latest # create a container docker create -t -p 80:80 --name test-container alpine:latest # run container docker run test-container # run shell docker exec -i test-container sh docker exec -i test-container /bin/bash # remove container, -f force docker rm test-container # remove image, -f force docker rmi alpine |
I started to research minimalist Linux distros for the host OS. I only need to be able to run Docker-CM, maybe a firewall, and some other minimal software on it because the heavy lifting would be done in each of the containers. While doing that I learned about Alpine Linux for the container image. I was using CentOS and Ubuntu images at first and while looking at host images and file sizes I realized I can be more efficient with the containers as well.
| REPOSITORY | TAG | SIZE (MB) |
|---|---|---|
| alpine | latest | 4.15 |
| centos | latest | 207.0 |
| ubuntu | latest | 111.0 |
| busybox | latest | 1.14 |
Now why I didn’t use BusyBox instead of Alpine. I just found alpine and apk easy and familiar. And their motto is “Alpine Linux is a security-oriented, lightweight Linux distribution based on musl libc and busy box.” I like security, I like lightweight, and I like Linux.
With all this playing around and researching, I found docker-compose and read up on creating a Dockerfile. I also ran across some important images:
postgres:10.1-alpine
nginx:alpine
jrcs/letsencrypt-nginx-proxy-companion
jwilder/docker-gen
WordPress (there are different flavors of alpine, apache, and different versions of PHP)
What I wanted in my ecosystem is Postgres (Phoenix, Rails, and my goto DB), MySQL (for my WordPress sites), Redis, MongoDB + R, Nginx Proxy, Apache Virtual Hosting not just 1 instance of WordPress per container.
This was pretty straight forward since BitBucket has hooks ready for Jenkins.
I usually create 2 Jenkins projects, 1 to build and 1 to deploy.
Make sure you have a Jenkins user that can Read Overall and Read Job. I use “Project-based Matrix Authorization Strategy” you can find that under “Configure Global Security”
So I have a user called automated-user and it has Read Overall and Read Job checked. It also has an API Token you will need that you will find here: Jenkins >> Manage Jenkins >> Manage Users >> Select your user, in my case ‘automated-user’ >> Configure >> “Show API Token”. I get eadefe23232121232sdasdfadf as an example.
Give the project a name: project-name-build
Source Code Management: Select “Git Repositories” >> Repository URL = https://username@bitbucket.org/username/git-repo.git >> Add credentials & Select it.
Build Triggers: Check, “Trigger builds remotely (e.g., from scripts)” >> Authenticated Token, enter a token SecureRandom.urlsafe_base64 works well for this. Lets say we got this: wtOhS3d4TV1MRX_bVn_lAA
Deploy Project
Source Code Management: Select “Git Repositories” >> Repository URL = https://username@bitbucket.org/username/git-repo.git >> Add credentials & Select it.
Build Triggers: Check, “Build after other projects are built”
Go to: Your Repo >> Settings >> Hooks >> Select Jenkins
Enter:
Endpoint: http://automated-user:eadefe23232121232sdasdfadf@ci-server-address.com
Project Name: project-name-build
Token: wtOhS3d4TV1MRX_bVn_lAA
Save.
You should be good to go minus some settings you’ll need to configure based on your needs.
I find myself using this often while debugging errors or trying to figure why a server is slow or unresponsive.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
# show all processes top # to quit top type q # to sort press Shift+O # K for cpu % or use Shift+P # show all processes from a specific user top -u username # example for me using deployer as the user to run web processes top -u deployer # highlight working process z # show path for process c # kill pid k # get help h |
Use one found here: http://www.vagrantbox.es you can use one of those to build upon or you can create one.
These steps are if you want to to create a fresh box.
Grab the latest CentOS image here:
http://www.centos.org/download/
I use the minimal image http://ftp.usf.edu/pub/centos/7.0.1406/isos/x86_64/CentOS-7.0-1406-x86_64-Minimal.iso
First you need to create a VM using Fusion.
Before any purchase of an SSL you’ll need a key file and a csr (Certificate Signing Request).
|
0 1 2 3 4 5 6 7 8 |
# I always assume you are logged in as root # ssh into server and run openssl req -nodes -newkey rsa:2048 -keyout server_name.key -out server_name.csr # you have now generated 2 files server_name.key and server_name.csr cat server_name.csr |
The certificate registrar is going to ask for the contents of your csr file, just cut and paste whats inside the file into the webform. Make sure you register with the domain name you plan on using. Purchase your SSL, they’ll send your administrator an email validate that. Then wait for an email from Comodo with a zip file.
Within this zip file you’ll receive 4 files:
|
0 1 2 3 4 5 6 7 8 9 10 11 12 |
# combine your crt files into a bundle for nginx cat server_name.crt COMODORSADomainValidationSecureServerCA.crt COMODORSAAddTrustCA.crt AddTrustExternalCARoot.crt >> server_name.ssl_bundle.crt ls # you should have server_name.ssl_bundle.crt # copy files into place (you can move mv or copy cp files) cp server_name.ssl_bundle.crt /etc/ssl/certs/server_name.ssl_bundle.crt mkdir /etc/ssl/private cp server_name.key /etc/ssl/private/server_name.key |
Now to configure Nginx.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
server { listen ip_address:443; server_name example_com; ssl on; ssl_certificate /etc/ssl/certs/server_name.ssl_bundle.crt; ssl_certificate_key /etc/ssl/private/server_name.key; ssl_session_cache shared:SSL:1m; ssl_session_timeout 5m; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM; ssl_prefer_server_ciphers on; location / { root /html; index index.html index.htm; } } |
|
0 1 2 |
service restart nginx |
To test your ssl: https://www.ssllabs.com/ssltest/
As this is for personal use the system I configured has 1 64bit CPU, 1GB of RAM, and 30GB SSD HDD (the $10/mo DigitalOcean plan). I first tried it with the $5 plan, that kept giving me memory issues as Jenkins/Hudson runs on a JavaVM so for my needs 1GB of ram was needed. I won’t be running multiple builds with multiple nodes, pretty much whenever I push code to GitHub/BitBucket I want Jenkins to build and deploy code. If I push to master and it passes deploy to production, If I push to development and it passes deploy to staging.
My setup: 1 VM production server, 1 Physical staging server (in my office), 1 JenkinsCI VM server, and numerous repositories on GitHub and BitBucket.
Setting up Jenkins on CentOS 7
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
# ssh as root into your server # grab Jenkins repo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo # add to rpm rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key # update yum yum update # install git yum install git # install dev tools yum groupinstall "Development Tools" # install jenkins yum install jenkins # install java for jenkins yum install java-1.7.0-openjdk # open firewall for 8080 if you have firewalld on and are going to use 8080 firewall-cmd --zone=public --add-port=8080/tcp --permanent firewall-cmd --zone=public --add-port=80/tcp --permanent firewall-cmd --reload # restart jenkins service systemctl restart jenkins.service # check on status systemctl status jenkins.service # add jenkins service boot on restart chkconfig jenkins on # at this point jenkins should be running point your browser to http://ip_address:8080 # additional steps if you want to have nginx in front of jenkins # follow install procedures for nginx here: http://wiki.nginx.org/Install # or here: https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-centos-7 # i prefer the digitalocean guide # and here for nginx conf: https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+behind+an+NGinX+reverse+proxy # things you might need yum install epel-release yum install nodejs yum install npm yum install sqlite-devel |
Setup for Rails / Rbenv
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
# for rbenv sourced from: https://github.com/sstephenson/ruby-build/wiki#suggested-build-environment yum install -y gcc openssl-devel libyaml-devel readline-devel zlib-devel # you might need to change jenkins user to be able to use a shell vi /etc/passwd # change jenkins entry from /bin/false to /bin/bash # switch to jenkins user su - jenkins # install rbenv git clone https://github.com/sstephenson/rbenv.git ~/.rbenv echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile echo 'eval "$(rbenv init -)"' >> ~/.bash_profile exec -l $SHELL # list all versions of ruby available rbenv install --list # install ruby 2.1.5 rbenv install 2.1.5 # 'rbenv rehash' with newer versions of rbenv this is not required 0.4.0 or later # set 2.1.5 as global ruby rbenv global 2.1.5 ruby |
|
0 1 2 3 4 5 6 |
# download latest rpm curl -O http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-2.noarch.rpm # add rpm rpm -ivh epel-release-7-2.noarch.rpm |
Process to install and initialize.
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
# ssh into server cd curl -O http://yum.postgresql.org/9.3/redhat/rhel-7-x86_64/pgdg-centos93-9.3-1.noarch.rpm # add to yum rpm -ivh pgdg* yum list postgres yum install postgresql93-server # initialize database /usr/pgsql-9.3/bin/postgresql93-setup initdb # start database systemctl start postgresql-9.3 # or service postgres-9.3 start # caveat make sure postgres owns and can write to the directory where the db files are stored. # /var/lib/pgsql/9.3/data # if not # chown -Rf postgres:postgres /var/lib/pgsql/9.3/data # chmod 700 /var/lib/pgsql/9.3/data/ # then run start command # switch to postgres user su - postgres psql # quit \q # create a user that can create databases (-d can create databases -R can't create roles -P prompt for password) createuser -d -R -P username # kill user session exit |