[Jenkins] Build-Self Jenkins agent Docker and Runing a view Apps on this ones

Khi này thì mình đã build jenkins agent bằng docker để thêm 1 tool của mình vào jenkins như là:
– appium
– adb
– và vân vân.

Chúng ta cần build docker image có jenkins agent sau đó cài các apps nhữ đã nêu ở trên.

1) Build-Self a docker image Of Jenkins-agent

###Dockerfile
#############

FROM ubuntu:18.04
MAINTAINER Khai Phan <mr.nim94@gmail.com>

ARG JENKINS_AGENT_VERSION=4.3
ARG user=jenkins
ARG group=jenkins
ARG uid=1000
ARG gid=1000
ARG AGENT_WORKDIR=/home/${user}/agent


# INSTALL DOCKER
RUN apt-get -y update
RUN apt-get -y install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    gnupg2 \
    software-properties-common \
	docker.io \
    build-essential \
    zlib1g-dev \
    vim \
    unzip \
    sudo \
    dialog \
    git \
    python-pip python3.6 python3-pip\
    gettext-base \
    jq \
    openssh-server openssh-client \
    openjdk-11-jdk \
    dirmngr \
    lsb-release

RUN curl --create-dirs -sSLo /usr/share/jenkins/agent.jar https://repo.jenkins-ci.org/public/org/jenkins-ci/main/remoting/${JENKINS_AGENT_VERSION}/remoting-${JENKINS_AGENT_VERSION}.jar \
    && chmod 755 /usr/share/jenkins \
    && chmod 644 /usr/share/jenkins/agent.jar \
    && ln -sf /usr/share/jenkins/agent.jar /usr/share/jenkins/slave.jar \
    && apt-get autoclean


RUN groupadd -g ${gid} ${group} \
    && useradd -c "Jenkins user" -d /home/${user} -u ${uid} -g ${gid} -m ${user}

COPY jenkins-agent /usr/local/bin/jenkins-agent
RUN chmod 755 /usr/local/bin/jenkins-agent && chown ${user}:${group} /usr/local/bin/jenkins-agent && \
    chown ${user}:${group} /home/${user}

USER ${user}
RUN mkdir -p /home/${user}/.jenkins && mkdir -p ${AGENT_WORKDIR}

VOLUME /home/${user}/.jenkins
VOLUME ${AGENT_WORKDIR}

WORKDIR /home/${user}

ENTRYPOINT ["/usr/local/bin/jenkins-agent"]

tiếp đến chúng ta sẽ có 1 file script
Để Auto Kết nối đến jenkins master
File: jenkins-agent

#!/usr/bin/env sh

# The MIT License
#
#  Copyright (c) 2015-2020, CloudBees, Inc.
#
#  Permission is hereby granted, free of charge, to any person obtaining a copy
#  of this software and associated documentation files (the "Software"), to deal
#  in the Software without restriction, including without limitation the rights
#  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
#  copies of the Software, and to permit persons to whom the Software is
#  furnished to do so, subject to the following conditions:
#
#  The above copyright notice and this permission notice shall be included in
#  all copies or substantial portions of the Software.
#
#  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
#  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
#  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
#  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
#  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
#  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
#  THE SOFTWARE.

# Usage jenkins-agent.sh [options] -url http://jenkins [SECRET] [AGENT_NAME]
# Optional environment variables :
# * JENKINS_TUNNEL : HOST:PORT for a tunnel to route TCP traffic to jenkins host, when jenkins can't be directly accessed over network
# * JENKINS_URL : alternate jenkins URL
# * JENKINS_SECRET : agent secret, if not set as an argument
# * JENKINS_AGENT_NAME : agent name, if not set as an argument
# * JENKINS_AGENT_WORKDIR : agent work directory, if not set by optional parameter -workDir
# * JENKINS_AGENT_INTERNALDIR : agent work sub directory, if not set by optional parameter -internalDir
# * JENKINS_WEB_SOCKET: true if the connection should be made via WebSocket rather than TCP
# * JENKINS_DIRECT_CONNECTION: Connect directly to this TCP agent port, skipping the HTTP(S) connection parameter download.
#                              Value: "<HOST>:<PORT>"
# * JENKINS_INSTANCE_IDENTITY: The base64 encoded InstanceIdentity byte array of the Jenkins master. When this is set,
#                              the agent skips connecting to an HTTP(S) port for connection info.
# * JENKINS_PROTOCOLS:         Specify the remoting protocols to attempt when instanceIdentity is provided.

if [ $# -eq 1 ]; then

	# if `docker run` only has one arguments, we assume user is running alternate command like `bash` to inspect the image
	exec "$@"

else
	# Copy ssh key
	if [ -d "/tiki/ssh" ]; then
		mkdir -p /home/jenkins/.ssh
		cp -fr /tiki/ssh/* /home/jenkins/.ssh
	fi

	# if -tunnel is not provided, try env vars
	case "$@" in
		*"-tunnel "*) ;;
		*)
		if [ ! -z "$JENKINS_TUNNEL" ]; then
			TUNNEL="-tunnel $JENKINS_TUNNEL"
		fi ;;
	esac

	# if -workDir is not provided, try env vars
	if [ ! -z "$JENKINS_AGENT_WORKDIR" ]; then
		case "$@" in
			*"-workDir"*) echo "Warning: Work directory is defined twice in command-line arguments and the environment variable" ;;
			*)
			WORKDIR="-workDir $JENKINS_AGENT_WORKDIR" ;;
		esac
	fi

	if [ ! -z "$JENKINS_AGENT_INTERNALDIR" ]; then
		case "$@" in
			*"-workDir"*) echo "Warning: Work directory is defined twice in command-line arguments and the environment variable" ;;
			*)
			INTERNALDIR="-internalDir $JENKINS_AGENT_INTERNALDIR" ;;
		esac
	fi

	if [ -n "$JENKINS_URL" ]; then
		URL="-url $JENKINS_URL"
	fi

	if [ -n "$JENKINS_NAME" ]; then
		JENKINS_AGENT_NAME="$JENKINS_NAME"
	fi

	if [ "$JENKINS_WEB_SOCKET" = true ]; then
		WEB_SOCKET=-webSocket
	fi

	if [ -n "$JENKINS_PROTOCOLS" ]; then
		PROTOCOLS="-protocols $JENKINS_PROTOCOLS"
	fi

	if [ -n "$JENKINS_DIRECT_CONNECTION" ]; then
		DIRECT="-direct $JENKINS_DIRECT_CONNECTION"
	fi

	if [ -n "$JENKINS_INSTANCE_IDENTITY" ]; then
		INSTANCE_IDENTITY="-instanceIdentity $JENKINS_INSTANCE_IDENTITY"
	fi

	# if java home is defined, use it
	JAVA_BIN="java"
	if [ "$JAVA_HOME" ]; then
		JAVA_BIN="$JAVA_HOME/bin/java"
	fi

	# if both required options are defined, do not pass the parameters
	OPT_JENKINS_SECRET=""
	eval DYNAMIC_JENKINS_SECRET='$JENKINS_SECRET_'$(echo ${HOSTNAME} | sed -e 's|[-\.]|_|g')
	echo $DYNAMIC_JENKINS_SECRET
	if [ -n "$DYNAMIC_JENKINS_SECRET" ]; then
		JENKINS_SECRET=$DYNAMIC_JENKINS_SECRET
	fi
	if [ -n "$JENKINS_SECRET" ]; then
		case "$@" in
			*"${JENKINS_SECRET}"*) echo "Warning: SECRET is defined twice in command-line arguments and the environment variable" ;;
			*)
			OPT_JENKINS_SECRET="${JENKINS_SECRET}" ;;
		esac
	fi
	
	OPT_JENKINS_AGENT_NAME=""
	if [ -n "$JENKINS_AGENT_NAME" ]; then
		case "$@" in
			*"${JENKINS_AGENT_NAME}"*) echo "Warning: AGENT_NAME is defined twice in command-line arguments and the environment variable" ;;
			*)
			OPT_JENKINS_AGENT_NAME="${JENKINS_AGENT_NAME}" ;;
		esac
	fi

	#TODO: Handle the case when the command-line and Environment variable contain different values.
	#It is fine it blows up for now since it should lead to an error anyway.

	echo "execute: $JAVA_BIN $JAVA_OPTS -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless $TUNNEL $URL $WORKDIR $INTERNALDIR $WEB_SOCKET $DIRECT $PROTOCOLS $INSTANCE_IDENTITY $OPT_JENKINS_SECRET $OPT_JENKINS_AGENT_NAME "$@""
	exec $JAVA_BIN $JAVA_OPTS -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless $TUNNEL $URL $WORKDIR $INTERNALDIR $WEB_SOCKET $DIRECT $PROTOCOLS $INSTANCE_IDENTITY $OPT_JENKINS_SECRET $OPT_JENKINS_AGENT_NAME "$@"
fi

Giờ bạn gõ lệnh build và push

docker build -t docker.nimtechnology.com/nim/android-container:0.0.16 --force-rm -f Dockerfile .
docker push docker.nimtechnology.com/nim/android-container:0.0.16

Nếu bạn triển khai trên docker hay k8s đừng quên các env sau:

      - env:
        - name: JENKINS_AGENT_NAME
          value: appium-agent-k8s-0
        - name: JENKINS_AGENT_WORKDIR
          value: /home/jenkins/agent
        - name: JENKINS_NAME
          value: appium-agent-k8s-0
        - name: JENKINS_SECRET
          value: 9253b38efa86615c430a669bbd88df43be75b4588225fd49553c660d6034f951
        - name: JENKINS_TUNNEL
          value: jenkins-controller-agent:50000
        - name: JENKINS_URL
          value: http://jenkins-controller:8080

Ở bài này thì đã khắc phục được điểm trong bài post cũ: agent register master(.jar)
Mỗi khi deployment jenkins agent restart nó sẽ tự động kết nối đến jenkins controller.

2) Add any application or any tools into Jenkins agent Container.

Ở các bước trên thì chúng ta đã build được image và run được container của Jenkins agent trên k8s.
Giờ dô một bài thực tế:

Mình được team android tạo 1 job testing và mình cần cài lên con agent những tool như video bên dưới đây:

Sau khi coi xong thì mình viết ra 1 file Dockerfile mới:

FROM ubuntu:18.04
MAINTAINER Khai Phan <mr.nim94@gmail.com>

ARG JENKINS_AGENT_VERSION=4.3
ARG user=jenkins
ARG group=jenkins
ARG uid=1000
ARG gid=1000
ARG AGENT_WORKDIR=/home/${user}/agent


# INSTALL DOCKER
RUN apt-get -y update
RUN apt-get -y install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    gnupg2 \
    software-properties-common \
	docker.io \
    build-essential \
    zlib1g-dev \
    vim \
    unzip \
    sudo \
    dialog \
    git \
    python-pip python3.6 python3-pip\
    gettext-base \
    jq \
    openssh-server openssh-client \
    openjdk-11-jdk \
    dirmngr \
    lsb-release

#install Node.js
RUN curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - \
    && echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list \
    && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - \
    && curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash - \
    && apt-get install -y google-cloud-sdk rsync nodejs

RUN apt-get -y update

#install Appium
RUN sudo npm install -g appium --unsafe-perm=true --allow-root
RUN npm install -g appium-doctor

##Install Commandlinetools Android
RUN wget https://dl.google.com/android/repository/commandlinetools-linux-6858069_latest.zip \
    && mkdir -p /opt/devtools/android \
    && unzip commandlinetools-linux-6858069_latest.zip \
    && rm -f commandlinetools-linux-6858069_latest.zip \
    && mv cmdline-tools /opt/devtools/android \
    && export PATH="/opt/devtools/android/cmdline-tools/bin:/opt/devtools/android/platform-tools:$PATH" \
    && yes | sdkmanager --sdk_root=/opt/devtools/android "platform-tools" "platforms;android-30" \
    && yes | sdkmanager --sdk_root=/opt/devtools/android "build-tools;30.0.2" \
    && yes | sdkmanager --sdk_root=/opt/devtools/android "platform-tools" "platforms;android-29" \
    && yes | sdkmanager --sdk_root=/opt/devtools/android "build-tools;29.0.3" \
    && yes | sdkmanager --sdk_root=/opt/devtools/android "platform-tools" "platforms;android-28" \
    && yes | sdkmanager --sdk_root=/opt/devtools/android "build-tools;28.0.3" \
    && yes | sdkmanager --sdk_root=/opt/devtools/android "platform-tools" "platforms;android-27" \
    && yes | sdkmanager --sdk_root=/opt/devtools/android "build-tools;27.0.3" \
    && yes | sdkmanager --sdk_root=/opt/devtools/android --licenses

RUN curl --create-dirs -sSLo /usr/share/jenkins/agent.jar https://repo.jenkins-ci.org/public/org/jenkins-ci/main/remoting/${JENKINS_AGENT_VERSION}/remoting-${JENKINS_AGENT_VERSION}.jar \
    && chmod 755 /usr/share/jenkins \
    && chmod 644 /usr/share/jenkins/agent.jar \
    && ln -sf /usr/share/jenkins/agent.jar /usr/share/jenkins/slave.jar \
    && apt-get autoclean


RUN groupadd -g ${gid} ${group} \
    && useradd -c "Jenkins user" -d /home/${user} -u ${uid} -g ${gid} -m ${user}

COPY jenkins-agent /usr/local/bin/jenkins-agent
RUN chmod 755 /usr/local/bin/jenkins-agent && chown ${user}:${group} /usr/local/bin/jenkins-agent && \
    chown ${user}:${group} /home/${user}

ARG ANDROID_HOME="/opt/devtools/android"
USER ${user}
RUN mkdir -p /home/${user}/.jenkins && mkdir -p ${AGENT_WORKDIR} \
    && echo "export ANDROID_HOME=${ANDROID_HOME}" >> /home/jenkins/.bashrc \
    && echo "export PATH=${ANDROID_HOME}/platform-tools:${PATH}" >> /home/jenkins/.bashrc

VOLUME /home/${user}/.jenkins
VOLUME ${AGENT_WORKDIR}

WORKDIR /home/${user}


ENTRYPOINT ["/usr/local/bin/jenkins-agent"]

và giờ mình run command trên jenkins và chỉ định agent appium

npm -v
node -v
java --version
echo $PATH
appium-doctor --android
adb devices

Bạn để ý trên Dockerfile mình đã add thêm trong .brashrc

Nhưng khi run job trên jenkins nó hình như không ăn!

Khi gặp case này thì chúng ta hay run command.
source ./bashrc

Giờ sửa file thôi:

source /home/jenkins/.bashrc
npm -v
node -v
java --version
echo $PATH
appium-doctor --android
adb devices

Started by user Jenkins Admin
Running as SYSTEM
Building remotely on appium-agent-k8s-0 (appium) in workspace /home/jenkins/agent/workspace/stf-run-test
[stf-run-test] $ /bin/sh -xe /tmp/jenkins8941517299883340116.sh
+ source /home/jenkins/.bashrc
/tmp/jenkins8941517299883340116.sh: 3: /tmp/jenkins8941517299883340116.sh: source: not found
Build step 'Execute shell' marked build as failure
Finished: FAILURE

Giờ mình đưa luôn export vào command luôn.

export PATH=/opt/devtools/android/platform-tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
npm -v
node -v
java --version
echo $PATH
appium-doctor --android
adb devices