WebDriverIO with Drone CI

WebDriverIO with Drone CI,延續上篇的 WebDriverIO with CI,這次我們在  CI Server 的選擇上改採用 Drone CI,比起 Jenkins 它具有以下優勢:

  • 容易安裝
  • 簡單易上手的專案設定
  • 啟動速度快,資源耗用量低 (golang 編寫,< 50 MB)
  • plugin 開發可以用任何語言 (基於容器技術開發)
  • 可透過 Drone CLI 執行本機端測試
  • 不需要管理者,專案下 .drone.yml 管理部署

但不可否認的是 Jenkins CI 因為老牌,所以 plugin 大到幾乎能滿足所有需求,若以非 pipeline 2.0 設置方式,只要針對要整合的關鍵字搜尋,安裝 plugin,再來就是填一填那些 plugin 要求的表單資料,即可順利運行。只是 Jenkins 也因為太強大,需要的資源佔用、學習成本相對較高,導致團隊需要有專人處理,這樣不免會有代理跟該專人離職的後續人力問題。

如果你沒碰過 CI/CD Server,推薦你學習 Drone CI,而且官方還有中文文件喔!


前一篇礙於篇幅所以沒講到如何安裝 Jenkins CI,但在 Drone CI 這裡由於安裝實在太簡單,所以我這邊便也直接放置安裝步驟,充充版面:

安裝 Drone

  1. 首先你需要環境上有 Docker engine,這邊就不多加贅述,安裝方式請參考官網
  2. 你需要有 Docker Compose,讓我們安裝 Drone CI 更加方便 (因為有 Drone server + Drone agent 的至少兩項執行個體),安裝方式請參考官網 (也可以用 binary install 方式,但那是 go 開發者所使用的)
  3. 在環境目錄下創建一個 drone 資料夾,在該資料夾下創建一個 docker-compose.yml 檔案
    version: '2'
    
    services:
      drone-server:
        image: drone/drone:0.8
        ports:
          - 8090:8000
          - 9000 # 跟 agent 溝通的 port,在同台 server 的話可以不用
        volumes:
          - ./:/var/lib/drone/ # sqlite database,需要目錄寫入
        restart: always
        environment:
          - DRONE_HOST=${DRONE_HOST} # run drone 的 public domain
          - DRONE_OPEN=true
          - DRONE_SECRET=${DRONE_SECRET} # drone server 與 agent 交換的 key
          - DRONE_ADMIN=${DRONE_ADMIN} # 管理者設定,有兩個帳號用 "," 區分
          # GitHub Config
          - DRONE_GITHUB=true # 啟動 Github 模組,可直接與 Github 串接
          - DRONE_GITHUB_CLIENT=${DRONE_GITHUB_CLIENT}
          - DRONE_GITHUB_SECRET=${DRONE_GITHUB_SECRET}
    
    
      drone-agent:
        image: drone/agent:0.8
        restart: always
        depends_on:
          - drone-server
        volumes:
          - /var/run/docker.sock:/var/run/docker.sock # docker in docker 方式,需要 volume
        environment:
          - DRONE_SERVER=drone-server:9000
          - DRONE_SECRET=${DRONE_SECRET}
          - DRONE_MAX_PROCS=3 # 一個 drone engine 可以 build 幾個專案,記憶體越高這裡可以調越高
    
  4. 執行指令
    $ docker-compose up
    
  5. Drone 畫面設置:勾選要啟動的專案
  6. Drone 畫面設置:專案勾選 Trusted

整合 Github

這裡範例我使用 Github 整合 Drone 方式,在 Github 頁面上的 Settings / Developer settings / OAuth Apps 下,新增 OAuth App,如下畫面:

這裡的 drone public URL 我使用 ngrok 方式,按下註冊 application後,接著 Github 便會提供給你 client ID 與 Secret,這兩項就是用來補上前面 drone 安裝步驟的 docker-compose.yml ${DRONE_GITHUB_CLIENT} 與 ${DRONE_GITHUB_SECRET},這樣便能連結你的帳戶並在 drone dashboard 畫面上選擇該帳戶下的 repository (組織的話需要另外要求授權)

安裝  Drone CLI

為了能實現本機端測試,我們需要安裝 drone CLI,也能在本機上設置相關 secret,但在這裡不會用到,我們僅會使用到 drone exec 的指令。

$ brew tap drone/drone
$ brew install drone --devel

我是在 Mac OSX 下安裝的,若有其他環境需求,請參考官網

創建 .drone.yml

pipeline:
  install:
    image: node:8
    commands:
      - sleep 5
      - yarn install --pure-lockfile

  test:
    image: node:8
    commands:
      - npm test

  report:
    image: joakimbeng/java-node
    commands:
      - npm run gen-report
    when:
      status: [ success, failure ]

  slack:
    image: plugins/slack
    channel: ${SLACK_CHANNEL}
    webhook: ${SLACK_WEBHOOK_URL}
    when:
      status: [ success, failure ]


services:
  hub:
    image: selenium/hub:3.13.0-argon
    ports:
      - "4444:4444"

  firefox:
    image: selenium/node-firefox:3.13.0-argon
    detach: true
    volumes:
      - /dev/shm:/dev/shm
    environment:
      HUB_HOST: hub

  chrome:
    image: selenium/node-chrome:3.13.0-argon
    detach: true
    volumes:
      - /dev/shm:/dev/shm
    environment:
      HUB_HOST: hub

這裡簡單描述一下流程步驟:

  1. clone 專案
  2. 啟動 selenium server 服務 (chrome and firefox)
  3. 安裝 node 依賴
  4. 執行測試
  5. 產出 Allure 報表
  6. 寄發 Slack 通知

修改 wdio.conf.js

這裡做了一點調整,能隨開發環境與產品環境有不同設置:

const NODE_ENV = process.env.NODE_ENV || 'development';

let config = {

  //
  // ==================
  // Specify Test Files
  // ==================
  // Define which test specs should run. The pattern is relative to the directory
  // from which `wdio` was called. Notice that, if you are calling `wdio` from an
  // NPM script (see https://docs.npmjs.com/cli/run-script) then the current working
  // directory is where your package.json resides, so `wdio` will be called from there.
  //
  specs: [
    './test/specs/**/*.js'
  ],
  // Patterns to exclude.
  exclude: [
    // 'path/to/excluded/files'
  ],
  //
  // ============
  // Capabilities
  // ============
  // Define your capabilities here. WebdriverIO can run multiple capabilities at the same
  // time. Depending on the number of capabilities, WebdriverIO launches several test
  // sessions. Within your capabilities you can overwrite the spec and exclude options in
  // order to group specific specs to a specific capability.
  //
  // First, you can define how many instances should be started at the same time. Let's
  // say you have 3 different capabilities (Chrome, Firefox, and Safari) and you have
  // set maxInstances to 1; wdio will spawn 3 processes. Therefore, if you have 10 spec
  // files and you set maxInstances to 10, all spec files will get tested at the same time
  // and 30 processes will get spawned. The property handles how many capabilities
  // from the same test should run tests.
  //
  maxInstances: 10,
  //
  // If you have trouble getting all important capabilities together, check out the
  // Sauce Labs platform configurator - a great tool to configure your capabilities:
  // https://docs.saucelabs.com/reference/platforms-configurator
  //
  capabilities: [
    {
      // maxInstances can get overwritten per capability. So if you have an in-house Selenium
      // grid with only 5 firefox instances available you can make sure that not more than
      // 5 instances get started at a time.
      maxInstances: 5,
      //
      browserName: 'chrome'
    },
    {
      // maxInstances can get overwritten per capability. So if you have an in-house Selenium
      // grid with only 5 firefox instances available you can make sure that not more than
      // 5 instances get started at a time.
      maxInstances: 5,
      //
      browserName: 'firefox'
    },
    // {
    //   maxInstances: 1,
    //   browserName: 'MicrosoftEdge',
    //   version: '14',
    //   platform: 'WINDOWS',
    //   name: 'e2e testing'
    // }
  ],
  //
  // ===================
  // Test Configurations
  // ===================
  // Define all options that are relevant for the WebdriverIO instance here
  //
  // By default WebdriverIO commands are executed in a synchronous way using
  // the wdio-sync package. If you still want to run your tests in an async way
  // e.g. using promises you can set the sync option to false.
  sync: true,
  //
  // Level of logging verbosity: silent | verbose | command | data | result | error
  logLevel: 'error',
  //
  // Enables colors for log output.
  coloredLogs: true,
  //
  // Warns when a deprecated command is used
  deprecationWarnings: true,
  //
  // If you only want to run your tests until a specific amount of tests have failed use
  // bail (default is 0 - don't bail, run all tests).
  bail: 0,
  //
  // Saves a screenshot to a given path if a command fails.
  screenshotPath: './errorShots/',
  //
  // Set a base URL in order to shorten url command calls. If your `url` parameter starts
  // with `/`, the base url gets prepended, not including the path portion of your baseUrl.
  // If your `url` parameter starts without a scheme or `/` (like `some/path`), the base url
  // gets prepended directly.
  baseUrl: 'http://localhost',
  //
  // Default timeout for all waitFor* commands.
  waitforTimeout: 10000,
  //
  // Default timeout in milliseconds for request
  // if Selenium Grid doesn't send response
  connectionRetryTimeout: 90000,
  //
  // Default request retries count
  connectionRetryCount: 3,
  //
  // Initialize the browser instance with a WebdriverIO plugin. The object should have the
  // plugin name as key and the desired plugin options as properties. Make sure you have
  // the plugin installed before running any tests. The following plugins are currently
  // available:
  // WebdriverCSS: https://github.com/webdriverio/webdrivercss
  // WebdriverRTC: https://github.com/webdriverio/webdriverrtc
  // Browserevent: https://github.com/webdriverio/browserevent
  // plugins: {
  //     webdrivercss: {
  //         screenshotRoot: 'my-shots',
  //         failedComparisonsRoot: 'diffs',
  //         misMatchTolerance: 0.05,
  //         screenWidth: [320,480,640,1024]
  //     },
  //     webdriverrtc: {},
  //     browserevent: {}
  // },
  //
  // Test runner services
  // Services take over a specific job you don't want to take care of. They enhance
  // your test setup with almost no effort. Unlike plugins, they don't add new
  // commands. Instead, they hook themselves up into the test process.

  // services: ['testingbot'],
  // host: 'hub',
  // port: 4444,
  // user: process.env.TESTINGBOT_KEY,
  // key: process.env.TESTINGBOT_SECRET,

  // Framework you want to run your specs with.
  // The following are supported: Mocha, Jasmine, and Cucumber
  // see also: http://webdriver.io/guide/testrunner/frameworks.html
  //
  // Make sure you have the wdio adapter package for the specific framework installed
  // before running any tests.
  framework: 'mocha',
  //
  // Test reporter for stdout.
  // The only one supported by default is 'dot'
  // see also: http://webdriver.io/guide/reporters/dot.html
  reporters: ['dot', 'spec'],
  //
  // Options to be passed to Mocha.
  // See the full list at http://mochajs.org/
  mochaOpts: {
    ui: 'bdd',
    timeout: 600000
  },
  //
  // =====
  // Hooks
  // =====
  // WebdriverIO provides several hooks you can use to interfere with the test process in order to enhance
  // it and to build services around it. You can either apply a single function or an array of
  // methods to it. If one of them returns with a promise, WebdriverIO will wait until that promise got
  // resolved to continue.
  /**
   * Gets executed once before all workers get launched.
   * @param {Object} config wdio configuration object
   * @param {Array.<Object>} capabilities list of capabilities details
   */
  // onPrepare: function (config, capabilities) {
  // },
  /**
   * Gets executed just before initialising the webdriver session and test framework. It allows you
   * to manipulate configurations depending on the capability or spec.
   * @param {Object} config wdio configuration object
   * @param {Array.<Object>} capabilities list of capabilities details
   * @param {Array.<String>} specs List of spec file paths that are to be run
   */
  // beforeSession: function (config, capabilities, specs) {
  // },
  /**
   * Gets executed before test execution begins. At this point you can access to all global
   * variables like `browser`. It is the perfect place to define custom commands.
   * @param {Array.<Object>} capabilities list of capabilities details
   * @param {Array.<String>} specs List of spec file paths that are to be run
   */
  // before: function (capabilities, specs) {
  // },
  /**
   * Runs before a WebdriverIO command gets executed.
   * @param {String} commandName hook command name
   * @param {Array} args arguments that command would receive
   */
  // beforeCommand: function (commandName, args) {
  // },

  /**
   * Hook that gets executed before the suite starts
   * @param {Object} suite suite details
   */
  // beforeSuite: function (suite) {
  // },
  /**
   * Function to be executed before a test (in Mocha/Jasmine) or a step (in Cucumber) starts.
   * @param {Object} test test details
   */
  // beforeTest: function (test) {
  // },
  /**
   * Hook that gets executed _before_ a hook within the suite starts (e.g. runs before calling
   * beforeEach in Mocha)
   */
  // beforeHook: function () {
  // },
  /**
   * Hook that gets executed _after_ a hook within the suite starts (e.g. runs after calling
   * afterEach in Mocha)
   */
  // afterHook: function () {
  // },
  /**
   * Function to be executed after a test (in Mocha/Jasmine) or a step (in Cucumber) starts.
   * @param {Object} test test details
   */
  // afterTest: function (test) {
  // },
  /**
   * Hook that gets executed after the suite has ended
   * @param {Object} suite suite details
   */
  // afterSuite: function (suite) {
  // },

  /**
   * Runs after a WebdriverIO command gets executed
   * @param {String} commandName hook command name
   * @param {Array} args arguments that command would receive
   * @param {Number} result 0 - command success, 1 - command error
   * @param {Object} error error object if any
   */
  // afterCommand: function (commandName, args, result, error) {
  // },
  /**
   * Gets executed after all tests are done. You still have access to all global variables from
   * the test.
   * @param {Number} result 0 - test pass, 1 - test fail
   * @param {Array.<Object>} capabilities list of capabilities details
   * @param {Array.<String>} specs List of spec file paths that ran
   */
  // after: function (result, capabilities, specs) {
  // },
  /**
   * Gets executed right after terminating the webdriver session.
   * @param {Object} config wdio configuration object
   * @param {Array.<Object>} capabilities list of capabilities details
   * @param {Array.<String>} specs List of spec file paths that ran
   */
  // afterSession: function (config, capabilities, specs) {
  // },
  /**
   * Gets executed after all workers got shut down and the process is about to exit.
   * @param {Object} exitCode 0 - success, 1 - fail
   * @param {Object} config wdio configuration object
   * @param {Array.<Object>} capabilities list of capabilities details
   */
  // onComplete: function(exitCode, config, capabilities) {
  // }
};

if (NODE_ENV === 'production') {
  config.reporters = ['dot', 'spec', 'allure'];
  config.reporterOptions = {
    junit: {
      outputDir: './allure-results'
    }
  };
  config.host = 'hub';
  config.port = 4444;
}

exports.config = config;

修改 package.json

{
  "name": "wdio-demo",
  "version": "1.0.0",
  "scripts": {
    "dev-test": "npm run e2e-test",
    "test": "cross-env NODE_ENV=production npm run e2e-test",
    "e2e-test": "wdio wdio.conf.js",
    "watch-test": "chimp --mocha --watch --path=test",
    "test-with-drone": "drone exec",
    "gen-report": "allure generate --clean",
    "open-report": "allure open"
  },
  "keywords": [],
  "author": "eden90267",
  "license": "ISC",
  "devDependencies": {
    "allure-commandline": "^2.5.0",
    "app-root-path": "^2.0.1",
    "chai": "^4.1.2",
    "mocha": "^5.0.0",
    "wdio-allure-reporter": "^0.6.0",
    "wdio-mocha-framework": "^0.5.12",
    "wdio-spec-reporter": "^0.1.3",
    "wdio-testingbot-service": "^0.1.7",
    "webdriverio": "^4.10.1"
  },
  "dependencies": {
    "cross-env": "^5.1.4"
  }
}

執行本機端測試

接下來是令人興奮的時刻,不過在 commit 前,我們先在本機端測試看看是否運作正常 (若 drone server 安裝在本機要先關掉,不然會有 docker networking 衝突)

$ rm -rf node_modules

$ docker pull node:8
$ docker pull joakimbeng/java-node
$ docker pull plugins/slack
$ docker pull selenium/hub:3.9.1
$ docker pull selenium/node-firefox:3.9.1
$ docker pull selenium/node-chrome:2.53.1

$ drone exec

如果不先 pull 好需要的 images,drone exec 不會回饋在 pull 的訊息,看起來像當掉一樣…

如何?成功執行測試與驗證嗎?至少我這裡是可以的,不過寄發的 Slack 訊息會很本機的樣子,不用太緊張。

執行 CI Server 測試

勇敢的 commit 吧!讓 Drone CI Server 抓到你的 repository commit and push,並實施真正的整合自動化測試:

這裡是 Drone CI 寄發的 Slack 通知 channel 畫面:

最後,我們的產出 Allure 報表是遠超過 Jenkins junit 報表的樣式的,透過 drone exec 後的報表產出資料夾 “allure-report”,你可以開啟他的 index.html 畫面 (or 執行 npm run open-report),如下:

大功告成,至於由主專案 repo 的 master 有變更便 trigger 測試 repo 的做法,請將下列的 .drone.yml 放置該主專案的 repo:

trigger:
  image: plugins/downstream
  server: https://cbc1164b.ngrok.io
  token: ${GLOBAL_TOKEN}
  fork: true
  repositories:
    - eden90267/wdio-demo
  when:
    branch: master

這樣便能實現 repo -> repo 觸發。

 


後記:

感謝 appleboy 提供的資源,過程中我也向他請教了不少問題,譬如說 .drone.yml 目前還不支援 depends_on 寫法的因應方式、組織專案如何授權給 Drone CI、trigger repo 的做法,讓我能順利的達成此次研究目的。Drone CI 說他成熟但也有其限制,希望能更為蓬勃發展,讓我們這些開發者在持續整合上能更為慵懶方便!

發佈留言

發佈留言必須填寫的電子郵件地址不會公開。 必填欄位標示為 *

這個網站採用 Akismet 服務減少垃圾留言。進一步了解 Akismet 如何處理網站訪客的留言資料