WebDriverIO with Drone CI,延續上篇的 WebDriverIO with CI,這次我們在 CI Server 的選擇上改採用 Drone CI,比起 Jenkins 它具有以下優勢:
- 容易安裝
- 簡單易上手的專案設定
- 啟動速度快,資源耗用量低 (golang 編寫,< 50 MB)
- plugin 開發可以用任何語言 (基於容器技術開發)
- 可透過 Drone CLI 執行本機端測試
- 不需要管理者,專案下 .drone.yml 管理部署
但不可否認的是 Jenkins CI 因為老牌,所以 plugin 大到幾乎能滿足所有需求,若以非 pipeline 2.0 設置方式,只要針對要整合的關鍵字搜尋,安裝 plugin,再來就是填一填那些 plugin 要求的表單資料,即可順利運行。只是 Jenkins 也因為太強大,需要的資源佔用、學習成本相對較高,導致團隊需要有專人處理,這樣不免會有代理跟該專人離職的後續人力問題。
如果你沒碰過 CI/CD Server,推薦你學習 Drone CI,而且官方還有中文文件喔!
前一篇礙於篇幅所以沒講到如何安裝 Jenkins CI,但在 Drone CI 這裡由於安裝實在太簡單,所以我這邊便也直接放置安裝步驟,充充版面:
安裝 Drone
- 首先你需要環境上有 Docker engine,這邊就不多加贅述,安裝方式請參考官網
- 你需要有 Docker Compose,讓我們安裝 Drone CI 更加方便 (因為有 Drone server + Drone agent 的至少兩項執行個體),安裝方式請參考官網 (也可以用 binary install 方式,但那是 go 開發者所使用的)
- 在環境目錄下創建一個 drone 資料夾,在該資料夾下創建一個 docker-compose.yml 檔案
version: '2' services: drone-server: image: drone/drone:0.8 ports: - 8090:8000 - 9000 # 跟 agent 溝通的 port,在同台 server 的話可以不用 volumes: - ./:/var/lib/drone/ # sqlite database,需要目錄寫入 restart: always environment: - DRONE_HOST=${DRONE_HOST} # run drone 的 public domain - DRONE_OPEN=true - DRONE_SECRET=${DRONE_SECRET} # drone server 與 agent 交換的 key - DRONE_ADMIN=${DRONE_ADMIN} # 管理者設定,有兩個帳號用 "," 區分 # GitHub Config - DRONE_GITHUB=true # 啟動 Github 模組,可直接與 Github 串接 - DRONE_GITHUB_CLIENT=${DRONE_GITHUB_CLIENT} - DRONE_GITHUB_SECRET=${DRONE_GITHUB_SECRET} drone-agent: image: drone/agent:0.8 restart: always depends_on: - drone-server volumes: - /var/run/docker.sock:/var/run/docker.sock # docker in docker 方式,需要 volume environment: - DRONE_SERVER=drone-server:9000 - DRONE_SECRET=${DRONE_SECRET} - DRONE_MAX_PROCS=3 # 一個 drone engine 可以 build 幾個專案,記憶體越高這裡可以調越高
- 執行指令
$ docker-compose up
- Drone 畫面設置:勾選要啟動的專案
- Drone 畫面設置:專案勾選 Trusted
整合 Github
這裡範例我使用 Github 整合 Drone 方式,在 Github 頁面上的 Settings / Developer settings / OAuth Apps 下,新增 OAuth App,如下畫面:
這裡的 drone public URL 我使用 ngrok 方式,按下註冊 application後,接著 Github 便會提供給你 client ID 與 Secret,這兩項就是用來補上前面 drone 安裝步驟的 docker-compose.yml ${DRONE_GITHUB_CLIENT} 與 ${DRONE_GITHUB_SECRET},這樣便能連結你的帳戶並在 drone dashboard 畫面上選擇該帳戶下的 repository (組織的話需要另外要求授權)
安裝 Drone CLI
為了能實現本機端測試,我們需要安裝 drone CLI,也能在本機上設置相關 secret,但在這裡不會用到,我們僅會使用到 drone exec 的指令。
$ brew tap drone/drone $ brew install drone --devel
我是在 Mac OSX 下安裝的,若有其他環境需求,請參考官網。
創建 .drone.yml
pipeline: install: image: node:8 commands: - sleep 5 - yarn install --pure-lockfile test: image: node:8 commands: - npm test report: image: joakimbeng/java-node commands: - npm run gen-report when: status: [ success, failure ] slack: image: plugins/slack channel: ${SLACK_CHANNEL} webhook: ${SLACK_WEBHOOK_URL} when: status: [ success, failure ] services: hub: image: selenium/hub:3.13.0-argon ports: - "4444:4444" firefox: image: selenium/node-firefox:3.13.0-argon detach: true volumes: - /dev/shm:/dev/shm environment: HUB_HOST: hub chrome: image: selenium/node-chrome:3.13.0-argon detach: true volumes: - /dev/shm:/dev/shm environment: HUB_HOST: hub
這裡簡單描述一下流程步驟:
- clone 專案
- 啟動 selenium server 服務 (chrome and firefox)
- 安裝 node 依賴
- 執行測試
- 產出 Allure 報表
- 寄發 Slack 通知
修改 wdio.conf.js
這裡做了一點調整,能隨開發環境與產品環境有不同設置:
const NODE_ENV = process.env.NODE_ENV || 'development'; let config = { // // ================== // Specify Test Files // ================== // Define which test specs should run. The pattern is relative to the directory // from which `wdio` was called. Notice that, if you are calling `wdio` from an // NPM script (see https://docs.npmjs.com/cli/run-script) then the current working // directory is where your package.json resides, so `wdio` will be called from there. // specs: [ './test/specs/**/*.js' ], // Patterns to exclude. exclude: [ // 'path/to/excluded/files' ], // // ============ // Capabilities // ============ // Define your capabilities here. WebdriverIO can run multiple capabilities at the same // time. Depending on the number of capabilities, WebdriverIO launches several test // sessions. Within your capabilities you can overwrite the spec and exclude options in // order to group specific specs to a specific capability. // // First, you can define how many instances should be started at the same time. Let's // say you have 3 different capabilities (Chrome, Firefox, and Safari) and you have // set maxInstances to 1; wdio will spawn 3 processes. Therefore, if you have 10 spec // files and you set maxInstances to 10, all spec files will get tested at the same time // and 30 processes will get spawned. The property handles how many capabilities // from the same test should run tests. // maxInstances: 10, // // If you have trouble getting all important capabilities together, check out the // Sauce Labs platform configurator - a great tool to configure your capabilities: // https://docs.saucelabs.com/reference/platforms-configurator // capabilities: [ { // maxInstances can get overwritten per capability. So if you have an in-house Selenium // grid with only 5 firefox instances available you can make sure that not more than // 5 instances get started at a time. maxInstances: 5, // browserName: 'chrome' }, { // maxInstances can get overwritten per capability. So if you have an in-house Selenium // grid with only 5 firefox instances available you can make sure that not more than // 5 instances get started at a time. maxInstances: 5, // browserName: 'firefox' }, // { // maxInstances: 1, // browserName: 'MicrosoftEdge', // version: '14', // platform: 'WINDOWS', // name: 'e2e testing' // } ], // // =================== // Test Configurations // =================== // Define all options that are relevant for the WebdriverIO instance here // // By default WebdriverIO commands are executed in a synchronous way using // the wdio-sync package. If you still want to run your tests in an async way // e.g. using promises you can set the sync option to false. sync: true, // // Level of logging verbosity: silent | verbose | command | data | result | error logLevel: 'error', // // Enables colors for log output. coloredLogs: true, // // Warns when a deprecated command is used deprecationWarnings: true, // // If you only want to run your tests until a specific amount of tests have failed use // bail (default is 0 - don't bail, run all tests). bail: 0, // // Saves a screenshot to a given path if a command fails. screenshotPath: './errorShots/', // // Set a base URL in order to shorten url command calls. If your `url` parameter starts // with `/`, the base url gets prepended, not including the path portion of your baseUrl. // If your `url` parameter starts without a scheme or `/` (like `some/path`), the base url // gets prepended directly. baseUrl: 'http://localhost', // // Default timeout for all waitFor* commands. waitforTimeout: 10000, // // Default timeout in milliseconds for request // if Selenium Grid doesn't send response connectionRetryTimeout: 90000, // // Default request retries count connectionRetryCount: 3, // // Initialize the browser instance with a WebdriverIO plugin. The object should have the // plugin name as key and the desired plugin options as properties. Make sure you have // the plugin installed before running any tests. The following plugins are currently // available: // WebdriverCSS: https://github.com/webdriverio/webdrivercss // WebdriverRTC: https://github.com/webdriverio/webdriverrtc // Browserevent: https://github.com/webdriverio/browserevent // plugins: { // webdrivercss: { // screenshotRoot: 'my-shots', // failedComparisonsRoot: 'diffs', // misMatchTolerance: 0.05, // screenWidth: [320,480,640,1024] // }, // webdriverrtc: {}, // browserevent: {} // }, // // Test runner services // Services take over a specific job you don't want to take care of. They enhance // your test setup with almost no effort. Unlike plugins, they don't add new // commands. Instead, they hook themselves up into the test process. // services: ['testingbot'], // host: 'hub', // port: 4444, // user: process.env.TESTINGBOT_KEY, // key: process.env.TESTINGBOT_SECRET, // Framework you want to run your specs with. // The following are supported: Mocha, Jasmine, and Cucumber // see also: http://webdriver.io/guide/testrunner/frameworks.html // // Make sure you have the wdio adapter package for the specific framework installed // before running any tests. framework: 'mocha', // // Test reporter for stdout. // The only one supported by default is 'dot' // see also: http://webdriver.io/guide/reporters/dot.html reporters: ['dot', 'spec'], // // Options to be passed to Mocha. // See the full list at http://mochajs.org/ mochaOpts: { ui: 'bdd', timeout: 600000 }, // // ===== // Hooks // ===== // WebdriverIO provides several hooks you can use to interfere with the test process in order to enhance // it and to build services around it. You can either apply a single function or an array of // methods to it. If one of them returns with a promise, WebdriverIO will wait until that promise got // resolved to continue. /** * Gets executed once before all workers get launched. * @param {Object} config wdio configuration object * @param {Array.<Object>} capabilities list of capabilities details */ // onPrepare: function (config, capabilities) { // }, /** * Gets executed just before initialising the webdriver session and test framework. It allows you * to manipulate configurations depending on the capability or spec. * @param {Object} config wdio configuration object * @param {Array.<Object>} capabilities list of capabilities details * @param {Array.<String>} specs List of spec file paths that are to be run */ // beforeSession: function (config, capabilities, specs) { // }, /** * Gets executed before test execution begins. At this point you can access to all global * variables like `browser`. It is the perfect place to define custom commands. * @param {Array.<Object>} capabilities list of capabilities details * @param {Array.<String>} specs List of spec file paths that are to be run */ // before: function (capabilities, specs) { // }, /** * Runs before a WebdriverIO command gets executed. * @param {String} commandName hook command name * @param {Array} args arguments that command would receive */ // beforeCommand: function (commandName, args) { // }, /** * Hook that gets executed before the suite starts * @param {Object} suite suite details */ // beforeSuite: function (suite) { // }, /** * Function to be executed before a test (in Mocha/Jasmine) or a step (in Cucumber) starts. * @param {Object} test test details */ // beforeTest: function (test) { // }, /** * Hook that gets executed _before_ a hook within the suite starts (e.g. runs before calling * beforeEach in Mocha) */ // beforeHook: function () { // }, /** * Hook that gets executed _after_ a hook within the suite starts (e.g. runs after calling * afterEach in Mocha) */ // afterHook: function () { // }, /** * Function to be executed after a test (in Mocha/Jasmine) or a step (in Cucumber) starts. * @param {Object} test test details */ // afterTest: function (test) { // }, /** * Hook that gets executed after the suite has ended * @param {Object} suite suite details */ // afterSuite: function (suite) { // }, /** * Runs after a WebdriverIO command gets executed * @param {String} commandName hook command name * @param {Array} args arguments that command would receive * @param {Number} result 0 - command success, 1 - command error * @param {Object} error error object if any */ // afterCommand: function (commandName, args, result, error) { // }, /** * Gets executed after all tests are done. You still have access to all global variables from * the test. * @param {Number} result 0 - test pass, 1 - test fail * @param {Array.<Object>} capabilities list of capabilities details * @param {Array.<String>} specs List of spec file paths that ran */ // after: function (result, capabilities, specs) { // }, /** * Gets executed right after terminating the webdriver session. * @param {Object} config wdio configuration object * @param {Array.<Object>} capabilities list of capabilities details * @param {Array.<String>} specs List of spec file paths that ran */ // afterSession: function (config, capabilities, specs) { // }, /** * Gets executed after all workers got shut down and the process is about to exit. * @param {Object} exitCode 0 - success, 1 - fail * @param {Object} config wdio configuration object * @param {Array.<Object>} capabilities list of capabilities details */ // onComplete: function(exitCode, config, capabilities) { // } }; if (NODE_ENV === 'production') { config.reporters = ['dot', 'spec', 'allure']; config.reporterOptions = { junit: { outputDir: './allure-results' } }; config.host = 'hub'; config.port = 4444; } exports.config = config;
修改 package.json
{ "name": "wdio-demo", "version": "1.0.0", "scripts": { "dev-test": "npm run e2e-test", "test": "cross-env NODE_ENV=production npm run e2e-test", "e2e-test": "wdio wdio.conf.js", "watch-test": "chimp --mocha --watch --path=test", "test-with-drone": "drone exec", "gen-report": "allure generate --clean", "open-report": "allure open" }, "keywords": [], "author": "eden90267", "license": "ISC", "devDependencies": { "allure-commandline": "^2.5.0", "app-root-path": "^2.0.1", "chai": "^4.1.2", "mocha": "^5.0.0", "wdio-allure-reporter": "^0.6.0", "wdio-mocha-framework": "^0.5.12", "wdio-spec-reporter": "^0.1.3", "wdio-testingbot-service": "^0.1.7", "webdriverio": "^4.10.1" }, "dependencies": { "cross-env": "^5.1.4" } }
執行本機端測試
接下來是令人興奮的時刻,不過在 commit 前,我們先在本機端測試看看是否運作正常 (若 drone server 安裝在本機要先關掉,不然會有 docker networking 衝突)
$ rm -rf node_modules $ docker pull node:8 $ docker pull joakimbeng/java-node $ docker pull plugins/slack $ docker pull selenium/hub:3.9.1 $ docker pull selenium/node-firefox:3.9.1 $ docker pull selenium/node-chrome:2.53.1 $ drone exec
如果不先 pull 好需要的 images,drone exec 不會回饋在 pull 的訊息,看起來像當掉一樣…
如何?成功執行測試與驗證嗎?至少我這裡是可以的,不過寄發的 Slack 訊息會很本機的樣子,不用太緊張。
執行 CI Server 測試
勇敢的 commit 吧!讓 Drone CI Server 抓到你的 repository commit and push,並實施真正的整合自動化測試:
這裡是 Drone CI 寄發的 Slack 通知 channel 畫面:
最後,我們的產出 Allure 報表是遠超過 Jenkins junit 報表的樣式的,透過 drone exec 後的報表產出資料夾 “allure-report”,你可以開啟他的 index.html 畫面 (or 執行 npm run open-report),如下:
大功告成,至於由主專案 repo 的 master 有變更便 trigger 測試 repo 的做法,請將下列的 .drone.yml 放置該主專案的 repo:
trigger: image: plugins/downstream server: https://cbc1164b.ngrok.io token: ${GLOBAL_TOKEN} fork: true repositories: - eden90267/wdio-demo when: branch: master
這樣便能實現 repo -> repo 觸發。
後記:
感謝 appleboy 提供的資源,過程中我也向他請教了不少問題,譬如說 .drone.yml 目前還不支援 depends_on 寫法的因應方式、組織專案如何授權給 Drone CI、trigger repo 的做法,讓我能順利的達成此次研究目的。Drone CI 說他成熟但也有其限制,希望能更為蓬勃發展,讓我們這些開發者在持續整合上能更為慵懶方便!