Merge pull request #34 from magento/nightly-builds

Configured basic and extended testsuites for PR and nightly builds

Author: paliarush

.travis.yml

@@ -3,7 +3,10 @@ sudo: required
 dist: xenial
 # This moves Kubernetes specific config files.
 env:
-  - CHANGE_MINIKUBE_NONE_USER=true
+  global:
+    - CHANGE_MINIKUBE_NONE_USER=true
+    # Uncomment the following line to execute full test suite on Travis
+    # - RUN_EXTENDED_TEST_SUITE=true
 
 install:
   # Make root mounted as rshared to fix kube-dns issues.

CONTRIBUTING.md

@@ -1 +1 @@
-Everyone is welcome to contribute improvements and fixes to this project. There are no specific requirements, just create a pull request with proposed modifications and try to run [tests](https://github.com/paliarush/magento2-devbox-for-developers-tests) on your local environment.
+Everyone is welcome to contribute improvements and fixes to this project. There are no specific requirements, just create a pull request with proposed modifications and try to run [tests](https://github.com/paliarush/magento2-vagrant-for-developers-tests) on your local environment.

README.md

@@ -3,7 +3,7 @@
 [![Build Status](https://travis-ci.com/magento/magento2-kubernetes-devbox.svg?branch=master)](https://travis-ci.com/magento/magento2-kubernetes-devbox)
 <!--[![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)-->
 [![Semver](http://img.shields.io/SemVer/2.0.0.png?color=blue)](http://semver.org/spec/v2.0.0.html)
-<!--[![Latest GitHub release](docs/images/release_badge.png)](https://github.com/paliarush/magento2-devbox-for-developers/releases/latest)-->
+<!--[![Latest GitHub release](docs/images/release_badge.png)](https://github.com/paliarush/magento2-vagrant-for-developers/releases/latest)-->
 
  * [What You get](#what-you-get)
  * [How to install](#how-to-install)
@@ -33,6 +33,7 @@
    * [Redis for caching](#redis-for-caching)
    * [Reset environment](#reset-environment)
    * [Switch NodeJS Versions](#switch-nodejs-versions)
+ * [DevBox tests](#devbox-tests)
  * [FAQ](#faq)
 
 ## What You get
@@ -320,7 +321,7 @@ bash m-composer update
 ### Running Magento tests
 
 Not available yet.
-<!--See [draft](https://github.com/paliarush/magento2-devbox-for-developers/issues/120)-->
+<!--See [draft](https://github.com/paliarush/magento2-vagrant-for-developers/issues/120)-->
 
 
 ## Environment configuration
@@ -394,31 +395,26 @@ n <version>
 
 Note: See [Working with npm](https://www.npmjs.com/package/n#working-with-npm) if after switching versions with `n`, `npm` is not working properly.
 -->
+
+### DevBox tests
+
+The tests are executed on every PR on Travis CI. It is possible to configure the same tests to run on the forked repository.
+
+An extended testsuite is executed nightly on the master branch. On every PR by default only basic smoke tests are executed.
+It is possible to execute an extended testsuite on every PR build by uncommenting `RUN_EXTENDED_TEST_SUITE=true` in the [.travis.yaml](./.travis.yml)
+
+The same tests can be run on local using the following command. :warning: only one devbox can be running on the same host at the same time. The tests will destroy existing devbox installation.
+
+```
+cd tests
+bash ./testsuite-basic.sh # OR bash ./testsuite-extended.sh
+```
+
 ### FAQ
-<!--
  1. To debug any CLI script in current Devbox project, set `debug:devbox_project` option in [config.yaml](etc/config.yaml.dist) to `1`
- 1. Is Windows 10 supported? Yes, but you may face the same issue as described [here](https://github.com/paliarush/magento2-devbox-for-developers/issues/36) or [here](https://github.com/paliarush/magento2-devbox-for-developers/issues/173). Also Virtual box may not work on Windows 10 in headless mode, see how to [enable GUI mode](https://www.devboxup.com/docs/virtualbox/configuration.html)
- 1. ![](docs/images/linux-icon.png)![](docs/images/osx-icon.png) On OSX and \*nix hosts NFS will be used by default to sync your project files with guest. On some hosts Devbox cannot configure NFS properly, in this case it is possible to deploy project without NFS by setting `use_nfs` option in [config.yaml](etc/config.yaml.dist) to `0` <br />
- 1. ![](docs/images/windows-icon.png) On Windows hosts you might face `Composer Install Error: ZipArchive::extractTo(): Full extraction path exceed MAXPATHLEN (260)` exception during `composer install`. This can be fixed in 2 ways: decrease path length to the project directory or set `composer_prefer_source` option in [config.yaml](etc/config.yaml.dist) to `1`
  1. Make sure that you used `magento2-devbox` directory as project root in PHP Storm (not `magento2-devbox/magento`)
  1. If project opened in PhpStorm looks broken, close PhpStorm  and remove `magento2-devbox/.idea`. Run `bash magento2-devbox/scripts/host/configure_php_storm.sh`. After opening project in PhpStorm again everything should look good
- 1. If code is not synchronized properly on Windows hosts (or when NFS mode is disabled in [config.yaml](etc/config.yaml.dist) explicitly), make sure that PhpStorm is running before making any changes in the code. This is important because otherwise PhpStorm will not be able to detect changes and upload them to the guest machine
  1. Please make sure that currently installed software, specified in [requirements section](#requirements), meets minimum version requirement
  1. Be careful if your OS is case-insensitive, NFS might break the symlinks if you cd into the wrong casing and you power the devbox up. Just be sure to cd in to the casing the directory was originally created as.
- 1. Cannot run unit tests from PHPStorm on Magento 2.2, see possible solution [here](https://github.com/paliarush/magento2-devbox-for-developers/issues/167)
- 1. [Permission denied (publickey)](https://github.com/paliarush/magento2-devbox-for-developers/issues/165)
- 1. If during a devbox reload, the following message appears:
- 
-    >There was a problem while downloading the metadata for your box
-    to check for updates. This is not an error, since it is usually due
-    to temporary network problems. This is just a warning. The problem
-    encountered was:
-    The requested URL returned error: 404 Not Found
-    
-    It is likely that your devbox cli is caching an old url. Perform the following cli commands:
-    
-    ```bash
-    sed -i -- 's/atlas.hashicorp/devboxcloud/g' ~/.devbox.d/boxes/{name of your paliarush/ubuntu image}/metadata_url
-    mv ~/.devbox.d/boxes/{name of your paliarush/ubuntu image}/metadata_url2 ~/.devbox.d/boxes/{name of your paliarush/ubuntu image}/metadata_url
-    ``` 
--->
+ 1. Cannot run unit tests from PHPStorm on Magento 2.2, see possible solution [here](https://github.com/paliarush/magento2-vagrant-for-developers/issues/167)
+ 1. [Permission denied (publickey)](https://github.com/paliarush/magento2-vagrant-for-developers/issues/165)

init_project.sh

@@ -302,7 +302,7 @@ info "$(bold)[Important]$(regular)
 #if [[ ${host_os} == "Windows" ]] || [[ ${use_nfs} == 0 ]]; then
 #    info "$(bold)[Optional]$(regular)
 #    To verify that deployment configuration for $(bold)${magento_ce_dir}$(regular) in PhpStorm is correct,
-#        use instructions provided here: $(bold)https://github.com/paliarush/magento2-devbox-for-developers/blob/2.0/docs/phpstorm-configuration-windows-hosts.md$(regular).
+#        use instructions provided here: $(bold)https://github.com/paliarush/magento2-vagrant-for-developers/blob/2.0/docs/phpstorm-configuration-windows-hosts.md$(regular).
 #    If not using PhpStorm, you can set up synchronization using rsync"
 #fi
 

scripts/guest/check_mounted_directories

@@ -9,7 +9,7 @@ incrementNestingLevel
 
 if [[ ! -f "${MAGENTO_ROOT}/bin/magento" ]]; then
     error "Directory '${MAGENTO_ROOT}' was not mounted as expected and Magento code base is not accessible on the guest machine.
-        If your host is OSX or *nix, please make sure that Devbox is able to mount NFS shares on your environment (see https://github.com/paliarush/magento2-devbox-for-developers/issues/88#issuecomment-254854019 ).
+        If your host is OSX or *nix, please make sure that Devbox is able to mount NFS shares on your environment (see https://github.com/paliarush/magento2-vagrant-for-developers/issues/88#issuecomment-254854019 ).
         Also remove any stale declarations from /etc/exports on the host."
     exit 1
 fi

tests/include/configuration.sh.dist

@@ -10,6 +10,6 @@ devbox_project_local_path="${project_root_dir}"
 devbox_project_repository_url="git@github.com:magento/magento2-devbox-for-developers.git"
 devbox_project_branch="master"
 
-delete_test_project_on_tear_down=0
+delete_test_project_on_tear_down=1
 
 cd ${original_dir}

tests/include/helpers.sh

@@ -43,7 +43,7 @@ function downloadBaseVersionOfDevboxProject()
     echo "${grey}## downloadBaseVersionOfDevboxProject${regular}"
     echo "## downloadBaseVersionOfDevboxProject" >>${current_log_file_path}
     cd ${tests_dir}
-    git clone git@github.com:paliarush/magento2-devbox-for-developers.git "${devbox_dir}" >>${current_log_file_path} 2>&1
+    git clone git@github.com:paliarush/magento2-vagrant-for-developers.git "${devbox_dir}" >>${current_log_file_path} 2>&1
     cd "${devbox_dir}"
     git checkout tags/v2.2.0 >>${current_log_file_path} 2>&1
     # Make sure that older box version is used
@@ -174,7 +174,7 @@ function clearTestTmp()
     echo "## clearTestTmp" >>${current_log_file_path}
     if [ -e "${devbox_dir}" ]; then
         cd "${devbox_dir}"
-        devbox destroy -f &>/dev/null
+        minikube delete &>/dev/null
         cd ${tests_dir}
         rm -rf "${devbox_dir}"
     fi

tests/run-tests-on-travis.sh

@@ -1,8 +1,9 @@
 #!/bin/bash
-set -ev
+set -e
 
-bash ./testsuite.sh
-
-if [ "${TRAVIS_PULL_REQUEST}" = "false" ]; then
-  bundle exec rake test:integration
+echo "# Running Basic test suite"
+bash ./testsuite-basic.sh
+if [[ "${TRAVIS_EVENT_TYPE}" = "cron" ]] || [[ "${RUN_EXTENDED_TEST_SUITE}" = "true" ]]; then
+    echo '# Running extended test suite'
+    bash ./testsuite-extended.sh
 fi

tests/testsuite.sh renamed to tests/testsuite-basic.sh

@@ -39,7 +39,7 @@ function tearDown()
         clearTestTmp
     fi
 
-    # TODO: change globally when https://github.com/paliarush/magento2-devbox-for-developers/issues/58 is unblocked
+    # TODO: change globally when https://github.com/paliarush/magento2-vagrant-for-developers/issues/58 is unblocked
     devbox_dir="${tests_dir}/tmp/test/magento2-devbox"
 }
 

tests/testsuite-extended.sh

@@ -0,0 +1,71 @@
+#! /usr/bin/env bash
+
+cd "$(dirname "${BASH_SOURCE[0]}")/.." && project_root_dir=$PWD
+
+tests_dir="${project_root_dir}/tests"
+cd ${tests_dir}
+
+## Includes
+source include/global_variables.sh
+source include/configuration.sh
+source include/helpers.sh
+source include/assertions.sh
+
+original_devbox_dir="${devbox_dir}"
+source ./../scripts/functions.sh
+devbox_dir=${original_devbox_dir}
+cd ${tests_dir}
+
+## Setup and tear down
+
+function oneTimeSetUp
+{
+    clearLogs
+}
+
+function setUp()
+{
+    debug_devbox_project=0
+    skip_codebase_stash=0
+}
+
+function tearDown()
+{
+    assertNoErrorsInLogs
+
+    if [[ ${delete_test_project_on_tear_down} -eq 1 ]]; then
+        stashLogs
+        stashMagentoCodebase
+        clearTestTmp
+    fi
+
+    # TODO: change globally when https://github.com/paliarush/magento2-vagrant-for-developers/issues/58 is unblocked
+    devbox_dir="${tests_dir}/tmp/test/magento2-devbox"
+}
+
+function oneTimeTearDown()
+{
+    echo "
+See logs in ${logs_dir}"
+}
+
+## Tests
+
+function testNoCustomConfigExtendedTest()
+{
+    current_config_name="no_custom_config"
+    current_codebase="ce"
+    installEnvironment
+#    assertVarnishDisabled
+    executeBasicCommonAssertions
+    assertMagentoEditionIsCE
+    assertCeSampleDataNotInstalled
+    assertTestsConfigured
+    assertDebugConfigurationWork
+    assertRedisCacheIsEnabled
+
+    executeExtendedCommonAssertions
+}
+
+## Call and Run all Tests
+source lib/shunit2/shunit2