Skip to content

MQE-987: Decouple MFTF from Magento #134

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 30, 2018
Merged

MQE-987: Decouple MFTF from Magento #134

merged 1 commit into from
May 30, 2018

Conversation

imeron2433
Copy link
Contributor

  • add command parity to mftf bin command
  • provide support for local bin execution in funcitonal tests

Description

Fixed Issues (if relevant)

  1. magento/magento2-functional-testing-framework#<issue_number>: Issue title
  2. ...

Contribution checklist

  • Pull request has a meaningful description of its purpose
  • All commits are accompanied by meaningful commit messages
  • All new or changed code is covered with unit/verification tests (if applicable)
  • All automated tests passed successfully (all builds on Travis CI are green)
  • Changes to Framework doesn't have backward incompatible changes for tests or have related Pull Request with fixes to tests

@imeron2433 imeron2433 requested a review from lenaorobei May 23, 2018 15:36
@coveralls
Copy link

coveralls commented May 23, 2018

Coverage Status

Coverage decreased (-0.1%) to 58.344% when pulling 1af30e0 on MQE-987 into 486a5be on develop.

@okolesnyk
Copy link
Member

@lenaorobei please review

Copy link
Contributor

@lenaorobei lenaorobei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see my comments above.

foreach ($_ENV as $key => $var) {
defined($key) || define($key, $var);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing new line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.


return $exists;
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing new line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}

if ($output->isVerbose()) {
$output->writeLn("All Suites Generated");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This message looks like not really "verbose". Suggest outputting messages about all suits and leaving All Suites Generated by default.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

->setDescription('This command generates all test files and suites based on xml declarations')
->addArgument('name', InputArgument::OPTIONAL | InputArgument::IS_ARRAY, 'name(s) of specific tests to generate')
->addOption("config", null, InputOption::VALUE_REQUIRED, 'default, singleRun, or parallel', 'default')
->addOption("force", null,InputOption::VALUE_NONE, 'force generation of tests regardless of Magento Instance Configuration')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be good to add shortcuts for options (second parameter in the addOption method).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

SuiteGenerator::getInstance()->generateAllSuites($testManifest);
$testManifest->generate();

print "Generate Tests Command Run" . PHP_EOL;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$output->writeLn can be used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

if ($output->isVerbose()) {
// build a list of the paths
$afterState = $this->getFileSystemState($finder);
$deletedFiles = array_diff($beforeState, $afterState);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have doubts that you need $beforeState and $afterState to get the list of deleted files.
$fileSystem->remove() throws an Exception if the file cannot be deleted, so you can rely on the list of files you are intended to delete.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I understand this comment. From what I understand if a file does not exist and filesystem attempts to remove there is no exception, how can I rely on this list? Additionally I am deleting php files in certain directories, that will not tell me explicit file deletion to output.

Copy link
Contributor

@lenaorobei lenaorobei May 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm missing something, but this is how I understood it should work.

protected function execute(InputInterface $input, OutputInterface $output)
    {
        $fileSystem = new Filesystem();

        // delete the files mftf generates during test execution in TESTS_BP
        $fileSystem->remove(self::GENERATED_FILES);
        if ($output->isVerbose()) {
            $output->writeln("Folder with generated files deleted.");
        }
        
        $finder = new Finder();
        $files = $finder->files()->name('*.php')->in(FW_BP . '/src/Magento/FunctionalTestingFramework/Group/');
        foreach ($files as $file) {
            $path = $file->getRealPath();
            $fileSystem->remove($file);
            if ($output->isVerbose()) {
                $output->writeln(sprintf('File %s deleted.', $path));
            }
        }

        // delete config files if user specifies a hard reset
        $isHardReset = $input->getOption('hard');
        if ($isHardReset) {
            foreach (self::CONFIGURATION_FILES as $configFile) {
                if (file_exists($configFile)) {
                    $fileSystem->remove($configFile);
                    if ($output->isVerbose()) {
                        $output->writeln("Config file $configFile deleted.");
                    }
                }
            }
        }

        $output->writeln('<info>Project cleaned successfully.<info>');
    }

Copy link
Contributor Author

@imeron2433 imeron2433 May 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay I see what you are saying, I didn't want to explicitly loop the files since the filesystem command will go ahead and delete the files included by the finder object on its own but you're right if we want to have the explicit print we'll have to loop the files ourselves and output.

$this->setName('generate:tests')
->setDescription('This command generates all test files and suites based on xml declarations')
->addArgument('name', InputArgument::OPTIONAL | InputArgument::IS_ARRAY, 'name(s) of specific tests to generate')
->addOption("config", null, InputOption::VALUE_REQUIRED, 'default, singleRun, or parallel', 'default')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use single quotes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

{
$tests = $input->getArgument('name');
$config = $input->getOption('config');
$json = $input->getOption('tests');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest validating input parameters here.
For example, you try to decode JSON in the parseTestsConfigJson method and it causes a lot of redundant code is executed. If JSON is invalid, execution should stop immediately.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

$process->setWorkingDirectory(TESTS_BP);
$process->run(
function ($type, $buffer) use ($output) {
$output->write($buffer);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

writeLn will be more readable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

*/
private function getGroupAndSuiteConfiguration(array $groups)
{
$testConfiguration['tests'] = [];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems strange to me that $testConfiguration['tests'] = [] but $testConfiguration['suites'] = null.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like them both to be null, but I cannot merge any relevant tests to a value of null. I could do a check for the "first value" and replace instead of merge but this seems like elegant to me. Additionally I could make this command exclusive to test groups and point users to the run suite command if they attempt to run a group that is in fact a suite.

@magento magento deleted a comment from lenaorobei May 24, 2018
- add new mftf console commands
- include new bootstrap from mftf autoloading
- update tests and pathing to be windows compatible
@imeron2433
Copy link
Contributor Author

Hi @lenaorobei, I believe i have addressed all of your comments. I am going to merge this code in at the End of the Day if you don't have any other concerns.

@imeron2433 imeron2433 merged commit 343009b into develop May 30, 2018
@imeron2433 imeron2433 deleted the MQE-987 branch May 30, 2018 15:36
magento-devops-reposync-svc pushed a commit that referenced this pull request Nov 30, 2021
MQE-2669: Seprated a run:failed command to generate:failed and run:fa…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants