Adds a new --exclude parameter to img-proof test, that allows to pass a comma-separated list of test expressions to skip during a test run.

This allows for using of common patterns e.g. test_sles without running certain tests. Before one needed to state an explicit list of test runs with the risk of not including new test runs as they are included into img-proof.

Currently we do the assumption, that the resource group name is equal to the name of the VM. This isn't always the case.

1. We should add a parameter --resource-group for azure.
2. We could allow using full instance id's like /subscriptions/c011786b-59d7-4817-880c-7cd8a6ca4b19/resourceGroups/openqa-suse-de-1ec3f5a05b7c0712/providers/Microsoft.Compute/virtualMachines/openqa-suse-de-1ec3f5a05b7c0712 and parse resource-group and instance name from it.

Use user data to add SSH public key to instance on EC2 launch when ssh key name is not provided.

• SSH public key is generated based on private key file.
• ssh_key_name arg is optional but takes precedence if provided.

• The cloud-regionsrv-client API for the is_registered() function has changed from expecting an object to expecting a string. Change the test implementation to handle both cases.

ERROR 2019-05-20 20:14:03,773 TestingService Pass[1]: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ipa/ipa_cloud.py", line 636, in test_im age client File "/usr/lib/python3.6/site-packages/ipa/ipa_utils.py", line 262, in get_hos t_key_fingerprint client.get_transport().get_remote_server_key().get_fingerprint() File "/usr/lib/python3.6/site-packages/paramiko/transport.py", line 761, in ge t_remote_server_key raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/mash/services/testing/ipa_helper.py", l ine 95, in ipa_test timeout=ipa_timeout File "/usr/lib/python3.6/site-packages/ipa/ipa_controller.py", line 131, in te st_image return cloud.test_image() File "/usr/lib/python3.6/site-packages/ipa/ipa_cloud.py", line 644, in test_im age 'An error occurred retrieving host key: %s' % error ipa.ipa_exceptions.IpaCloudException: An error occurred retrieving host key: No existing session

mash job id: 4f8c81dd-cfd4-4697-be5f-1777c6c4b220

The suse specific repository test should provide more details. With the switch to the modules and the various flavors of repositories, source, debuginfo etc. it has pretty much become impossible to manually verify registration success. IPA provides the necessary functionality. However, the registration test has a pass/fail status leaving the user/verifyer with the original problem. the test shold produce a diff list in the results if the repo data does not match the expected content. This should hopefully allow the person verifying the test results an opportunity to understand the failure and address it.

Consider migrating EC2 and Azure to Apache Libcloud to minimize dependencies and make providers more uniform.

This would also make adding new providers much easier so long as they are supported by libcloud.

Todo:

• [x] GCE
• [x] EC2
• [ ] Azure

Allows the exclude parameter to be any iterable and string. This fixes a regression in which passing any exclude parameter crashed the program.

Fixes https://github.com/SUSE-Enceladus/img-proof/issues/345

Set root disk to non-default size of 50GB except in OCI (TBD). And test confirms that the filesystem properly expands.

I could not find info in OCI SDK docs about changing root disk parameters when launching instances. Issue opened upstream.

This change adds support for testing Red Hat based distributions, and adds a Fedora module for testing Fedora cloud images.

This PR is the beginning of investigating using img_proof for automatically testing Fedora Cloud Edition images by the Fedora Cloud SIG.

(cc: @dustymabe, @major)

Since https://github.com/torvalds/linux/commit/712ff25450bd01366301eef81c33e865d901e7b7 the uuid is created with lowercase letters.

I see failing test with SLE15-SP1: https://openqa.suse.de/tests/2340648#step/EC2_test_sles_ec2_uuid_py__test_sles_ec2_uuid/1

Hi,

when installing img-proof from the http://download.opensuse.org/repositories/Cloud:/Tools/openSUSE_Leap_15.2/ repository, I get the following error message:

[email protected]:~> img-proof --version
Traceback (most recent call last):
  File "/usr/bin/img-proof", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3123, in <module>
    @_call_aside
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3107, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3136, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 578, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 895, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 781, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'configparser>=3.5.0b1' distribution was not found and is required by oci

I followed the installation procedure from the README.md

zypper ar http://download.opensuse.org/repositories/Cloud:/Tools/openSUSE_Leap_15.2/ PC_Tools_15_2
zypper ref
zypper in python3-img-proof

If a user supplies the .py extension in a test description yaml file or in command line it would be helpful for img-proof to drop this by default prior to doing the test search. Or it would be nice to provide more info with the test not found message to make it clear that the extension is not needed.

Currently when running the SLES test suite on BYOS images we might reboot to soon. Prior to reboot img-proof should wait for the initialization code to finish:

• In Azure this is waagent
• In EC2 and OCI it's cloud-init-final
• In GCE it's google-startup-scripts

If ipa handles listing results from an archive then the user should just give us the archive name, i.e.

ipa results list --archive $PATH_TO_MY_ARCHIVE ipa results show 5 --archive $PATH_TO_MY_ARCHIVE

And ipa should just extract the file on the fly to get the results.

Question is should this be to a temp dir which is deleted after the command or should it be cached somehow for subsequent calls?

It would be a nice enhancement to have a config setup option in CLI. This would help the user setup the ipa config and/or all config files with info for testing. Since a user may not have config setup for ec2utils, azurectl or Google.

ipa setup

Series of prompts requesting config data.