Skip to content

[Simulator] Add zone wide storage#5986

Merged
nvazquez merged 2 commits into
apache:mainfrom
shapeblue:simulatorzonewide
Feb 17, 2022
Merged

[Simulator] Add zone wide storage#5986
nvazquez merged 2 commits into
apache:mainfrom
shapeblue:simulatorzonewide

Conversation

@nvazquez

@nvazquez nvazquez commented Feb 11, 2022

Copy link
Copy Markdown
Contributor

Description

This PR allows adding zone-wide primary storage on simulator

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

Before it was not possible to add a zone-wide storage pool on the simulator
Screen Shot 2022-02-11 at 10 48 45

After the changes:
Screen Shot 2022-02-11 at 10 49 54

How Has This Been Tested?

Add zone wide primary storage, select Simulator on Hypervisor when Scope = 'Zone'

@nvazquez

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@nvazquez nvazquez self-assigned this Feb 11, 2022
@DaanHoogland

Copy link
Copy Markdown
Contributor

looks good, @nicolas. I'll test this (with low prio ;))

@nvazquez

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 2583

@nvazquez

Copy link
Copy Markdown
Contributor Author

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-3308)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 32239 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5986-t3308-kvm-centos7.zip
Smoke tests completed. 92 look OK, 0 have errors
Only failed tests results shown below:

Test Result Time (s) Test File

Comment thread server/src/main/java/com/cloud/storage/StorageManagerImpl.java Outdated
@DaanHoogland

Copy link
Copy Markdown
Contributor

testing

$ python3 tools/marvin/marvin/deployDataCenter.py -i setup/dev/advtwopodzone.cfg

failed with

 <marvin.cloudstackAPI.createStoragePool.createStoragePoolCmd object at 0x7f8d9bdae0b0>
Exception Occurred: ['Traceback (most recent call last):\n', '  File "/home/dahn/apache/cloudstack/tools/marvin/marvin/deployDataCenter.py", line 282, in createPrimaryStorages\n    ret = self.__apiClient.createStoragePool(primarycmd)\n', '  File "/home/dahn/.local/lib/python3.10/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 3540, in createStoragePool\n    response = self.connection.marvinRequest(command, response_type=response, method=method)\n', '  File "/home/dahn/.local/lib/python3.10/site-packages/marvin/cloudstackConnection.py", line 381, in marvinRequest\n    raise e\n', '  File "/home/dahn/.local/lib/python3.10/site-packages/marvin/cloudstackConnection.py", line 376, in marvinRequest\n    raise self.__lastError\n', '  File "/home/dahn/.local/lib/python3.10/site-packages/marvin/cloudstackConnection.py", line 308, in __parseAndGetResponse\n    ret = jsonHelper.getResultObj(\n', '  File "/home/dahn/.local/lib/python3.10/site-packages/marvin/jsonHelper.py", line 155, in getResultObj\n    raise cloudstackException.CloudstackAPIException(respname, errMsg)\n', 'marvin.cloudstackException.CloudstackAPIException: Execute cmd: createstoragepool failed, due to: errorCode: 530, errorText:Failed to add data store: Failed to create storage pool as it is not accessible to hosts.\n']

cfg:

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
{
    "zones": [
        {
            "name": "zim1",
            "guestcidraddress": "10.100.1.0/24",
            "dns1": "10.147.100.6",
            "physical_networks": [
                {
                    "broadcastdomainrange": "Zone",
                    "vlan": "1100-1200",
                    "name": "z1-pnet",
                    "traffictypes": [
                        {
                            "typ": "Guest"
                        },
                        {
                            "typ": "Management"
                        },
                        {
                            "typ": "Public"
                        }
                    ],
                    "providers": [
                        {
                            "broadcastdomainrange": "ZONE",
                            "name": "VirtualRouter"
                        },
                        {
                            "broadcastdomainrange": "ZONE",
                            "name": "VpcVirtualRouter"
                        },
                        {
                            "broadcastdomainrange": "ZONE",
                            "name": "InternalLbVm"
                        }
                    ],
                    "isolationmethods": [
                             "VLAN"
                    ]
                }
            ],
            "ipranges": [
                {
                    "startip": "192.168.100.2",
                    "endip": "192.168.100.200",
                    "netmask": "255.255.255.0",
                    "vlan": "50",
                    "gateway": "192.168.100.1"
                }
            ],
            "networktype": "Advanced",
            "pods": [
                {
                    "endip": "172.16.100.200",
                    "name": "Z1P1",
                    "startip": "172.16.100.2",
                    "netmask": "255.255.255.0",
                    "clusters": [
                        {
                            "clustername": "Z1P1C1",
                            "hypervisor": "simulator",
                            "hosts": [
                                {
                                    "username": "root",
                                    "url": "http://sim1/c1/h1",
                                    "password": "password"
                                }
                            ],
                            "clustertype": "CloudManaged"
                        }
                    ],
                    "gateway": "172.16.100.1"
                }
            ],
            "internaldns1": "10.147.100.6",
            "primaryStorages": [
                {
                    "url": "nfs://10.147.100.6:/export/home/sandbox/z2p3",
                    "name": "Z1PS1",
                    "hypervisor": "Any"
                }
            ],
            "secondaryStorages": [
                {
                    "url": "nfs://10.147.100.6:/export/home/sandbox/z1secondary",
                    "provider" : "NFS"
                }
            ]
        }
    ],
    "dbSvr": {
        "dbSvr": "localhost",
        "passwd": "cloud",
        "db": "cloud",
        "port": 3306,
        "user": "cloud"
    },
    "logger":
        {
            "LogFolderPath": "/tmp"
        },
    "mgtSvr": [
        {
            "mgtSvrIp": "localhost",
            "passwd": "password",
            "user": "root",
            "port": 8096,
            "hypervisor": "simulator",
            "useHttps": "False",
            "certCAPath":  "NA",
            "certPath":  "NA"
        }
    ]
}

@DaanHoogland

Copy link
Copy Markdown
Contributor

it worked with "hypervisor": "Simulator" it cannot be "hypervisor": "Any"

Comment thread server/src/main/java/com/cloud/storage/StorageManagerImpl.java Outdated
@nvazquez

Copy link
Copy Markdown
Contributor Author

Thanks @GutoVeronezi @DaanHoogland
@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 2607

@sureshanaparti sureshanaparti left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code LGTM

@nvazquez

Copy link
Copy Markdown
Contributor Author

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-3339)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45871 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5986-t3339-kvm-centos7.zip
Smoke tests completed. 91 look OK, 1 have errors
Only failed tests results shown below:

Test Result Time (s) Test File
test_01_invalid_upgrade_kubernetes_cluster Failure 3605.98 test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster Failure 3608.56 test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster Failure 0.08 test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster Failure 0.04 test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_cluster Failure 0.04 test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster Failure 0.03 test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster Failure 0.04 test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster Failure 0.03 test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster Failure 0.03 test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown Error 91.14 test_kubernetes_clusters.py

@apache apache deleted a comment from blueorangutan Feb 16, 2022
@apache apache deleted a comment from blueorangutan Feb 16, 2022
@nvazquez

Copy link
Copy Markdown
Contributor Author

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian Build Failed (tid-3351)

@nvazquez

Copy link
Copy Markdown
Contributor Author

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-3358)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 38383 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5986-t3358-kvm-centos7.zip
Smoke tests completed. 86 look OK, 6 have errors
Only failed tests results shown below:

Test Result Time (s) Test File
test_01_add_primary_storage_disabled_host Error 0.58 test_primary_storage.py
test_01_primary_storage_nfs Error 0.10 test_primary_storage.py
ContextSuite context=TestStorageTags>:setup Error 0.18 test_primary_storage.py
test_01_invalid_upgrade_kubernetes_cluster Failure 3610.45 test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster Failure 34.98 test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster Failure 38.00 test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster Failure 35.98 test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_cluster Failure 40.13 test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster Failure 4.22 test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster Failure 4.21 test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster Failure 3.21 test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster Failure 4.29 test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown Error 123.81 test_kubernetes_clusters.py
test_01_secure_vm_migration Error 154.42 test_vm_life_cycle.py
test_02_unsecure_vm_migration Error 270.18 test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration Error 141.93 test_vm_life_cycle.py
test_08_migrate_vm Error 43.76 test_vm_life_cycle.py
test_01_nic Error 135.63 test_nic.py
test_02_list_snapshots_with_removed_data_store Error 9.48 test_snapshots.py
test_02_list_snapshots_with_removed_data_store Error 9.48 test_snapshots.py
test_hostha_enable_ha_when_host_in_maintenance Error 301.72 test_hostha_kvm.py

@nvazquez

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 2634

@nvazquez

Copy link
Copy Markdown
Contributor Author

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-3367)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31660 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5986-t3367-kvm-centos7.zip
Smoke tests completed. 91 look OK, 1 have errors
Only failed tests results shown below:

Test Result Time (s) Test File
test_01_nic Error 141.46 test_nic.py

@nvazquez

Copy link
Copy Markdown
Contributor Author

Merging based on approvals and test results, error not related to this PR

@nvazquez nvazquez merged commit ed30683 into apache:main Feb 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants