network is not setup correctly in vFW (Openstack Ocata)

 
1
0
-1

I went through the tutorial up to a point where I get the vFW stack provisioned in Openstack. As I cannot get to the port 667 of the sink, I had a look in the snk VM and discovered, that only eth0 is configured. I was able to add the config for the two other interfaces in vsn_vm.

Trying to do the same on vpg and vfw failed. I can see (with lspci) there are four network cards in vfw, but if I look in /proc/net/dev, I can only see eth0. Similar for vpg. Not sure this is general ubuntu vs. Openstack question, but maybe somebody in the community ran into the same issue and has a resolution for me.

  1. Josef Reisinger

    vsn

    ubuntu@demofwl01snk:~$ cat /proc/net/dev
    Inter-|   Receive                                                |  Transmit
     face |bytes    packets errs drop fifo frame compressed multicast|bytes    packets errs drop fifo colls carrier compressed
      eth0:   86221     880    0    0    0     0          0         0    61776     491    0    0    0     0       0          0
      eth1:     434       2    0    0    0     0          0         0      962       9    0    0    0     0       0          0
      eth2:  157478    3727    0    0    0     0          0         0      962       9    0    0    0     0       0          0
        lo:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
    ubuntu@demofwl01snk:~$ lspci
    00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
    00:04.0 Ethernet controller: Red Hat, Inc Virtio network device
    00:05.0 Ethernet controller: Red Hat, Inc Virtio network device

    vpg

    ubuntu@demofwl01pgn:~$ cat /proc/net/dev
    Inter-|   Receive                                                |  Transmit
     face |bytes    packets errs drop fifo frame compressed multicast|bytes    packets errs drop fifo colls carrier compressed
       br0:       0       0    0    0    0     0          0         0      648       8    0    0    0     0       0          0
    tap111:       0       0    0    0    0     0          0         0     1296      16    0    0    0     0       0          0
      eth0: 239510279  180090    0    0    0     0          0         0  4494227   60541    0    0    0     0       0          0
        lo:    1300      26    0    0    0     0          0         0     1300      26    0    0    0     0       0          0
    ubuntu@demofwl01pgn:~$ lspci
    00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
    00:04.0 Ethernet controller: Red Hat, Inc Virtio network device
    00:05.0 Ethernet controller: Red Hat, Inc Virtio network device



CommentAdd your comment...

1 answer

  1.  
    1
    0
    -1

    Josef Reisinger looks like openstack provided correct number of interfaces to KVM. By default vFW heat templates has no configuration script present to bring up the eth1, eth2 and eth3 interfaces. We need to add that like below


    user_data:
    str_replace:
    params:
    __dcae_collector_ip__ : { get_param: dcae_collector_ip }
    __dcae_collector_port__ : { get_param: dcae_collector_port }
    __repo_url_blob__ : { get_param: repo_url_blob }
    __repo_url_artifacts__ : { get_param: repo_url_artifacts }
    __demo_artifacts_version__ : { get_param: demo_artifacts_version }
    $vfw_private_0_port: { get_attr: [vfw_private_0_port, fixed_ips, 0, ip_address] }
    $vfw_private_1_port: { get_attr: [vfw_private_1_port, fixed_ips, 0, ip_address] }
    $vfw_private_2_port: { get_attr: [vfw_private_2_port, fixed_ips, 0, ip_address] }
    template: |
    #!/bin/bash

    DCAE_COLLECTOR_IP=__dcae_collector_ip__
    DCAE_COLLECTOR_PORT=__dcae_collector_port__
    REPO_URL_BLOB=__repo_url_blob__
    REPO_URL_ARTIFACTS=__repo_url_artifacts__
    DEMO_ARTIFACTS_VERSION=__demo_artifacts_version__

    echo $vfw_private_0_port
    echo "auto eth1
    iface eth1 inet static
    address $vfw_private_0_port
    netmask 255.255.255.0" >> /etc/network/interfaces.d/eth1.cfg
    ifconfig eth1 $vfw_private_0_port/24

    echo $vfw_private_1_port
    echo "auto eth2
    iface eth2 inet static
    address $vfw_private_1_port
    netmask 255.255.255.0" >> /etc/network/interfaces.d/eth2.cfg
    ifconfig eth2 $vfw_private_1_port/24

    echo $vfw_private_2_port
    echo "auto eth3
    iface eth3 inet static
    address $vfw_private_2_port
    netmask 255.255.255.0" >> /etc/network/interfaces.d/eth3.cfg
    ifconfig eth3 $vfw_private_2_port/8


    1. kranthi guttikonda

      Or add the configuration files in /etc/network/interfaces.d/ directory for each interface and reboot the VM or bring up each interface using "ifup"

    2. Josef Reisinger

      kranthi guttikonda, the issue is if you don't have the device in /proc/net/dev, you cannot ifup; You cannot even simply bring it up without config, because the network layer does not know about it.

      root@demofwl01pgn:/etc/network/interfaces.d# ifconfig eth1 up
eth1: ERROR while getting interface flags: No such device

      What I do not understand is that the "hardware" layer in the kernel identified three ethernet devices on the PCI bus, but only one of them is recognized by the network layer.


    3. kranthi guttikonda

      Josef Reisinger hmm. Ubuntu detects pci slots but no /proc entry. Never saw this issue. What does ls -lart /sys/devices/pci0000\:00/ show?

    4. Josef Reisinger

      kranthi guttikonda I never saw this issue too .. and what is confusing is that from three identical images one works, two not

      root@demofwl01pgn:~# ls -lart /sys/devices/pci0000\:00/
total 0
drwxr-xr-x  3 root root    0 Jun 22 16:44 pci_bus
drwxr-xr-x  4 root root    0 Jun 22 16:44 0000:00:07.0
drwxr-xr-x  5 root root    0 Jun 22 16:44 0000:00:06.0
drwxr-xr-x  3 root root    0 Jun 22 16:44 0000:00:05.0
drwxr-xr-x  3 root root    0 Jun 22 16:44 0000:00:04.0
drwxr-xr-x  5 root root    0 Jun 22 16:44 0000:00:03.0
drwxr-xr-x  3 root root    0 Jun 22 16:44 0000:00:02.0
drwxr-xr-x  3 root root    0 Jun 22 16:44 0000:00:01.3
drwxr-xr-x  4 root root    0 Jun 22 16:44 0000:00:01.2
drwxr-xr-x  5 root root    0 Jun 22 16:44 0000:00:01.1
drwxr-xr-x  3 root root    0 Jun 22 16:44 0000:00:01.0
drwxr-xr-x  3 root root    0 Jun 22 16:44 0000:00:00.0
drwxr-xr-x 12 root root    0 Jun 22 16:44 ..
drwxr-xr-x 15 root root    0 Jun 22 16:44 .
-rw-r--r--  1 root root 4096 Jun 22 16:44 uevent
drwxr-xr-x  2 root root    0 Jun 22 16:49 power
lrwxrwxrwx  1 root root    0 Jun 22 16:49 firmware_node -> ../LNXSYSTM:00/device:00/PNP0A03:00
    5. kranthi guttikonda

      Josef Reisinger Totally weird. /sys/devices/ looks fine. Just do a blind way once. try rebooting the problematic VMs and see if the problem disappears. 

    6. Josef Reisinger

      kranthi guttikonda, thanks for your continous replies. Well appreciated

      I've done a reboot before I did the output above.

      Another observation:

      I have two ONAP instances running, one of a Pureflex in our lab, one in a Softlayer Data center. They are hardware-wise pretty different from each other, on both run same Openstack Ocata, and on both I have the same symptom: snk network is OK, the other two not. I just rebuild the pg & fw VMs from the 14.0 image on the Softlayer instance and voila... there the interfaces are. I assume the rebuild does not invoke the cloud init scripts, so I guess there is something very weird happening in cloud init.

      I thought I could see that the VMs missing the additional interfaces have a high number of

      [  125.791824] igb_uio: Unknown symbol __uio_register_device (err 0)
[  125.791845] igb_uio: Unknown symbol uio_unregister_device (err 0)

      in the output of dmesg.

      What I see now are the above error messages in pg & fw on Softlayer, but not getting the configuration from dhcp:


      In summary: snk run in both environments out-of-the-box (after adding .cfg files), in both  environments, for fw & pg the additional eth devices are known as hardware, but not to the network layer. If I rebuild the image for pg & fw in the Softlayer env, I can see the eth devices in network layer, but they do not reliably get their config from DHCP. I would assume the  same on the Pureflex, will try this tomorrow.

    7. Josef Reisinger

      Yet another observation: I extracted the HEAT template for the vFW from the zip file i used to create the Software Product in SCP and created a stack with openstack cli. All three VMs have all expected interfaces visible.

      Something must be strange with the stack creation via MSO. I'll file a JIRA tomorrow, it is late over here...

      kranthi guttikonda, once again thanks for your attention.

    CommentAdd your comment...