Only_Dead_Fish_Go_With_The_Flow: August 2011

Monday, August 29, 2011

Be strong. Be alive. Be an example. Be a leader.

Here is a short video by Nick Vujicic.

So, do not be sad.

Do not complain.

Do not consider you to be unlucky.

Do not give up.

Learn.

Many people need your help.

Enjoy every single moment.

Because there are so many beautiful things.

Life is beautiful.

Marco ( @mgua )

PS: the Video is not shown in some places. Check this one instead.
http://www.youtube.com/watch?v=wOlTdkYXuzE

.

Thursday, August 25, 2011

Network interface bonding and trunking VLANs with different MTUs on RedHat Linux

(Last updated on aug 30 2011)

Configuration of Red Hat Enterprise Linux Server 6.1 (Hardware: HP DL380G7, 2 CPU Intel core) connected with two 10Gbit/sec ethernet interfaces (HP NC522SFP Dual Port 10GbE Server Adapter Board), with interface bonding, high availability, bandwidth aggregation and multiple VLAN transport.

Two 10Gbit/s physical network interfaces have to be connected to two FEX (Nexus Fabric EXtension "switches") upstreaming to two different Nexus 7000 cores, in a high availability configuration, allowing also to use both links simultaneously, and to transport different VLANs.
The different VLANs have then to be mapped on different linux logical subinterfaces.

The two physical interface are joined to form a bond, called bond0.
A specific channel configuration exists on the cisco nexus switches, so to allow these ports to form a single trunk channel, and to transport on it tagged frames from the selected VLANs, complying to the 802.1q transport.

Here are the settings on the Cisco Nexus Side (only one "side" is shown, the other is symmetric):

Core to FEX2232

  interface port-channel131
    switchport
    switchport mode fex-fabric
    fex associate 131
    mtu 9216



  FEX to server

  interface Ethernet131/1/5
    description srv
    switchport
    switchport mode trunk
    switchport trunk allowed vlan 10-13
    flowcontrol send off
    channel-group 88 mode active
    no shutdown

On the RedHat linux side, my two physical interfaces are eth0 and eth2. I check with ethtool that on both I can see the link, and that the speed and duplex settings are ok (autonegotiation has been disabled both on the switch side and on the server).

The two physical interfaces are joined in bond0

Here are the configuration files of the physical and logical interfaces, in /etc/sysconfig/network-scripts
bond0.10 and bond0.11 are the two logical interfaces sitting on VLANs 10 and 11
bond0.10 has MTU 1500
bond0.11 has MTU 9000 so to allow more efficient traffic of large blocks of data on the storage VLAN

-------
[root@srv network-scripts]# cat ifcfg-eth0
DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
HWADDR=78:e3:b5:f4:e0:50
TYPE=Ethernet
IPV6INIT=no

[root@srv network-scripts]# cat ifcfg-eth2
DEVICE=eth2
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
HWADDR=78:E3:B5:F4:76:E0
TYPE=Ethernet
IPV6INIT=no


[root@srv network-scripts]# cat ifcfg-bond0
DEVICE=bond0
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
BONDING_OPTS="mode=4 miimon=100"
IPV6INIT=no
USERCTL=no
MTU=9000


[root@srv network-scripts]# cat ifcfg-bond0.10
IPADDR=192.168.145.248
NETMASK=255.255.254.0
DEVICE=bond0.10
ONBOOT=yes
VLAN=yes
TYPE=Ethernet
HWADDR=78:e3:b5:f4:e0:50
BOOTPROTO=none
GATEWAY=192.168.144.1
IPV6INIT=no
USERCTL=no
MTU=1500

[root@srv network-scripts]# cat ifcfg-bond0.11
IPADDR=192.168.148.85
NETMASK=255.255.255.0
DEVICE=bond0.11
ONBOOT=yes
VLAN=yes
TYPE=Ethernet
BOOTPROTO=none
IPV6INIT=no
USERCTL=no
MTU=9000




[root@srv network-scripts]# ifconfig eth0 && ifconfig eth2 && ifconfig bond0 && ifconfig bond0.10 && ifconfig bond0.11
eth0      Link encap:Ethernet  HWaddr 78:E3:B5:F4:E0:50
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1
          RX packets:30750 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12992 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3590043 (3.4 MiB)  TX bytes:3577302 (3.4 MiB)
          Interrupt:53

eth2      Link encap:Ethernet  HWaddr 78:E3:B5:F4:E0:50
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1
          RX packets:36472 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7248 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:27483898 (26.2 MiB)  TX bytes:906669 (885.4 KiB)
          Interrupt:61

bond0     Link encap:Ethernet  HWaddr 78:E3:B5:F4:E0:50
          inet6 addr: fe80::7ae3:b5ff:fef4:e050/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:9000  Metric:1
          RX packets:67224 errors:0 dropped:0 overruns:0 frame:0
          TX packets:20242 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:31074081 (29.6 MiB)  TX bytes:4484967 (4.2 MiB)

bond0.10  Link encap:Ethernet  HWaddr 78:E3:B5:F4:E0:50
          inet addr:192.168.145.248  Bcast:192.168.145.255  Mask:255.255.254.0
          inet6 addr: fe80::7ae3:b5ff:fef4:e050/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:51880 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13135 errors:0 dropped:10 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:8963296 (8.5 MiB)  TX bytes:3972360 (3.7 MiB)

bond0.11  Link encap:Ethernet  HWaddr 78:E3:B5:F4:E0:50
          inet addr:192.168.148.85  Bcast:192.168.148.255  Mask:255.255.255.0
          inet6 addr: fe80::7ae3:b5ff:fef4:e050/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:9000  Metric:1
          RX packets:14599 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6891 errors:0 dropped:7 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:20770093 (19.8 MiB)  TX bytes:486535 (475.1 KiB)



[root@srv network-scripts]# ethtool eth0
Settings for eth0:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supports auto-negotiation: No
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Speed: 10000Mb/s
        Duplex: Full
        Port: FIBRE
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: off
        Supports Wake-on: g
        Wake-on: g
        Current message level: 0x00000005 (5)
        Link detected: yes

[root@srv network-scripts]# ethtool eth2
Settings for eth2:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supports auto-negotiation: No
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Speed: 10000Mb/s
        Duplex: Full
        Port: FIBRE
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: off
        Supports Wake-on: g
        Wake-on: g
        Current message level: 0x00000005 (5)
        Link detected: yes


-------

With this configurations, the server keeps correctly the configuration at reboot, and all the interfaces have the correct MTU sizes.

The definition of the MTU is only needed in the bond0 and in the proper subinterface. The physical interfaces do not have any MTU settings attached.

The behaviour of ethtool command appeared consistent. mii-tool was not.

ethtool is the command to be used.

Performance metering:
With this setup, I was able to squeeze an amazing end-to-end tcp speed of 9.8Gbit/sec

here is the nuttcp test output.

[root@srv network-scripts]# nuttcp -t -v -T 60 192.168.148.85
  nuttcp-t: v6.1.2: socket
  nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> 192.168.148.85
  nuttcp-t: time limit = 60.00 seconds
  nuttcp-t: connect to 192.168.148.85 with mss=8948, RTT=0.277 ms
  nuttcp-t: send window size = 28440, receive window size = 87380
  nuttcp-t: available send window = 21330, available receive window = 65535
  nuttcp-t: 70086.0000 MB in 60.00 real seconds = 1196131.87 KB/sec = 9798.7123 Mbps
  nuttcp-t: retrans = 0
  nuttcp-t: 1121376 I/O calls, msec/call = 0.05, calls/sec = 18689.56
  nuttcp-t: 0.2user 29.7sys 1:00real 49% 0i+0d 468maxrss 0+2pf 25844+75csw
 
  nuttcp-r: v6.1.2: socket
  nuttcp-r: buflen=65536, nstream=1, port=5001 tcp
  nuttcp-r: accept from 192.168.148.86
  nuttcp-r: send window size = 28440, receive window size = 87380
  nuttcp-r: available send window = 21330, available receive window = 65535
  nuttcp-r: 70086.0000 MB in 60.00 real seconds = 1196065.83 KB/sec = 9798.1712 Mbps
  nuttcp-r: 2319818 I/O calls, msec/call = 0.03, calls/sec = 38661.42
  nuttcp-r: 0.4user 33.9sys 1:00real 57% 0i+0d 330maxrss 0+17pf 1103994+76csw
  [root@srv network-scripts]#

which is a quite good result!
during the nuttcp test, the servers were having about a 50KHz interrupt rate and a very low average load.

here is the vmstat 1 output of the transmitting server during the test:

  procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
   r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
   0  0      0 15100956  44948 525388    0    0     0     0  280  152  0  0 100  0  0
   0  0      0 15100956  44948 525388    0    0     0     0  159   73  0  0 100  0  0
   1  0      0 15096676  44948 525388    0    0     0     0 44407  536  0  2 98  0  0
   1  0      0 15097296  44956 525384    0    0     0    44 51087  455  0  6 94  0  0
   1  0      0 15097792  44956 525388    0    0     0     0 51025  556  0  2 98  0  0
   1  0      0 15098056  44956 525388    0    0     0     0 50067  656  0  7 93  0  0
   1  0      0 15098304  44956 525388    0    0     0     0 49978  774  0  2 98  0  0
   1  0      0 15098676  44956 525388    0    0     0     0 50264  769  0  6 94  0  0
   1  0      0 15098676  44956 525388    0    0     0  1076 50428  843  0  2 98  0  0
   1  0      0 15098676  44956 525388    0    0     0     0 50832 1202  0  5 95  0  0
   0  0      0 15099296  44956 525388    0    0     0     0 51193 1350  0  2 98  0  0
   1  0      0 15099544  44956 525388    0    0     0     0 51228 1198  0  7 93  0  0
   0  0      0 15102948  44956 525388    0    0     0     0 6756  681  0  0 100  0  0
   0  0      0 15103072  44964 525380    0    0     0   360  684  462  0  0 100  0  0
   0  0      0 15103400  44964 525388    0    0     0     0  234  147  0  0 100  0  0

Marco ( @mgua )

Friday, August 19, 2011

Youth

Youth, by Samuel Ullman (1840-1924)

Youth is not a time of life; it is a state of mind; it is not a matter of rosy cheeks, red lips and supple knees; it is a matter of the will, a quality of the imagination, a vigor of the emotions; it is the freshness of the deep springs of life.

   Youth means a temperamental predominance of courage over timidity of the appetite, for adventure over the love of ease. This often exists in a man of sixty more than a body of twenty. Nobody grows old merely by a number of years. We grow old by deserting our ideals.

   Years may wrinkle the skin, but to give up enthusiasm wrinkles the soul. Worry, fear, self-distrust bows the heart and turns the spirit back to dust.

   Whether sixty or sixteen, there is in every human being's heart the lure of wonder, the unfailing child-like appetite of what's next, and the joy of the game of living. In the center of your heart and my heart there is a wireless station; so long as it receives messages of beauty, hope, cheer, courage and power from men and from the Infinite, so long are you young.

   When the aerials are down, and your spirit is covered with snows of cynicism and the ice of pessimism, then you are grown old, even at twenty, but as long as your aerials are up, to catch the waves of optimism, there is hope you may die young at eighty.