So, this will be a multi-part series documenting my adventures in deploying OpenStack Mitaka on Ubuntu 14.04 leveraging EMC’s ScaleIO storage (along with some iSCSI sitting on ZFS). This post is meant to document how I’ve set up my environment, and any issues i’ve encountered.<\/p>\n
The environment:<\/strong><\/p>\n I have 6 total servers in the environment all connected via Mellanox 20Gb\/s Infiniband and 1Gb\/s IP.<\/p>\n Each of these servers contains an identical base build at this point.<\/p>\n The Setup:<\/strong><\/p>\n When installing Ubuntu, the only server role i’ve given them is OpenSSH server. Once the base install is complete, I’ve installed the opensm package in order to bring up the Subnet Manager in the IB network, and I’ve loaded the appropriate modules for my environment:<\/p>\n Once the OpenSM package is installed, I load the appropriate modules:<\/p>\n I also add these modules to the existing \/etc\/modules file so it looks like this:<\/p>\n Now that I have my modules loaded, I can set up my network interfaces for both my OpenStack provider network, and my IB storage network. I edit the \/etc\/network\/interfaces file to this:<\/p>\n Once the \/etc\/network\/interfaces file is ready, simply bring up the eth1 and ib0 interfaces to ensure they’re operational.<\/p>\n Next we want to update our packages using apt-get. DO NOT upgrade the Kernel just yet.<\/p>\n Now, we want to update the Kernel to the latest SUPPORTED Kernel for ScaleIO. as of this post, the latest SUPPORTED for Ubuntu 14.04 is 4.2.0-30. If you blindly do a apt-get dist-upgrade, you will get a newer kernel than what is supported. This is BAD, the ScaleIO Kernel module for the SDC’s will not load. To get around this, we will install the 4.2.0-30 kernel, then ensure that apt-get HOLDS at that level.<\/p>\n nbsp;sudo apt-get install linux-headers-4.2.0-30-generic linux-headers-4.2.0-30 linux-image-4.2.0-30-generic linux-image-extra-4.2.0-30-generic <\/code><\/p>\n Once the kernel install completes, we will set apt-get to HOLD at this level<\/p>\n REBOOT<\/strong><\/p>\n **NOTE: There doesn’t appear to be a similar way to hold packages with the apt command. I know there’s a way to hold packages with dpkg, but I havent tested this with apt.<\/em><\/p>\n Installing components necessary for ScaleIO<\/strong><\/p>\n There’s not a lot in terms of prerequisites fro ScaleIO on Ubuntu. we need to install Java 8, numactl, libaio, and bash-completion.<\/p>\n Do this on each MDM, SDS, and SDC node you plan to use in your environment.<\/p>\n Additionally, it appears that you need to enable the root account and enable it to login via SSH for the Gateway installer to work. I don’t believe this is a requirement for a manual install.<\/p>\n Allow root to login via ssh by modifying the \/etc\/ssh\/sshd_config file with the following parameter:<\/p>\n Restart the ssh service.<\/p>\n Repeat this fro each nod you will be using as an MDM, SDS, or SDC in your environment.<\/p>\n You are now ready to begin deploying ScaleIO. I will cover that in Part 2<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":" So, this will be a multi-part series documenting my adventures in deploying OpenStack Mitaka on Ubuntu 14.04 leveraging EMC’s ScaleIO storage (along with some iSCSI sitting on ZFS). This post is meant to document how I’ve set up my environment, and any issues i’ve encountered. The environment: I have 6 total servers in the environment … Continue reading Adventures in ScaleIO and OpenStack – Part 1 – Base OS setup<\/span> \n
$ sudo apt-get install opensm<\/code><\/pre>\n$ sudo modprobe mlx4_ib\n$ sudo modprobe ib_umad\n$ sudo modprobe ib_ipoib<\/code><\/pre>\n$ cat \/etc\/modules\n# \/etc\/modules: kernel modules to load at boot time.\n#\n# This file contains the names of kernel modules that should be loaded\n# at boot time, one per line. Lines beginning with \"#\" are ignored.\n# Parameters can be specified after the module name.\n\nlp\nrtc\nmlx4_ib\nib_umad\nib_ipoib<\/code><\/pre>\n# This file describes the network interfaces available on your system\n# and how to activate them. For more information, see interfaces(5).\n# The loopback network interface\nauto lo\niface lo inet loopback\n\n# The Management network interface\nauto eth0\niface eth0 inet static\n address 10.0.0.10\/24\n gateway 10.0.0.1\n dns-nameservers 10.0.0.2\n dns-search battledome.lcl\n\n# The Provider network Interface\nauto eth1\niface eth1 inet manual\n up ip link set dev $IFACE up\n down ip link set dev $IFACE down\n\n# The Infiniband Network\n auto ib0\n iface ib0 inet static\n address 10.1.1.10\/24\n post-up echo connected > \/sys\/class\/net\/ib0\/mode\n post-up ifconfig ib0 mtu 65520<\/code><\/pre>\n$ sudo ifup eth1 && sudo ifup ib0<\/code><\/pre>\n$ sudo apt-get update\n$sudo apt-get upgrade<\/code><\/pre>\n\n<\/pre>\n
$ sudo apt-mark hold install linux-headers-4.2.0-30-generic linux-headers-4.2.0-30 linux-image-4.2.0-30-generic linux-image-extra-4.2.0-30-generic <\/code><\/pre>\n$ sudo add-apt-repository ppa:webupd8team\/java\n$ sudo apt-get update\n$ sudo apt-get install oracle-java8-installer\n$ sudo apt-get install numactl libaio1 bash-comletion <\/code><\/pre>\n$ sudo passwd root<\/code><\/pre>\nPermitRootLogin yes<\/code><\/pre>\n$ sudo service ssh restart<\/code><\/pre>\n