跳转至

使用 packstack 在 CentOS 8 上部署 OpenStack 集群

文档时效性说明

本文为早期笔记，可能存在版本过时、命令失效、链接失效、最佳实践变化等问题。请以官方最新文档为准。

原英文标题：Deploying OpenStack Cluster on CentOS 8 with packstack

## 环境描述

需要 5 台可访问外网的 CentOS 8.3 服务器。下表描述了硬件需求。

角色 主机名 IP 地址 CPU 内存 磁盘
Controller controller 192.168.8.10 2vCPUs 8GB 100GB
Compute compute01 192.168.8.20 4vCPUs 16GB 200GB
Compute compute02 192.168.8.21 4vCPUs 16GB 200GB
Network network 192.168.8.30 2vCPUs 8GB 100GB
NTP ntp 192.168.8.40 1vCPU 2GB 20GB

网络配置如下

  • 网关 IP 地址: 192.168.8.1
  • 子网地址: 192.168.8.0/24
  • 可用 IP 地址: 192.168.8.50 - 150
  • DNS 服务器: 192.168.8.1

1. 设置静态 IP 地址

在所有主机上

1
2
[root@192 ~]# nmcli connection modify ens33 ipv4.method manual ipv4.dns 192.168.8.1 ipv4.gateway 192.168.8.1 autoconnect yes ipv4.addresses <ip_addr>/24
[root@192 ~]# nmcli connection up ens33

2. 配置主机名

编写 /etc/hosts

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Static table lookup for hostnames.
# See hosts(5) for details.
#
127.0.0.1    localhost
::1          localhost

192.168.0.10 controller
192.168.0.20 compute01
192.168.0.21 compute02
192.168.0.30 network
192.168.0.40 ntp

将 /etc/hosts 分发到所有主机

1
for server in {controller,compute01,compute02,network,ntp};do scp /etc/hosts root@$server:/etc/hosts;done

设置主机名

1
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "hostnamectl set-hostname $server";done

3. 禁用防火墙、SELinux、NetworkManager

在所有主机上

1
2
3
4
5
6
7
8
9
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "systemctl disable --now firewalld";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "systemctl mask firewalld";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "sed -i 's/SELINUX=enforcing/SELinux=disabled/' /etc/sysconfig/selinux";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "sed -i 's/SELINUX=enforcing/SELinux=disabled/' /etc/selinux/config";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "setenforce 0";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "dnf install -y network-scripts";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "systemctl disable --now NetworkManager";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "systemctl enable --now network";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "systemctl reboot";done

4. 启用 NTP

在 ntp 主机上，编辑 /etc/chrony.conf

1
2
# Allow NTP client access from local network.
allow 192.168.8.0/24

在其他服务器上，编辑 /etc/chrony.conf

1
2
3
4
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
# pool 2.centos.pool.ntp.org iburst
pool ntp iburst

在所有主机上

1
2
systemctl enable --now chronyd.service
systemctl restart chronyd.service

验证

1
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "chronyc sources";done

5. 安装 OpenStack 软件源

在所有 OpenStack 节点上

1
2
3
4
5
6
for server in {controller,compute01,compute02,network};do ssh root@$server "dnf config-manager --enable powertools";done
for server in {controller,compute01,compute02,network};do ssh root@$server "dnf remove -y epel-release";done
for server in {controller,compute01,compute02,network};do ssh root@$server "sudo dnf install -y https://www.rdoproject.org/repos/rdo-release.el8.rpm";done
# Not needed but rdo project recommend to install
# for server in {controller,compute01,compute02,network};do ssh root@$server "sudo dnf install -y centos-release-openstack-victoria";done
for server in {controller,compute01,compute02,network};do ssh root@$server "dnf -y update";done

6. 安装 Packstack

在 controller 上

1
dnf install -y openstack-packstack

7. 生成应答文件

在 controller 上

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
packstack \
--os-neutron-ml2-tenant-network-types=vxlan \
--os-neutron-l2-agent=openvswitch \
--os-neutron-ml2-type-drivers=vxlan,flat \
--os-neutron-ml2-mechanism-drivers=openvswitch \
--keystone-admin-passwd=redhat \
--nova-libvirt-virt-type=kvm \
--provision-demo=n \
--cinder-volumes-create=y \
--os-heat-install=y \
--os-swift-storage-size=20G \
--gen-answer-file /root/answers.txt

本例中，admin 密码设置为 redhat，编辑 /root/answers.txt

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Specify 'y' if you want to run OpenStack services in debug mode;
# otherwise, specify 'n'. ['y', 'n']
CONFIG_DEBUG_MODE=y

# Server on which to install OpenStack services specific to the
# controller role (for example, API servers or dashboard).
CONFIG_CONTROLLER_HOST=192.168.0.10

# List the servers on which to install the Compute service.
CONFIG_COMPUTE_HOSTS=192.168.0.20,192.168.0.21

# List of servers on which to install the network service such as
# Compute networking (nova network) or OpenStack Networking (neutron).
CONFIG_NETWORK_HOSTS=192.168.0.30

8. 使用应答文件部署

在 controller 上

1
packstack --answer-file /root/answers.txt --timeout=3000

9. 允许非管理员用户查看实例状态

在 controller 上，编辑 /etc/nova/policy.json，添加

1
"os_compute_api:os-extended-server-attributes": ""

另请参见 https://docs.openstack.org/nova/latest/configuration/policy.html

10. 添加 bash 补全

1
openstack complete | sudo tee /etc/bash_completion.d/osc.bash_completion > /dev/null

11. 将网桥接入外部网络

在 network 节点上，编辑 /etc/sysconfig/network-scripts/ifcfg-ens33

1
2
3
4
5
6
7
TYPE=OVSPort
NAME=ens33
DEVICE=ens33
ONBOOT=yes
TYPE=OVSPort
DEVICETYPE=ovs
OVS_BRIDGE=br-ex

创建 /etc/sysconfig/network-scripts/ifcfg-br-ex

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
DEVICE=br-ex
BOOTPROTO=none
ONBOOT=yes
TYPE=OVSBridge
DEVICETYPE=ovs
USERCTL=yes
PEERDNS=yes
IPV6INIT=no
IPADDR=192.168.8.10
PREFIX=24
GATEWAY=192.168.8.1
DNS1=192.168.8.1

重启网络

1
systemctl restart network

12. 修改 noVNC 地址

在所有 compute 节点上，编辑 /etc/nova/nova.conf

1
server_proxyclient_address=<compute_node_ip>

重启服务

1
systemctl restart openstack-nova-compute.service

13. 测试部署

在 controller 上

1
source ~/keystonerc_admin

查看所有服务

1
openstack service list

创建私有网络

1
openstack network create private

为私有网络创建子网

1
2
3
4
5
6
openstack subnet create \
--network private \
--allocation-pool start=172.10.10.50,end=172.10.10.200 \
--dns-nameserver 223.5.5.5 \
--dns-nameserver 223.6.6.6 \
--subnet-range 172.10.10.0/24 private_subnet

创建公共网络

1
2
3
4
openstack network create \
--provider-network-type flat \
--provider-physical-network extnet \
--external public

为公共网络创建子网

1
2
3
4
5
openstack subnet create \
--network public \
--allocation-pool start=192.168.8.50,end=192.168.8.150 \
--no-dhcp \
--subnet-range 192.168.8.0/24 public_subnet

创建路由器

1
openstack router create private_router

添加路由器网关

1
openstack router set --external-gateway public private_router

将路由器关联到子网

1
openstack router add subnet private_router private_subnet

在 network 节点上，查看路由器信息，运行

1
ip netns show

检查连通性，运行

1
ip netns exec qrouter-<your_router_uuid_above> ping -c 1 baidu.com

在 controller 上，启动一个实例

1
2
3
mkdir images
cd images
wget http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img

上传镜像

1
2
3
4
openstack image create \
--disk-format qcow2 \
--container-format bare --public \
--file ./cirros-0.5.1-x86_64-disk.img "Cirros-0.5.1"

列出镜像

1
openstack image list

创建安全组

1
2
3
openstack security group create permit_all --description "Allow all ports"
openstack security group rule create --protocol TCP --dst-port 1:65535 --remote-ip 0.0.0.0/0 permit_all
openstack security group rule create --protocol ICMP --remote-ip 0.0.0.0/0 permit_all

列出安全组

1
openstack security group list

上传 SSH 密钥

1
openstack keypair create --public-key ~/.ssh/id_rsa.pub admin

列出 SSH 密钥

1
openstack keypair list

列出规格

1
openstack flavor list

启动实例

1
2
3
4
5
6
7
openstack server create \
--flavor m1.tiny \
--image "Cirros-0.5.1" \
--network private \
--key-name admin \
--security-group permit_all \
mycirros

列出所有服务器

1
openstack server list

分配浮动 IP

1
openstack floating ip create public

列出所有浮动 IP

1
openstack floating ip list

将浮动 IP 分配给实例

1
openstack server add floating ip mycirros 192.168.8.60

登录到实例

1
ssh cirros@192.168.8.60

14. 原文（English）

1. Deploying OpenStack Cluster on CentOS 8 with packstack

文档时效性说明

本文为早期笔记，可能存在版本过时、命令失效、链接失效、最佳实践变化等问题。请以官方最新文档为准。

## Environment Description

5 CentOS 8.3 with external network access is needed. The following table describes the hardware requirements.

Role hostname IP address CPU RAM Disk
Controller controller 192.168.8.10 2vCPUs 8GB 100GB
Compute compute01 192.168.8.20 4vCPUs 16GB 200GB
Compute compute02 192.168.8.21 4vCPUs 16GB 200GB
Network network 192.168.8.30 2vCPUs 8GB 100GB
NTP ntp 192.168.8.40 1vCPU 2GB 20GB

Network configuration is described below

  • Gateway IP Address: 192.168.8.1
  • Subnet Address: 192.168.8.0/24
  • Available IP Address: 192.168.8.50 - 150
  • DNS Server: 192.168.8.1

1.1 Set Static IP Address

On all hosts

1
2
[root@192 ~]# nmcli connection modify ens33 ipv4.method manual ipv4.dns 192.168.8.1 ipv4.gateway 192.168.8.1 autoconnect yes ipv4.addresses <ip_addr>/24
[root@192 ~]# nmcli connection up ens33

1.2 Configure Hostname

Write /etc/hosts

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Static table lookup for hostnames.
# See hosts(5) for details.
#
127.0.0.1    localhost
::1          localhost

192.168.0.10 controller
192.168.0.20 compute01
192.168.0.21 compute02
192.168.0.30 network
192.168.0.40 ntp

Distribute /etc/hosts to all hosts

1
for server in {controller,compute01,compute02,network,ntp};do scp /etc/hosts root@$server:/etc/hosts;done

Set hostname

1
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "hostnamectl set-hostname $server";done

1.3 Disable Firewall, SELinux, NetworkManager

On all hosts

1
2
3
4
5
6
7
8
9
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "systemctl disable --now firewalld";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "systemctl mask firewalld";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "sed -i 's/SELINUX=enforcing/SELinux=disabled/' /etc/sysconfig/selinux";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "sed -i 's/SELINUX=enforcing/SELinux=disabled/' /etc/selinux/config";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "setenforce 0";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "dnf install -y network-scripts";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "systemctl disable --now NetworkManager";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "systemctl enable --now network";done
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "systemctl reboot";done

1.4 Enable NTP

On ntp, edit /etc/chrony.conf

1
2
# Allow NTP client access from local network.
allow 192.168.8.0/24

On other servers, edit /etc/chrony.conf

1
2
3
4
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
# pool 2.centos.pool.ntp.org iburst
pool ntp iburst

On all hosts

1
2
systemctl enable --now chronyd.service
systemctl restart chronyd.service

Verify

1
for server in {controller,compute01,compute02,network,ntp};do ssh root@$server "chronyc sources";done

1.5 Install OpenStack Repository

On all OpenStack attenders

1
2
3
4
5
6
for server in {controller,compute01,compute02,network};do ssh root@$server "dnf config-manager --enable powertools";done
for server in {controller,compute01,compute02,network};do ssh root@$server "dnf remove -y epel-release";done
for server in {controller,compute01,compute02,network};do ssh root@$server "sudo dnf install -y https://www.rdoproject.org/repos/rdo-release.el8.rpm";done
# Not needed but rdo project recommend to install
# for server in {controller,compute01,compute02,network};do ssh root@$server "sudo dnf install -y centos-release-openstack-victoria";done
for server in {controller,compute01,compute02,network};do ssh root@$server "dnf -y update";done

1.6 Install Packstack

On controller

1
dnf install -y openstack-packstack

1.7 Generate answer file

On controller

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
packstack \
--os-neutron-ml2-tenant-network-types=vxlan \
--os-neutron-l2-agent=openvswitch \
--os-neutron-ml2-type-drivers=vxlan,flat \
--os-neutron-ml2-mechanism-drivers=openvswitch \
--keystone-admin-passwd=redhat \
--nova-libvirt-virt-type=kvm \
--provision-demo=n \
--cinder-volumes-create=y \
--os-heat-install=y \
--os-swift-storage-size=20G \
--gen-answer-file /root/answers.txt

in this case, admin password is set to redhat, Edit /root/answers.txt

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Specify 'y' if you want to run OpenStack services in debug mode;
# otherwise, specify 'n'. ['y', 'n']
CONFIG_DEBUG_MODE=y

# Server on which to install OpenStack services specific to the
# controller role (for example, API servers or dashboard).
CONFIG_CONTROLLER_HOST=192.168.0.10

# List the servers on which to install the Compute service.
CONFIG_COMPUTE_HOSTS=192.168.0.20,192.168.0.21

# List of servers on which to install the network service such as
# Compute networking (nova network) or OpenStack Networking (neutron).
CONFIG_NETWORK_HOSTS=192.168.0.30

1.8 Deploy with answer file

on controller

1
packstack --answer-file /root/answers.txt --timeout=3000

1.9 Permit Non-admin User to View Instance Status

on controller, edit /etc/nova/policy.json, add

1
"os_compute_api:os-extended-server-attributes": ""

See also https://docs.openstack.org/nova/latest/configuration/policy.html

1.10 Add bash-completion

1
openstack complete | sudo tee /etc/bash_completion.d/osc.bash_completion > /dev/null

1.11 Bridge Network to External

on network node, edit /etc/sysconfig/network-scripts/ifcfg-ens33

1
2
3
4
5
6
7
TYPE=OVSPort
NAME=ens33
DEVICE=ens33
ONBOOT=yes
TYPE=OVSPort
DEVICETYPE=ovs
OVS_BRIDGE=br-ex

create /etc/sysconfig/network-scripts/ifcfg-br-ex

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
DEVICE=br-ex
BOOTPROTO=none
ONBOOT=yes
TYPE=OVSBridge
DEVICETYPE=ovs
USERCTL=yes
PEERDNS=yes
IPV6INIT=no
IPADDR=192.168.8.10
PREFIX=24
GATEWAY=192.168.8.1
DNS1=192.168.8.1

restart network

1
systemctl restart network

1.12 Modify noVNC Address

on all compute nodes, edit /etc/nova/nova.conf

1
server_proxyclient_address=<compute_node_ip>

restart service

1
systemctl restart openstack-nova-compute.service

1.13 Test the deployment

on controller

1
source ~/keystonerc_admin

see all services

1
openstack service list

create private network

1
openstack network create private

create subnet for private network

1
2
3
4
5
6
openstack subnet create \
--network private \
--allocation-pool start=172.10.10.50,end=172.10.10.200 \
--dns-nameserver 223.5.5.5 \
--dns-nameserver 223.6.6.6 \
--subnet-range 172.10.10.0/24 private_subnet

create public network

1
2
3
4
openstack network create \
--provider-network-type flat \
--provider-physical-network extnet \
--external public

create subnet for public network

1
2
3
4
5
openstack subnet create \
--network public \
--allocation-pool start=192.168.8.50,end=192.168.8.150 \
--no-dhcp \
--subnet-range 192.168.8.0/24 public_subnet

create a router

1
openstack router create private_router

add router gateway

1
openstack router set --external-gateway public private_router

link router to subnet

1
openstack router add subnet private_router private_subnet

on network node, to show router information, run

1
ip netns show

to check connectivity, run

1
ip netns exec qrouter-<your_router_uuid_above> ping -c 1 baidu.com

on controller, fire up an instance

1
2
3
mkdir images
cd images
wget http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img

upload image

1
2
3
4
openstack image create \
--disk-format qcow2 \
--container-format bare --public \
--file ./cirros-0.5.1-x86_64-disk.img "Cirros-0.5.1"

list images

1
openstack image list

create a security group

1
2
3
openstack security group create permit_all --description "Allow all ports"
openstack security group rule create --protocol TCP --dst-port 1:65535 --remote-ip 0.0.0.0/0 permit_all
openstack security group rule create --protocol ICMP --remote-ip 0.0.0.0/0 permit_all

list security group

1
openstack security group list

upload a ssh key

1
openstack keypair create --public-key ~/.ssh/id_rsa.pub admin

list ssh keys

1
openstack keypair list

list flavors

1
openstack flavor list

launch instance

1
2
3
4
5
6
7
openstack server create \
--flavor m1.tiny \
--image "Cirros-0.5.1" \
--network private \
--key-name admin \
--security-group permit_all \
mycirros

list all servers

1
openstack server list

allocate a floating IP

1
openstack floating ip create public

list all floating IPs

1
openstack floating ip list

assign floating IP to instance

1
openstack server add floating ip mycirros 192.168.8.60

shell into instance

1
ssh cirros@192.168.8.60