Ubuntu 22.04部署OpenStack私有云实战指南
1. 项目概述
在当今云计算时代,OpenStack作为开源云计算平台的标杆,已经成为企业构建私有云和混合云的首选方案。作为一名在云计算领域摸爬滚打多年的工程师,我最近在Ubuntu 22.04 LTS上完成了一个OpenStack部署项目,实现了多租户虚拟化和弹性伸缩功能。这个方案特别适合中小型企业、教育机构和研发团队,能够以较低成本获得接近商业云服务的体验。
Ubuntu 22.04 LTS作为长期支持版本,提供了稳定的基础环境,而OpenStack则带来了完整的IaaS能力。通过这个组合,我们可以实现虚拟机实例的快速部署、网络隔离、存储管理和自动扩缩容等核心功能。本文将详细记录我从零开始部署的全过程,包括环境准备、核心组件配置、性能优化以及实际使用中遇到的坑和解决方案。
2. 环境准备与基础配置
2.1 硬件需求规划
OpenStack对硬件资源有一定要求,合理的规划可以避免后续性能瓶颈。根据我的经验,最小化生产环境需要:
- 控制节点:至少16核CPU,64GB内存,500GB SSD(系统+数据库),1TB HDD(镜像存储)
- 计算节点:根据虚拟机密度规划,每台建议32核CPU,128GB内存,2TB SSD(实例存储)
- 网络:万兆网卡(至少2块),建议使用bonding提高可靠性
提示:如果是测试环境,可以使用一台配置较高的服务器(如32核/128GB)部署All-in-One模式,但要注意这种配置不适合生产环境。
2.2 操作系统基础配置
首先在Ubuntu 22.04 LTS上做一些必要的系统级优化:
# 更新系统 sudo apt update && sudo apt upgrade -y # 安装基础工具 sudo apt install -y git python3-dev python3-pip libffi-dev gcc libssl-dev # 禁用防火墙(OpenStack会管理自己的防火墙规则) sudo systemctl stop ufw && sudo systemctl disable ufw # 配置NTP时间同步 sudo apt install -y chrony sudo systemctl enable chrony && sudo systemctl start chrony # 修改主机名并更新/etc/hosts sudo hostnamectl set-hostname controller echo "127.0.0.1 controller" | sudo tee -a /etc/hosts2.3 网络规划与配置
网络是OpenStack最复杂的部分之一,建议提前规划好以下网络:
- 管理网络:用于节点间通信(如10.0.0.0/24)
- 租户网络:虚拟机使用的网络(如192.168.1.0/24)
- 外部网络:提供浮动IP(如公共IP段)
配置网络接口(以双网卡为例):
# /etc/netplan/00-installer-config.yaml network: version: 2 renderer: networkd ethernets: enp1s0: addresses: [10.0.0.10/24] gateway4: 10.0.0.1 nameservers: addresses: [8.8.8.8, 8.8.4.4] enp2s0: {} bonds: bond0: interfaces: [enp2s0] addresses: []应用配置:sudo netplan apply
3. OpenStack核心组件部署
3.1 数据库与消息队列安装
OpenStack依赖MySQL和RabbitMQ:
# 安装MySQL sudo apt install -y mariadb-server python3-pymysql # 配置MySQL sudo mysql_secure_installation # 创建OpenStack数据库和用户 sudo mysql -u root -p <<EOF CREATE DATABASE keystone; CREATE DATABASE glance; CREATE DATABASE nova; CREATE DATABASE neutron; CREATE DATABASE cinder; CREATE DATABASE heat; GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' IDENTIFIED BY 'KEYSTONE_DBPASS'; GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'localhost' IDENTIFIED BY 'GLANCE_DBPASS'; GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' IDENTIFIED BY 'NOVA_DBPASS'; GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' IDENTIFIED BY 'NEUTRON_DBPASS'; GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'localhost' IDENTIFIED BY 'CINDER_DBPASS'; GRANT ALL PRIVILEGES ON heat.* TO 'heat'@'localhost' IDENTIFIED BY 'HEAT_DBPASS'; EOF # 安装RabbitMQ sudo apt install -y rabbitmq-server sudo rabbitmqctl add_user openstack RABBIT_PASS sudo rabbitmqctl set_permissions openstack ".*" ".*" ".*"3.2 Keystone身份服务部署
Keystone是OpenStack的身份认证核心:
# 安装Keystone sudo apt install -y keystone apache2 libapache2-mod-wsgi-py3 # 配置/etc/keystone/keystone.conf [database] connection = mysql+pymysql://keystone:KEYSTONE_DBPASS@controller/keystone [token] provider = fernet # 初始化数据库 sudo su -s /bin/sh -c "keystone-manage db_sync" keystone # 初始化Fernet密钥 keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone keystone-manage credential_setup --keystone-user keystone --keystone-group keystone # 启动服务 keystone-manage bootstrap --bootstrap-password ADMIN_PASS \ --bootstrap-admin-url http://controller:5000/v3/ \ --bootstrap-internal-url http://controller:5000/v3/ \ --bootstrap-public-url http://controller:5000/v3/ \ --bootstrap-region-id RegionOne # 配置Apache echo "ServerName controller" | sudo tee /etc/apache2/conf-available/servername.conf sudo a2enconf servername sudo systemctl restart apache23.3 Glance镜像服务部署
Glance管理虚拟机镜像:
# 创建Glance用户和端点 openstack user create --domain default --password GLANCE_PASS glance openstack role add --project service --user glance admin openstack service create --name glance --description "OpenStack Image" image openstack endpoint create --region RegionOne image public http://controller:9292 openstack endpoint create --region RegionOne image internal http://controller:9292 openstack endpoint create --region RegionOne image admin http://controller:9292 # 安装Glance sudo apt install -y glance # 配置/etc/glance/glance-api.conf [database] connection = mysql+pymysql://glance:GLANCE_DBPASS@controller/glance [keystone_authtoken] www_authenticate_uri = http://controller:5000 auth_url = http://controller:5000 memcached_servers = controller:11211 auth_type = password project_domain_name = Default user_domain_name = Default project_name = service username = glance password = GLANCE_PASS [paste_deploy] flavor = keystone [glance_store] stores = file,http default_store = file filesystem_store_datadir = /var/lib/glance/images/ # 初始化数据库 sudo su -s /bin/sh -c "glance-manage db_sync" glance # 启动服务 sudo systemctl enable glance-api sudo systemctl start glance-api # 上传测试镜像 wget http://download.cirros-cloud.net/0.5.2/cirros-0.5.2-x86_64-disk.img openstack image create "cirros" \ --file cirros-0.5.2-x86_64-disk.img \ --disk-format qcow2 --container-format bare \ --public4. 计算服务与网络部署
4.1 Nova计算服务配置
Nova负责虚拟机生命周期管理:
# 创建Nova用户和端点 openstack user create --domain default --password NOVA_PASS nova openstack role add --project service --user nova admin openstack service create --name nova --description "OpenStack Compute" compute openstack endpoint create --region RegionOne compute public http://controller:8774/v2.1 openstack endpoint create --region RegionOne compute internal http://controller:8774/v2.1 openstack endpoint create --region RegionOne compute admin http://controller:8774/v2.1 # 安装Nova组件 sudo apt install -y nova-api nova-conductor nova-novncproxy nova-scheduler # 配置/etc/nova/nova.conf [api_database] connection = mysql+pymysql://nova:NOVA_DBPASS@controller/nova_api [database] connection = mysql+pymysql://nova:NOVA_DBPASS@controller/nova [default] transport_url = rabbit://openstack:RABBIT_PASS@controller:5672/ my_ip = 10.0.0.10 use_neutron = true firewall_driver = nova.virt.firewall.NoopFirewallDriver [api] auth_strategy = keystone [keystone_authtoken] www_authenticate_uri = http://controller:5000/ auth_url = http://controller:5000/ memcached_servers = controller:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = nova password = NOVA_PASS [vnc] enabled = true server_listen = $my_ip server_proxyclient_address = $my_ip [glance] api_servers = http://controller:9292 [oslo_concurrency] lock_path = /var/lib/nova/tmp [placement] region_name = RegionOne project_domain_name = Default project_name = service auth_type = password user_domain_name = Default auth_url = http://controller:5000/v3 username = placement password = PLACEMENT_PASS # 初始化数据库 sudo su -s /bin/sh -c "nova-manage api_db sync" nova sudo su -s /bin/sh -c "nova-manage cell_v2 map_cell0" nova sudo su -s /bin/sh -c "nova-manage cell_v2 create_cell --name=cell1 --verbose" nova sudo su -s /bin/sh -c "nova-manage db sync" nova # 启动服务 sudo systemctl enable nova-api nova-conductor nova-novncproxy nova-scheduler sudo systemctl start nova-api nova-conductor nova-novncproxy nova-scheduler4.2 Neutron网络服务配置
Neutron提供网络虚拟化功能:
# 创建Neutron用户和端点 openstack user create --domain default --password NEUTRON_PASS neutron openstack role add --project service --user neutron admin openstack service create --name neutron --description "OpenStack Networking" network openstack endpoint create --region RegionOne network public http://controller:9696 openstack endpoint create --region RegionOne network internal http://controller:9696 openstack endpoint create --region RegionOne network admin http://controller:9696 # 安装Neutron组件 sudo apt install -y neutron-server neutron-plugin-ml2 \ neutron-linuxbridge-agent neutron-l3-agent neutron-dhcp-agent \ neutron-metadata-agent # 配置/etc/neutron/neutron.conf [database] connection = mysql+pymysql://neutron:NEUTRON_DBPASS@controller/neutron [default] core_plugin = ml2 service_plugins = router allow_overlapping_ips = true transport_url = rabbit://openstack:RABBIT_PASS@controller:5672/ auth_strategy = keystone [keystone_authtoken] www_authenticate_uri = http://controller:5000 auth_url = http://controller:5000 memcached_servers = controller:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = neutron password = NEUTRON_PASS [oslo_concurrency] lock_path = /var/lib/neutron/tmp # 配置/etc/neutron/plugins/ml2/ml2_conf.ini [ml2] type_drivers = flat,vlan,vxlan tenant_network_types = vxlan mechanism_drivers = linuxbridge,l2population extension_drivers = port_security [ml2_type_flat] flat_networks = provider [ml2_type_vxlan] vni_ranges = 1:1000 [securitygroup] enable_ipset = true # 配置/etc/neutron/plugins/ml2/linuxbridge_agent.ini [linux_bridge] physical_interface_mappings = provider:enp2s0 [vxlan] enable_vxlan = true local_ip = 10.0.0.10 l2_population = true [securitygroup] enable_security_group = true firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver # 配置/etc/neutron/l3_agent.ini [default] interface_driver = linuxbridge # 配置/etc/neutron/dhcp_agent.ini [default] interface_driver = linuxbridge dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq enable_isolated_metadata = true # 配置/etc/neutron/metadata_agent.ini [default] nova_metadata_host = controller metadata_proxy_shared_secret = METADATA_SECRET # 链接配置文件 sudo ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini # 初始化数据库 sudo su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf \ --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head" neutron # 重启服务 sudo systemctl restart nova-api sudo systemctl enable neutron-server neutron-linuxbridge-agent neutron-dhcp-agent \ neutron-metadata-agent neutron-l3-agent sudo systemctl start neutron-server neutron-linuxbridge-agent neutron-dhcp-agent \ neutron-metadata-agent neutron-l3-agent5. 多租户与弹性伸缩实现
5.1 多租户隔离配置
OpenStack通过项目和用户实现多租户隔离:
# 创建示例租户和用户 openstack project create --description "Demo Project" demo openstack user create --domain default --password DEMO_PASS demo openstack role add --project demo --user demo user # 创建网络和安全组 openstack network create demo-net openstack subnet create --network demo-net --subnet-range 192.168.1.0/24 \ --gateway 192.168.1.1 --dns-nameserver 8.8.8.8 demo-subnet openstack security group create demo-sg openstack security group rule create --ingress --protocol tcp --dst-port 22 demo-sg openstack security group rule create --ingress --protocol icmp demo-sg # 创建路由器并连接外部网络 openstack router create demo-router openstack router set --external-gateway public demo-router openstack router add subnet demo-router demo-subnet5.2 弹性伸缩配置
使用Heat和Ceilometer实现自动伸缩:
# 安装Ceilometer和Heat sudo apt install -y ceilometer-api ceilometer-collector \ ceilometer-agent-central ceilometer-agent-notification \ ceilometer-alarm-evaluator ceilometer-alarm-notifier \ python3-ceilometerclient heat-api heat-api-cfn heat-engine # 配置/etc/ceilometer/ceilometer.conf [database] connection = mysql+pymysql://ceilometer:CEILOMETER_DBPASS@controller/ceilometer [default] transport_url = rabbit://openstack:RABBIT_PASS@controller:5672/ [service_credentials] auth_url = http://controller:5000/v3 project_name = service username = ceilometer password = CEILOMETER_PASS interface = internalURL region_name = RegionOne # 配置/etc/heat/heat.conf [database] connection = mysql+pymysql://heat:HEAT_DBPASS@controller/heat [default] transport_url = rabbit://openstack:RABBIT_PASS@controller:5672/ heat_metadata_server_url = http://controller:8000 heat_waitcondition_server_url = http://controller:8000/v1/waitcondition [keystone_authtoken] www_authenticate_uri = http://controller:5000 auth_url = http://controller:5000 memcached_servers = controller:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = heat password = HEAT_PASS [trustee] auth_type = password auth_url = http://controller:5000 username = heat password = HEAT_PASS user_domain_name = default [clients_keystone] auth_uri = http://controller:5000 [oslo_messaging_notifications] driver = messagingv2 transport_url = rabbit://openstack:RABBIT_PASS@controller:5672/ # 初始化数据库 sudo su -s /bin/sh -c "ceilometer-upgrade --skip-gnocchi-resource-types" ceilometer sudo su -s /bin/sh -c "heat-manage db_sync" heat # 启动服务 sudo systemctl enable ceilometer-api ceilometer-collector \ ceilometer-agent-central ceilometer-agent-notification \ ceilometer-alarm-evaluator ceilometer-alarm-notifier \ heat-api heat-api-cfn heat-engine sudo systemctl start ceilometer-api ceilometer-collector \ ceilometer-agent-central ceilometer-agent-notification \ ceilometer-alarm-evaluator ceilometer-alarm-notifier \ heat-api heat-api-cfn heat-engine5.3 创建自动伸缩模板
创建一个HOT模板实现基于CPU使用率的自动伸缩:
# autoscaling.yaml heat_template_version: 2015-04-30 description: Auto-scaling example with CPU monitoring parameters: image: type: string default: cirros flavor: type: string default: m1.tiny network: type: string default: demo-net subnet: type: string default: demo-subnet security_group: type: string default: demo-sg scaling_min: type: number default: 1 scaling_max: type: number default: 3 cpu_threshold: type: number default: 50.0 resources: server_group: type: OS::Heat::AutoScalingGroup properties: min_size: { get_param: scaling_min } max_size: { get_param: scaling_max } resource: type: OS::Nova::Server properties: image: { get_param: image } flavor: { get_param: flavor } networks: - network: { get_param: network } security_groups: - { get_param: security_group } cpu_alarm_high: type: OS::Ceilometer::Alarm properties: description: Scale-up if CPU > 50% for 1 minute meter_name: cpu_util statistic: avg period: 60 evaluation_periods: 1 threshold: { get_param: cpu_threshold } comparison_operator: gt alarm_actions: - { get_attr: [server_group, scaleup_url] } matching_metadata: {'metadata.user_metadata.stack': {get_param: "OS::stack_id"}} cpu_alarm_low: type: OS::Ceilometer::Alarm properties: description: Scale-down if CPU < 15% for 5 minutes meter_name: cpu_util statistic: avg period: 300 evaluation_periods: 1 threshold: 15.0 comparison_operator: lt alarm_actions: - { get_attr: [server_group, scaledown_url] } matching_metadata: {'metadata.user_metadata.stack': {get_param: "OS::stack_id"}} outputs: group_size: description: Current size of the server group value: { get_attr: [server_group, current_size] }部署模板:openstack stack create -t autoscaling.yaml autoscale-demo
6. 性能优化与运维技巧
6.1 数据库优化
OpenStack重度依赖数据库,优化MariaDB配置:
# /etc/mysql/mariadb.conf.d/99-openstack.cnf [mysqld] innodb_buffer_pool_size = 4G innodb_log_file_size = 512M innodb_flush_log_at_trx_commit = 2 innodb_flush_method = O_DIRECT innodb_file_per_table = ON innodb_thread_concurrency = 0 query_cache_size = 0 max_connections = 1000重启MySQL:sudo systemctl restart mariadb
6.2 消息队列优化
调整RabbitMQ配置提高性能:
# /etc/rabbitmq/rabbitmq.conf vm_memory_high_watermark.relative = 0.6 disk_free_limit.absolute = 2GB loopback_users = none重启RabbitMQ:sudo systemctl restart rabbitmq-server
6.3 Nova调度优化
修改Nova调度策略提高效率:
# /etc/nova/nova.conf [scheduler] driver = filter_scheduler available_filters = nova.scheduler.filters.all_filters enabled_filters = RetryFilter, AvailabilityZoneFilter, ComputeFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter [filter_scheduler] max_attempts = 10重启Nova调度器:sudo systemctl restart nova-scheduler
6.4 网络性能优化
针对Neutron网络组件的优化建议:
- 启用巨帧(Jumbo Frame)提高网络吞吐:
sudo ip link set enp2s0 mtu 9000- 调整Linux网桥参数:
echo "net.bridge.bridge-nf-call-iptables=0" | sudo tee -a /etc/sysctl.conf echo "net.bridge.bridge-nf-call-ip6tables=0" | sudo tee -a /etc/sysctl.conf sudo sysctl -p- 优化Neutron的DHCP租期:
# /etc/neutron/dhcp_agent.ini [DEFAULT] dhcp_lease_duration = 864007. 常见问题排查与解决
7.1 虚拟机启动失败
现象:实例状态显示为"Error",日志中出现"no valid host was found"
排查步骤:
- 检查Nova计算服务状态:
nova service-list - 查看调度日志:
grep "Filtering" /var/log/nova/nova-scheduler.log - 检查计算节点资源:
nova hypervisor-show <hypervisor-id>
解决方案:
- 确保计算节点有足够资源
- 检查网络连接和防火墙规则
- 验证镜像和规格兼容性
7.2 网络连接问题
现象:虚拟机无法获取IP或无法访问外网
排查步骤:
- 检查DHCP代理:
neutron agent-list | grep DHCP - 查看命名空间:
ip netns list - 检查路由表:
neutron router-show <router-id>
解决方案:
- 重启Neutron服务:
sudo systemctl restart neutron-* - 检查外部网络配置
- 验证安全组规则
7.3 弹性伸缩不生效
现象:CPU负载高但未触发扩容
排查步骤:
- 检查Ceilometer数据:
ceilometer statistics -m cpu_util -q "resource_id=<instance-id>" - 查看告警状态:
openstack alarm list - 检查Heat引擎日志:
journalctl -u heat-engine
解决方案:
- 确保Ceilometer服务正常运行
- 验证告警阈值设置
- 检查自动伸缩组策略
7.4 性能监控与日志收集
建议部署以下工具辅助运维:
- Prometheus + Grafana监控:
# 安装Node Exporter wget https://github.com/prometheus/node_exporter/releases/download/v1.3.1/node_exporter-1.3.1.linux-amd64.tar.gz tar xvfz node_exporter-*.tar.gz cd node_exporter-* ./node_exporter &- ELK日志收集:
# 安装Filebeat curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.4.1-amd64.deb sudo dpkg -i filebeat-8.4.1-amd64.deb # 配置/etc/filebeat/filebeat.yml filebeat.inputs: - type: log enabled: true paths: - /var/log/nova/*.log - /var/log/neutron/*.log - /var/log/keystone/*.log- 定期数据库备份:
# 创建备份脚本 #!/bin/bash mysqldump -u root -p --all-databases > /backup/openstack-db-$(date +%Y%m%d).sql8. 安全加固建议
8.1 Keystone安全配置
- 启用多因素认证:
openstack identity provider create --remote-id https://idp.example.com --domain default example_idp openstack mapping create --rules rules.json example_mapping openstack federation protocol create --identity-provider example_idp --mapping example_mapping saml2- 限制管理员访问:
# /etc/keystone/keystone.conf [oslo_policy] enforce_scope = true enforce_new_defaults = true8.2 实例安全最佳实践
- 使用SSH密钥对:
openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey- 最小化安全组规则:
openstack security group rule create --protocol tcp --dst-port 22 --ingress default- 定期更新镜像:
openstack image set --property os_distro=ubuntu --property os_version=22.04 my-image8.3 网络隔离策略
- 启用安全组:
# /etc/neutron/plugins/ml2/ml2_conf.ini [securitygroup] enable_security_group = true firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver- 配置网络分段:
openstack network segment create --network demo-net --physical-network physnet1 \ --network-type vlan --segment 100 demo-segment- 启用端口安全:
# /etc/neutron/plugins/ml2/ml2_conf.ini [ml2] extension_drivers = port_security9. 升级与维护策略
9.1 版本升级路径
Ubuntu 22.04 LTS支持的OpenStack版本升级顺序:
- 备份所有数据库和配置文件
- 按顺序升级:Queens → Rocky → Stein → Train → Ussuri → Victoria → Wallaby → Yoga
- 每次升级后运行健康检查
9.2 日常维护任务
- 数据库维护:
nova-manage db archive_deleted_rows --max_rows 1000 neutron-db-manage --config-file /etc/neutron/neutron.conf purge- 日志轮转配置:
# /etc/logrotate.d/openstack /var/log/nova/*.log { daily missingok rotate 7 compress delaycompress notifempty create 640 nova nova }- 资源清理脚本:
#!/bin/bash # 清理超过30天的未使用镜像 openstack image list --status active --property created_at=gt:$(date -d "30 days ago" +%Y-%m-%d) -f value -c ID | xargs -r openstack image delete9.3 监控指标阈值建议
关键监控指标及告警阈值:
| 指标 | 警告阈值 | 严重阈值 | 检查频率 |
|---|---|---|---|
| CPU使用率 | 70% | 90% | 1分钟 |
| 内存使用率 | 80% | 95% | 1分钟 |
| 磁盘空间使用率 | 85% | 95% | 5分钟 |
| API响应时间 | 500ms | 1000ms | 1分钟 |
| 数据库连接数 | 80%最大连接数 | 90%最大连接数 | 5分钟 |
在实际部署中,我发现OpenStack的性能很大程度上取决于底层硬件的质量。特别是网络性能,使用万兆网卡和适当的交换机配置可以显著提高虚拟机的网络吞吐量。另一个关键点是数据库优化,通过合理配置InnoDB缓冲池大小和日志文件大小,可以大幅提升整体性能。