Moving from Cloud to Self-Hosted Infrastructure: Enterprise Reliability at Value Prices

1. Introduction
2. Why Consider Moving Away from the Cloud
3. Proxmox Clustering: Enterprise Features on a Budget
4. Enterprise Hardware on a Budget
5. Self-Hosted Cloud Alternatives
6. Migration Strategies and Considerations
7. Real-World Example: Setting Up a Complete Stack
8. Cost Comparison: Cloud vs. Self-Hosted
9. Conclusion and Next Steps

Cost Predictability and Control

Cloud pricing models can lead to unpredictable bills, especially with services charged by usage. Self-hosting provides fixed, predictable costs after the initial investment.

Performance Consistency

Avoid "noisy neighbor" issues common in shared cloud environments. With dedicated hardware, you control the entire stack and can tune it specifically for your workloads.

Data Sovereignty and Compliance

For organizations with strict data residency requirements, self-hosting ensures complete control over where data is stored and processed.

Total Cost of Ownership

While cloud makes sense for variable workloads, stable operations with predictable resource needs often become significantly less expensive when self-hosted—particularly at scale.

High Availability (HA)

Configure critical VMs and containers to automatically restart on other nodes in the event of hardware failure.

Live Migration

Move running VMs between physical hosts with no downtime—perfect for maintenance windows or load balancing.

Centralized Management

Manage your entire infrastructure from a single web interface, with role-based access control for team environments.

Distributed Storage

Native support for Ceph provides software-defined storage with replication across nodes, eliminating single points of failure.

Backup and Restore

Scheduled backups with retention policies, with support for full and incremental backups to various storage types.

1. Install Proxmox VE on each physical server (minimum 3 nodes recommended for true HA)
2. Ensure all nodes have the same Proxmox version and are on the same subnet
3. Create a cluster on the first node using the web interface
4. Generate join information for additional nodes
5. Join remaining nodes to the cluster
6. Configure shared storage (critical for features like live migration)

When creating multiple VMs in Proxmox, it's efficient to clone from a template and then initialize each new VM. This script handles post-clone initialization tasks:

#!/bin/bash
# proxmox-vm-init.sh
# Post-clone initialization script for Proxmox VMs
# Usage: Edit NEWNAME variable and run on freshly cloned VM
# Example: NEWNAME="app1" ./proxmox-vm-init.sh

# Set this before running
NEWNAME="control"

# Run this block as-is
sudo hostnamectl set-hostname $NEWNAME
echo $NEWNAME | sudo tee /etc/hostname
sudo sed -i "s/app1/$NEWNAME/" /etc/hosts
sudo rm /etc/ssh/ssh_host_*
sudo dpkg-reconfigure openssh-server
sudo rm /etc/machine-id
sudo systemd-machine-id-setup

This script handles common post-cloning tasks like setting a unique hostname, regenerating SSH host keys, and creating a new machine ID to ensure the clone is properly individualized.

Dell PowerEdge R720/R730

Excellent balance of performance, expandability, and value. These 2U servers support high memory configurations (up to 768GB on the R730) and plenty of PCIe slots for expansion.

HP ProLiant DL380 G8/G9

Robust and widely available 2U servers with similar capabilities to the Dell PowerEdge series. Well-supported with a strong community.

Supermicro Systems

Often available at lower prices than Dell/HP with similar specifications. Less polish but excellent reliability and flexibility.

Tier 1: NVMe SSDs

Use for highest-performance workloads like databases. Even consumer-grade NVMe drives offer exceptional performance for most workloads. Consider enterprise options like Intel Optane for write-intensive applications.

Tier 2: SATA SSDs

Excellent for VM operating systems and applications. Enterprise SSDs (like Intel DC series) offer better endurance but consumer drives often provide sufficient performance at much lower costs.

Tier 3: Enterprise HDDs

"Used spinning rust" is perfect for bulk storage. 10-12TB SAS drives offer an excellent balance of capacity and value. Consider RAID 6 or RAID 10 for improved reliability.

10Gb Ethernet

Used 10GbE switches from Dell, Cisco, or Mellanox are widely available at reasonable prices. Essential for storage traffic and live migration.

Direct Attached Copper (DAC) Cables

Significantly less expensive than optical transceivers for short runs (under 5 meters). Perfect for rack deployments.

Network Segregation

Consider separate networks for management, VM traffic, and storage replication to prevent congestion and improve security.

MinIO provides S3-compatible object storage that can run on your own hardware. Features include:

• S3-compatible API—works with existing tools and libraries
• Distributed deployment option for high availability
• Erasure coding for data protection with minimal overhead
• Supports encryption, identity management, and object lifecycle policies

Coolify is an open-source, self-hostable Heroku/Netlify alternative that provides:

• One-click deployments from Git repositories
• Automated builds and deployments
• Support for various application types (Node.js, Python, PHP, etc.)
• Built-in database management
• Simple scaling and monitoring

MongoDB

Self-hosted MongoDB provides document database capabilities without the cost of MongoDB Atlas.

PostgreSQL + Patroni

Create a highly available PostgreSQL cluster using Patroni for automatic failover.

Redis

Run Redis clusters for caching, session storage, and real-time features.

Apache Kafka

Self-hosted Kafka provides robust event streaming capabilities for complex data pipelines.

RabbitMQ

Excellent for traditional message queueing needs with lower resource requirements than Kafka.

Prometheus + Grafana

The standard for metrics collection and visualization in self-hosted environments.

ELK Stack (Elasticsearch, Logstash, Kibana)

Comprehensive log collection, indexing, and analysis—a replacement for cloud logging services.

Zabbix

Mature, enterprise-ready monitoring solution with extensive alerting capabilities.

1. Resource Assessment: Analyze current cloud resource usage to determine hardware requirements
2. Service Inventory: Document all cloud services in use and identify self-hosted alternatives
3. Dependency Mapping: Understand how services interact to plan migration order
4. Network Architecture: Design network architecture, including VLANs, routing, and firewall rules
5. Backup Strategy: Establish backup procedures before starting migration

Phase 1: Infrastructure Setup

Build and test the physical infrastructure, including servers, networking, and storage.

Phase 2: Core Services

Deploy fundamental services like DNS, monitoring, and backup systems.

Phase 3: Stateless Applications

Move web servers and other stateless applications first, as they're typically easier to migrate.

Phase 4: Stateful Services

Carefully migrate databases and other stateful services, often requiring specialized migration tools.

Phase 5: Validation and Optimization

Test thoroughly and optimize performance, then decommission cloud resources.

• TTL Adjustment: Lower TTLs before migration to allow faster DNS propagation
• Blue/Green Deployment: Set up both environments and switch traffic when ready
• Weighted Routing: Gradually shift traffic from cloud to self-hosted infrastructure
• Reverse Proxy: Use a proxy layer to redirect traffic without DNS changes

3x Dell PowerEdge R730 Servers

• 2x Intel Xeon E5-2690 v3 (12 cores each)
• 256GB DDR4 RAM
• 2x 1TB NVMe (local OS and high-performance storage)
• 6x 4TB SAS HDDs (for distributed storage)
• 10GbE network interfaces

Networking

• 1x 24-port 10GbE switch (Dell S4048-ON or similar)
• 1x 48-port 1GbE switch for management network
• DAC cables for 10GbE connections

1. Install Proxmox VE on all three servers
2. Configure separate networks for management, VM traffic, and storage
3. Create Proxmox cluster and join all nodes
4. Set up Ceph storage across the three nodes for resilient storage
5. Configure VM templates for common OS distributions

Create the following VMs/Containers:

• 3x HAProxy/Nginx load balancers (configured as an HA cluster)
• 4x Application servers
• 3x PostgreSQL database servers (with Patroni for automatic failover)
• 3x MinIO servers (distributed deployment)
• 2x Monitoring servers (Prometheus, Grafana, Alertmanager)
• 2x Log management servers (Elasticsearch, Logstash, Kibana)
• 2x Backup servers (with appropriate storage allocation)

Here's how to set up a highly available PostgreSQL cluster using Patroni:

# On each PostgreSQL node

# Install required packages
sudo apt update
sudo apt install -y postgresql-14 python3-pip python3-dev libpq-dev

# Install Patroni and requirements
pip3 install patroni[etcd] psycopg2-binary

# Create Patroni configuration
sudo mkdir -p /etc/patroni
sudo cat > /etc/patroni/config.yml << EOF
scope: postgres-cluster
namespace: /db/
name: postgresql-1  # unique for each node

restapi:
  listen: 0.0.0.0:8008
  connect_address: 192.168.1.11:8008  # Replace with node IP

etcd:
  hosts: 192.168.1.21:2379,192.168.1.22:2379,192.168.1.23:2379  # etcd endpoints

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    postgresql:
      use_pg_rewind: true
      parameters:
        max_connections: 500
        shared_buffers: 4GB
        effective_cache_size: 12GB
        maintenance_work_mem: 1GB
        
  initdb:
    - encoding: UTF8
    - data-checksums
    
postgresql:
  listen: 0.0.0.0:5432
  connect_address: 192.168.1.11:5432  # Replace with node IP
  data_dir: /var/lib/postgresql/14/main
  bin_dir: /usr/lib/postgresql/14/bin
  
  authentication:
    replication:
      username: replicator
      password: replPass123
    superuser:
      username: postgres
      password: adminPass123
EOF

# Create Patroni systemd service
sudo cat > /etc/systemd/system/patroni.service << EOF
[Unit]
Description=Patroni PostgreSQL Cluster
After=network.target

[Service]
Type=simple
User=postgres
Group=postgres
ExecStart=/usr/local/bin/patroni /etc/patroni/config.yml
KillMode=process
TimeoutSec=30
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

# Enable and start the service
sudo systemctl enable patroni
sudo systemctl start patroni

Initial Investment

Ongoing Monthly Costs

3-Year Total Cost of Ownership

Variable Workloads

Applications with significant traffic spikes or seasonal variation benefit from cloud elasticity.

Early-Stage Startups

When capital preservation is crucial and infrastructure needs are evolving rapidly.

Globally Distributed Applications

When you need presence in multiple regions without establishing your own global data center network.

Specialized Services

When using cloud-specific services like AI/ML platforms, IoT hubs, or specialized analytics that would be complex to self-host.

• Proxmox VE Wiki - Comprehensive documentation for Proxmox
• ServeTheHome - Excellent resource for server hardware reviews and guides
• MinIO Documentation - Guides for setting up S3-compatible storage
• r/homelab - Community for home lab enthusiasts with excellent guides and discussions
• r/selfhosted - Resource for self-hosted alternatives to cloud services