Case Study - Hospitality: Ellerman House
Ellerman House is “Cape Town’s Most Exclusive Address”, a world class boutique hotel based in that according to General Manager Paul Bruce-Brand is “in a unique position of being able to offer our guests the very best of South Africa; From generous hospitality and refined service, to exceptional experiences and the most representative South African art collection in existence.”
In February 2014, Carol Kohne, the Financial Manager approached Precision Technology after referral by its ISP, Internet Solutions to discuss the general state of the hotels IT infrastructure.
While Ellerman House offered its guests a world class holiday experience, its IT infrastructure at that time was anything but ”world class”. For example the main booking system called Opera was regularly crashing and then the server on which it was hosted was taking incredibly long to start back up. This was having serious implications on the hotels ability to function effectively and causing intense frustrations amongst guests and staff. Other challenges included poor and unreliable wifi coverage, slow and problematic internet access
The initial discussions in February were more about the issues being experienced and exploring options to stabalise the situation. By June 2014, it was decided that Precision Technology would take over management of the IT infrastructure with the signing of a SLA agreement.
Upon taking over the management of the hotels IT infrastructure in June 2014, different goals were set in order to migrate the IT infrastructure over to a more appropriate and stable set up as per the summary of goals and action plane below:
Short term goals
- Audit the IT infrastructure as per discussions with management. Also identify and address any urgent/immediate IT problems and deal with them as best as possible. Lock down the servers and network on takeover.
- Audit and tweak or “fix” all the onsite PCs/Laptops/Macs.
- Sort/tidy any cabling or infrastructure that needs to be addressed. Sort any network issues.
- Identify and sort backup strategies for servers and data, and any risks to Ellerman House should a critical system fail.
- Familiarise ourselves with the onsite Opera booking program.
- Get control of DNS, MX records, network etc.
Medium to longer term goals
- Upgrade the main network Firewall
- Install adequate network storage.
- Replace “unsuitable” network switches with appropriate options. Rebuild the network core.
- Assess and ensure adequate there is a power backup solution.
- Clone and virtualise all servers
- Add real time monitoring onto the network and systems.
- Segment the network and move the Guest wireless network away from staff functionality. Sort any wireless coverage problems.
Based on these goals an action plan was implemented and key areas were upgraded as follows:
- Because of the sheer scale of the site with multiple IT rooms located over the property, the initial step was to map out the network in order to understand how it was connected and assist in identifying areas that needed urgent attention. This was achieved in the first few weeks.
- 3 Cisco switches were added to the network layer 3 Core switches from which the network was managed. Unreliable and low spec switches in remote locations were replaced with suitable POE Netgear alternatives and these acted as Access switches connected to the Cisco Core switches with fibre.
- The network was redesigned and architected into a “classic” structured, design with Core and Access switches connected using Lag/redundant were used on the fibre links to the scattered IT rooms. Any “spaghetti like” or poor network design configurations were removed. The network was also segregated into vlans for better management and control.
- A Cyberoam was installed at the “front” of the networks to introduce better internet security and control. This was connected to a 1:1 fibre connection with Wiband failover.
- An advanced managed wireless network was introduced using Ruckus technology across the site.
- Network cabling was tidied up throughout the site. Critical cabinets were rebuilt.
- The three standalone Dell Servers were rebuilt as virtual server hosts using Proxmox. Much of the server functionality was segmented into VM (virtual machines) for better control and management. For example the Opera server became an “Opera VM”.
- A new Domain Controller VM was introduced for staff and network management.
- Remote network Nas device was introduced on the network to provide remote onsite backup storage.
- A new DC server has been built on one of the Dell VM hosts
- A 5 KVA UPS with several battery packs were introduced into the main server rooms to keep the servers and core network up in case of outages. Later a large onsite generator was introduced that switched on after several minutes in the case of an outage.
- Smaller UPSs were introduced in the smaller IT room locations.
Within a few months of the upgrade, the IT systems (network and servers) were stabilized and has operated “reasonably smoothly” since then. The slow access to the servers and unpredictable wifi became a thing of the past.
In August 2018 a decision was made to upgrade the server architecture and introduce a Nutanix Cluster for the existing Proxmox setup.
Essentially a new server was purchased and incorporated with two of the existing Dell servers (upgraded) into a “high availability” cluster. The three servers operate together in as a fast Distributed storage fabric. The three servers in the cluster were connected using a 10 Gbps backbone core network.
A fourth server was turned into a Nutanix DR (Disaster Recovery) box and hosted in a separate server room from the main cluster.
Initial reports after introduction of the cluster was that the speed and performance experience for staff using the VMs (Opera) has radically improved.