Blog

An Enterprise-Level Company Shares the Challenges of Relocating Their Network: TBI’s Very Own Case Study

With a 230-person workforce and more than 20 open positions across sales, marketing, support finance, and engineering, it is safe to say TBI is a rapidly growing organization. As we know, growth in business goes hand-in-hand with growth in headcount. It is because of this growth, TBI decided it was time for a bigger headquarters.

In June, TBI moved to a new office building, a nearly 28,000 square foot, single floor office space that enables our Chicagoland team to work more collaboratively. Our swanky new digs offer several upgraded features, including the Technology Evaluation Center (TEC), a lab showcasing technology solutions, hands-on training, and a place to perform proof-of-concept solution testing. We also installed a fully-equipped video studio that enables our videographer to professionally produce in-house video content for TBI as well as our vendors and partners.

It is no secret that moving a server environment can be painful, time-consuming and a nearly overwhelming amount of work, but it doesn't have to be. With proper preparation and the right resources, this process can simple, straightforward and secure. We recommend utilizing the best practices we put in place when moving a server environment; working with a master agent or technology consultant who has experience in relocating an entire server environment.

2019 Newsletter September Blog 2

Before purchasing equipment, TBI took an inventory to determine the overall port density we would require and the number of PoE and non-PoE ports that would be needed in our core switch. This was done to make sure we had an adequate port density across our chassis switch line cards while maintaining adequate room to grow as our headcount continued to increase. We evaluated the security market to determine the best firewall infrastructure for our edge. Ultimately, we decided on Fortinet based on the advanced security feature set and support for high bandwidth optical networking between our core switch and carrier CPEs.

From a network perspective, TBI took great care to ensure that we developed a well-thought-out Layer 2 and Layer 3 addressing scheme to ensure our network and corresponding departments were segmented. We put all our departments on separate VLANs and subnets to ensure network resources were restricted to only those who need them and to make network management simpler. We also put our voice and IoT devices, such as lighting, printers, and cameras, on separate VLANs and networks. IoT devices like smart TVs and lights are inherently insecure, so we wanted to ensure that they could not communicate with anything else on the network, where a device could potentially be compromised. 

Additionally, we built out a very detailed port mapping scheme and Visio diagram to show how our equipment would be racked in our server closet and how all the corresponding endpoints and devices would link back to our network core. Because our network footprint is small relatively speaking, we collapsed our access, distribution and core layers into just one core layer to guarantee high performance and fast packet switching across our LAN.

TBI was very determined to make certain we possessed a high-availability environment to maximize our network uptime in the event of any sort of outages. As a result, we have large stacks of TrippLite UPS systems for power outages that provide our network and server equipment enough time to back up their configurations and save any running data in the event of complete power failure. Typical power trips in buildings like ours only last 15-20 minutes; we, therefore, made sure to have an adequate battery backup to keep our devices running without power cycling and bringing the network down in case of an outage.

Our security consists of a high-availability pair of Fortinet firewalls that instantly failover between each other in the event of either device failing for whatever reason. Our HA pair of firewalls have redundant trunk links to our Cisco 9410R chassis switch spread across multiple line cards for further redundancy. Also, our Cisco chassis switch is advanced enough to specifically failover between PSUs and line cards based on the importance of the services running off the interfaces across the line cards, thus ensuring our critical systems remain up as long as possible. We ran multiple LAG groups across our PoE switches, running our Aruba WiFi APs so that if one switch dies, coverage is not significantly impacted and our WiFi continues to run. For added security/segmentation, our guest network was completely isolated from our production network. We accomplished this with a circuit that was diverse from our two primaries, going into its own separate firewall.

Even with a great deal of preparation, TBI faced some roadblocks during the move. Our main challenges came from the deliverables of our ISP vendors. We have two fully disparate ISP connections for redundancy and high uptime, but both missed their due date for circuit delivery and TTU. Further, after extensive testing we determined that we were not getting the throughput that we were contracted for—we were getting only 50-60% of the bandwidth we had contracted for—and needed to troubleshoot with the service delivery teams to determine why our ISP connections seemed to be bottlenecked.

2019 Newsletter September Blog

Due to our strong relationship with the ISPs we contracted through our master agent contracts, we were able to engage various levels of escalation to receive priority support on remediation. Before 

intrusive testing,we isolated each ISP circuit to a single interface on our core switch to bypass our firewall and confirm no security services were hampering the connection. Once we established that our firewalls were not impacting the performance of the circuits, we were able to conduct more extensive testing to determine that there were auto-negotiation issues on the carrier side that were preventing us from getting our allotted bandwidth. With the escalations provided by our carrier contacts, we were able to get Tier 3 engineering support that diagnosed the issue and remedied the negotiation issues to bring our connections up to their full capacity.

Even though we faced some challenges during the move, it was an overall successful e

xperience with many lessons learned. We now know first-hand what goes into a well-orchestrated move that occurs in a short timeframe while also configuring a completely new network. TBI's knowledgeable, experienced staff, specifically our IT and engineering teams, as well as our master agent contracts, were the elements that made this move so successful. Moreover, having not only professional but personal relationships with our vendors made the process that much easier.

Set up a discovery call with one of TBI's engineers to discuss the technology infrastructure of your customers or your own business. Whether a company move, an equipment purchase, a solution upgrade, or an overall infrastructure assessment, we've got your back!

Watch a video recap of us moving into our new home!

 

About TBI

TBI is the nation’s leading third-party technology distributor. Since 1991, it has assisted Systems Integrators, VARs, MSPs, IT consultants and more in advising and sourcing the right technology solutions. TBI serves as a partner’s advocate, ensuring the proper provisioning of cloud, Internet, data, mobility, voice, and managed services from best-in-class service providers to achieve clients’ desired business outcomes. Through training and marketing programs focused on the benefits of technology to the business, TBI empowers its partners to be the foremost authority to advise and source all of their clients’ technology needs. With the largest back-office in the industry, TBI partners are fully supported by certified solutions engineers, pre- and post-sales operations, and project managers. For more information visit www.tbicom.com.