cici

I would like to throw my hat into the ring. Today, I think we're in a better position versus where we were before. We have complete control over our server versus having NFO managing our server which I volunteered to do to lessen the server's cost. Subsequently, managing the server allows us to get a real-time perspective on resource utilization we didn't have before when NFO managed the server. Consequently, the consistent lag/pauses we were experiencing have disappeared. Additionally, I volunteered to work on the points plugin which I believe is in a good state today. I believe the points plugin functionally works the same prior to the lag issue with some new features added to it. This work included recreating the missing sections of the source code we were running for the points plugin which I did when I started looking at the points plugin source code. I was able to recreate a complete version of the points plugin source code utilizing the running plugin we had and now there is a workable plugin we are able to modify and improve upon. Subsequently, new functionality and fixes were implemented in the points plugin and new plugins have been added to the server to complement game play. The effort to work on the points plugin we are using today is not just from the work I did but with assistance from Archer and Soda. Also, I worked on upgrading the server to the latest stable branch of SourceMod version 1.11. This effort included transitioning the points plugin source code to the SourceMod’s new syntax utilized in version 1.11. Subsequently, transitioning to version 1.11 of SourceMod revealed issues with other third-party plugins we are utilizing, hence I worked on addressing those issues as the arose and converting those third-party plugins to SourceMod 1.11. This was a necessary step for our server because development of SourceMod plugins are being developed for version 1.11 versus older versions of SourceMod. Therefore, I felt it was worth the effort to transition to SourceMod 1.11 thereby ensuring we are compatible with current and future plugins we may want to utilize. This process was enjoyable to work on and I want to continue being involved with that work. As it pertains to funding the servers, I can provide funds to supplement a shortage if necessary but I’m hoping there is an increase in membership to cover the cost. Also, Soda I did add the 'world model turn rate fix’ plugin you recommended to the server. I'm not sure if the plugin really addressed the issue Mercman mentioned but we are running it today.

Soda I'm curious if you're planning on implementing changes to the gameplay? I get the impression you want to make the game more challenging based on our discussions. However, I may be wrong. Can you elaborate on if you're planning on making changes and what those changes are?

Latest Update: Update @ 3:40am CST on 1/16: The facility reports that five out of six chillers are operational, and the datacenter is within a normal temperature range again. They manually rebooted our primary router, and it came back online; we've now shifted loads back onto it. Our audits have not identified any equipment that is not functioning properly, so everything appears to be back to normal now at this facility. We will continue to monitor, however, and follow up with the facility as they work to repair the sixth chiller and improve their overall cooling systems.

I see the servers are accessible now. However, NFO states 'we're not out of the woods yet' in their last update. Also, NFO mentioned 'automatic CPU throttling may occur on machines at the physical system level, limiting performance'. Therefore, I'm thinking we may see performance issues on the server until the cooling system is fully restored. Additional Updates: Update @ 9pm CST: We are told that one of the broken chillers is back online now and that temperatures have stabilized at 120 degrees F. It is still not possible for us to effectively troubleshoot downed equipment, but we are monitoring very closely and will try to bring everything back online as soon as we are able to. Update @ 9:41pm CST: The facility says that another chiller is back online and that temperatures are slowly decreasing now, but we have not seen a change yet in our equipment status. We are continuing to monitor and wait. Update @ 2am CST: As the temperature slowly goes down, our router is going longer before its network adapter overheats and it kills the connection. We are observing about 7 minutes of connectivity before it goes offline for a minute. Our primary router and one of our network switches are still offline. We have asked the facility to investigate these, but they have told us that they will not turn back on any equipment for customers until temps drop further. We will pursue them. Update @ 3:42am CST: The ambient temperature dropped a little further and our secondary router has not had a high-temperature disconnect error for a bit over 30 minutes now. This means that most customers have connectivity again. We still have one switch offline, and our primary router offline, and we are lobbying the facility to investigate these ASAP. The router being offline is not causing customers problems because it is redundant, but the switch being offline is leading to some customers' machines or VDSes being inaccessible. So far, we have seen a couple of machines that rebooted due to the heat, but we haven't noted any total hardware failures apart from the switch and primary router. We will be performing a complete audit of all equipment after temperatures are back in the normal range and the facility has restored the downed switch. Update @ 3:57am CST: The switch that was offline is now online again; it seems to have left a temperature protection mode as the ambient temperature dropped. We are continuing to investigate the downed router and to look for any other equipment that might be having problems. Please also note that because the facility's temperature is still high -- we are told that it is 88F now -- automatic CPU throttling may occur on machines at the physical system level, limiting performance. This should automatically resolve as the temperature drops further. Update @ 9:08am CST: Customer equipment stayed online through the night, but the facility itself has not yet fully recovered, so we're not out of the woods yet. Data center says that there was a slight increase in temperature during the night when two chillers failed and had to be restarted, and that they are in the process of installing additional portable coolers. They have not yet worked on our offline router.

The L4D2 server is inaccessible because the cooling system has failed in the Chicago data center. NFO Servers who hosts our VDS provided the following information: Jan 14 2024 11:26:31 AM PT: Investigating packet loss and routing reconvergence, we found that a link to one of our upstreams in Chicago was flapping today. We have shut down that link while we investigate further, which should prevent further impact to customers. Jan 14 2024 11:48:10 AM PT: The issues in Chicago are actually due to components in our routers overheating, due to a major datacenter outage with their cooling system. We are unable to stop the packet loss for now as a result. We are actively monitoring for updates from the facility on a fix. Update @ 3:48pm CST: The facility reports that they are continuing to try to recover from multiple failed chillers, which went down due to the low temperatures. Currently, temperatures are still rising in the facility, wreaking havoc on our equipment, including our routers. Our last look at our main router showed an internal case temperature of around 160 degrees F. We have tried workarounds, but both of our redundant routers are seeing serious problems, including failed power supplies, due to the heat issue, and all of our upstream connections are intermittently offline because the network cards and cables are so hot. Only when they have brought temperatures down again will we be able to assess what is permanently broken and adapt to it. Update @ 6:31pm CST: The facility still has not brought down temperatures, though they are trying with fans. Our primary router seems to have entirely failed, likely because it overheated; we are transitioning to exclusive use of our secondary router, but do not know how long it will hold out. Its upstream links are also still frequently going down. Update @ 6:53pm CST: As the facility continues to have temperature rises, we are seeing switches and machines fail. We will have to assess these when the cooling is restored. We hope that they are not entirely dead.

As a follow-on to Flitterkill's previous post 'GC Status and pseudo-wind-down' on the status of our virtual dedicated server (VDS), we have transitioned to a new VDS. Overall, the new VDS lessens the annual cost of hosting GC's L4D2 dedicated servers per Flitterkill's previous post 'GC Status and pseudo-wind-down'. Although, to minimize the cost requires us to maintain and manage our VDS versus using an NFO hosted L4D2 instance, which I volunteered to do. The new VDS's IP address is '74.91.115.164' (with a DNS entry of l4d2.gcftw.com). Once the new VDS was online, I started the process of transitioning over server two ('gamrs.co | !buy 10v10 #2') and server three (gamrs.co | !buy 10v10 #3 dev) to the new VDS. I was able to complete the transition fairly quickly and started testing the new VDS that same day. Minimal testing convinced me the transition went well and the server is capable of hosting our L4D2 dedicated servers. Additionally, I was able to continue testing with some members/regulars to see if there were any issues but did not notice an issue myself. However, testing with a full server will be the real test and should clearly indicate if there is an issue using the new VDS. Now that the new VDS is online and working, our current VDS will be shut down on July 27th, 2023. Therefore, you will need to adjust your connection to the new VDS if you want to continue playing on GC's L4D2 servers. To facilitate hosting more than one L4D2 dedicated server on the new VDS, I'm taking advantage of using different ports to connect to different L4D2 dedicated servers, which luckily Valves' server browser is capable of today. To point to the L4D2 dedicated servers hosted on the new VDS you will need to connect to the following: 1) 'Server #1' is hosted at 'l4d2.gcftw.com:27015' (without the single quotes). 2) 'Server #2' is hosted at 'l4d2.gcftw.com:27016' (without the single quotes). 3) 'Dev Server #3' is hosted at 'l4d2.gcftw.com:27020' (without the single quotes). Although there are three L4D2 dedicated servers configured on the new VDS, the new VDS can only comfortably host two servers concurrently based on my testing. Therefore, I'll leave the development server off and only enable it when needed for testing purposes. Also, if hosting two L4D2 dedicated servers concurrently is an issue we can disable one of the L4D2 dedicated servers if needed. In conclusion, there is a new VDS online with IP address '74.91.115.164' hosting GC's L4D2 dedicated servers. Additionally, the current VDS will be shut-off on July 27th, 2023.