Linus Tech Tips - The $1,000,000 PC ruined our server room
The video details the process of managing the heat output from a high-performance server cluster, nicknamed 'million-doll PC,' which generates 7,000 to 8,000 watts of heat. The team initially used a proof-of-concept heat exchanger setup with APC data center cooling towers to manage the heat. However, this setup was inefficient and took up a lot of space. To improve efficiency, they decided to attach radiators directly to the back of the server rack, which would save space and potentially work better. The video documents the challenges faced during this process, such as leaks and the need for specific fittings, and how they were resolved. The team also added fans to enhance airflow through the radiators. The final setup, which includes a custom-built water door with radiators, successfully reduces the temperature of the air exiting the server rack, demonstrating an effective cooling solution for high-performance computing environments.
Key Points:
- The server cluster generates significant heat, requiring innovative cooling solutions.
- Initial cooling setup was inefficient and space-consuming, prompting a redesign.
- Directly attaching radiators to the server rack saves space and improves cooling efficiency.
- Challenges included leaks and fitting issues, which were resolved through trial and error.
- The final cooling system effectively reduces server room temperature, enhancing performance.
Details:
1. 🔥 Server Cooling Challenges
- The server cluster, nicknamed 'milliondoll PC,' generates 7,000 to 8,000 watts of heat, creating severe cooling challenges in the server room.
- A proof of concept heat exchanger setup using APC data center cooling towers has improved the situation but still leaves the room warmer than desired.
- The heat exchanger setup captures heat from the server cluster and uses a second cooling tower to expel it into a warehouse, which reduces the need for additional heating in that space.
- The current cooling setup occupies significant space and is not ideal for the server room, which lacks separate hot and cold aisles.
- A cardboard duct system is used to direct hot air to the cooling tower radiators, but this method is inefficient and poses a fire hazard.
- There is potential to improve space efficiency and cooling effectiveness by attaching radiators directly to the back of the server rack.
- Consider exploring alternative cooling technologies such as liquid cooling systems or server room redesign to include hot and cold aisle separation for enhanced efficiency.
2. 🔧 Setup and Sponsors
2.1. Projector Setup Features
2.2. Sponsorship and Promotions
3. 💡 Current Cooling System Improvements
- The current cooling system is operational with effective filtering and automatic top-up, showing no leaks or immediate issues, indicating no major changes required to these components.
- The existing pump, made of cast iron, is unsuitable for a mixed metal loop due to the absence of corrosion inhibitors, necessitating an upgrade.
- The upgraded pump will be constructed from stainless steel, mitigating corrosion risks within a mixed metal system, and will have a capacity of one-sixth horsepower.
- The new pump aims to deliver approximately 25 gallons per minute with 10 feet of head pressure, markedly improving efficiency over typical garden hose systems.
- Stainless steel's use over cast iron is strategic, offering enhanced durability and compatibility in diverse metal environments, thus extending system longevity.
4. 🔩 Troubleshooting Radiators
- Radiators require specific screwdriver bit sets, such as a torque set, available at the LTP store.
- Radiator connections include one inlet and two outlets, requiring careful measurement for fittings.
- Despite initial assumptions, copper fittings needed are 5/8 and 3/4 inches, not 7/8 and 3/4 inches, indicating the importance of precise measurement.
- Common radiator issues include leaks and inconsistent heating, which can often be resolved by checking and tightening connections.
- Ensure the correct fitting size to prevent leaks, which often occur due to incorrect measurements or worn-out seals.
5. 🧰 Plumbing Challenges and Solutions
5.1. Identifying the Right Pipe Size
5.2. Acquiring Necessary Fittings
5.3. Measurement
5.4. Adjusting Pipe Fittings
5.5. Finalizing the Solution
5.6. Innovative Solution Consideration
6. 🔍 Leak Testing and Adjustments
- Radiator fittings have been soldered and are being tested for leaks using an air pump with a gauge and Windex spray to detect bubbles.
- Initial leak tests show no bubbles, indicating no leaks and that the solder joints are clean and effective.
- Reworking radiators instead of purchasing new ones due to high cost and lack of response from vendors, with quotes as high as $60,000 for new radiators.
- The decision to salvage and repair existing radiators is seen as both cost-effective and more engaging, although potential problems are acknowledged.
7. 🔄 Radiator Installation and Testing
- Utilize 3/4 NPT to 3/4 ID barbs for simple connection, enabling easy twisting on, which simplifies the installation process.
- Incorporate T fittings and right angles at the top of each radiator to facilitate complex connections, ensuring seamless integration with the bottom radiator.
- A swivel barb priced at $11 is crucial for effective connection setup, providing necessary flexibility and ease of use.
- Strategically route the installation to accommodate door movement without relying on flexible hoses, which involves planning additional length for bending to prevent kinking.
- Connect two hoses from PEX tubing in the building to designated inlet and outlet points, ensuring a streamlined flow with minimal splitting, enhancing efficiency.
8. 🌬️ Airflow Dynamics and Improvements
8.1. Installation Challenges
8.2. Testing Outcomes
9. 🚪 Building a Water-Cooled Server Door
- The water-cooled server door is designed to capture and cool hot air from servers using water at approximately 20°C, which can potentially improve cooling efficiency and reduce the need for larger cooling setups.
- By positioning the cooling door closer to the heat source, space usage is optimized, potentially enhancing overall cooling performance.
- Initial tests of the server door indicate that airflow may not be strong enough, suggesting the need for additional fans to achieve optimal cooling efficiency.
- To ensure secure connections and prevent leaks, the transition from shark bite fittings to crimp fittings is being implemented, which involves using a metal ring and a tool to compress the plastic tubing tightly.
- Further testing is needed to evaluate the effectiveness of these improvements and to ensure the setup provides adequate cooling.
10. 💨 Final Adjustments and Testing
- Utilized room temperature water (~20°C) for cooling, ensuring effective heat dissipation.
- The cooled air is noticeably colder than ambient room temperature, indicating successful cooling.
- Servers tested using fresh air intake, optimizing thermal management and possibly reducing energy consumption.
- Employed Noctua industrial fans with a speed of 30,000 RPM to maximize cooling efficiency.
- Configured four sets of double fans, which, while barely adequate for one radiator, represent a significant improvement in cooling capability.
- Air output is slightly warm, demonstrating high air throughput and effective cooling performance.
11. 🔌 Power Supply and Server Room Setup
- The server setup experienced zero leaks in the current configuration, improving reliability.
- Fans are effectively pulling heat through, indicating a need for more active cooling to maintain optimal temperature.
- The back of the servers are operating at temperatures between 45 to 55 degrees Celsius, suggesting room for thermal optimization.
- The existing 24-watt power supply is insufficient to power more than three fans safely, indicating the need for a higher capacity power solution.
- A 500-watt HDLEX gallium nitride power supply is being considered, which is compact, easily mountable, and capable of being daisy-chained for enhanced power distribution.
- The current setup uses a Molex connection capable of handling approximately 100 watts, suitable for running multiple fans.
- Consideration of additional cooling strategies, such as liquid cooling or higher efficiency fans, could further enhance thermal management.
12. 📊 Performance Assessment and Results
12.1. Air Conditioning Setup
12.2. Performance Metrics
12.3. Cooling Improvement
12.4. Thermal Analysis
12.5. Temperature Measurements
12.6. Server Performance
13. 📽️ Sponsor Message
- The projector boasts 2600 ANSI lumens for ultra-bright visuals and a delta E of less than one for precise color accuracy, delivering studio-grade picture quality.
- Its compact design is comparable to a shoe box, with a handle for easy portability, making it ideal for both home and on-the-go use.
- Projects screen sizes from 80 inches to 150 inches, with support for Dolby Vision and HDR10+ for superior image quality, enhancing movie nights and presentations.
- Smart AI screen adaptation includes auto-screen fit, intelligent obstacle avoidance, and autofocus, providing a seamless user experience.
- Equipped with two built-in 15-watt speakers, offering quality sound output that enriches the viewing experience without needing external speakers.
- Features three HDMI 2.1 inputs, allowing diverse device connectivity for gaming consoles, laptops, and streaming devices.
- Includes a hidden compartment for a Fire TV Stick, enhancing streaming capabilities and keeping the setup tidy.
14. 🔚 Conclusion and Wrap-Up
- The project involved a 2-week effort to build a water door, demonstrating commitment and project management over an extended period.
- Encourages audience engagement by asking viewers to like and subscribe, indicating a strategy for increasing engagement and audience retention.
- References to previous work (wholesome water cooling project) suggests a series of related content, which can help in building a consistent viewer base.