For 10.5 Leopard: The GeForce 8800 GT is the undisputed champion for speed

[chrisNova777]

Yes, it is very safe to say the GeForce 8800 GT was the superior choice for Mac OS X 10.5 Leopard.

While the HD 5000 series is the king of Snow Leopard, the 8800 GT was the absolute performance leader for the Leopard era, especially for gaming and 3D work.

Why the 8800 GT ruled Leopard (10.5)
Raw Gaming Power: In 2008, the 8800 GT was significantly faster in OpenGL benchmarks (like Doom 3, Quake 4, and Halo) than the stock ATI Radeon HD 2600 XT or even the high-end Radeon X1900 XT.

The "G92" Architecture: This was NVIDIA’s first massive leap into modern unified shaders. It handled Leopard’s UI effects and 3D applications much more smoothly than the older ATI architectures.

Native Stability: Because Apple sold this card as a high-end "Upgrade Kit" for both the 1st Gen (2006/2007) and 2nd Gen (2008) Mac Pros, the drivers were deeply integrated into Leopard. It was the most stable "power user" card of that specific OS version.

The One Exception: "Pro Apps"
There was a small "catch" at the time:

Core Image: Apple’s "Pro Apps" (like Aperture, Motion, and Final Cut Pro) relied heavily on an engine called Core Image.

ATI cards (like the later Radeon HD 3870) were actually better optimized for Core Image than NVIDIA cards. If you were doing heavy video editing or photo work in 10.5, the ATI 3870 could sometimes outperform the 8800 GT in those specific tasks, even though the 8800 GT was faster for everything else.

Summary for your Builds
For 10.5 Leopard: The GeForce 8800 GT is the undisputed champion for speed and compatibility.

For 10.6.8 Snow Leopard: The ATI HD 5770 / 5870 takes the crown because it supports newer technologies like OpenCL that 10.6 introduced.

Since you've been working with the ASUS P5B Deluxe, the 8800 GT is a perfect period-accurate match. It feels "right" for that hardware and will give you the most authentic 2008-era Hackintosh experience.

[chrisNova777]

compare the 8800 GT with the GTX 260 the difference is earlier leopard support?
Gemini said
That is exactly right. The most significant functional difference between these two cards for a legacy project is where their support begins and ends within the macOS ecosystem.

While both are based on the NVIDIA "Tesla" architecture, they represent two different generations (G92 vs. GT200). Here is how they compare specifically for your builds:

1. The "Leopard" Factor (OS X 10.5)
8800 GT: This is a native Leopard hero. Because Apple sold a "Mac Edition" of this card specifically for the 2008 Mac Pro, the drivers are baked into OS X 10.5.2 and later. It is rock-solid for that OS.

GTX 260: This card arrived later. It has zero native support in 10.5. To get it working, you usually have to "hack" it by injecting device IDs into the 8800 GT's drivers or using experimental web drivers, which often results in stability issues or no hardware acceleration at all.

2. Performance & Power (The "Brute Force" Difference)
If you move up to 10.6.8 Snow Leopard, the GTX 260 shows its strength:

Speed: The GTX 260 is roughly 30-50% faster than the 8800 GT. It has more than double the CUDA cores (192 or 216 vs. 112) and a much wider memory bus (448-bit vs. 256-bit).

VRAM: The GTX 260 usually has 896MB, which is much better for dual-monitor setups than the 512MB found on most 8800 GTs.

Power: The 8800 GT is a "single-slot" card that requires one 6-pin power connector. The GTX 260 is a massive "dual-slot" card that requires two 6-pin connectors and generates significantly more heat.

3. Compatibility with your Hardware (P5B / P5PE-VM)
ASUS P5B Deluxe: Both cards will work well here, but the GTX 260 is a large card. Ensure your power supply can handle the 180W+ draw of a GTX 260.

ASUS P5PE-VM: I would not recommend the GTX 260 for this board. That board is quite compact and relies on older power delivery standards. The 8800 GT is the much safer, more period-accurate choice for a 10.5/10.6 build on that specific hardware.

Comparison Summary
Feature   NVIDIA GeForce 8800 GT   NVIDIA GeForce GTX 260
First OS Support   10.5.2 (Native)   10.6.0 (Usually needs Web Drivers)
Max OS Support   10.7.5 (Lion)   10.7.5 (Lion)
Power Connectors   1x 6-pin   2x 6-pin
Best Use Case   Leopard (10.5) Perfection   Snow Leopard (10.6) Powerhouse
Verdict: If your goal is to run 10.5 Leopard, the 8800 GT is the winner because it is "plug-and-play." If you are staying strictly on 10.6.8 and have a beefy power supply, the GTX 260 is the faster card, but it's a "dead end" for the older OS versions you've been exploring.

[chrisNova777]

was the 8800 GT the first to support 2 30 inch cinema displays?
Gemini said
Actually, no—while the 8800 GT was a popular and powerful choice for the 30-inch Cinema Display era, it was not the first card to support two of them.

To drive a 30-inch Apple Cinema Display at its native resolution of 2560 x 1600, a graphics card requires a Dual-Link DVI port. To drive two of them simultaneously, the card needs two independent Dual-Link DVI pipelines.
+1

The True Pioneers
Several cards preceded the 8800 GT (released in late 2007) in offering dual 30-inch support:

NVIDIA Quadro FX 4500 (2005): This was the high-end workstation beast for the PCIe Power Mac G5 and the first Intel Mac Pro. It was specifically marketed for its ability to drive two 30-inch Cinema HD Displays right out of the box.
+1

ATI Radeon X1900 XT (2006): When the first Intel Mac Pro launched, this was the "pro-sumer" upgrade option that featured two Dual-Link DVI ports, allowing for dual 30-inch setups long before the 8800 GT arrived.

NVIDIA GeForce 7800 GT/GTX (2005): While many consumer versions only had one Dual-Link and one Single-Link port, certain high-end versions (and the Mac-specific 7800 GT for the G5) were capable of dual 2560 x 1600 output.

Why the 8800 GT is often remembered for this:
The 8800 GT holds a special place in Mac history because it was the first affordable, single-slot card that made dual 30-inch displays accessible to more users.

Space Efficiency: Previous cards like the X1900 XT or Quadro FX 4500 were massive, "double-slot" cards that took up a lot of room and generated immense heat.

The "Upgrade" Era: It became the go-to upgrade for first-generation Mac Pro owners who wanted to run modern (at the time) dual-monitor setups without the reliability issues of the older ATI cards.

Technical Note: If you are setting this up today, remember that "Dual-Link DVI" refers to the number of pins and the bandwidth of the cable, not the number of monitor ports on the card. A standard DVI-to-HDMI adapter will usually limit you to 1920 x 1200, so you must use a true Dual-Link DVI cable to get that full 2560 x 1600 resolution on your Cinema Displays.