efg's Graphics -- Very Large Bitmap Experiment

Purpose
The purpose of this experiment is to find the Windows "breaking point" for creation of in-memory bitmaps. (Windows cannot handle very large bitmaps and the "breaking point" differs considerably among various PCs.) A secondary purpose is to explore the performance of an algorithm to count the number of unique colors in a 24-bit bitmap.

Delphi Version	Windows	CPU	System Memory [MB]	Video Adapter	N-by-N Bitmap Size [pixels]	Creation Time [seconds]	Creation / Count Colors Time [seconds]	Unique Colors	By
D3	98 2nd Ed	PII 450	128	STB nVIDIA TNT (16 MB)	5372	0.7	19.0	16,777,216	efg (PixelFormat First)
D3	98	PII 450	128	STB nVIDIA TNT (16 MB)	2043	0.3	3.3	4,173,849	efg (PixelFormat Last)
D4	98	PII 450	128	STB nVIDIA TNT (16 MB)	2043	0.3	3.0	4,173,849	efg (PixelFormat Last)
D3	95	P 166	32	Matrox MGA Millennium PowerDesk (8 MB)	2360	17.4	41.7	5,569,600	efg (PixelFormat Last)
D3	98	P 166	32	S3 Vision968 PCI	2890	32.5	168.9	8,352,100	efg (PixelFormat Last)
D3	98	P 120	32	Cirrus Logic 7543 PCI	2890	40.6	116.4	8,352,100	efg (PixelFormat Last)
D3	NT 4.0	P 166	32?	MGA Millennium	>4096?	857.9			efg (PixelFormat Last)
D4	Windows 2000 Beta	Dual PII 350	128	Diamond Viper 330 AGP (4 MB RAM).	6272	378.6	459.6	16,777,216	EC
D3	NT 4.0 SP4	P 200	96	Homemade system with Asus VX Motherboard	4352	Not reported	Not reported. System unstable after "successful" test	16,777,216	LR
D3	NT 4 Build 1381 SP 5	PII 350	256	Elsa Erazor II, 16 MB VRAM (TNT2)	9216	98.8	225	16,777,216	MS
D3	Win 2000	P166 MX	96	Matrix Millennium (2 MB)	6400	18.0	N/A	16,777,216	LP
D3	Win 2000	P166 MX	96	Matrix Millennium (2 MB)	10,112	59.3	591.8	16,777,216	LP
D3	Win 98	PII 400	256	Rage II AGP (4 MB)	10,240	40.0	640	16,777,216	CZ
D3	Win NT 4.0 SP4	500MHz P3	512	SGI Visual Workstation 320: There is no graphic card inside, as the whole machine is THE graphic card.	4,096	0.8	10.9	16,777,216	RT
					10,240	5.1	64.8
					11,776	6.6	82.9
					12,800	212.4	N/A-
					16,384	311.0	N/A-
D5	Win 2000	650 MHz P3 (Dell Inspiron 7500)^*	192	ATI Rage Mobility-P AGP2X (8 MB)	8,192	16.9	88.5	16,777,216	efg
					10,240	23.8	213.7
					12,800	55.6	468.4
					15,040	105.8	799.2
D5?	Win 2000	Intel dual P3-500-xeon	768	GeForce2GTS w/64MB	14,336	3.4	N/A	-	BD
					15,360	15.5	N/A
					16,384	28.5	N/A
D3	NT4 SP6	Athlon 850	512	Asus V7700 (NVIDIA GL, 32 M)	4,096	0.4	7.0	16,777,216	LR
					8,192	1.5	27.1
					12,288	8.7	65.8
D?	Win 2000 SP2	Pentium 4 Xeon 1.7 GHz	256	Dell Workstation PWS530	16,384	67-69			DSW
					14,336	47.9			DSW
					12,288	31.4			DSW
					10,240	14.5			DSW
					8,1992	2.0			DSW
D?	Win XP	Dell Latitude C840, P4, 1.8 GHz	512	GeForce 4 GO 440, 64 MB	16,384		372.8	16,777,216	MP
D?	Win 200 SP 3	Athlon XP 1.4 GHz	896	3Asus V7700 (NVIDIA GL, 32 M)	8,192	0.8	15.2	16,777,216	LR
					10,240	1.2	21.8
					12,288	1.8	31.4
					14,336	2.4	44.2
					16,384	6.6	57.9

Note: efg = Earl F. Glynn, EC=Esben Carlsen, LR=Leif Rudd, MS=Marco Schmidt, CZ=Claus Ziegler, RT = Ing. Buero R. Tschaggelar, LP = Lazikas o Pontios, BD = Ben Discoe, DSW = Daren Scot Wilson, MP = Mickey Petersen

For PixelFormat First, see the code shown below with the comment "// Do this first". This statement appears after assigning Height and Width with PixelFormat Last.

Determine Video Adapter via:
My Computer | Control Panel | System | Device Manager | Display Adapters

^*In Windows 2000, the Windows Task Manager has a Performance tab with graphs for CPU Usage History and Memory Usage History. For very large bitmaps, the CPU usage is quite low and the memory usage is pegged near the top of the graph. With the 12,800-by-12,800 bitmap and above, Windows 2000 gave a message "Your system is low on virtual memory."

If the largest bitmap that you can create is either larger or smaller than shown in the table above, please send me your results and I'll extend the table.

Discussion
Many programmers naively think that a bitmap of any size can be created in Windows. Some Delphi programmers immediately blame this problem on Borland, but the limitation is caused by Windows and the video device driver.

Part of the CreateBitmapClick method simple tries to create a bitmap of the specified size and traps any EOutOfResources exception:

The Do this first comment above was suggested by Alexandre Bento Freire (see below) for much better performance. When a bitmap is created by assigning the Bitmap.Width and Bitmap.Height first, followed by the Bitmap.PixelFormat, significant additional resources are required than if the PixelFormat is specified first (as suggested above by Alexandre).

Once the large pf24bit bitmap has been created, each pixel is assigned a unique color (assuming the bitmap has 16,777,216) using Scanline:

CONST
MaxPixelCount = 65536;

TYPE
// For pf24bit Scanlines
pRGBTripleArray = ^TRGBTripleArray;
TRGBTripleArray = ARRAY[0..MaxPixelCount-1] OF TRGBTriple;

VAR
index: INTEGER;
i,j : INTEGER;
row : pRGBTripleArray;
...
// Assign unique color to first 16,777,216 pixels
index := 0;
FOR j := 0 to Bitmap.Height-1 DO
BEGIN
row := Bitmap.Scanline[j];
FOR i := 0 TO Bitmap.Width-1 DO
BEGIN
    WITH Row[i] DO
    BEGIN
      rgbtRed    := index AND $000000FF;
      rgbtGreen := (index AND $0000FF00) SHR 8;
      rgbtBlue   := (index AND $00FF0000) SHR 16
    END;
    INC(index)
END
END;

Once the large bitmap is created and filled with unique colors, the number of unique colors is counted (when requested) using a 2D array of TBits:

// Count number of unique R-G-B triples in a pf24bit Bitmap.
//
// Use 2D array of TBits objects -- when (R,G) combination occurs
// for the first time, create 256-bit array of bits in blue dimension.
// So, overall this is a fairly sparse matrix for most pictures.
// Tested with pictures created with a known number of colors, including
// a specially constructed image with 1024*1024 = 1,048,576 colors.
//
// efg, October 1998.
FUNCTION CountColors(CONST Bitmap: TBitmap): INTEGER;
VAR
    Flags: ARRAY[BYTE, BYTE] OF TBits;
    i    : INTEGER;
    j    : INTEGER;
    k    : INTEGER;
    rowIn: pRGBTripleArray;
BEGIN
// Be sure bitmap is 24-bits/pixel
ASSERT (Bitmap.PixelFormat = pf24Bit);

// Clear 2D array of TBits objects
FOR j := 0 TO 255 DO
    FOR i := 0 TO 255 DO
      Flags[i,j] := NIL;

// Step through each scanline of image setting bits
FOR j := 0 TO Bitmap.Height-1 DO
BEGIN
    rowIn := Bitmap.Scanline[j];
    FOR i := 0 TO Bitmap.Width-1 DO
    BEGIN
      WITH rowIn[i] DO
      BEGIN

        IF NOT Assigned(Flags[rgbtRed, rgbtGreen])
        THEN BEGIN
          // Create 3D column when needed
          Flags[rgbtRed, rgbtGreen] := TBits.Create;
          Flags[rgbtRed, rgbtGreen].Size := 256;
        END;

        // Mark this R-G-B triple
        Flags[rgbtRed,rgbtGreen].Bits[rgbtBlue] := TRUE
      END
    END
END;

RESULT := 0;
// Count and Free TBits objects
FOR j := 0 TO 255 DO
BEGIN
    FOR i := 0 TO 255 DO
    BEGIN

      IF Assigned(Flags[i,j])
      THEN BEGIN
        FOR k := 0 TO 255 DO
          IF Flags[i,j].Bits[k]
          THEN INC(RESULT);
        Flags[i,j].Free;
      END

    END
END

END {CountColors};

This 2D array of TBits is usually a sparse array for a "real world" picture, but the performance is still satisfactory when the array is not sparse at all.

The warning "DO NOT BE SURPRISED IF YOUR SYSTEM CRASHES. USE CAUTIOUSLY." is for those that can't read smaller print. (In past experiments I mentioned I was stressing system resources and then I received E-mail from some that were surprised with the program crashed Windows!)

This experiment was performed using 24-bit (pf24bit) bitmaps. Larger bitmaps should be possible using smaller PixelFormats.

Example of scrolling in-memory TBitmaps as an image marquee. Creating one large "horizontal" bitmap would break something, but scrolling in-memory TBitmaps is relatively easy using CopyRect. ImageMarquee.ZIP

Borland's FAQ 2418D, "Handling bitmap display," discusses size limitations of Bitmaps.

Comments from Joe Hecht (aka ZeppinHood in UseNet post)
Why not just read the Microsoft article that explains the limitation and live with it? I'm sure you have read some of my posts that sum it up as:

Suggestion by Alexandre Bento Freire:
When I tried to:
        Bitmap.Width := W;
   Bitmap.Height := H;                                 (1)
   Bitmap.PixelFormat := pf24bit;

I got Out of Resources in large bitmaps.

But when i modify the code to:

   Bitmap.PixelFormat := pf24bit;
   Bitmap.Width := W;                                 (2)
   Bitmap.Height := H;

That boundary enlarged a lot. That is: bitmaps that resulted on Out of Resources, using (1), now with (2), they created ok. With this method i can even create 4000x4000 bitmaps on large number of computers.

I hope this can be relevant.
Alexandre Bento Freire

[Thanks Alexandre. I assume this creates a DIB directly instead of a DDB first that is converted to a DIB. -- efg]

Alexandre Bento Freire pointed out that if the bit depth is set before the bitmap size, the allowable bitmap size is increased.

I tested this on my old Chips LCD, Cyrix p200+, win98, 130mb RAM, Delphi 3 C/S system. First I ran the VeryLargeBitmap.exe and was able to create bitmaps of a maximum size of 2816^2 pixels.

By changing the code from :
   Bitmap.Width := SpinEditPixels.Value;
   Bitmap.Height := SpinEditPixels.Value;
   Bitmap.PixelFormat := pf4bit;

To:
   Bitmap.PixelFormat := pf4bit;
   Bitmap.Width := SpinEditPixels.Value;
   Bitmap.Height := SpinEditPixels.Value;

I was able to create bitmaps of max size 7808^2 pixels!!!

Pretty dramatic difference eh?

Comments by Steve Bliss (in UseNet Post)
"The max size of a bitmap in Windows varies from video card to video card, and I do not mean the screen resolution being used on a system, I mean it is limited by the video card driver. As a rule of thumb, anything bigger than about 2,000 x 1,600 will fail on some systems. In your case I would use four images instead of one to be safe."

Comments by Rick Rogers (in UseNet Post)
"You can't make "really BIG" graphics -- Windows and/or your video drivers won't allow it. Frequently, you won't be able to make a bitmap larger than the screen. However, you really don't ever need to do so."

"You can fairly easily make something *appear* to be a large graphic, while in fact just drawing what is necessary onto the screen. For your example of a large Gantt chart, you will be maintaining the data to be represented in the Gantt chart somewhere, such as a database. You'll create some routines to pull the data out of the database and draw it onto a canvas. You will need to keep track of the user's position within the Gannt chart (such as current date). Then you simply draw onto the screen the relevant subset of the Gantt chart data. When the user changes their current position (for example, by scrolling), you simply redraw onto the screen the next subset of the Gantt chart data. You can set the scrollbar's Max property to an appropriately large number (such as the number of days covered by the Gantt chart), and then simply redraw the current subset onto the screen."

"The process that I've described -- keeping track of current position and drawing a current subset onto the screen -- is how just about every single Windows application with scrollbars works."

Comments by Peter Below (in UseNet Post):
"The maximum bitmap size actually depends on the video driver. The only safe assumption you can make here is that you can create a bitmap with the number of pixels equivalent to the screens size in the maximum color depth the video driver supports. This is a limitation of device-dependent bitmaps. Device-independent bitmaps are not limited this way, so try to set the bitmaps HandleType to bmDIB and select an appropriate PixelFormat before you draw on it."

Comments by Daren Scot Wilson
"...why would anyone need a bitmap bigger than the screen? Try looking at some of data from NASA's Terra satellite. Here at ARC Science, we routinely work on images of 4Kx4K, 8K x 8K, or bigger, and for some projects involving mosaics of several satellite photos, even 40K or so pixels on a side. Of course to view these we StretchBlt or scroll these images, but the custom software we work with as well as the commercially available tools must handle this data as whole images. More generally, anyone working in Earth sciences or geography using GIS tools and satellite data may need to work with such huge images."

Conclusions
There are significant limitations in creating a very large bitmap in Windows 95/98. Surprisingly, having a lot of CPU memory and video card memory doesn't necessarily mean a larger bitmap can be created. Resource limitations (mostly available memory) appear to be the only restriction for Windows NT/2000/XP.

One of the largest image databases is Microsoft's Terraserver at http://terraserver.microsoft.com. Note that Microsoft breaks a large bitmap into a number of smaller bitmaps arranged in a matrix.

Keywords
TBitmap, EOutOfResources, Scanline, pf24Bit, TRGBTripleArray, pRGBTripleArray, TBits

		Very Large Bitmap Experiment		Lab Report
Chinese Translation by Hector Xiang

Experiment in Creating Very Large Bitmaps in Windows

	Above, a Dell XPS R450 with 128 MB memory and 16 MB on video adapter cannot create a 2044-by-2044 24-bit bitmap. However, an older Dell XPS P166s with 32 MB memory and a lesser video adapter can create a 2890-by-2890 bitmap.