You said there are 4 browns and 3 reds in Menacing...
However the only colors I'm allowed to use for vgaspec.dat files are red,orange,green,yellow,blue,cyan and purple...
No, no, no. Sorry, looks like there's a misunderstanding.
myvgaspec will take the first 7 color it finds in your bitmap, but they can be any colors. The example used the rainbow colors to help highlight the fact that there are 7 colors, but you're not stuck with just those specific 7.
I tried making a vgaspec.dat file with white pixels,it didn't work.I tried doing the same in orange and it didn't work either.
The problem might have nothing to do with colors. For example, what's the dimensions (width and height) of the bitmap you're trying to use? The width must be 960 or less. Also, your bitmap has to be of BMP format, and must be a 24 bpp BMP (in Windows, open up your graphics in MS Paint and do a Save As, making sure to select under "Type" the one that has "24" mentioned).
myvgaspec should give you fairly descriptive error messages pinpointing why it doesn't accept your bitmap. Tell me which error message it gave you.