Steel strings sound terrible on harps, as you've observed. There seem to be several reasons. To see why you need to think about how strings work, and how harps work differently from guitars.
When you pluck a string it actually produces three different periodic forces signals on the end point at the soundboard. If you think of the string as vibrating 'up and down' it's pulling that point up and down along with it. It can also, of course, move from side to side, and that component of the motion doesn't produce sound from this signal. This 'transverse' signal is, in general, the strongest one the string produces.
As the string vibrates it gets longer and shorter, and that makes the tension change. Because the string gets longer twice for every full cycle of vibration, this signal is 'frequency doubled'; it has energy at even multiples of the string's fundamental pitch. This signal has much less force than the 'transverse', usually about 1/7 of the amplitude.
The third signal is the 'longitudinal' wave, a compression wave that's produced in a string any time you pluck it off center in it's length. The frequency of this is almost solely a product of the string material and length, and bears no necessary relationship to the pitch of the string. It' usually somewhere around the 7th or 8th partial of the string. In pianos they call this the 'clang tone'; on guitars it's the 'zip tone' that you hear when you run a finger along a wound string. It's a little hard to compare the strength of this signal with the other two, for one thing, it tends to come and go periodically depending on a bunch of factors. It does seem to be pretty large in amplitude in some cases.
Guitars are almost solely driven by the 'transverse' signal. It's pulling the bridge 'up and down', which moves the top like a loudspeaker cone. It's fairly easy to do, and effective at making sound. Both the 'tension' and 'longitudinal' signals work by rocking the bridge fore and aft. This is hard to do; guitars are designed to resist this. It's also less efficient, since part of the soundboard moves 'up' as another part moves 'down'. Guitars thus get some energy from the 'tension' and 'longitudinal' signals, but not enough to add any real power; it mostly produces changes in the timbre, particularly depending on the height of the strings off the top.
Harps are another matter. Because the strings pull upward on the soundboard all of the string signals, the 'transverse' , 'tension' and 'longitudinal', produce sound about equally effectively depending on how much force they put in. The proportional effectiveness depends to some degree on the exact string angle. As a string material, steel tends to have relatively more energy in the 'tension' signal as compared with the 'transverse'. This will not be a problem on a harp, and my actually produce a nicer sound. The 'longitudinal' signal seems to be the main culprit.
The problem here is two fold; the 'longitudinal' signal is stronger on steel strings for the same reasons the 'tension' signal is: the difficulty of stretching a steel string. More than that, though, is the fact that it's at a higher frequency on steel than nylon. This in itself tends to make it more audible and irritating: your hearing tends to be more sensitive at those high frequencies, and pitches up in the upper octave of the range (say, above 15kHz) are particularly bad. You'd be in that range of frequencies with the 'clang' tone of your middle octave with steel strings. The fact that the frequency is generally dissonant only adds insult to injury.
Metal strung harps use brass or bronze wire in general. The 'clang tone' is lower in pitch due to the properties of the material, and not as powerful. These strings generally tune about a fourth or fifth lower in pitch than nylon or steel, so even though the clang tone pitch is still dissonant, at least it's not in such an irritating range.
You may have noticed that I said that the 'clang tone' is driven when you pluck (or strike) the string off center in it's length. Harpists are often told to play in the middle of the string, and that will cut the 'clang tone' down to nothing, or nearly so. Playing a guitar string in the middle sounds lousy: it lacks all of the even partials because you've plucked it just where the string wants to be still for those tones. They're there in the 'tension' signal, but not very effective at producing sound. With the strings pulling upward on the soundboard on a harp the 'tension' signal can get into the act in a useful way. Plucking the string in the middle actually maximizes the power of the tension signal relative to the transverse, and can produce a full tone with all the partials on a harp.
All of which is the long way around to saying: "There are reasons behind tradition".
As far as using 'better' strings; so far as I know all of the string manufacturers get their material from the same big companies, like DuPont or BASF. There are dozens of kinds of 'nylon' with different properties, and the string makers pick the one that will make the sound they want. There can be problems with the stock they get from the manufacturers: often the nylon in the middle of a spool will be squashed out of round, which makes it vibrate in an odd way (to put in mildly!). Some of the guitar string makers try to correct this by grinding the strings round, and sometimes it works. It's not a different material, though. Harp string suppliers sell the same stuff, and they buy some of it colored blue and red. That's hard to do with plain clear nylon.