I got HPPD after my first trip (3/4 tab of quality acid) and since then (about a year ago) I have probably smoked 3-4 ounces of grass, used alchohol 3-4 times a week, had two mild doses of speed, done e once (2 pills) acid twice (1/3 tab and 1/2 tab) nitrous oxide 20-30 times and salvia (leaf and 5x extract) 8-10 times. My symptoms have not gotten much worse, despite reports that use of other drugs (particularly psychedlics and grass) make them so. I have not had a particulaly strong psychedlic experience (the acid I took after getting HPPD was pretty crap) and do not use drugs to excess (apart from alcohol and dope).
The doses of drugs I have taken have been pretty mild, and my symptoms have worsened a little- just not to the extent described elsewhere.