Here are some graphical programming environments that you can use for realtime manipulation or triggering of video, audio, etc. which are dandy tools for making installations or for VJ (video jockey) visuals. (Did I explain that right? I think so.)
Max/DSP
EyesWeb
Pure Data.