With the ubiquity of powerful, mobile computers and rapid advances in sensing and robot technologies, there exists a great potential for creating advanced, perceptual computing environments. In this work, we investigate techniques for integrating passive, vision-based sensing into such environments, which include both conventional interfaces and large-scale environments. We discuss a methodology for vision-based human-computer interaction called the Visual Interaction Cues (VICs) paradigm. In the VICs paradigm, users interact with the computer system through natural hand gestures. We capture a "language of interaction" in which individual gestures are the words and sequences of gestures are the sentences. We give examples of both individual gestures and the gesture language, and present them on our next-generation interaction platform, the 4D Touchpad. Ultimately, our work takes a concrete step toward perceptual user interfaces.