A vector that lies in the vertical space at is termed a vertical vector at .
We typically use the notions of vertical space and vertical vector for a bundle map, where we are thinking of the base as sitting horizontally, and the fibers as sitting vertically. Then, a vertical vector is a vector that is literally vertical, and the vertical space is the vertical subspace of te tangent space.
There is no natural notion of a horizontal space in general. However, when we have a principal bundle, and if we provide a connection on it, then we do get a notion of horizontal space at each point, and this is a direct sum complement to the vertical space.