Visual perceptual learning models, as constrained by orientation and location specificities, propose that learning either reflects changes in V1 neuronal tuning or reweighting specific V1 inputs in either the visual cortex or higher areas. Here we demonstrate that, with a training-plus-exposure procedure, in which observers are trained at one orientation and either simultaneously or subsequently passively exposed to a second transfer orientation, perceptual learning can completely transfer to the second orientation in tasks known to be orientation-specific. However, transfer fails if exposure precedes the training. These results challenge the existing specific perceptual learning models by suggesting a more general perceptual learning process. We propose a rule-based learning model to explain perceptual learning and its specificity and transfer. In this model, a decision unit in high-level brain areas learns the rules of reweighting the V1 inputs through training. However, these rules cannot be applied to a new orientation/location because the decision unit cannot functionally connect to the new V1 inputs that are unattended or even suppressed after training at a different orientation/ location, which leads to specificity. Repeated orientation exposure or location training reactivates these inputs to establish the functional connections and enable the transfer of learning. Copyright © 2010 the authors.