The ability to dynamically adapt to changes in stimulus statistics is one of the defining features of biological sensory systems. However, despite a long history of experimental research, it is not known how to characterize the dynamics of adaptation from an information processing perspective. Here we introduce a theoretical framework for optimizing and analyzing the temporal structure of adaptation in neural sensory codes. We show that sensory codes optimized for performing task-relevant computations are typically different from codes optimized for adapting to changes in the stimulus distributions that underlie these computations. These differences are manifested in the speed of adaptation, the accuracy of the code during periods of adaptation, and the accuracy in the adapted state. We derive encoding schemes that interpolate between task performance and adaptability, and can leverage the advantages of both objectives. Our results provide a unifying perspective on adaptation across a range of sensory systems, environments, and sensory tasks.