In order to interpret the phonon-assisted optical transitions in semiconductor quantum dots, a theory is developed comprising the exciton interaction with both adiabatic and Jahn-Teller phonons and also the external nonadiabaticity (pseudo-Jahn-Teller effect). The effects of nonadiabaticity of the exciton-phonon system are shown to lead to a significant enhancement of phonon-assisted transition probabilities and to multiphonon optical spectra that are considerably different from the Franck-Condon progression. The calculated relative intensity of the phonon satellites and its temperature dependence compare well with the experimental data on the photoluminescence of CdSe quantum dots, both colloidal and embedded in glass.