Consider the class QS of all non-degenerate planar quadratic differential systems and its subclass QSE of all systems possessing an invariant ellipse. In this paper we classify the family QSE according to their geometric properties encoded in the configurations of invariant ellipses and invariant straight lines which these systems could possess. The classification, which is taken modulo the action of the group of real affine transformations and time rescaling, is given in terms of algebraic and geometric invariants and also in terms of invariant polynomials and it yields a total of 35 distinct such configurations. This classification is also an algorithm which makes it possible to verify for any given real quadratic differential system if it has invariant ellipses or not and to specify its configuration of invariant ellipses and straight lines. This work will prove helpful in studying the integrability of the systems in QSE. It is also a stepping stone for studying the topological classification of this family. Since it is known that the maximum number of limit cycles occurring in systems of QSE is 1, this goal is thus not out of reach at the moment.