J. Fluid Mech. (2010), vol. 661, pp. 130–158. c  Cambridge University Press 2010 doi:10.1017/S0022112010003204 Elliptical instability in rotating cylinders: liquid metal experiments under imposed magnetic field W. HERREMAN 1,2 †, D. CEBRON 2 , S. LE DIZ ` ES 2 AND P. LE GAL 2 1 LRA, Ecole Normale Sup´ erieure, 24, rue Lhomond, F-75005 Paris, France 2 IRPHE, CNRS, Aix-Marseille Universit´ e, 49, rue Joliot Curie, F-13013 Marseille, France (Received 17 December 2009; revised 9 June 2010; accepted 10 June 2010; first published online 23 August 2010) In this paper, we present new theoretical and experimental results on the elliptical instability in a liquid metal contained in a rotating deformable cylinder in the presence of an imposed magnetic field. The imposed field, which is aligned with the rotating axis, has a double interest. On the one hand, it permits an analysis of the inertial waves excited by the elliptical instability by measuring their induced magnetic fields. On the other hand, it permits the control of the instability by acting on the Joule damping. In this paper, firstly an analytical calculation of the magnetic field induced by the flow and its associated Joule damping is presented. Also, the linear and weakly nonlinear theories of the elliptical instability are extended to include magnetic field effects. Then, the description of the experiments starts by the presentation of the effect of the imposed magnetic field strength. Close to the instability threshold, both super- and subcritical bifurcations are identified. When the imposed field is decreased, we observe a transition towards complex nonlinear evolutions that we describe with the help of two-dimensional phase diagrams. In a second set of experiments, we vary the eccentricity of the elliptic deformation over a large range in order to demonstrate that far from the instability threshold, the mean inertial wave amplitude is uncorrelated to the eccentricity. We show that, for a given eccentricity, this mean amplitude decreases when the rotation rate increases. In a last series of experiments, we focus on the description of the nonlinear evolution of an oscillatory eigenmode which is different from the principal stationary mode. Key words: magnetic fluids, vortex instability 1. Introduction The instability of flows with elliptic streamlines, named the elliptical instability, is recognized as a generic mechanism for the generation of small-scale turbulence in vortex systems (see for example the review of Kerswell 2002). The linear instability mechanism is understood as a parametric resonance of pairs of inertial waves. For small eccentricities, asymptotic theory leads to linear growth rate formulas, which are now available for many different elliptic vortices, in the presence of imposed magnetic fields (Kerswell 1993; Lebovitz & Zweibel 2004; Mizerski & Bayer 2009), density † Email address for correspondence: wietze.herreman@lra.ens.fr