East Asian Math. J. Vol. 31 (2015), No. 3, pp. 383–391 http://dx.doi.org/10.7858/eamj.2015.031 SYSTEM OF GENERALIZED NONLINEAR MIXED VARIATIONAL INCLUSIONS INVOLVING RELAXED COCOERCIVE MAPPINGS IN HILBERT SPACES Byung-Soo Lee and Salahuddin Abstract. We considered a new system of generalized nonlinear mixed variational inclusions in Hilbert spaces and define an iterative method for finding the approximate solutions of this class of system of general- ized nonlinear mixed variational inclusions. We also established that the approximate solutions obtained by our algorithm converges to the exact solutions of a new system of generalized nonlinear mixed variational in- clusions. 1. Introduction It is well known that the variational inequality theory and complementar- ity problems are very powerful tools of the current mathematical technology. In recent years, classical variational inequality and complementarity problems have been extended and generalized into a large variety of problems arising in mechanics, physics, optimization and control theory etc., see [3, 4, 6, 10, 11]. Hassouni and Moudafi [12] introduced and studied a class of mixed type vari- ational inequalities with single-valued mappings which was called variational inclusions. Verma [22, 23, 24] studied some system of variational inequali- ties with single-valued mappings and suggested some iterative algorithms to compute approximate solutions of these systems in Hilbert spaces. As an ap- plication of system of variational inclusions Pang [20] showed that the traffic equilibrium problem, the Nash equilibrium problem and the general equilibrium problem, can be modeled as a system of variational inequalities and inclusions, see [5, 9, 16, 19]. Inspired and motivated by recent research works in this field, see [1, 2, 7, 8, 13, 14, 15, 17, 21, 25], we introduce a new system of generalized nonlinear mixed variational inclusions and suggest an iterative algorithm. By the defini- tion of relaxed cocoersive mapping and resolvent operator techniques, we find Received September 29, 2014; Accepted March 17, 2015. 2010 Mathematics Subject Classification. 49J40, 47H06. Key words and phrases. System of generalized nonlinear mixed variational inclusions, co- coercive mappings, Lipschitz continuity, resolvent operator, iterative sequences, Hausdorff metric, Hilbert spaces. c 2015 The Youngnam Mathematical Society (pISSN 1226-6973, eISSN 2287-2833) 383