Research Article Ground State Solutions for a Nonlocal System in Fractional Orlicz-Sobolev Spaces Hamza El-Houari , Hicham Moussa, and Lalla Saˆ adia Chadli Sultan Moulay Slimane, Beni Mellal 23000, Morocco Correspondence should be addressed to Hamza El-Houari; h.elhouari94@gmail.com Received 19 October 2021; Revised 18 January 2022; Accepted 16 March 2022; Published 20 May 2022 Academic Editor: Sining Zheng Copyright © 2022 Hamza El-Houari et al. is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. We consider an elliptic system driven by the fractional a(.)-Laplacian operator, with Dirichlet boundary conditions type. By using the Nehari manifold approach, we get a nontrivial ground state solution on fractional Orlicz–Sobolev spaces. 1. Introduction We consider the following fractional elliptic system (−Δ) s a 1 (.) u F u (x, u, v) in Ω, (−Δ) s a 2 (.) v F v (x, u, v) in Ω, u v 0 onR N /Ω , (1) where Ω is a bounded open subset of R N with Lipschitz boundary zΩ, s (0, 1), and (−Δ) s a i1,2 (.) is the nonlocal fractional a i (.)-Laplacian operator of elliptic type intro- duced in [1] and is defined as (−Δ) s a i (.) u(x) P.V R N a i |u(x)− u(y)| |x y| s u(x)− u(y) |x y| s dy |x y| N , (2) for all x R N , where P.V is the principal value and a i1,2 : R + R + are of class C 2 and satisfies the following conditions: (ϕ 1 ) lim t0 ta i (t) 0, and lim t⟶∞ ta i (t) . (ϕ 2 ) t ta i (t) is strictly increasing. For example, when we choose a 1 (t)|t| p(x)− 2 t and a 2 (t)|t| q(x)− 2 t for t > 0, then a i satisfies (ϕ 1 ) and (ϕ 2 ). In this case, the operator (2) is named fractional p(.)-Laplacian operator and reads as (−Δ) s p(.) u(x) lim ϵ⟶0 R N \B ϵ |u(x)− u(y)| p(x)− 2 (u(x)− u(y)) |x y| N+(s1)p(x,y) dy , for x R N , (3) and the system (1) reduces to the fractional (p(.),q(.))-Laplacian system studied in [2] and given as (−Δ) s p(.) u F u (x, u, v) in Ω, (−Δ) s q(.) v F v (x, u, v) in Ω, u v 0 on R N /Ω. (4) e existence of solutions for systems like (3) has also received a wide range of interests. For this, we find that, in the literature, many researchers have studied this type of system using some important methods, such as variational method, Nehari manifold and fibering method, and three critical points theorem (see for instance [3–6]). is kind of operator can be used for many purposes, such as phase transition phenomena, population dynamics, and Hindawi International Journal of Differential Equations Volume 2022, Article ID 3849217, 16 pages https://doi.org/10.1155/2022/3849217