1 RSMA Precoding Design Based on Interference Nulling and Sum Rate Upper Bound Elaheh Sadeghabadi, Student Member, IEEE, Steven Blostein, Senior Member, IEEE Abstract Rate splitting multiple access (RSMA), a recent generalization and merging of spatial division multiple access (SDMA) and superposition coding has exponential complexity. Hierarchical streams that transmit a subset of possible streams is proposed for rate splitting and analyzed. To reduce precoding complexity, RSMA with interference nulling (RSMA-IN), that nulls the portion of interference that is not decoded is investigated by formulating a sum rate maximization problem subject to a total power constraint. To solve this non-convex problem, a method that combines interference nulling and stream SNR maximization is proposed. A convex upper bound problem is formulated for the sum rate to compare existing algorithms, and conditions under which the upper bound problem is tight are determined. Taking advantage of the tight cases, we propose an upper bound aided (UBA) enhancement of existing precoding designs by exploiting optimal solutions in the tight cases. It is also shown that the existing augmented weighted mean square error (AWMSE) algorithms cannot converge to a local optimum in sum rate maximization. Simulation results indicate that the proposed approaches enhance performance over existing ones as well as approach their upper bounds in certain instances. Index Terms Rate-splitting multiple access, Broadcast channel, Multiple-input multiple-output communications, Nonorthogonal multiple access, Space-division multiple access. E. Sadeghabadi and S. Blostein are with Department of Electrical and Computer Engineering Queens’ University, Canada. Email: 19es6@queensu.ca and steven.blostein@queensu.ca. This work was sponsored by Huawei Technologies Canada. November 28, 2022 DRAFT