%0 Journal Article %T Necessary Conditions for Optimal Control of Forward-Backward Stochastic Systems with Random Jumps %A Jingtao Shi %J International Journal of Stochastic Analysis %D 2012 %I Hindawi Publishing Corporation %R 10.1155/2012/258674 %X This paper deals with the general optimal control problem for fully coupled forward-backward stochastic differential equations with random jumps (FBSDEJs). The control domain is not assumed to be convex, and the control variable appears in both diffusion and jump coefficients of the forward equation. Necessary conditions of Pontraygin's type for the optimal controls are derived by means of spike variation technique and Ekeland variational principle. A linear quadratic stochastic optimal control problem is discussed as an illustrating example. 1. Introduction 1.1. Basic Notations Throughout this paper, we denote by the space of -dimensional Euclidean space, by the space of matrices, and by the space of symmetric matrices. and denote the scalar product and norm in the Euclidean space, respectively. appearing in the superscripts denotes the transpose of a matrix. Let be a complete filtered probability space satisfying the usual conditions, where the filtration is generated by the following two mutually independent processes:(i)a one-dimensional standard Brownian motion ;(ii)a Poisson random measure on , where is a nonempty open set equipped with its Borel field , with compensator , such that is a martingale for all satisfying . £¿ is assumed to be a -finite measure on and is called the characteristic measure. Let be fixed and be a nonempty subset of . We denote . Any generic point in is denoted by . Let be the set of all -predictable processes such that . Any is called an admissible control process. We denote by or the set of integrable functions with norm . We define We denote Clearly, is a Banach space. Any process in is denoted by , whose norm is defined by 1.2. Formulation of the Optimal Control Problem and Basic Assumptions For any and , we consider the following fully coupled forward-backward stochastic control system: with the cost functional given by Here, takes value in and For any and , we refer to as the state process corresponding to the admissible control if FBSDEJ (1.4) admits a unique adapted solution. For controlled FBSDEJ (1.4) with cost functional (1.5), we consider the following. Problem C. Find , such that Any satisfying (1.7) is called an optimal control process of Problem C, and the corresponding state process, denoted by , is called optimal state process. We also refer to as an optimal 5-tuple of Problem C. Our main goal in this paper is to derive some necessary conditions for the optimal control of Problem C, which is called the stochastic maximum principle of Pontraygin¡¯s type. For this target, we first introduce the following basic %U http://www.hindawi.com/journals/ijsa/2012/258674/