Fast replacement models (or surrogates) have been widely applied in the recent years to accelerate simulation-driven design procedures in microwave engineering. The fundamental reason is a considerable-and often prohibitive-CPU cost of massive full-wave electromagnetic (EM) analyses related to solving common tasks such as parametric optimization or uncertainty quantification. The most popular class of surrogates are data-driven models, which are fast to evaluate, versatile, and easy to handle. Notwithstanding, the curse of dimensionality as well as the utility demands (e.g., so that the model covers sufficiently broad ranges of the system operating conditions), limit the applicability of conventional methods. A performance-driven modeling paradigm allows for mitigating these issue by focusing the surrogate setup process in a constrained domain encapsulating designs being of high quality w.r.t. the assumed figures of interest. The nested kriging framework capitalizing on this idea, renders the constrained surrogate using kriging interpolation, and has been shown to surpass traditional approaches. In pursuit of further accuracy improvements, this work incorporates the performance-driven concept into the fully-connected regression model (FRCM). The latter has been recently introduced in the context of frequency selective surfaces, and combined deep neural networks with Bayesian optimization, the latter employed to determine the network architecture and hyper-parameters. Using two examples of miniaturized microstrip couplers, our methodology is demonstrated to outperform both conventional modeling techniques and nested kriging, with reliable models constructed over multi-dimensional parameters spaces using just a few hundreds of samples.