Rapid advances in high-throughput molecular profiling such as DNA microarrays have created unprecedented opportunities to unravel the mechanisms that orchestrate the activities of genes and proteins in cells, where reconstruction of condition-specific biological networks directly from data has attracted great interest. In parallel, significant efforts have also been made to manually curate molecular interactions in cells, such as protein-protein interactions and biological pathways, providing constantly accumulated rich domain knowledge. Novel incorporation of biological prior knowledge into network learning algorithms can effectively leverage domain knowledge and make data-driven inference more robust and biologically relevant. However, biological prior knowledge is neither condition-specific nor context-specific, only serving as an aggregated source of partially-validated evidence under diverse experimental conditions. Hence, direct incorporation of imperfect and non-specific prior knowledge in specific problems is prone to errors and may lead to false positives. To address this challenge, we formulate the inference of condition-specific network structures that incorporates relevant prior knowledge as a convex optimization problem, and develop an efficient learning algorithm. We also propose a sampling scheme to estimate the expected error rate due to "random" knowledge and develop a strategy to manage such error in our algorithm that fully exploits the benefit of prior knowledge while remaining robust to the false positive edges in the knowledge. We test the proposed method on two simulation data sets and demonstrate the effectiveness of this method. The experimental results are consistent with our theoretical analysis. Finally, we apply our method to real ovarian cancer microarray data and obtain biologically plausible results.