The unstable feature of the widely employed organic hole‐transporting materials (HTMs) (e.g., spiro‐MeOTAD) significantly limits the practical application of perovskite solar cells (PSCs). Therefore, it is desirable to design new structured PSCs with stable HTMs presenting excellent carrier extraction and transfer properties. This work demonstrates a new inverted PSC configuration. The new PSC has a graded band alignment and bilayered inorganic HTMs (i.e., compact NiOx and mesoporous CuGaO2). In comparison with planar‐structured PSCs, the mesoporous CuGaO2 can effectively extract holes from perovskite due to the increased contact area of the perovskite/HTM. The graded energy alignment constructed in the ultrathin compact NiOx, mesoporous CuGaO2, and perovskite can facilitate carrier transfer and depress charge recombination. As a result, the champion device based on the newly designed mesoscopic PSCs yields a stabilized efficiency of ≈20%, which is considered one of the best results for inverted PSCs with inorganic HTMs. Additionally, the unencapsulated PSC device retains more than 80% of its original efficiency when subjected to thermal aging at 85 °C for 1000 h in a nitrogen atmosphere, thus demonstrating superior thermal stability of the device. This study may pave a new avenue to rational design of highly efficient and stable PSCs.