To a torus action on a complex vector space, Gelfand, Kapranov and Zelevinsky introduce a system of differential equations, which are now called the GKZ hypergeometric system. Its solutions are GKZ hypergeometric functions. We study the $p$-adic counterpart of the GKZ hypergeometric system. The $p$-adic GKZ hypergeometric complex is a twisted relative de Rham complex of over-convergent differential forms with logarithmic poles. It is an over-holonomic object in the derived category of arithmetic $\mathcal D$-modules with Frobenius structures. Traces of Frobenius on fibers at Techm\"uller points of the GKZ hypergeometric complex define the hypergeometric function over the finite field introduced by Gelfand and Graev. Over the non-degenerate locus, the GKZ hypergeometric complex defines an over-convergent $F$-isocrystal. It is the crystalline companion of the $\ell$-adic GKZ hypergeometric sheaf that we constructed before. Our method is a combination of Dwork's theory and the theory of arithmetic $\mathcal D$-modules of Berthelot.