This paper examines a novel binary feature referred to as the Local Hybrid Patterns (LHP) that is generated by mixing highly discriminative bits of the binary local pattern features (BLPFs) such as the Local Binary Patterns (LBP), Local Gradient Patterns (LGP), and Mean LBP (MLBP). Starting with the most discriminative BLPF selected, the LHP generating algorithm iteratively updates the bits of the selected BLPF by replacing the least discriminative bit with the most discriminative bit of all the candidate BLPFs. At the expense of a small increase in computation, the LHP is guaranteed to give smaller or equal empirical error compared to any BLPFs considered in the pool. Experimental comparison of different sets of features consistently shows that the LHP leads to better performance than previously proposed methods under the AdaBoost face detection framework on MIT+CMU and FDDB benchmark datasets.