Computer-aided detection of polyps in a colonoscopy video may enhance overall polyp detection. We investigated the feasibility of a boundary classification approach for automatic polyp detection. Given a colonoscopy image, the main idea is to identify the edge pixels that lie on the boundary of polyps. To do so, we first use the Canny edge detector to form a crude set of edge pixels, and then apply a set of boundary classifiers to remove a large portion of irrelevant edges. The polyp locations are determined by a novel vote accumulation scheme that operates on the positively classified edge pixels. Our results are comparable to the state-of-the-art and demonstrate the feasibility and promises of a boundary classification approach for automatic polyp detection.