Fabric defect inspection is necessary and essential for quality control in the textile industry. Traditionally, fabric inspection to assure textile quality is done by humans, however, in the past years, researchers have paid attention to PC-based automatic inspection systems to improve the detection efficiency. This paper proposes a novel automatic inspection scheme for the warp knitting machine using smart visual sensors. The proposed system consists of multiple smart visual sensors and a controller. Each sensor can scan 800 mm width of web, and can work independently. The following are considered in dealing with broken-end defects caused by a single yarn: first, a smart visual sensor is composed of a powerful DSP processor and a 2-megapixel high definition image sensor. Second, a wavelet transform is used to decompose fabric images, and an improved direct thresholding method based on high frequency coefficients is proposed. Third, a proper template is chosen in a mathematical morphology filter to remove noise. Fourth, a defect detection algorithm is optimized to meet real-time demands. The proposed scheme has been running for six months on a warp knitting machine in a textile factory. The actual operation shows that the system is effective, and its detection rate reaches 98%.