Magnonic crystals are prototype magnetic metamaterials designed for the control of spin wave propagation. Conventional magnonic crystals are composed of single domain elements. If magnetization textures, such as domain walls, vortices and skyrmions, are included in the building blocks of magnonic crystals, additional degrees of freedom over the control of the magnonic band structure can be achieved. We theoretically investigate the influence of domain walls on the spin wave propagation and the corresponding magnonic band structure. It is found that the rotation of magnetization inside a domain wall introduces a geometric vector potential for the spin wave excitation. The corresponding Berry phase has quantized value $4 n_w \pi$, where $n_w$ is the winding number of the domain wall. Due to the topological vector potential, the magnonic band structure of magnonic crystals with domain walls as comprising elements differs significantly from an identical magnonic crystal composed of only magnetic domains. This difference can be utilized to realize dynamic reconfiguration of magnonic band structure by a sole nucleation or annihilation of domain walls in magnonic crystals.