Mammalian gut microbiota are integral to host health. However, how this association began remains unclear. We show that in basal chordates the gut space is radially compartmentalized into a luminal part where food microbes pass and an almost axenic peripheral part, defined by membranous delamination of the gut epithelium. While this membrane, framed with chitin nanofibers, structurally resembles invertebrate peritrophic membranes, proteome supports its affinity to mammalian mucus layers, where gut microbiota colonize. In ray-finned fish, intestines harbor indigenous microbes, but chitinous membranes segregate these luminal microbes from the surrounding mucus layer. These data suggest that chitin-based barrier immunity is an ancient system, the loss of which, at least in mammals, provided mucus layers as a novel niche for microbial colonization. These findings provide a missing link for intestinal immune systems in animals, revealing disparate mucosal environment in model organisms and highlighting the loss of a proven system as innovation.