Ethnic segregation is generally seen as an obstacle to immigrants’ integration. Therefore, it is not surprising that immigrants’ who live in a neighborhood with high shares of co-ethnics show low identification with the host country society (national identification). Previous research, however, is rather vague concerning the underlying mechanisms of this pattern. Building on a general model of identity formation by Deaux and Martin (2003), we seek to clarify why and how ethnic segregation can be assumed to affect national identification. In particular, we argue that immigrant groups differ with respect to the compatibility of national identity and ethnic group identity. Our main hypothesis is that immigrants from groups with less compatible national and ethnic identities who strongly identify with their ethnic group are more strongly affected by the share of co-ethnic neighbors than immigrants who do not or only weakly identify with their ethnic group. We test this hypothesis using newly available large-scale data from the National Educational Panel Study (NEPS) in Germany. Focusing on the descendants of the main immigrant groups in Germany, i.e. on children of parents from the former Soviet Union, Turkey, former Yugoslavia and Eastern as well as Southern Europe, our findings confirm our hypothesis; the effects of ethnic segregation on national identification indeed seems to be conditional on the level of ethnic identification and the (in)compatibility of ethnic and national identity.